Sending as much data as you want for the price of 1 data input

Is this possible?

Yes, that’s possible. Today I will teach you how you can save money by sending multiple data inputs as if it was only 1. To do this, we must build a custom Payload Parser, you may never have done this before, but I will demonstrate just how simple it is.

image

Why should I build a custom Payload Parser?

The main reason for you to do this is that it enables you to merge your variables into the metadata of a “main variable”. And so we are taking advantage of the fact that you can put whatever you want into the metadata of the variables we want and have it only count as 1 data input.

Ok, but what are the trade-offs?

This practice saves you money, but there are some disadvantages to it. The first disadvantage is that you will not be able to use variables within the metadata in Actions. So, if you need to use Actions on a variable, don’t put it in the metadata. Another disadvantage is that the data within the metadata will not be indexed, which means that you will not be able to work with it in analysis.

As you can see, if your data will need some processing like Analysis or Actions, it is not recommended that you submit it within metadata. Otherwise, you can send them through metadata and show it in widgets later using the Formula feature. I explain this process in more depth in this post (Using Formula to display metadata values for my variable).

How can I implement it into my application?

Finally, I will show you how to implement this method into your application. Imagine that you have 4 variables that are: current, frequency, voltage and power. Of these 4 variables, your Actions and Analyzes only process the voltage variable. From what we saw earlier, we now know that the other 3 variables can be placed in the voltage variable metadata.

So here is what we are going to do; Go to the Device that is sending the data and go to the Payload Parser tab. Then, enable the option “Run your own parser”.

Note that in the image above, when you enable this option, a snippet comes ready in the editor. With some changes to this snippet, we will be ready to send multiple data inputs as one. The next step is to configure the variable that I highlighted in the image above: ignore_vars. This variable is what will ignore data that we don’t want to be counted.

So, since I want the variables current, frequency and power to be in the metadata of the voltage variable, and I don’t want them to be counted as a data input, I will put these variables inside that array and it will look like this:

const ignore_vars = ['current', 'frequency', 'power']; 

After that, I search for the variables I want and insert them into the voltage variable metadata. With all this, the code looks like this:

// Add ignorable variables in this array. 
const ignore_vars = ['current', 'frequency', 'power']; 
 
// Getting the variables I want to insert into the tension variable metadata 
const current = payload.find((x) => x.variable === 'current'); 
const frequency = payload.find((x) => x.variable === 'frequency'); 
const power = payload.find((x) => x.variable === 'power'); 
 
// Getting the index of the tension variable 
const tensionIndex = payload.findIndex((x) => x.variable === 'tension'); 
 
// If the tension does not have metadata field, I add a 
// new object to the metadata field 
if (tensionIndex !== -1 && !payload[tensionIndex].hasOwnProperty('metadata')) { 
  payload[tensionIndex].metadata = {} 
} 
 
// After all that is complete, I add my variables into the metadata 
if (tensionIndex !== -1) { 
  payload[tensionIndex].metadata.current = current.value; 
  payload[tensionIndex].metadata.frequency = frequency.value; 
  payload[tensionIndex].metadata.power = power.value; 
} 

// Remove unwanted variables. 
payload = payload.filter(x => !ignore_vars.includes(x.variable)); 

The code is all commented upon so that you may understand it better and so that you can adapt it to use with the variables you want. If you have any questions related to the code, ask here in this post.

Well, that’s what I wanted to teach you in this post. Remember, to display the data that is in the metadata, you must use the Formula feature. Take a look at this post talking about it (Using Formula to display metadata values for my variable).

Thanks!

Do you have an example of a payload parser such as convert raw json to tagi json snippit when these commands are integrated?

Hi @dave

Unfortunately I don’t have a snippet like the one you want integrated with these commands to decrease the number of data inputs. The script available in the post was built to be used as a payload parser in addition to the payload parsers on your device. So, if you choose your device from the Add Device list, you will already have an internal script that will convert the data to the TagoIO format, just use the post script to reduce the data inputs.

I hope this can help you, thanks!

Hello, community

I following the instrunction for this issue, however it s almost working fine: I using device simulator do add to device

[
{“variable”:
“m1s1p”,
“unit”: “mg”,
“value”: 2.055,
“metadata”: {
“m1s1d”: 45.211,
“m1sdf”: 59.211
}
}
]
The problem is :- from widget on dashboard not show the values contained into metadata, however if testing the same payload, in the menu formula into wdiget option test it - the value is showed
[
{“variable”:
“m1s1p”,
“unit”: “mg”,
“value”: 2.055,
“metadata”: {
“m1s1d”: 45.211,
“m1sdf”: 59.211
}
}
]

Adding information regarding the parser payload that’s used:
//Add ignorable variables in this array.
const ignore_vars = [‘m1s1d’, ‘m1sdf’];

// Getting the variables I want to insert into the tension variable metadata
const m1s1d = payload.find((x) => x.variable === ‘m1s1d’);
const m1sdf = payload.find((x) => x.variable === ‘m1sdf’);

// Getting the index of the tension variable
const tensionIndex = payload.findIndex((x) => x.variable === ‘m1s1p’);

// If the tension does not have metadata field, I add a
// new object to the metadata field
if (tensionIndex !== -1 && !payload[tensionIndex].hasOwnProperty(‘metadata’)) {
payload[tensionIndex].metadata = {}
}

// After all that is complete, I add my variables into the metadata
if (tensionIndex !== -1) {
payload[tensionIndex].metadata.m1s1d = m1s1d;
payload[tensionIndex].metadata.m1sdf = m1sdf;

}

// Remove unwanted variables.
payload = payload.filter(x => !ignore_vars.includes(x.variable));

Hi, @tulio.souza

There are some errors in the way you are sending the simulated data and also in your parser. Take a look below at what you should fix in order for everything to work normally.

1 - Change your parser so that it inserts only the value field in the metadata and not the entire object:

Fixed Parser
 //Add ignorable variables in this array.
 const ignore_vars = ['m1s1d', 'm1sdf'];

 // Getting the variables I want to insert into the m1s1p variable metadata
 const m1s1d = payload.find((x) => x.variable === 'm1s1d');
 const m1sdf = payload.find((x) => x.variable === 'm1sdf');
 
 // Getting the index of the m1s1p variable
 const m1s1pIndex = payload.findIndex((x) => x.variable === 'm1s1p');
 
 // If the m1s1p does not have metadata field, I add a
 // new object to the metadata field
 if (m1s1pIndex !== -1 && !payload[m1s1pIndex].hasOwnProperty('metadata')) {
 payload[m1s1pIndex].metadata = {}
 }
 
 // After all that is complete, I add my variables into the metadata
 if (m1s1pIndex !== -1) {
 payload[m1s1pIndex].metadata.m1s1d = m1s1d.value;
 payload[m1s1pIndex].metadata.m1sdf = m1sdf.value;
 
 }
 
 // Remove unwanted variables.
 payload = payload.filter(x => !ignore_vars.includes(x.variable));

2 - Change the scope your are testing in the Device Emulator to the following

New Scope
[{
  "variable": "m1s1p",
  "unit": "mg",
  "value": 2.055
},
{
  "variable": "m1s1d",
  "value": 45.211
},{
  "variable": "m1sdf",
  "value": 59.211
}]

This scope after going through your new parser will become the following data:

[
    {
        "variable": "m1s1p",
        "unit": "mg",
        "value": 2.055,
        "metadata": {
            "m1s1d": 45.211,
            "m1sdf": 59.211
        }
    }
]

Let me know if you have anymore questions, thanks!

Hi @kelvinwelter, thanks for quickly reply,

I tested the new scope using the device simulator and live inspector show that the parsed insert 1 data added correctly, however is it not show the same results into widgets, see below:

image

BR, </>

tulio

Adding information the value 2,06 mg above, refer the variable; m1s1p , but the variable value m1s1d is not shown. However using the function test it formular inside widgets the value is show;


BR</>

Hi, @tulio.souza

The parser you are using are inserting your data into the M1S1P metadata. So, if you want to get the values within the M1S1P metadata, you need to select this variable in your widget. Check an example image from a correct display widget.

After that, you can go to the Formula configuration and select the variable you want from the metadata.

I imagine that this clarified what was happening. Let me know if you have other problems.

Note: don’t forget to change the variable alias to the correct name of the variable you are showing on your display. :grin:

Thanks!