Payload Parser problem

I’m sending binary data without any encoding from my board(cellular) to Tago using MQTT.

I think I’m seeing different bytes arriving at the Payload Parser from what I actually send.

For example, I send this array as a payload:
{ 0x6D,0x00, 0x74, 0x89, 0xC6, 0x5E, 0xCF, 0x02, 0x34, 0x0B, 0x4A, 0x16, 0x00, 0x00, 0x3B }

Live inspector shows me this:

The code I’m using is:
const ignore_vars = [];

payload = payload.filter(x => !ignore_vars.includes(x.variable));

// Payload is an environment variable. Is where what is being inserted to your device comes in.
// Payload always is an array of objects. [ { variable, value…}, {variable, value…} …]
const payload_raw = payload.find(x => x.variable === ‘payload_raw’ || x.variable === ‘payload’ || x.variable === ‘data’);
if (payload_raw) {
// Get a unique serie for the incoming data.
const { value, serie, time } = payload_raw;

// Parse the payload_raw to JSON format (it comes in a String format)
if (value) {
console.log(value);
const buf = Buffer.from(value,‘latin1’);
console.log(buf.toString(‘hex’));
//payload = payload.concat(parsePayload(value).map(x => ({ …x, serie, time: x.time || time })));
}
}

In this particular example, it seems that the bytes 0x89, 0xC6, 0xCF are substituted by 0xFD bytes. I’ve tried other encodings(even with no encoding) on the Buffer.from(), but the result is worst.

Am I missing something?

Thank you.

hi @bruno, every request is transform to json object, so binary type is not a primary type of json (it’s a type on node.js), the server is setted to utf-8 encoded, you can tried convert from utf-8 string to buffer, or change your payload to hex in string, to be easily to convert it.

Hi Felipe,

Thank you for your answer.

When you say to convert from UTF-8 to Buffer, you mean const buf = Buffer.from(value,'utf8'); ? If yes, I’ve already did this, but the converted buffer comes with a lot of ‘wrong’ bytes.

No chance on converting the data to a human-readable format. The same devices can be configured for send the data through satellite(Iridium) as well, so every bit counts.

Last year I worked with Tago and a device that send data in pretty much the same binary format, and everything worked well. The only difference is that this application didn’t use MQTT, but some type of custom connector (I think) made by Vitor (don’t know if he still works at Tago).

I think sending binary data in constrained devices is vital, are you sure that Tago only expects human-readable packets?

thank you!

hi @bruno I will take a look on our end, and come back to you soon. thanks.

Hi Felipe!

If you need something or some specific test, please let me know.

Thank you!

Hi @bruno,

We deploy a new version of MQTT Connector with payload option, now you can choose to use payload with HEX format, you should create a new device, and select HEX in payload type. if you publish a message back to your device, it should be in hex format on string type (the same way you received). We’re still working on documentation, but you can start using it right now.

Hi Felipe,

Working perfectly now, thank you very much for the effort!

BTW, the interface is way more faster than when I worked with Tago last year, congratulations!

Best Regards,

1 Like