MQTT Payload Help Please

Hi Guys

So I have a LoRaWAN soil moisture sensor working perfectly. I have decided to go nb-iot instead and am trying to configure the equivalent device (same manufacturer, same sensor, NB version).

I have managed to get it connecting and sending a payload but I’m having trouble decoding it. I’m a looong way from being a coder but I’d managed to get the LoRaWAN device going with a payload parser in TheThingsNetwork. I’ve tried using the same parser in Tago but it’s not working.

Please help with what I need to do to get this device decoding and storing data.

This is what’s coming through in the Live Inspector:

This is from the user manual:
The payload is ASCII string, representative same HEX:
0x72403155615900640c7817075e0a8c02f900 where:
➢ Device ID: 0x 724031556159 = 724031556159
➢ Version: 0x0064=100=1.0.0
➢ BAT: 0x0c78 = 3192 mV = 3.192V
➢ Singal: 0x17 = 23
➢ Soil Moisture: 0x075e= 1886 = 18.86 %
➢ Soil Temperature:0x0a8c =2700=27 °C
➢ Soil Conductivity(EC) = 0x02f9 =761 uS /cm
➢ Interrupt: 0x00 = 0

This is the payload parser used in TTN for the other (LoRaWAN) device.

function Decoder(bytes, port) {
// Decode an uplink message from a buffer
// (array) of bytes to an object of fields.
var value=(bytes[0]<<8 | bytes[1]) & 0x3FFF;
var batV=value/1000;//Battery,units:V

value=bytes[2]<<8 | bytes[3];
if(bytes[2] & 0x80)
{value |= 0xFFFF0000;}
var temp_DS18B20=(value/10);//DS18B20,temperature,units:℃

value=bytes[4]<<8 | bytes[5];
var water_SOIL=(value/100);//water_SOIL,Humidity,units:%

value=bytes[6]<<8 | bytes[7];
var temp_SOIL;
if((value & 0x8000)>>15 === 0)
temp_SOIL=(value/100);//temp_SOIL,temperature,units:°C
else if((value & 0x8000)>>15 === 1)
temp_SOIL=((value-0xFFFF)/100).toFixed(2);//temp_SOIL,temperature,units:°C

value=bytes[8]<<8 | bytes[9];
var conduct_SOIL=(value);//conduct_SOIL,conductivity,units:uS/cm

return {
Bat:batV,
TempC_DS18B20:temp_DS18B20,
water_SOIL:water_SOIL,
temp_SOIL:temp_SOIL,
conduct_SOIL:conduct_SOIL
};
}

Thanks in advance

Hi @rob,
Please, take a look in this two tutorials:
In-depth guide to Payload Parser - How to - TagoIO Community
How to build a MQTT Payload Parser - How to - TagoIO Community

I’m assuming you are sending a hexadecimal in order to use the decoder from TTN. All decoders from TTN works with a buffer, which means you will need to get the step from the second tutorial to use with an hexadecimal value. Then you can get the buffer variable and send to the decoder in the bytes parameter.

Also, set the payload = Decoder(buffer); in order to replace the payload that will be added to TagoIO.

After that, you also need to fix the “return” in your function. TagoIO has a specific data structure that is mentioned in all parser tutorial. The examples also has the data variable which exemplifies the structure you need to follow:

  const data = [
    { variable: 'protocol_version',  value: buffer.readInt8(0) },
    { variable: 'temperature',  value: buffer.readInt16BE(1) / 100, unit: '°C' },
    { variable: 'humidity',  value: buffer.readUInt16BE(3) / 100, unit: '%' },
  ];

Thanks Vitor. I have read through those but they didn’t really help me much (my lack of skills). I couldn’t see any examples of what to do with an ASCII string payload to use as a base example.

I also probably confused the issue in my post. I’m not running through TTN anymore, I ‘was’ successfully coming via TTN for my LoRaWAN device (and with that parser in TTN) but for this new nb-iot device I’m connecting directly to Tago MQTT. I mentioned it becuase I’d assumed the parser would be very similar due to the device being almost identical.

I’ll have another read and another tinker and see what I can do.
Thanks

As I mentioned, you can use the decoder from TTN and only change the return section.

It will be something like this:

function Decoder(bytes, port) {
  // Decode an uplink message from a buffer
  // (array) of bytes to an object of fields.
  let value = ((bytes[0] << 8) | bytes[1]) & 0x3fff;
  const batV = value / 1000; // Battery,units:V

  value = (bytes[2] << 8) | bytes[3];
  if (bytes[2] & 0x80) {
    value |= 0xffff0000;
  }
  const temp_DS18B20 = value / 10; // DS18B20,temperature,units:℃

  value = (bytes[4] << 8) | bytes[5];
  const water_SOIL = value / 100; // water_SOIL,Humidity,units:%

  value = (bytes[6] << 8) | bytes[7];
  let temp_SOIL;
  if ((value & 0x8000) >> 15 === 0) temp_SOIL = value / 100;
  // temp_SOIL,temperature,units:°C
  else if ((value & 0x8000) >> 15 === 1) temp_SOIL = ((value - 0xffff) / 100).toFixed(2); // temp_SOIL,temperature,units:°C

  value = (bytes[8] << 8) | bytes[9];
  const conduct_SOIL = value; // conduct_SOIL,conductivity,units:uS/cm

  // Normalize your data to TagoIO format as follow:
  return [
    { variable: "bat", value: batV },
    { variable: "TempC_ds18b20", value: temp_DS18B20 },
    { variable: "water_soil", value: water_SOIL },
    { variable: "temp_soil", value: temp_SOIL },
    { variable: "conduct_soil", value: conduct_SOIL },
  ];
}

const payload_raw = payload.find((data) => data.variable === "payload");
if (payload_raw) {
  // transform your hexadecimal payload to an array buffer.
  const bytes = Buffer.from(payload_raw.value, "hex");
  const decoded_payload = Decoder(bytes);

  // concatenate the payload with your decoded data.
  payload = payload.concat(decoded_payload);

  // Add serie to all data, so it can be grouped.
  const serie = String(new Date().getTime());
  payload = payload.map((data) => ({ ...data, serie }));
}

Also, I did notice from your Live Inspector logs that you are sending a integer value, that’s why you see that 3.43131… You need to add quotation marks to your payload when you publish it, so it doesn’t convert to a number.

“343131303536363…”

And also, convert it to hexadecimal before sending. Or you can just remove the “hex” parameter in the Buffer.from function, such as:

const bytes = Buffer.from(payload.value)

But I’m unsure it will work.


I also noticed that your decoder is complety wrong in the offsets that you showed for your example. So you may need to fix that by yourself. For example

  value = (bytes[6] << 8) | bytes[7];
  let temp_SOIL;
  if ((value & 0x8000) >> 15 === 0) temp_SOIL = value / 100;
  // temp_SOIL,temperature,units:°C
  else if ((value & 0x8000) >> 15 === 1) temp_SOIL = ((value - 0xffff) / 100).toFixed(2); // temp_SOIL,temperature,units:°C

Is getting bytes[6] and bytes[7], which is respectively the bytes 0064 and not 0a8c from 72403155615900640c7817075e0a8c02f900

Thanks for the clarification. Yeah I think I’ll be ok modifying the TTN parser once I can stop it sending as an integer. That seems to be causing me all the grief and the first hurdle to overcome.

I can’t see in the device doco how to change that though, how to add quotes. Can this be coverted back in the parser?

Thanks again for all your help.

Well, there are two ways:

  1. You can add the quotation marks on your device firmware.
  2. You can go the Configuration Parameters of your device at TagoIO, and set the payload_type value to hex instead of “auto”.

Thanks!

  • I’d been looking through all available firmware doc and couldn’t see anywhere to make that change.
  • I also had already tried adding the mqtt device as auto and as hex in the ‘custom MQTT add device’ window.
  • I had hex as a device parameter.

**BUT… **

I just manually changed that device parameter to ‘auto’ and it seems to be using the parser and returning variables! WooHoo!

I just need to tweak the parser little now (which I think I can do fine) and I’m all good.

Thanks for all your help.

1 Like