# In-depth guide to Payload Parser

**URL:** <https://community.tago.io/t/in-depth-guide-to-payload-parser/898>\
**Category:** Tutorials\
**Tags:** tutorials\
**Created:** [January 22, 2022, 7:46pm UTC](https://community.tago.io/t/in-depth-guide-to-payload-parser/898 "2022-01-22T19:46:07Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![system](https://yyz2.discourse-cdn.com/flex034/user_avatar/community.tago.io/system/32/24_2.png) [@system](https://community.tago.io/u/system)\
**Post date:** [January 22, 2022, 7:46pm UTC](https://community.tago.io/t/in-depth-guide-to-payload-parser/898/1 "2022-01-22T19:46:07Z")

</div>

# Overview

The Payload Parser is a data transformation tool in TagoIO that processes incoming device data before it’s stored in your device’s bucket. It acts as an intermediary layer that can decode, normalize, and modify data from IoT devices, ensuring it’s in the correct format for storage and analysis.

When IoT devices send data to TagoIO, they often transmit it in formats that aren’t immediately usable—such as hexadecimal strings, raw bytes, or proprietary encoding schemes. A Payload Parser is a JavaScript function that transforms this raw data into TagoIO’s standardized JSON format, making it accessible for dashboards, analytics, and other platform features.

**Supported Operations**

- **Data Decoding:** Convert hexadecimal payloads and byte arrays into readable values
- **Format Normalization:** Transform any data structure into TagoIO’s standard JSON format
- **Value Conversion:** Perform calculations, unit conversions, and data type transformations
- **Protocol Support:** Works with any communication protocol (LoRaWAN, Sigfox, MQTT, HTTP)
- **Data Filtering:** Remove unwanted variables before storage
- **Variable Addition:** Create new variables based on calculations or static values
- **Device Context Access:** Read device tags and configuration parameters during parsing

**Technical Constraints**

- **No Historical Data Access:** Cannot retrieve previously stored data from the device’s bucket
- **Resource Isolation:** Cannot access other TagoIO resources (dashboards, actions, other devices)
- **Execution Time Limit:** Code must complete within 2 seconds
- **No External Requests:** Cannot make HTTP requests or connect to external services
- **Synchronous Only:** No support for asynchronous JavaScript operations

* * *

## How Payload Parsers Work

> Device → Network Server → Payload Parser → Bucket Storage

When data arrives at your device:

1. The network server (LoRaWAN, Sigfox, etc.) normalizes the data if applicable
2. Your Payload Parser receives the normalized data
3. The parser transforms the data according to your code
4. Only the parser’s output is stored in the bucket

### **Execution Context**

The Payload Parser runs as an evaluated function with two parameters:

```auto
// Your code runs inside this context:
function payloadParser(payload, device) {
  // payload: Array of data sent to the device
  // device: Object containing device metadata

  // Your transformation code here

  payload = [] // Replaces the Payload to return the transformed data
}

```

* * *

# Getting Started

### Enabling Payload Parser

1. Navigate to your device in the TagoIO platform
2. Select the Payload Parser tab
3. Toggle the feature to Enable
4. Choose a template from the snippet dropdown menu:
  - **Parse example for LoRaWAN:** For LoRaWAN devices
  - **Parse example for Sigfox:** For Sigfox devices
  - **Convert Raw JSON to TagoIO JSON:** For HTTP devices sending JSON data
  - **Parse Example for MQTT Hexadecimal:** If you’re sending a hexadecimal using MQTT

### Code Structure

```auto
// Available packages (pre-imported)
// - dayjs: Lightweight date library
// - timeUtils: TagoIO proprietary library to handle timezones

// Function parameters
// payload: Array of incoming data objects
// device: Object with device metadata
// - device.tags: Array of device tags [{key, value}]
// - device.params: Array of config parameters [{key, value, sent}]

// Example: Filter unwanted variables
payload = payload.filter(item => item.variable !== 'unwanted_var');

// Payload is automatically returned to the system with the new data packet

```

* * *

# Working with Binary Data

### Hexadecimal to Buffer Conversion

Most IoT sensors send data as hexadecimal strings. Convert them to buffers for easier manipulation:

```auto
const payload_raw = payload.find(x => x.variable === 'payload');
const buffer = Buffer.from(payload_raw.value, 'hex');

// Example hex: "08FB383F"
// Buffer array: [0x08, 0xFB, 0x38, 0x3F]
// Decimal values: [8, 251, 56, 63]

```

### Buffer Reading Methods

There are some methods available that help us when reading bytes from a buffer:

| Method | Description | Bytes Read | Endianness |
| --- | --- | --- | --- |
| `buffer.readInt8(offset)` | Signed 8-bit integer | 1 | N/A |
| `buffer.readUInt8(offset)` | Unsigned 8-bit integer | 1 | N/A |
| `buffer.readInt16BE(offset)` | Signed 16-bit integer | 2 | Big Endian |
| `buffer.readInt16LE(offset)` | Signed 16-bit integer | 2 | Little Endian |
| `buffer.readUInt16BE(offset)` | Unsigned 16-bit integer | 2 | Big Endian |
| `buffer.readUInt16LE(offset)` | Unsigned 16-bit integer | 2 | Little Endian |
| `buffer.readInt32BE(offset)` | Signed 32-bit integer | 4 | Big Endian |
| `buffer.readInt32LE(offset)` | Signed 32-bit integer | 4 | Little Endian |
| `buffer.readUInt32BE(offset)` | Unsigned 32-bit integer | 4 | Big Endian |
| `buffer.readUInt32LE(offset)` | Unsigned 32-bit integer | 4 | Little Endian |

### Bitwise Operations

Extract specific bits using masks:

```auto
// Extract lower 4 bits
const lowerNibble = buffer\[0\] & 0x0F;
  
    
// Extract upper 4 bits
const upperNibble = (buffer\[0\] & 0xF0) >> 4;
   
    
// Check if specific bit is set
const isBitSet = (buffer\[0\] & 0x80) !== 0;

```

* * *

# Debugging Tools

An important step when running your payload parser script is to be able to test and debug it for any issue. In your device’s page, you should also see the Emulator and the Live Inspector section.

### Live Inspector

The Live Inspector provides real-time visibility into data flow:

- **Incoming data:** Raw payloads received by the device
- **Parser execution** : Console output and errors
- **Stored data** : Final data written to the bucket
- **MQTT events:** Connection status and publishes

By using a console.log inside your payload parser, you can print content of variables and arithmetic results when the payload parser runs for any data that being stored to your device. This is very helpful when trying to understand why you’re receiving errors or unwanted results.

```auto
console.log('Raw payload:', payload);
console.log('Device tags:', device.tags);
console.log('Parsed temperature:', temperature);

```

 ![Live Inspector Example](https://canada1.discourse-cdn.com/flex034/uploads/tagoio1/original/1X/776447f077759790d18b50b32f8265d40831884a.png "Live Inspector Example")

### Emulator

The Emulator allows you to test your parser without sending actual device data:

1. Copy a raw payload from Live Inspector or craft test data
2. Paste into the Emulator input field
3. Execute to see parser results

Example emulator inputs:

```auto
// LoRaWAN format
[
  { "variable": "payload", "value": "001100" },
  { "variable": "port", "value": 100 }
]

```

```auto
// MQTT format
[
  {
    "variable": "payload",
    "value": "0109611395",
    "metadata": { "mqtt_topic": "data" }
  }
]

```

 ![MQTT Emulator Example](https://canada1.discourse-cdn.com/flex034/uploads/tagoio1/original/1X/81808ee0e513301e474018f5ca521aeb03fcb3c9.png "MQTT Emulator Example")

* * *

# Complete Example

Here’s a practical parser for a temperature/humidity sensor:

```javascript
// Find the payload in incoming data
const payload_raw = payload.find(x =>
  x.variable === 'payload_raw' ||
  x.variable === 'payload' ||
  x.variable === 'data'
);

if (payload_raw) {
  try {
    // Convert hex to buffer
    const buffer = Buffer.from(payload_raw.value, 'hex');

    // Decode according to device protocol:
    // Byte 0: Protocol version
    // Bytes 1-2: Temperature (int16, big-endian, scaled by 100)
    // Bytes 3-4: Humidity (uint16, big-endian, scaled by 100)

    const data = [
      {
        variable: 'protocol_version',
        value: buffer.readInt8(0)
      },
      {
        variable: 'temperature',
        value: buffer.readInt16BE(1) / 100,
        unit: '°C'
      },
      {
        variable: 'humidity',
        value: buffer.readUInt16BE(3) / 100,
        unit: '%'
      }
    ];

    // Preserve metadata from original payload
    payload = payload.concat(
      data.map(x => ({
        ...x,
        group: String(payload_raw.serie || payload_raw.group),
        time: String(payload_raw.time)
      }))
    );

  } catch (e) {
    console.error('Parser error:', e);
    payload = [{ variable: 'parse_error', value: e.message }];
  }
}

```

* * *

# Scaling with Connectors

### What are Connectors?

Connectors are reusable Payload Parser templates that can be applied to multiple devices. Instead of copying parser code to each device, you create a connector once and reference it from any compatible device.

### Creating a Connector

1. Navigate to [Integrations](https://admin.tago.io/integrations/connector) in your account menu
2. Click Create Connector
3. Select supported networks (LoRaWAN, Sigfox, MQTT, etc.)
4. Paste your tested Payload Parser code
5. Add documentation and metadata (optional)
6. Save the connector

### Benefits

- Centralized Updates: Modify parser logic in one place
- Consistency: Ensure all devices use the same parsing logic
- Sharing: Make connectors public for community use
- Documentation: Include setup instructions and examples

More information regarding the connectors can be found at _[Connector Overview - TagoIO](https://help.tago.io/portal/en/kb/articles/466-connector-overview)_

## Additional Resources

- [LoRaWAN/Sigfox Payload Parser tutorial](https://help.tago.io/portal/en/community/topic/how-to-build-a-lorawan-sigfox-payload-parser)
- [MQTT Payload Parser Tutorial](https://help.tago.io/portal/en/community/topic/how-to-build-a-mqtt-payload-parser)

---

<div class="post-metadata">

**Author:** ![caioangelomotta](https://yyz2.discourse-cdn.com/flex034/user_avatar/community.tago.io/caioangelomotta/32/420_2.png) [@caioangelomotta](https://community.tago.io/u/caioangelomotta)\
**Post date:** [February 7, 2022, 6:47am UTC](https://community.tago.io/t/in-depth-guide-to-payload-parser/898/2 "2022-02-07T06:47:57Z")

</div>

Is there a way to modify the default parser? I have loaded a profile for the Dragino LT-22222-L using the Helium network and there are a few errors in the decode (voltage and current are swapped).

It seems like a small issue so I would like to modify the default parser instead of rewriting the whole thing from scratch.

---

<div class="post-metadata">

**Author:** ![felipe.gutierre](https://yyz2.discourse-cdn.com/flex034/user_avatar/community.tago.io/felipe.gutierre/32/1596_2.png) [@felipe.gutierre](https://community.tago.io/u/felipe.gutierre)\
**Post date:** [February 7, 2022, 12:13pm UTC](https://community.tago.io/t/in-depth-guide-to-payload-parser/898/3 "2022-02-07T12:13:43Z")

</div>

Hi Caio,

I noticed you opened a ticket reporting this issue. Our team will analyze the decode for Dragino LT-22222 and get back to you through the ticket system.

Thank you!

Phil

---

<div class="post-metadata">

**Author:** ![jpouza](https://avatars.discourse-cdn.com/v4/letter/j/a8b319/32.png) [@jpouza](https://community.tago.io/u/jpouza)\
**Post date:** [August 25, 2022, 10:50pm UTC](https://community.tago.io/t/in-depth-guide-to-payload-parser/898/4 "2022-08-25T22:50:14Z")

</div>

I need help with the parser for location variable on my moko lw001 bg Pro.

---

<div class="post-metadata">

**Author:** ![vitorfdl](https://yyz2.discourse-cdn.com/flex034/user_avatar/community.tago.io/vitorfdl/32/1728_2.png) [@vitorfdl](https://community.tago.io/u/vitorfdl)\
**Post date:** [August 29, 2022, 7:07pm UTC](https://community.tago.io/t/in-depth-guide-to-payload-parser/898/5 "2022-08-29T19:07:13Z")

</div>

Hi jpouza,

We are happy to help you. Can you provide more details? What questions do you have ?

---

<div class="post-metadata">

**Author:** ![achim.kraus](https://avatars.discourse-cdn.com/v4/letter/a/94ad74/32.png) [@achim.kraus](https://community.tago.io/u/achim.kraus)\
**Post date:** [November 19, 2024, 6:06pm UTC](https://community.tago.io/t/in-depth-guide-to-payload-parser/898/6 "2024-11-19T18:06:02Z")

</div>

Is there an example to convert a http request with plain text into [tago.io](http://tago.io) json?

e.g.:

curl --data-ascii ‘3652 mV 32%’ -H ‘Content-Type: text/plain; charset=ISO-8859-1’ https://..