Help on how to access data from external URL

What is the best way to access data from external URL from analysis?

The URL: http://dadosabertos.rio.rj.gov.br/apiTransporte/apresentacao/rest/index.cfm/onibus/315

provides data as CSV (just as example):

{“COLUMNS”:[“DATETIME”,“ORDER”,“LINE”,“LATITUDE”,“LONGITUDE”,“VELOCITY”,“DIRECTION”],“DATA”:[[“05-20-2020 03:15:30”,“C41400”,315.0,-22.87533,-43.24192,0.0,18.0],[“05-20-2020 13:21:05”,“C47537”,315.0,-22.95374,-43.35023,0.0,355.0],[“05-20-2020 13:26:51”,“C47604”,315.0,-22.95353,-43.34998,0.0,""],[“05-20-2020 16:15:07”,“A41301”,315.0,-22.87445,-43.24132,0.0,285.0],[“05-20-2020 16:20:54”,“C41294”,315.0,-22.87405,-43.24176,0.0,335.0],[“05-20-2020 16:28:06”,“A41047”,315.0,-22.96223,-43.35703,9.0,0.0],[“05-20-2020 16:31:02”,“C41366”,315.0,-22.90471,-43.19278,0.0,90.0],[“05-20-2020 16:31:07”,“C41331”,315.0,-23.01054,-43.45313,0.0,244.0],[“05-20-2020 16:31:28”,“C41265”,315.0,-22.90908,-43.20403,70.0,242.0],[“05-20-2020 16:32:13”,“A41444”,315.0,-23.01033,-43.35065,0.0,87.0],[“05-20-2020 16:32:21”,“A41027”,315.0,-23.02176,-43.48258,0.0,172.0],[“05-20-2020 16:32:23”,“C47627”,315.0,-22.95308,-43.35062,0.0,180.0],[“05-20-2020 16:32:24”,“C41290”,315.0,-22.88221,-43.23764,53.0,330.0],[“05-20-2020 16:32:25”,“C41092”,315.0,-23.00172,-43.41906,30.0,59.0],[“05-20-2020 16:32:31”,“C41373”,315.0,-22.90476,-43.1926,0.0,231.0],[“05-20-2020 16:32:33”,“C41355”,315.0,-23.0219,-43.48257,0.0,147.0],[“05-20-2020 16:32:33”,“C41367”,315.0,-23.0258,-43.45934,19.0,56.0],[“05-20-2020 16:32:37”,“C41080”,315.0,-22.87166,-43.25863,0.0,96.0],[“05-20-2020 16:32:46”,“A41434”,315.0,-22.9531,-43.18905,23.0,56.0],[“05-20-2020 16:32:46”,“C41022”,315.0,-23.01366,-43.31529,49.0,113.0],[“05-20-2020 16:32:46”,“C41325”,315.0,-23.02898,-43.48038,6.0,171.0],[“05-20-2020 16:32:47”,“C41361”,315.0,-22.90454,-43.19227,0.0,192.0],[“05-20-2020 16:32:52”,“A41445”,315.0,-23.0304,-43.47397,45.0,239.0],[“05-20-2020 16:32:52”,“C41343”,315.0,-22.99585,-43.36616,53.0,180.0]]}

Our idea is to write a script/analysis to access, parse to JSON and load into bucket.

Is this possible? Any easy way to access external URLs?

Thanks in advance for any help

hi @mmoritz,

If I understood right. You need to create an HTTP request to your URL and parse da data to JSON and insert it into TagoIO.

To create an HTTP request you can use a lib called “axios” (https://www.npmjs.com/package/axios). I’ll leave an example below.

const axios = require('axios');
const TagoDevice = require('tago/device');
const TagoAnalysis = require('tago/analysis');


async function integration(context) {
  const request_conf = {
    url: 'YOUR_URL,
    method: 'get',
  };
  const req = await axios(request_conf);
  console.log(req.data); // CSV RESPONSE TO PARSE TO JSON
  const datatotago = // TODO parse req.data to json
  const dev = new TagoDevice('YOUR_TAGO_DEVICE_TOKEN');
  await dev.insert(datatotago);
  context.log('success');
}

module.exports = new TagoAnalysis(integration, 'your-analysis-token-here');

I hope it helps you.

Thx,
Fabiano Eger

Thanks Fabiano, sorry for taking some time to get back, I was aware of axios but just wanted some more clarifications. I’m not that technical and kind of slow to learn. Again just to clarify the script can be implemented as an Analysis within TagoIO platform, is that a correct assumption?

Thanks is advance for your help

Márcio Moritz

Zane

---- Ativado Seg, 25 mai 2020 19:09:12 -0300 Fabiano Eger via TagoIO Community tagoio@discoursemail.com escreveu ----

Hi @mmoritz,
Yes! The script can be implemented as an Analysis. :slightly_smiling_face:

Thanks Aline.

Márcio Moritz

Zane

---- Ativado Qui, 28 mai 2020 09:34:15 -0300 Aline Tusi via TagoIO Community tagoio@discoursemail.com escreveu ----