HTTP modbus data to Mango

[Robmalone]

Hi all,
I am trying to get modbus data from a Teltonika RUT955 router that has a modbus master & data to server function using HTTP. I can successfully received data on the same PC Mango is installed using Hercules (testing sw) but cannot get mango to receive any data. Here is what Hercules is receiving from the modbus server/router:

POST /httpds/RUT955 HTTP/1.1
Host: 192.168.1.129:1883
Accept: /
Content-Length: 120
Content-Type: application/x-www-form-urlencoded

[{"TS":"27/03/2020 12:33:36","ST":"7007","VR":"[2.388783e+02,]"},{"TS":"27/03/2020 12:33:36","ST":"7501","VR":"[678,]"}]

and a screenshot of the router setup

I am trying to receive 2 variables voltage & energy they are denoted by the modbus registers 7007 and 7051.

I am very new to http so could anyone please tell me how to set up the Mango side so I can get this working? Mango Version is 3.6.4.
Thanks.
Rob

[cwangv]

@robmalone
i notice you have 'http(s)' selected, can it be 'http' instead? also, you need to change the modbus port setting inside mango from '502' to '1883' as shown in your screenshot.
below is where the port setting is in mango.

hope that helps.

[Robmalone]

Hi and thanks for the reply. The http(s) selection is for both http and https, depends on how you input the URL.
That screenshot was actually from when I was testing with Hercules with port 1883. I have tried with port 8080 and no port. http://192.168.1.129:8080/httpds and http://192.168.1.129/httpds.

I am not polling the modbus slave from Mango, the router polls the salve device and then sends the data to Mango by http. Do I still need a modbus data point set up in Mango?

I have also noticed an error in ma.log which is most likely the root of the issue - WARN 2020-03-27T15:55:58,809 (com.serotonin.m2m2.web.mvc.spring.security.MangoAccessDeniedHandler.handle:53) - Denying access to Mango resource /httpds/ to IP 192.168.1.1
Do I need to open/allow anything in Mango to allow incoming http?

Thanks again.

[cwangv]

@robmalone
Yes, you need to create a Modbus tcp/ip data source and points for these to work.

[Robmalone]

Sorry, I'm not sure what you mean.
Even though the data is being sent to Mango in http format? I dont see how I can? what IP address should I point it to, The 3G router does not have a static public IP? I am not polling the modbus devices from Mango.
Any ideas how to reslove this error? - WARN 2020-03-27T15:55:58,809 (com.serotonin.m2m2.web.mvc.spring.security.MangoAccessDeniedHandler.handle:53) - Denying access to Mango resource /httpds/ to IP 192.168.1.1

Thanks.

[cwangv]

@robmalone
I can not help with you about the error code.
but is the Mango on the same LAN as the PLC? Or are you trying to poll data from a remote site into the Mango?

[Robmalone]

At the moment, just for testing purposes, it is in the same lan. Once I get it working it will be used in another location.

[craig]

Hi Rob,

I think you need http receiver since your device is going to POST data to in JSON format mango. It has nothing to do with modbus except in your device you are populating the JSON data with fields from the devices modbus map.

[MattFox]

Hi gentlemen, hope you don't mind if I join the party, Rob, looking at your config, you're going to want a HTTP JSON Receiver since the body of the message is in JSON.
Secondly, configure your whitelist in the datasource settings to allow your Teltonika to send data in by entering it's ip address:
192.168.1.1
Use * as your device identifier
I have learnt you need to amend the format of your JSON. The VR comes through as a broken array. namely: [2.388783e 02,]
To set the time, use the unix timestamp option so mango can parse it. %t
For ease of the jackson JSON format that mango uses:
Change your send url to: http://192.168.1.129/httpds and move the RUT955 or remove it completely
I did:

{"RUT955":[{"TS":"27/03/2020 12:33:36","ST":7007,"VR":[2.388783e 02,0]},{"TS":"27/03/2020 12:33:36","ST":7501,"VR":[678,0]}]}

LASTLY
Add a custom header: you need to overwrite the Content-Type: application/x-www-form-urlencoded
with:

Content-Type: application/json

once that's done we can look at your points. So far I can get them parsing with

HTTP Parameter name: /RUT955/0/VR for voltage as an alphanumeric point. I cannot parse them as numeric because the format of the register data is a string (text), not a number.

Let me know if you can further customise the data. Hope this helps

Fox

[Robmalone]

@MattFox Hi Matt,
Thanks for the assist. I can see data coming into Mango now using the HTTP receiver listener but that as far as I have got. The JSON output from the router is fairly customizable. What would be the best JSON data for Mango to receive in order to get those 2 data points? The variables I have are Timestamp(%d), Modbus Start Register (%s) (could be useful as its unique to each variable), Register Data(%a).
I could do something like {TS:%t, %s : %a} to give on output of:
{
"7007": "[2.409815e+02,]",
"TS": "1585705904"
},
{
"7501": "[91169,]",
"TS": "1585705904"
}

Thanks again MattFox.

[MattFox]

@robmalone said in HTTP modbus data to Mango:

{
"7007": "[2.409815e+02,]",
"TS": "1585705904"
},

This I like; However the issue we have is the format of your numerical data.
This sucker:

"[2.409815e+02,]",

JSON will not parse this because it's a broken array (the ,] denouncing there is nothing in the second array element)
Secondly the 2.409815e+02 will be parsed into HTTP as 2.409815e 02 because the '+' is treated as a space.
Can you not alter the format of the stored data? secondly if that second element is always empty, can you not associate the affiliated register to always return a 0 so the element isn't left blank?

That's all for now. Once you've got through that I'll think you've cracked it.

Set up a link via https://webhook.site/ to post your data to; to both view and share. flick me a link via chat and I can look at what you're firing from your router.

Fox

[Robmalone]

@MattFox. I have spoken to the router manufacturer and unfortunately the issue with the scientific notation cant be changed at this point. Ill keep at it! Thanks again.

[MattFox]

@robmalone for a quick fix we could throw together a relay using another program.
Are you able to code in other languages?
If not I don't mind helping you knock together a quick solution
EDIT: Silly me, for the content type header, make it this: no wonder the '+' disappeared...
application/json;charset=utf-8
Fox

[Robmalone]

@MattFox I'm actually not trained in coding of any languages to be honest. I'm just working thing out as I go, sometimes successfully, sometimes not. I think ill try a couple of other ways before I waste any more of your time.

1. The router also has an MQTT sender option so Ill try that first. I suspect I'll run into the same issue with this.
2. I can use a different modbus slave device that populates modbus registers with integers instead of float points therefor no decimal places.

I might get back to you with some MQTT queries if that's OK. Thanks again for the help.

[MattFox]

@robmalone said in HTTP modbus data to Mango:

I think ill try a couple of other ways before I waste any more of your time.

Trust me, you're not. I enjoy this kind of work. Is your mango version free? I'm confident we can get the data in a two stage process, you'll need a meta point data source and then we can get those points. Trust me this will be easy. Have you finally decided on your data format?

[Robmalone]

Thanks a lot @MattFox. I think I'll try MQTT since its a 3G modem and I believe that should help reduce the data.
Ill take a look over the next day or 2 and get back to you. I am using a free version at home just for testing but we have a 1000 point server we use commercially. Once I get this working at home we will be deploying it for use with the main server.

[MattFox]

Righto, I'll be here!