I thought I would share a recent little side-project that I did, putting together a neat and tidy solution to take wind measurements over a three month period for a customer who is interested in installing small wind turbines at some of their rural radio repeater sites.

We built two almost-identical boxes for the customer, using a MangoES to log data from a Davis Instruments "Vantage Pro 2" weather station. All the equipment in the box is powered by an external DC supply (one site has 48VDC, one has 12VDC) provided by the customer.

The MangoES logs and records data from the weather station at a 30s interval. The data is then pushed to our Mango server using the Persistent TCP Mango module and a 3G VPN router (the bare PCB in the images below) provides connectivity via a Yagi antenna high on a mast.

An 'Ocean Controls' modbus gateway is used to connect the MangoES unit to the weather station using the built-in RS485 interface.
The MangoES is then connected, using ethernet, to our 3G router - which not only allows it to transmit data, but also allows us to log in (via our VPN) and check up on the units or make adjustments anytime.

The metal brackets are for mounting the weather station console/keypad on (which is actually required, unfortunately, to take measurements via RS485):

The data that we are getting from the units is very detailed. In total the MangoES is recording about 30 weather measurements, and we are even able to measure the DC supply voltage (as recorded by our 3G router) via the Mango SNMP module.

After the units were installed, I also set up Mango to send a weekly summary report to myself and the customer via email. The report shows the last week's worth of weather data on a chart and includes CSV files. It includes some average, mix and max values too. The email report is particularly convenient as it means the customer automatically receives the data and doesn't have to request it.

Interesting question. I would also like to find a good solution.

At the moment I am using statuscake.com to monitor the external availability of my mango server, though it will really only verify that the web-server is running.

I also use LibreNMS to chart my server stats - memory, disk, CPU, and have been meaning to set up alerts for these.

Perhaps something like a randomly changing watchdog value - which is posted to a remote server using the HTTP Publisher module, then a PHP or python listener script on the remote server alerts you if the value doesn't change in a while.

Or, set up a separate Mango server (on a VPS or something, which should be only a few dollars a month) and use the Persistent TCP publisher and listener modules to send a random number etc. across. You could make the generation of the random number quite complicated (using virtual data sources, meta points, point links, etc.) so that if any sub-system of mango breaks it will stop the value changing. On your remote mango server you could just set up a 'no change' alarm to email you.

Well done Joel, new forum looks and feels great. It's also really fast!

Hi Adaf/Asaf

There are some snippets from another user in this thread that might help:
http://infiniteautomation.com/forum/topic/1969/modbus4j-modbus-rtu-master/8

@hanse1pk said:

Managed to get it working.
Had to use 4 byte float swapped

Next item is for me to take in these integers such as in second example and have them be shown and recorded as what they represent.

This doc might be useful to you: http://wiki.infiniteautomation.com/doku.php?id=tipsandtricks:bestpractice

In my experience (having added probably a dozen different types of device to Mango using Modbus) there is always a bit of fluffing around initially to discover the magic transformation to get from the manufacturer's address to the address that Mango expects.

The first thing I usually do is look in the documentation for the modbus registers and find a register that can be easily identified - usually something like a 'hardware model number' or other known value.

Then you can use the 'Modbus Read Data' and 'Point Locator Test' functions on the Data Source page to test out different registers to try to hit the value you want.

In an extreme example, the manufacturer's documentation may say that a holding register is located at 43004, but you might find that the actual register you need to input into Mango is 3003 (43004 - 40001). Software such as 'Modbus poll' or 'CAS Modbus Scanner' also helps in figuring this out.

Looks great - can't wait to try!

Hi Mircea,

Is this for a MangoES? If so, the following link may help: https://wiki.debian.org/NetworkConfiguration#Setting_the_speed_and_duplex

As noted in that link, do be careful (particularly if this device is remote!) that you do not inadvertently lock yourself out of the device. Unstable link speeds can often be symptomatic of bad cabling, faulty termination, or intermittent noise and identifying and addressing the root cause may be called for.

One other thing to note is that many switches and other bits of network gear allow you to set a 'manual' speed by effectively disabling some rates, eg. you could disable Gigabit on the switch port and auto-negotiation would max out at 100Base-TX.

Brilliant, thanks Phil. Appreciate your explanation. That second regex works perfectly!

Hi

I am attempting to use the HTTP retriever to 'scrape' a HTML page for a number of values/datapoints. The page contains a table that looks like this, and my objective is to capture all of the numerical values as separate datapoints:

<table>

	<td>Gewicht Volk 1</td>
        <td><b><p class="right">44.6</p></b></td>
        <td>kg</td>
        <td>Zu/Abnahme</td>          
                    <td><b><p class="right">44.6</p></b></td>
         <td>kg</td></tr>
	<tr>
		<td>Gewicht Volk 2</td>
		<td><b><p class="right">29.4</p></b></td>
        <td>kg</td>
        <td>ab 00.00 Uhr</td> 
                    <td><b><p class="right">29.4</p></b></td>
        <td>kg</td></tr>
	<tr>
		<td>Luftdruck</td>
		<td><b><p class="right">1015</p></b></td>
        <td>mbar</td>	
	</tr>
        <td>Temperatur Drucksensor</td> 
        <td><b><p class="right">1.7</p></b></td>
        <td>°C</td>
	<tr>
        <td>Temperatur</td>
        <td><b><p class="right">6.2</p></b></td>
        <td>°C</td>
        <td>Tagesmin.</td> 
        <td><b><p class="right">0.0</p></b></td>
        <td>°C</td>
        <td>, Tagesmax.</td> 
        <td><b><p class="right">0.0</p></b></td>
        <td>°C</td>
</tr>
        <td>Brutraumtemperatur</td>
        <td><b><p class="right">0.0</p></b></td>
        <td>°C</td>
</tr>
        <td>Regensensor</td>
        <td><b><p class="right">17</p></b></td>
        <td>mm</td>   
        <td>Tagesmenge</td> 
        <td><b><p class="right">0.0</p></b></td>
        <td>mm</td>
</tr>
        <td>Luftfeuchtigkeit</td>
        <td><b><p class="right">80.6</p></b></td>
        <td>%</td>
	<tr>
        <td>Akku</td>
        <td><b><p class="right">12.0</p></b></td>
        <td>V</td>
    </tr>
     <tr>
        <td>CSQ (Signalqualität Antenne)</td>
        <td><b><p class="right">-1</p></b></td>
        <td> </td>
	 </tr>

If I use a regex along the lines of (?<=<p class="right">)(.*)(?=<\/p) I can match the first value (44.6), but I cannot get the second value or nth value by incrementing the 'value capture group' value in the data point properties. Adding a {n} index to the end of the regex to get the nth match doesn't seem to help either.

If I use a regex like <td>Luftfeuchtigkeit<\/td> <td><b><p class="right">(.*?)< (the forum has stripped the additional whitespaces) or even (?<=Luftdruck<\/td> <td><b><p class="right">)(.*)(?=<\/p>) I don't get any matches at all.

Attempts to 'learn' regex have come up short, so I am limited to copying examples from others and messing about by trial and error. Any suggestions about how to accomplish this would be very much appreciated!

@hayden_aus Great to hear you got things sorted out! Never used Digi but have previously looked at their kit (before landing on Mikrotik, actually) and it seems like nice gear.

Great, thanks Joel – I'll check that out. Mango reports that the JVM free memory is 144 MB and JVM maximum available memory is 241 MB.

Hey hayden_AUS,

Hmm - 1.178.148.xxx is not a private network (it is publicly routable) so if that is the IP address your Dlink device is getting (and as you likely already know, you'd need to look at the actual address on the device's 4G interface - using a service like 'whatismyipaddress.com' won't show you if you are behind NAT), carrier NAT should not be an issue here.

Because you're able to access it from your mobile hotspot I don't think that port blocking from Virgin is the issue either. Based on what you've said so far, my guess would be that it's either some strange issue with the Dlink's firewall (in which case, I'd try disabling any firewall functionality completely), some issue with the configuration of your Dlink itself (port forwarding rules or similar) or with your Dynamic DNS service (either the Dynamic DNS provider or the resolver that you use at your office might be slow to update).

I would continue troubleshooting by first logging in to your router from your (working) hotspot setup, confirming the current 4G/WAN IP address, then probing/attempting to connect to that IP address directly - without using DNS - from another connection.

I am not working in the industry right now but I have previously used m2mone (sister company of m2mconnectivity) and can't recommend them highly enough.

The pricing is comparable to what you might pay as a retail customer of Telstra (but you have access to very cheap, low data plans - like a few hundred MB for $3 per month) but of course you have the additional benefit of being able to immediately get access to a support person who is an expert in the field.

As a m2mone customer you also get access to an extremely powerful and useful web portal that you can use to manage your fleet of SIMs, run diagnostic checks, and even send some commands to the cell tower.

It's a real no-brainer in my opinion.

For a remote m2m application like yours, I would also go straight past Dlink and select something ruggedised and industrial from m2mcommunications. The additional investment pays itself off after it saves you making one trip back to your site. The Maestro E228 is probably a good entry-level unit. I personally used many Routerboard RB912 units (with Sierra MC8705 3G module) and while you may need to do some fiddling to get it set up, you cannot beat the value.

If you want to directly message me some additional details (as much as you are comfortable sharing) I'd be happy to take a quick look in to your networking issue from my end. As you can probably tell, I'm interested in this stuff!

Hi hayden_AUS

I have some experience with running Mango units behind 3G services (Telstra) and accessing remotely and via VPN.

I have a really vague recollection of running in to the same update issue and I think that it was due to the DNS resolver being unavailable. I would perhaps check the DNS settings on the MangoES (it may not be getting the correct one automatically, or may have previously been configured with a DNS server that is not reachable from your 4G connection).

If that doesn't help, one other thing might be to make sure that the system clock is set correctly on the MangoES. If the date/time is off, this could break SSL connections and that might explain why the IA store is 'timing out'.

The other connectivity issues you're having (from your office) are mysterious. What equipment are you using for your 4G connection? Are you using an APN that gives you a public IP or are you behind carrier NAT? If you can share a bit more detail about the set up of your remote site and your office I might be able to throw around a couple of ideas.

Hi there,

I have a test install of Mango 3.1.1 running in an AWS t2.micro (free tier) instance, with 512MB memory. Approx every week or so Mango will crash because Java runs out of memory:

Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00007f65c452d000, 12288, 0) failed; error='Cannot allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 12288 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /opt/mango/hs_err_pid25504.log

I am running Ubuntu 14.04.05 LTS with:

# JRE version: Java(TM) SE Runtime Environment (8.0_111-b14) (build 1.8.0_111-b14)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.111-b14 mixed mode linux-amd64 compressed oops)

Is there any suggested way I can configure Java (or Mango) so that it runs more happily on this server?

I should also add that I am running another application on this server (a PHP-based network monitor called LibreNMS) which I'm sure is not helping things.

Thanks!

Hi Mircea,

Is this for a MangoES? If so, the following link may help: https://wiki.debian.org/NetworkConfiguration#Setting_the_speed_and_duplex

As noted in that link, do be careful (particularly if this device is remote!) that you do not inadvertently lock yourself out of the device. Unstable link speeds can often be symptomatic of bad cabling, faulty termination, or intermittent noise and identifying and addressing the root cause may be called for.

One other thing to note is that many switches and other bits of network gear allow you to set a 'manual' speed by effectively disabling some rates, eg. you could disable Gigabit on the switch port and auto-negotiation would max out at 100Base-TX.

Hey Phil, the workaround didn't seem to work, but I got sidetracked with some other stuff and didn't try again.

I'm now running envcands 3.1.0 but don't see the date picker on the datasource page. Should it be there/is there anything special I need to do to see it?

Thanks Jared! I had somehow overlooked that in the examples - I will give that one a shot. Cheers!