Please Note This forum exists for community support for the Mango product family and the Radix IoT Platform. Although Radix IoT employees participate in this forum from time to time, there is no guarantee of a response to anything posted here, nor can Radix IoT, LLC guarantee the accuracy of any information expressed or conveyed. Specific project questions from customers with active support contracts are asked to send requests to support@radixiot.com.
Raspberry Pi as SCADA board - an attempt
-
TL;DR - is a RPI 3B (64bit processor at 1.2 ghz, ~950mb of ram) capable of running Mango successfully with the data rate that I need?
Hey all, I just wanted to give my perspective on the use of a Raspberry Pi 3B and my attempts to use it as a light weight, low power, all ways on SCADA. I tried out Mango about 4 years ago and was hoping to use it to form the base of a new supervisory system for a couple of distilleries.
First of all let my preface this by saying that this SCADA platform would be very very minimal in terms of C. All controls are handled by embedded PLC's and PACs and this was to be almost exclusively for Data acquisition and visualization. One of the distilleries is running experiments and being able to view 15-20 temperature sensors at a given time and have them graphing every 1-5 seconds is imperative.
I originally had some issues with installing Mango that was largely unrelated to the RPI so I'll not go into them other than TL; DR - it's possible to have an imported user profile get messed up. Delete it an make a new user.
Installation went fine. About 40 points were loaded (about 300 eventually) and at most 21 are viewed at a given time. For my tests I didn't bother making custom pages, just stuck with the watch lists. After running from anywhere from 10 minutes to 10 hours, the memory usage of Mango would rise to a terminal level and I'd have to kill process because commands would no longer work.
The next fix was to move the Mango install off of the memory card and onto an external USB drive. This went fine and re-importing all the points went just fine, but the same memory error persisted (not all that surprisingly). I enabled memory_small.sh and set the java memory a couple of ways. I even moved all of the RPI's physical memory that was devoted to the GPU over.
The result was the same, the memory usage start spiking, CPU hits 100% and then I have to kill the process. Next attempt at a fix was to make a nice big, juicy swap disk but that didn't do anything either.
I'm not a java expert but I'm really surprised to see how much this spikes considering that visualization should be happening client side. All administration of this board is being done remotely via SSH and VNC with Mango being viewed on intranet and externally via VPN. Network connections aren't factors here.
So here's my question - is a RPI 3B (64bit processor at 1.2 ghz, ~950mb of ram) capable of running Mango successfully with the data rate that I need. Right now my answerer seems to be no, but maybe someone else see's a glaring hole in my process or has been successful with this board.
Also, running the exact same points on a core i5 with 8gb of ram and an SSD (yes I know, a gun to a knife fight) does not even remotely replicate the issues above. The idea of having to run a relatively high powered box 24/7 kinda defeats the purpose.
-
@ordosays Interesting results, we are actually testing running Mango on a Raspberry Pi ourselves, in particular the compute module I believe. I am not personally involved so I'm sure someone else will tell you more.
Which version of Mango were you testing on? I know we have fixed several memory leaks recently. Also what was your
-Xmxsetting you tested with?Also the error message it was crashing with, I assume it was a OOME (Out of memory error)?
-
some details I left out:
the data points were "rolled up" using their average.
No more than 5 hour of points was viewed at a time and always with a "roll up"
polling was of a single modbus TCP/IP source
polling occurred over ethernet
data was set to be recorded for every change AND at an interval
data source was polled 1 times per second
there is only one other modbus device on that subnet. No bandwidth constraints
The i5 system is totally fine with the above, even when I hobble it.
PS, markup sucks. I've been trying to turn this into a semantic list for about 3 minutes.
-
@jared-wiltshire I tried everything from 300 to 2000 (with my big ass swap disk) for -Xmx and -Xms.
As for the error:
Java HotSpot(TM) Server VM warning: Exception java.lang.OutOfMemoryError occurred dispatching signal SIGTERM to handler- the VM may need to be forcibly terminated -
Hi ordosays,
I would expect it to be capable of what you describe, but there's a lot going on in your description and the current state isn't so clear, nor is the information about how a failure appears when it is happening. Is the device happily polling and then you open the chart you are describing, then things go off the rails?
As far as RAM settings, I would think for the amount of RAM you have specified the sweet spot will be between 650-850 and no swap space. Keep in mind the other things on the machine can impact what's available to Mango, so a light operating system is advisable (are you running headless or with a desktop?).
some details I left out:
You left out which version of Mango when Jared asked.
There are other SBCs in the intermediate space to the i5 8GB machine you make references to.
I would avoid using "ON_CHANGE_INTERVAL" logging if you're hitting the memory ceiling (I wouldn't do an interval average, either), but I wouldn't expect that to have a big impact unless the interval is very long (as there will be a lot of cancelled logging tasks hanging out in the timer until their time comes). You could post screenshots of the tabs on the
/internal/status.shtmpage. Perhaps one of them will reveal another place that memory could be going.I am running 300 virtual points on a compute module right now, polling on 200ms (doing ON_CHANGE_INTERVAL logging) and it's downloading data and charting fine. Did you design a dashboard for the frequently updating chart you describe? Can you share that code?
One of the distilleries is running experiments and being able to view 15-20 temperature sensors at a given time and have them graphing every 1-5 seconds is imperative.
Are you using the NoSQL module?
-
sorry to have left out some details. The mango instances have always been updated to current (in light of the chrome browser update screwing up parts of the data source interface, this is pretty much compulsory to use Mango).
for the majority of my sensors, a good deal of filtering is done at the sensor itself or at the PAC so logging on change isn't as crazy at it sounds. I'm probably only logging once every few minutes and then if process is stable for a long period of time, the log on interval serves to clean that up so when I zoom in on my charts, it's not all wonky. Exactly when the process starts moving is important because it may (and does) correlate to input changes in the control programs or feeds.
Yeah, there must be something up with the way this is being run on the Pi. On the i5 system (hackintosh running mac os 10.12 - same java version as the pi) I'm seeing the memory top out around 350MB and the processor sitting around 1-3%.
The Pi is running headless but with the desktop and all the trimmings installed. Honestly, the memory allocated to the GUI, which I can just kill off, is minimal. I mean, as soon as I run another program like a web browser, that goes out the window, but I'm not.
I stopped using the Pi for now but will post an error log from status.shtm when it dies again (if I can, it gets kinda unresponsive). If you have any configuration pointers, I'd love to know them.
-
Memory problems often appear as runaway utilization when they occur, but it does seem strange the memory usage would be so different. After I made that post, I requested a few hours of that data from a watchlist on the pi as a CSV and it ran out of memory while attempting to generate the file. So, it certainly has its limitations. Most often when people are using the Pi and Mango to my knowledge it's acting as a mere data collector and forwarding that on, with minimal data visualization (only when troubleshooting, otherwise its on some other larger server).
The Pi is running headless but with the desktop and all the trimmings installed.
Hmm. I really can't say if it's related, but the env.property setting
web.openBrowserOnStartup=trueis the default. You very probably want to set that false. I wonder if it is managing to launch anything.If you have any configuration pointers, I'd love to know them.
I've had good results with setting a memory threshhold around 70% of small system's memory. I would try it with 700MB as the Xms and Xmx settings.
-
little late to the response, but I would recommend that you consider not using RPi, but instead get a board with 2-4 GB of memory. There are quite a few now, below is a list from wikipedia to browse. I have a CuBox-i4 Pro with 2GB laying around somewhere that I will try once I find it. If that doesn't work I will be buying a 4GB board. You can get them in the $60-100 range. One I am interested in is the Libre Computer for pricing reasons. To me time is worth more than a few bucks, so just take the memory issue out of the picture.
It's good to know that Mango runs at all on the RPi, so thanks for sharing.
https://en.wikipedia.org/wiki/Comparison_of_single-board_computers
-
I'm just following up to let anyone that falls on this thread to know that I did find my CuBox-i4 Pro with 2GB and loaded Debian 7 with Java 8 and loaded mango. The system has only a few data points and has kept about 1.3 GB of memory free. I would say this is very workable for a small system with not too many users. Scrap RPi and find a 2GB board.
-
Hi jvaughters , thanks for the follow up! If you want to take explicit control of how much memory is being allocated to Java, you can do that with an Mango/bin/ext-enabled script like those in the Mango/bin/ext-available directory.
-
@phildunlap excellent tip, thx for pointing me those dirs, now I understand a new capability of Mango `,~)
