Please Note This forum exists for community support for the Mango product family and the Radix IoT Platform. Although Radix IoT employees participate in this forum from time to time, there is no guarantee of a response to anything posted here, nor can Radix IoT, LLC guarantee the accuracy of any information expressed or conveyed. Specific project questions from customers with active support contracts are asked to send requests to support@radixiot.com.

Extreme or incorrect values in the Mango data history have caused me a lot of problems (see the [url=http://mango.serotoninsoftware.com/forum/posts/list/1375.page Incorrect 0 point value...). We must have a way to locate and purge or modify selected data values in the PointValue history. I have deleted extreme values in the past via SQL, but these deletions are subject to errors (i.e. dangerous) and have often resulted in high-CPU usage problems that could only be rectified by restarting the Mango service. While the "Discard extreme values" datapoint function is useful, it has limitations: It only applies to new values; it does not remove extreme values from the point history In most cases the Discard feature is not enabled until extreme values have already been stored in the history The discard value will not discard invalid 0.00000 values; it will only discard extreme values If the Discard function is enabled, there is no way to detect the time, frequency, or value of the discarded data, which makes it difficult to troubleshoot and prevent extreme values from occurring in the future.[/list] It would be helpful if an optional alarm event were initiated each time an extreme value is discarded. The alarm would include the time stamp, the discarded value, and the discard threshold. What we need most of all is a point history editor that provides a means to select points based on their value and timestamp meeting certain criteria, and then either purge or modify those values (e.g. multiply or add to the point).

Whether I create a new meta datapoint or edit an existing one, when I click on Generate History that icon blinks for a while, and then a pop-up window displays the number of points generated. The number of points generated seems to correspond to the "Update event" interval and the inception of the first input context point; the wait time also seems to correlate to the number of points generated. However, the "Point Details" page does not show the newly generated points: neither in the point history nor in the graph. I suspect that the point history is being generated but not stored in the database. I use this feature a lot, both when fine tuning formulas or when creating new meta datapoints. Can you please look into this ASAP? Relevant modules: core 2.0.6, meta 1.4.0 On a related note, when I was using Mango 1.12.4, it seemed that I often had to purge all data before generating history. Does "generate history" always generate from Inception to Latest, or do I have to purge old data first? What time interval does it generate over if I already have points in the history? Thanks.

The nightly 'Medium Priority Work Items' that are causing 'datasource time value unchanged for 1 minute' alarms imply that there is a slowdown in the 1 second interval reading of the modbus data source. The problem isn't just the alarm, it is the data dropout that the alarm implies. This appears to be triggered every night by the automatic data purge: INFO 2013-05-04 03:05:00,001 (com.serotonin.m2m2.rt.maint.DataPurge.executeImpl:60) - Data purge started INFO 2013-05-04 03:08:31,832 (com.serotonin.m2m2.rt.maint.DataPurge.executeImpl:70) - Data purge ended, 2001716 point samples deleted A simple solution may be to assign a low (or lower) priority to the data purge process, or assign it with ionice so that it does not tax the disk IO. When could something like this be implemented? I have roughly 300 datapoints using Mango core 2.0.6 Thanks.

I'm using the free Fail2ban (https://en.wikipedia.org/wiki/Fail2ban) software to ban brute force ssh login attemps, and it is banning 1-3 IP addresses every day. Fail2ban scans log files (e.g. /var/log/apache/error_log) and bans IPs that show the malicious signs -- too many password failures, seeking for exploits, etc. Generally Fail2Ban then used to update firewall rules to reject the IP addresses for a specified amount of time, although any arbitrary other action (e.g. sending an email, or ejecting CD-ROM tray) could also be configured. Out of the box Fail2Ban comes with filters for various services (apache, curier, ssh, etc). http://www.fail2ban.org/ If you can implement the first feature: Write failed login attempts to a log file, including originating IP address ... then we can use [url=http://www.fail2ban.org/] fail2ban to watch the Mango log file for repeated login failures. Fail2ban will take care of banning the IP address if the number of failed login entries exceeds the allowed number. This would be a fast track to improved security. How about it? Source code at https://github.com/fail2ban/fail2ban (not that you would need it). Thanks.

I added the message directive as you suggested (line 6 below), but now all the email subject heading I receive are blank. /opt/mango/ftl/html$ **cat active.ftl ** <#-- Copyright (C) 2006-2011 Serotonin Software Technologies Inc. All rights reserved. @author Matthew Lohbihler --> <#include "include/eventHeader.ftl"> <@subject message=message/> <table> <tr><td colspan="3" class="smallTitle">${instanceDescription} - <@fmt key="ftl.eventActive"/></td></tr> <#include "include/eventData.ftl"> <#include "include/eventMessage.ftl"> </table> <#include "include/eventFooter.ftl"> Can you please paste a tested 'active.ftl' file that adds both the alarm level and the message to the subject field? How do you feel about making that the default subject heading template for 2.0.4? Thanks

Thanks for your suggestion. We will review this. If this is something you need addressed soon and is critical to your applications please email me. Thanks

It would be good if XID import status were displayed in different colors: currently all import results display in red, so errors do not stand out. This is especially a problem when there are a lot of points. e.g.: red=errors, green=added new XID, blue=updated existing XID Thanks.

Thanks! Are you referring to preventing the zero insertion during import, or to adding an option to the datapoint's "purge now" box to purge points meeting specified value and date criteria? The latter would be nice, especially since the "Discard extreme values" option is often not enabled until after invalid data has been stored in the point history. Additionally, enabling the discard function will prevent data problems from being detected and solved. Either way, I need to find out what SQL queries to use to purge or to modify points that fall meet value and date/age criteria. Thanks, PM

Great, thanks! Would that be 2.0.4? Where can I find the release notes or planned release date?

Can you please ensure that the datapoint import routine will check to see if there is already a value for the given time stamp, and if so, overwrite the existing value instead of creating duplicate time entries with different values? This will not only prevent data duplication if an already-imported point time entry is re-imported, it may also provide an easy way to correct erroneous values that may have crept into the time history. Thanks.

Can you please make sure that MA 2.0x can import multiple Mango M2M point values with no reformatting whatsoever? I have over 300 datapoints, each containing thousands of entries. If I have to reformat anything at all in the CSV format between the two versions, it will not be a simple copy/paste, will greatly slow down the process, and will make it more prone to errors. Thanks.

Hi I am having the same issue where mango seems to run on Derby but not mysql. We have a similar configuration on another machine that works, and comparing apples to apples, it all looks the same. This is the error I get: Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/comms/httpclient/Credentials at com.serotonin.m2m2.Main.main(Main.java:63) any clues? the db user created to access mysql has the GRANT ALL privileges so no access restrictions. Thanks!

Thanks, I'll try that script. However, it's far more tedious to edit the script than it is to just pick the values through the GUI, so in the long term it would be best to allow the multistate graphic object to display integer datapoints.

You're welcome. It will be in MA core 2.0.4. When you download it. Context help was added to the meta module.

Thanks for the link. I had searched for the other error message before posting, so I had not seen your helpful link. I backed up MangoDB when troubleshooting, and got the same error when running the backed up database under older Mango versions. For some reason, lately I could no longer get Mango to initialize when using the originally backed up MangoDB, so I had to start Mango with an empty database and import the json files to recreate my setup, and re-upload the graphical background images. Consequently, I lost all my historical data points. Actually, they're still in the backup MangoDB, but since Mango won't load when that database is used, I don't know how to retrieve that historical data, let alone import it into the new database. That data is not critical at the moment, but when historical data becomes important, it concerns me that I may not be able to retrieve it. I created new usernames and set their home pages, and this time I was able to update passwords of users with homepages and save without seeing the null error. However, I just got that error again when updating the admin password: ERROR 2011-06-01 20:57:54,121 (com.serotonin.mango.web.filter.ExceptionDetectionFilter.doFilter:52) - DWR invocation exception java.lang.NullPointerException at com.serotonin.mango.web.dwr.DataPointDetailsDwr.getPointData(DataPointDetailsDwr.java:66) at com.serotonin.mango.web.dwr.MiscDwr.doLongPoll(MiscDwr.java:343) at sun.reflect.GeneratedMethodAccessor1306.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.directwebremoting.impl.ExecuteAjaxFilter.doFilter(ExecuteAjaxFilter.java:34) at org.directwebremoting.impl.DefaultRemoter$1.doFilter(DefaultRemoter.java:428) at com.serotonin.mango.web.dwr.util.LoggedInAjaxMethodFilter.doFilter(LoggedInAjaxMethodFilter.java:52) at org.directwebremoting.impl.DefaultRemoter$1.doFilter(DefaultRemoter.java:428) at com.serotonin.web.dwr.LocalizationFilter.doFilter(LocalizationFilter.java:42) at org.directwebremoting.impl.DefaultRemoter$1.doFilter(DefaultRemoter.java:428) at com.serotonin.mango.web.filter.ExceptionDetectionFilter.doFilter(ExceptionDetectionFilter.java:40) at org.directwebremoting.impl.DefaultRemoter$1.doFilter(DefaultRemoter.java:428) at org.directwebremoting.impl.DefaultRemoter.execute(DefaultRemoter.java:431) at org.directwebremoting.impl.DefaultRemoter.execute(DefaultRemoter.java:283) at org.directwebremoting.servlet.PlainCallHandler.handle(PlainCallHandler.java:52) at org.directwebremoting.servlet.UrlProcessor.handle(UrlProcessor.java:101) at org.directwebremoting.servlet.DwrServlet.doPost(DwrServlet.java:146) at javax.servlet.http.HttpServlet.service(HttpServlet.java:637) at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Thread.java:636) This error occurred even though I had previously set a homepage for user admin.

Solution: In Debian/Ubuntu the serial ports are in group 'dialout.' However, user tomcat6 is not a member of group dialout, and consequently could not see the ports. $ ls -l /dev/ttyS* crw-rw---- 1 root dialout 4, 64 2011-03-23 20:42 /dev/ttyS0 crw-rw---- 1 root dialout 4, 65 2011-03-03 10:28 /dev/ttyS1 crw-rw---- 1 root dialout 4, 66 2011-03-03 10:28 /dev/ttyS2 crw-rw---- 1 root dialout 4, 67 2011-03-03 10:28 /dev/ttyS3 To fix the problem, user tomcat6 must be added to group dialout: root@ubuntuserver# usermod -G dialout tomcat6 Now Tomcat6 can see the serial ports, but Mango will not report them until it is restarted. To restart Mango: root@ubuntuserver# /etc/init.d/tomcat6 restart Stopping Tomcat servlet engine tomcat6 [ OK ] Starting Tomcat servlet engine tomcat6 Now Mango shows the serial ports in the pulldown menu When I searched for this in the forum, I had searched for 'NMEA.' Once I noticed the same 'missing ports' problem was also on the 'Modbus serial' data source page, I narrowed it to a serial port problem, and searched the forum for 'rxtx' and found out that [url=http://mango.serotoninsoftware.com/forum/posts/list/0/189.page#1362]'a reboot was mandatory,' so I restarted Tomcat. [solved] It would be useful to have a Mango wiki so that these issues can be organized and updated in a more structured manner than what a forum can support.