Does anybody know if it is possible to display an HTTP Image point from a webcam in DGLux? It doesn't seem to appear in the station tree as a point.

Cheers,
Matt.

Hi Terry,
Thank you for the quick response. I have just tried this on another PC and I don't get the error so it must be something cached.

Thank you for your help!
Matt.

Hi all,
I have just updated one of our MangoES units to 3.3.0 and now get the following error when trying to create (or modify) a persistent TCP publisher. "Missing method or missing parameter converters"

Everything is filled out correctly. Saving an existing publisher results in the same message as well. I have tried re-installing the Persistent module with the same outcome.

Help!

Cheers,
Matt.

Hi Phil,
Thanks for the tips.
I upgraded JAVA about a month ago on all systems so that shouldn't be the issue. Yes, 4 persistent sources (from 4 MangoES publishers) and one publisher (to a MangoES). I'll adjust the thread count and overlap values and see what happens.
Yes, the 6GB option has been enabled in the ext-enabled folder. Mango reports 5.3GB with 5.1GB free. The server has 8 GB total.

I did notice this while checking the memory usage.

I guess this could be due to the large thread count from the persistent publishers.

I will do some more probing in the console to see if I get a bit more info. At least I can trigger the fault easily by running a backup.

Will keep you posted.

Matt.

Hi Phil,
Thanks for your response.

The virtual machine has been upgraded recently and Mango now has access to up to 6GB of RAM (up from 1.5) so memory shouldn't be the issue.
We have plenty of hdd space with 17% used of 70GB.

I've turned off backups for now as they cause the system to crash. Errors will begin populating within the first minute of the backup starting.

The persistent data sync is running with the default settings so the thread count is set to 10. We currently have 4 persistent data sources on the main server and one persistent data publisher.

I'm not sure what you mean by the minimum window size. I cannot find this option in the publisher, source or system settings.

We're running core 3.2.1+ enterprise with NoSQL and all modules are up to date.

To me it feels like there is a bad data point somewhere in the system and the backup and persistent processes force Mango to probe it, causing an exception. If this is the case, how would I go about finding the problematic point?

Cheers,
Matt.

Hi all,
We're having an issue with our Mango instance crashing randomly. Data points are received from sources but not committed to the database. The issue can be re-created by running a NoSQL backup. The relevant logs are attached below with the 'Timer already cancelled.' error repeating until the process is rebooted. Rebooting the process gets things going again for another couple of days.

INFO  2017-09-25T09:03:36,695 (com.serotonin.m2m2.rt.maint.work.DatabaseBackupWorkItem.execute:109) - Starting database backup WorkItem. 
INFO  2017-09-25T09:03:36,697 (com.serotonin.m2m2.rt.maint.work.DatabaseBackupWorkItem.createLogOutputStream:224) - Writing upgrade log to /opt/mango/logs/com.serotonin.m2m2.rt.maint.work.DatabaseBackupWorkItem.log 
FATAL 2017-09-25T09:03:51,645 (com.serotonin.timer.TimerThread.run:41) - TimerThread failed 
java.lang.NullPointerException: null
	at com.serotonin.timer.OrderedThreadPoolExecutor$TimePriorityLimitedTaskQueue.add(OrderedThreadPoolExecutor.java:442) ~[mango-3.2.1.jar:?]
	at com.serotonin.timer.OrderedThreadPoolExecutor.execute(OrderedThreadPoolExecutor.java:204) ~[mango-3.2.1.jar:?]
	at com.serotonin.timer.OrderedTimerThread.executeTask(OrderedTimerThread.java:26) ~[mango-3.2.1.jar:?]
	at com.serotonin.timer.TimerThread.mainLoop(TimerThread.java:133) ~[mango-3.2.1.jar:?]
	at com.serotonin.timer.TimerThread.run(TimerThread.java:38) [mango-3.2.1.jar:?]
ERROR 2017-09-25T09:03:53,506 (com.serotonin.m2m2.rt.maint.BackgroundProcessing$RejectableWorkItemRunnable.run:559) - Error in work item 
java.lang.IllegalStateException: Timer already cancelled.
	at com.serotonin.timer.RealTimeTimer.scheduleImpl(RealTimeTimer.java:110) ~[mango-3.2.1.jar:?]
	at com.serotonin.timer.AbstractTimer.schedule(AbstractTimer.java:26) ~[mango-3.2.1.jar:?]
	at com.serotonin.m2m2.rt.maint.BackgroundProcessing.schedule(BackgroundProcessing.java:94) ~[mango-3.2.1.jar:?]
	at com.serotonin.m2m2.util.timeout.TimeoutTask.<init>(TimeoutTask.java:41) ~[mango-3.2.1.jar:?]
	at com.serotonin.m2m2.util.timeout.TimeoutTask.<init>(TimeoutTask.java:35) ~[mango-3.2.1.jar:?]
	at com.serotonin.m2m2.rt.event.detectors.TimeoutDetectorRT.scheduleJob(TimeoutDetectorRT.java:103) ~[mango-3.2.1.jar:?]
	at com.serotonin.m2m2.rt.event.detectors.TimeDelayedEventDetectorRT.scheduleJob(TimeDelayedEventDetectorRT.java:27) ~[mango-3.2.1.jar:?]
	at com.serotonin.m2m2.rt.event.detectors.DifferenceDetectorRT.pointData(DifferenceDetectorRT.java:42) ~[mango-3.2.1.jar:?]
	at com.serotonin.m2m2.rt.event.detectors.NoUpdateDetectorRT.pointUpdated(NoUpdateDetectorRT.java:22) ~[mango-3.2.1.jar:?]
	at com.serotonin.m2m2.rt.dataImage.DataPointEventMulticaster.pointUpdated(DataPointEventMulticaster.java:94) ~[mango-3.2.1.jar:?]
	at com.serotonin.m2m2.rt.dataImage.DataPointEventMulticaster.pointUpdated(DataPointEventMulticaster.java:94) ~[mango-3.2.1.jar:?]
	at com.serotonin.m2m2.rt.dataImage.DataPointEventMulticaster.pointUpdated(DataPointEventMulticaster.java:94) ~[mango-3.2.1.jar:?]
	at com.serotonin.m2m2.rt.dataImage.DataPointEventMulticaster.pointUpdated(DataPointEventMulticaster.java:94) ~[mango-3.2.1.jar:?]
	at com.serotonin.m2m2.rt.dataImage.DataPointRT$EventNotifyWorkItem.execute(DataPointRT.java:672) ~[mango-3.2.1.jar:?]
	at com.serotonin.m2m2.rt.maint.BackgroundProcessing$RejectableWorkItemRunnable.run(BackgroundProcessing.java:556) [mango-3.2.1.jar:?]
	at com.serotonin.timer.Task.runTask(Task.java:179) [mango-3.2.1.jar:?]
	at com.serotonin.timer.TaskWrapper.run(TaskWrapper.java:23) [mango-3.2.1.jar:?]
	at com.serotonin.timer.OrderedThreadPoolExecutor$OrderedTaskCollection.run(OrderedThreadPoolExecutor.java:307) [mango-3.2.1.jar:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_144]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_144]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_144]
ERROR 2017-09-25T09:03:55,954 (com.serotonin.m2m2.rt.maint.BackgroundProcessing$RejectableWorkItemRunnable.run:559) - Error in work item 
java.lang.IllegalStateException: Timer already cancelled.
	at com.serotonin.timer.RealTimeTimer.scheduleImpl(RealTimeTimer.java:110) ~[mango-3.2.1.jar:?]
	at com.serotonin.timer.AbstractTimer.schedule(AbstractTimer.java:26) ~[mango-3.2.1.jar:?]
	at com.serotonin.m2m2.rt.maint.BackgroundProcessing.schedule(BackgroundProcessing.java:94) ~[mango-3.2.1.jar:?]
	at com.serotonin.m2m2.util.timeout.TimeoutTask.<init>(TimeoutTask.java:41) ~[mango-3.2.1.jar:?]
	at com.serotonin.m2m2.util.timeout.TimeoutTask.<init>(TimeoutTask.java:35) ~[mango-3.2.1.jar:?]
	at com.serotonin.m2m2.rt.event.detectors.TimeoutDetectorRT.scheduleJob(TimeoutDetectorRT.java:103) ~[mango-3.2.1.jar:?]
	at com.serotonin.m2m2.rt.event.detectors.TimeDelayedEventDetectorRT.scheduleJob(TimeDelayedEventDetectorRT.java:27) ~[mango-3.2.1.jar:?]
	at com.serotonin.m2m2.rt.event.detectors.DifferenceDetectorRT.pointData(DifferenceDetectorRT.java:42) ~[mango-3.2.1.jar:?]
	at com.serotonin.m2m2.rt.event.detectors.NoUpdateDetectorRT.pointUpdated(NoUpdateDetectorRT.java:22) ~[mango-3.2.1.jar:?]
	at com.serotonin.m2m2.rt.dataImage.DataPointEventMulticaster.pointUpdated(DataPointEventMulticaster.java:94) ~[mango-3.2.1.jar:?]
	at com.serotonin.m2m2.rt.dataImage.DataPointEventMulticaster.pointUpdated(DataPointEventMulticaster.java:94) ~[mango-3.2.1.jar:?]
	at com.serotonin.m2m2.rt.dataImage.DataPointRT$EventNotifyWorkItem.execute(DataPointRT.java:672) ~[mango-3.2.1.jar:?]
	at com.serotonin.m2m2.rt.maint.BackgroundProcessing$RejectableWorkItemRunnable.run(BackgroundProcessing.java:556) [mango-3.2.1.jar:?]
	at com.serotonin.timer.Task.runTask(Task.java:179) [mango-3.2.1.jar:?]
	at com.serotonin.timer.TaskWrapper.run(TaskWrapper.java:23) [mango-3.2.1.jar:?]
	at com.serotonin.timer.OrderedThreadPoolExecutor$OrderedTaskCollection.run(OrderedThreadPoolExecutor.java:307) [mango-3.2.1.jar:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_144]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_144]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_144]

I've also noticed that the persistent data historical sync also gets held up when trying to fill the holes where data wasn't logged to the main server during these outages (we are using MangoES units on site for redundancy).

Any ideas on where to start looking? We need to get this sorted as we are going live to our clients in the next few days.

Thanks in advance,
Matt.

@phildunlap Thanks Phil. I'll do the update and let you know how I get on.
Interestingly I seem to be getting the following error repeating over and over, sometimes only 10 or so seconds apart.

High priority task: com.serotonin.m2m2.persistent.ds.PersistentDataSourceRT$StatusProvider was rejected because it is already running.

This is on the data source end or the persistent data source publisher. Could this be an issue of corruption?

Hi Phil,
I think you're right about the error being a symptom of something else.
Looking at the data I noticed that one of the persistent data points showed data up until the date the Mango server started to crash. Data on the MangoES was still being recorded. I'm guessing something had caused corruption in the database and when a historical sync was started it would get hung up in this data point as clearing all data point data and doing a complete re-sync with the MangoES on site seems to have solved things.

I'm guessing the file limit issues us due to too many hung up historical sync threads running. I'll look into the user limits though.

I'd still like a way of being able to work out if this happens again and how to fix it, especially if Mango is unable to start at all.

Many thanks for your support.
Matt.

Hi all,
I am having issues with the latest update.
It seems like after a random amount of time (usually around 30mins) Mango gets stuck in some sort of loop, consuming 100% of the CPU and not allowing HTTP access.
Attached is a screenshot of the Thread Monitoring page showing the runaway thread.

This corresponds to what I see in top

I'm getting a number of these errors in ma.log which may be the cause of the lockup or may be a side effect of it.

ERROR 2017-01-06 14:17:39,989 (com.infiniteautomation.datafilesource.rt.DataFileDataSourceRT.doPoll:269) -
java.lang.NullPointerException
at com.infiniteautomation.datafilesource.rt.DataFileDataSourceRT.loadNewFiles(DataFileDataSourceRT.java:182)
at com.infiniteautomation.datafilesource.rt.DataFileDataSourceRT.doPoll(DataFileDataSourceRT.java:259)
at com.infiniteautomation.datafilesource.rt.DataFileDataSourceRT.doPollNoSync(DataFileDataSourceRT.java:250)
at com.serotonin.m2m2.rt.dataSource.PollingDataSource.scheduleTimeout(PollingDataSource.java:134)
at com.serotonin.m2m2.util.timeout.TimeoutTask.run(TimeoutTask.java:69)
at com.serotonin.timer.TimerTask.runTask(TimerTask.java:148)
at com.serotonin.timer.OrderedTimerTaskWorker.run(OrderedTimerTaskWorker.java:29)
at com.serotonin.timer.OrderedThreadPoolExecutor$OrderedTask.run(OrderedThreadPoolExecutor.java:278)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Logs attached...0_1483665559469_ma.log

I've performed a clean install leaving only the databases and db folders behind and am still having issues.
I am running a Persistent TCP data source to sync with a remote MangoES.

Rebooting Mango seems to sort things out for 30 mins or so.

Cheers,
Matt.

I have done some more testing and I definitely think there is an issue with the file data source.
Every time I load the CSV I get a different number of data points cached and displayed correctly on the Data points page chart, depending on how full the cache buffer is at the time of the import but only a few points are actually committed to the database.

This screenshot for example shows data logged for 21 April, 04 April, 10 Mar, etc. This is daily data from the 14 Oct to the 21 April. The CSV file is of a constant format. You can see from the graph that daily data is being cached from roughly the 1st April to the 21st April but everything else is in bits. This data is missing from the database data.

Running the import again, this time with display cached data set to 150 I get the complete dataset but nothing more is recorded to the database (see history table).

The datasource is set to log all data so I would assume that if data is being cached correctly, unless there is a double up it should be logged to the database.

Regards,
Matt.

The points are set to log all data. My timestamps seem fine when looking at the cached data so I don't think that's the issue. The CSV that I am importing has a timestamp of yyyyMMdd so I am setting HHmm to 00:00 which seems to work OK looking at the cached data.