• Recent
    • Tags
    • Popular
    • Register
    • Login

    Please Note This forum exists for community support for the Mango product family and the Radix IoT Platform. Although Radix IoT employees participate in this forum from time to time, there is no guarantee of a response to anything posted here, nor can Radix IoT, LLC guarantee the accuracy of any information expressed or conveyed. Specific project questions from customers with active support contracts are asked to send requests to support@radixiot.com.

    Radix IoT Website Mango 3 Documentation Website Mango 4 Documentation Website Mango 5 Documentation Website

    Null Pointer Exception and Reports don't run after MangoNoSQL corruption repaired

    User help
    2
    6
    1.9k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • C
      craig
      last edited by craig

      It seems for some reason there is some corruption in the Mango NO SQL database as there are some log entries regarding repairing the corruption, but from that time on none of the scheduled reports (hourly) complete successfully so the queue fills up.

      WARN  2017-07-13T06:00:50,656 (com.serotonin.m2m2.util.timeout.TaskRejectionHandler.rejectedTask:77) - Rejected task: Interval logging: DP_452922 because Task Queue Full 
WARN  2017-07-13T06:01:11,041 (com.infiniteautomation.tsdb.impl.IasTsdbImpl.repairShard:1214) - Corruption detected in series 47 shard 698, repairing now. 
INFO  2017-07-13T06:01:11,869 (com.infiniteautomation.tsdb.impl.IasTsdbImpl.repairShard:1237) - Corruption repair report available at: /opt/mango/logs/tsdb-series-47-shard-698-scan-report_2017-07-13_06-01-11.log 
WARN  2017-07-13T18:01:00,199 (com.serotonin.m2m2.util.timeout.TaskRejectionHandler.rejectedTask:77) - Rejected task: Generating report: Hourly Compliance because Task Queue Full 
WARN  2017-07-13T19:01:00,152 (com.serotonin.m2m2.util.timeout.TaskRejectionHandler.rejectedTask:77) - Rejected task: Generating report: Hourly Compliance because Task Queue Full

      What showed up in the console though that isn't in the log is this NullPointerException, but it appears to happen before the database corruption:
      0_1500050991765_CTN mango NPE 2017-07-14.png

      Here is the contents of the corruption scan reports:

      mango@mangoES2147:/opt/mango/logs$ cat  tsdb-series-47-shard-698-scan-report_2017-07-13_06-01-11.log
-- Shard Scan Report Start: 07/13/2017 06:01:11.868 --
Callback was disordered by corruption: false

**********  SERIES 47 **********
Path: /opt/mango/databases/mangoTSDB/67/47
Runtime: 00:00:00.825
Shard Repair: None
mango@mangoES2147:/opt/mango/logs$cat  tsdb-series-178-shard-698-scan-report_2017-07-11_17-01-10.log
-- Shard Scan Report Start: 07/11/2017 17:01:11.897 --


**********  SERIES 178 **********
Path: /opt/mango/databases/mangoTSDB/50/178
Runtime: 00:00:01.03
Shard Repair: None

      Now in the excel reports tab there are a bunch of reports running and a bunch queued.

      The forum won't let me upload the ma.log or threads.json, so they are here
      https://pastebin.com/fQ92iAGm
      https://pastebin.com/C4HDzLZv

      C 1 Reply Last reply Reply Quote 0
      • C
        craig @craig
        last edited by

        mango won't shut down cleanly;

        mango@mangoES2147:/opt/mango$sudo ./bin/ma.sh stop
INFO  2017-07-14T13:57:47,780 (com.serotonin.m2m2.Lifecycle.terminate:361) - Mango Lifecycle terminating...
mango@mangoES2147:/opt/mango$INFO  2017-07-14T13:57:47,822 (com.serotonin.m2m2.rt.DataSourceGroupTerminator.terminate:72) - Terminating 8 NORMAL priority data sources in 8 threads.
INFO  2017-07-14T13:57:47,838 (com.serotonin.m2m2.rt.RuntimeManager.stopDataSourceShutdown:418) - Data source 'Cameras' stopped
INFO  2017-07-14T13:57:47,843 (com.serotonin.m2m2.rt.RuntimeManager.stopDataSourceShutdown:418) - Data source 'Mango Performance' stopped
INFO  2017-07-14T13:57:47,847 (com.serotonin.m2m2.rt.RuntimeManager.stopDataSourceShutdown:418) - Data source 'Unit 2' stopped
INFO  2017-07-14T13:57:47,851 (com.serotonin.m2m2.rt.RuntimeManager.stopDataSourceShutdown:418) - Data source 'Unit 1' stopped
INFO  2017-07-14T13:57:47,852 (com.serotonin.m2m2.rt.RuntimeManager.stopDataSourceShutdown:418) - Data source 'PLC0' stopped
INFO  2017-07-14T13:57:47,855 (com.serotonin.m2m2.rt.RuntimeManager.stopDataSourceShutdown:418) - Data source 'PLCi' stopped
INFO  2017-07-14T13:57:47,856 (com.serotonin.m2m2.rt.RuntimeManager.stopDataSourceShutdown:418) - Data source 'Mango Internal' stopped
INFO  2017-07-14T13:57:47,880 (com.serotonin.m2m2.rt.RuntimeManager.stopDataSourceShutdown:418) - Data source 'MangoES System' stopped
INFO  2017-07-14T13:57:47,926 (com.serotonin.m2m2.rt.DataSourceGroupTerminator.terminate:102) - Termination of 8 NORMAL priority data sources took 104ms
INFO  2017-07-14T13:57:55,953 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:462) - BackgroundProcessing waiting 60 more seconds for 1 active and 6 queued low priority tasks to complete.
INFO  2017-07-14T13:58:00,954 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:462) - BackgroundProcessing waiting 55 more seconds for 1 active and 6 queued low priority tasks to complete.
INFO  2017-07-14T13:58:05,955 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:462) - BackgroundProcessing waiting 50 more seconds for 1 active and 6 queued low priority tasks to complete.

INFO  2017-07-14T13:58:20,958 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:462) - BackgroundProcessing waiting 35 more seconds for 1 active and 6 queued low priority tasks to complete.
INFO  2017-07-14T13:58:25,959 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:462) - BackgroundProcessing waiting 30 more seconds for 1 active and 6 queued low priority tasks to complete.
INFO  2017-07-14T13:58:30,960 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:462) - BackgroundProcessing waiting 25 more seconds for 1 active and 6 queued low priority tasks to complete.
INFO  2017-07-14T13:58:35,961 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:462) - BackgroundProcessing waiting 20 more seconds for 1 active and 6 queued low priority tasks to complete.
INFO  2017-07-14T13:58:40,962 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:462) - BackgroundProcessing waiting 15 more seconds for 1 active and 6 queued low priority tasks to complete.
INFO  2017-07-14T13:58:45,963 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:462) - BackgroundProcessing waiting 10 more seconds for 1 active and 6 queued low priority tasks to complete.
INFO  2017-07-14T13:58:50,964 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:462) - BackgroundProcessing waiting 5 more seconds for 1 active and 6 queued low priority tasks to complete.
INFO  2017-07-14T13:58:55,966 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 60 more seconds for 24 active and 0 queued high priority tasks to complete.
INFO  2017-07-14T13:59:00,967 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 55 more seconds for 24 active and 0 queued high priority tasks to complete.
INFO  2017-07-14T13:59:05,968 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 50 more seconds for 24 active and 0 queued high priority tasks to complete.
INFO  2017-07-14T13:59:10,969 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 45 more seconds for 24 active and 0 queued high priority tasks to complete.
INFO  2017-07-14T13:59:15,970 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 40 more seconds for 24 active and 0 queued high priority tasks to complete.
INFO  2017-07-14T13:59:20,971 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 35 more seconds for 24 active and 0 queued high priority tasks to complete.
INFO  2017-07-14T13:59:25,972 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 30 more seconds for 24 active and 0 queued high priority tasks to complete.
INFO  2017-07-14T13:59:30,973 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 25 more seconds for 24 active and 0 queued high priority tasks to complete.
INFO  2017-07-14T13:59:35,974 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 20 more seconds for 24 active and 0 queued high priority tasks to complete.
INFO  2017-07-14T13:59:40,975 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 15 more seconds for 24 active and 0 queued high priority tasks to complete.
INFO  2017-07-14T13:59:45,976 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 10 more seconds for 24 active and 0 queued high priority tasks to complete.
INFO  2017-07-14T13:59:50,977 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 5 more seconds for 24 active and 0 queued high priority tasks to complete.
INFO  2017-07-14T13:59:54,978 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:481) - All high priority tasks exited gracefully.
INFO  2017-07-14T13:59:54,978 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:487) - All medium priority tasks exited gracefully.
INFO  2017-07-14T13:59:54,979 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:494) - 6 low priority tasks forcefully terminated.
INFO  2017-07-14T13:59:56,248 (com.infiniteautomation.nosql.MangoNoSqlProxy.shutdown:115) - Terminating NoSQL Batch Write Manager.
INFO  2017-07-14T13:59:56,248 (com.infiniteautomation.nosql.MangoNoSqlBatchWriteBehindManager.terminate:242) - Terminating NoSQL Point Value Mover.
INFO  2017-07-14T13:59:56,249 (com.infiniteautomation.nosql.MangoNoSqlBatchWriteBehindManager.terminate:249) - Terminating 22 Batch Writer Tasks.
INFO  2017-07-14T13:59:56,251 (com.infiniteautomation.nosql.MangoNoSqlBatchWriteBehindManager.terminate:258) - 22 Batch Writer Tasks awaiting termination.
WARN  2017-07-14T14:00:06,252 (com.infiniteautomation.nosql.MangoNoSqlBatchWriteBehind.joinTermination:169) - Waiting for Batch Writer Task 0 to stop
WARN  2017-07-14T14:00:16,253 (com.infiniteautomation.nosql.MangoNoSqlBatchWriteBehind.joinTermination:169) - Waiting for Batch Writer Task 0 to stop

        the last message repeats forever so I have to kill it

        upon restarting there are some out of memory exceptions:

        INFO  2017-07-14T14:40:45,060 (com.infiniteautomation.tsdb.impl.CorruptionScanner.seriesComplete:1022) - Scan of /opt/mango/databases/mangoTSDB/5/191 completed in 00:00:00.04
INFO  2017-07-14T14:40:45,623 (com.infiniteautomation.tsdb.impl.CorruptionScanner.seriesComplete:1008) - Completed folder 62 of 94
INFO  2017-07-14T14:40:45,624 (com.infiniteautomation.tsdb.impl.CorruptionScanner.seriesComplete:1022) - Scan of /opt/mango/databases/mangoTSDB/91/50 completed in 00:00:15.137
ERROR 2017-07-14T14:40:49,905 (com.infiniteautomation.tsdb.impl.CorruptionScanner.findBadTs:693) - Map failed
java.io.IOException: Map failed
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:907) ~[?:1.8.0_33]
        at com.infiniteautomation.tsdb.impl.ChecksumMappedByteBufferInputStream.<init>(ChecksumMappedByteBufferInputStream.java:41) ~[ias-tsdb-1.3.2.jar:?]
        at com.infiniteautomation.tsdb.impl.CorruptionScanner.openDataIn(CorruptionScanner.java:791) ~[ias-tsdb-1.3.2.jar:?]
        at com.infiniteautomation.tsdb.impl.CorruptionScanner.findBadTs(CorruptionScanner.java:671) [ias-tsdb-1.3.2.jar:?]
        at com.infiniteautomation.tsdb.impl.CorruptionScanner.checkFile(CorruptionScanner.java:484) [ias-tsdb-1.3.2.jar:?]
        at com.infiniteautomation.tsdb.impl.CorruptionScanner.checkShard(CorruptionScanner.java:434) [ias-tsdb-1.3.2.jar:?]
        at com.infiniteautomation.tsdb.impl.CorruptionScanner.checkSeriesDir(CorruptionScanner.java:336) [ias-tsdb-1.3.2.jar:?]
        at com.infiniteautomation.tsdb.impl.CorruptionScanner$CorruptionCheckTask.run(CorruptionScanner.java:300) [ias-tsdb-1.3.2.jar:?]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_33]
Caused by: java.lang.OutOfMemoryError: Map failed
        at sun.nio.ch.FileChannelImpl.map0(Native Method) ~[?:1.8.0_33]
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:904) ~[?:1.8.0_33]
        ... 8 more
ERROR 2017-07-14T14:40:49,929 (com.infiniteautomation.tsdb.impl.CorruptionScanner.findCorruption:531) - Map failed
java.io.IOException: Map failed
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:907) ~[?:1.8.0_33]
        at com.infiniteautomation.tsdb.impl.ChecksumMappedByteBufferInputStream.<init>(ChecksumMappedByteBufferInputStream.java:41) ~[ias-tsdb-1.3.2.jar:?]
        at com.infiniteautomation.tsdb.impl.CorruptionScanner.openDataIn(CorruptionScanner.java:791) ~[ias-tsdb-1.3.2.jar:?]
        at com.infiniteautomation.tsdb.impl.CorruptionScanner.findCorruption(CorruptionScanner.java:498) [ias-tsdb-1.3.2.jar:?]
        at com.infiniteautomation.tsdb.impl.CorruptionScanner.checkFile(CorruptionScanner.java:467) [ias-tsdb-1.3.2.jar:?]
        at com.infiniteautomation.tsdb.impl.CorruptionScanner.checkShard(CorruptionScanner.java:434) [ias-tsdb-1.3.2.jar:?]
        at com.infiniteautomation.tsdb.impl.CorruptionScanner.checkSeriesDir(CorruptionScanner.java:336) [ias-tsdb-1.3.2.jar:?]
        at com.infiniteautomation.tsdb.impl.CorruptionScanner$CorruptionCheckTask.run(CorruptionScanner.java:300) [ias-tsdb-1.3.2.jar:?]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_33]
Caused by: java.lang.OutOfMemoryError: Map failed
        at sun.nio.ch.FileChannelImpl.map0(Native Method) ~[?:1.8.0_33]
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:904) ~[?:1.8.0_33]
        ... 8 more
INFO  2017-07-14T14:40:51,325 (com.infiniteautomation.tsdb.impl.CorruptionScanner.seriesComplete:1008) - Completed folder 63 of 94
INFO  2017-07-14T14:40:51,327 (com.infiniteautomation.tsdb.impl.CorruptionScanner.seriesComplete:1022) - Scan of /opt/mango/databases/mangoTSDB/18/93 completed in 00:00:14.739
INFO  2017-07-14T14:40:51,362 (com.infiniteautomation.tsdb.impl.CorruptionScanner.seriesComplete:1008) - Completed folder 64 of 94
INFO  2017-07-14T14:40:51,363 (com.infiniteautomation.tsdb.impl.CorruptionScanner.seriesComplete:1022) - Scan of /opt/mango/databases/mangoTSDB/13/199 completed in 00:00:00.34
INFO  2017-07-14T14:40:51,367 (com.infiniteautomation.tsdb.impl.CorruptionScanner.seriesComplete:1008) - Completed folder 65 of 9
        1 Reply Last reply Reply Quote 0
        • phildunlapP
          phildunlap
          last edited by

          Hi Craig,

          Thanks for bringing this to our attention! It looks like the null pointer exception is a bug with the purge of failed or still running reports. Fixing.

          Is the problem with shutdown every time, or it is more like only once in a while? I may have found the cause of this.

          I will look into the corruption as a possible cause a little more.

          The OOM shouldn't be new. You could try changing your db.nosql.shardStreamType to RANDOM_ACCESS_FILE, increasing your memory allocated, or set db.nosql.runCorruptionOnStartupIfDirty to true if it is not.

          1 Reply Last reply Reply Quote 0
          • C
            craig
            last edited by

            Hi Phil,

            Thanks for getting in touch. I can report back to our customer that you are working on it.

            We made it a couple days before running in to this issue again. The first exception is in the excel report purge, then after it looks like the corruption scanner has an exception and then no more reports run after that, and then the queue fills up.

            I had the same problem not being able to shut mango down cleanly after the exceptions have occurred.

            I did change the runCorruptionOnStartupIfDirty to true after I the first bout of corruption occurred.

            Exception in the excel report purge:

            Exception in thread "high-pool-2-thread-10622" java.lang.NullPointerException
        at java.io.File.<init>(File.java:360)
        at com.infiniteautomation.mango.excelreports.ExcelReportsCommon.getReport(ExcelReportsCommon.java:174)
        at com.infiniteautomation.mango.excelreports.dao.ExcelReportDao.purgeReportsBefore(ExcelReportDao.java:193)
        at com.infiniteautomation.mango.excelreports.ExcelReportPurgeDefinition.execute(ExcelReportPurgeDefinition.java:30)
        at com.serotonin.m2m2.rt.maint.DataPurge.executeImpl(DataPurge.java:93)
        at com.serotonin.m2m2.rt.maint.DataPurge.execute(DataPurge.java:61)
        at com.serotonin.m2m2.rt.maint.DataPurge$DataPurgeTask.run(DataPurge.java:289)
        at com.serotonin.timer.Task.runTask(Task.java:179)
        at com.serotonin.timer.TaskWrapper.run(TaskWrapper.java:23)
        at com.serotonin.timer.OrderedThreadPoolExecutor$OrderedTaskCollection.run(OrderedThreadPoolExecutor.java:307)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

            Exception in the corruption scanner after which no reports run:

            WARN  2017-07-16T06:01:09,297 (com.infiniteautomation.tsdb.impl.IasTsdbImpl.repairShard:1214) - Corruption detected in series 68 shard 698, repairing now.
WARN  2017-07-16T06:01:09,297 (com.infiniteautomation.tsdb.impl.IasTsdbImpl.repairShard:1214) - Corruption detected in series 68 shard 698, repairing now.
WARN  2017-07-16T06:01:10,401 (com.serotonin.m2m2.util.timeout.TaskRejectionHandler.rejectedTask:77) - Rejected task: Interval logging: DP_194949 because Task Qu                        eue Full
WARN  2017-07-16T06:01:11,441 (com.serotonin.m2m2.util.timeout.TaskRejectionHandler.rejectedTask:77) - Rejected task: Interval logging: DP_745002 because Task Qu                        eue Full
WARN  2017-07-16T06:01:10,401 (com.serotonin.m2m2.util.timeout.TaskRejectionHandler.rejectedTask:77) - Rejected task: Interval logging: DP_194949 because Task Qu                        eue Full
WARN  2017-07-16T06:01:11,441 (com.serotonin.m2m2.util.timeout.TaskRejectionHandler.rejectedTask:77) - Rejected task: Interval logging: DP_745002 because Task Qu                        eue Full
ERROR 2017-07-16T06:01:11,443 (com.infiniteautomation.tsdb.impl.CorruptionScanner.findCorruption:531) - Map failed
java.io.IOException: Map failed
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:907) ~[?:1.8.0_33]
        at com.infiniteautomation.tsdb.impl.ChecksumMappedByteBufferInputStream.<init>(ChecksumMappedByteBufferInputStream.java:41) ~[ias-tsdb-1.3.2.jar:?]
        at com.infiniteautomation.tsdb.impl.CorruptionScanner.openDataIn(CorruptionScanner.java:791) ~[ias-tsdb-1.3.2.jar:?]
        at com.infiniteautomation.tsdb.impl.CorruptionScanner.findCorruption(CorruptionScanner.java:498) [ias-tsdb-1.3.2.jar:?]
        at com.infiniteautomation.tsdb.impl.CorruptionScanner.checkFile(CorruptionScanner.java:467) [ias-tsdb-1.3.2.jar:?]
        at com.infiniteautomation.tsdb.impl.CorruptionScanner.checkShard(CorruptionScanner.java:434) [ias-tsdb-1.3.2.jar:?]
        at com.infiniteautomation.tsdb.impl.IasTsdbImpl.repairShard(IasTsdbImpl.java:1226) [ias-tsdb-1.3.2.jar:?]
        at com.infiniteautomation.tsdb.impl.IasTsdbImpl.multiQuery(IasTsdbImpl.java:529) [ias-tsdb-1.3.2.jar:?]
        at com.infiniteautomation.nosql.MangoNoSqlPointValueDao.getPointValuesBetween(MangoNoSqlPointValueDao.java:238) [mangoNoSqlDatabase-3.1.1.jar:?]
        at com.infiniteautomation.mango.excelreports.rt.ExcelReportWorkItem.execute(ExcelReportWorkItem.java:501) [excel-reports-3.1.2.jar:?]
        at com.serotonin.m2m2.rt.maint.BackgroundProcessing$RejectableWorkItemRunnable.run(BackgroundProcessing.java:556) [mango-3.1.1.jar:?]
        at com.serotonin.timer.Task.runTask(Task.java:179) [mango-3.1.1.jar:?]
        at com.serotonin.timer.TaskWrapper.run(TaskWrapper.java:23) [mango-3.1.1.jar:?]
        at com.serotonin.timer.OrderedThreadPoolExecutor$OrderedTaskCollection.run(OrderedThreadPoolExecutor.java:307) [mango-3.1.1.jar:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_33]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_33]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_33]
Caused by: java.lang.OutOfMemoryError: Map failed
        at sun.nio.ch.FileChannelImpl.map0(Native Method) ~[?:1.8.0_33]
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:904) ~[?:1.8.0_33]
        ... 16 more
ERROR 2017-07-16T06:01:11,443 (com.infiniteautomation.tsdb.impl.CorruptionScanner.findCorruption:531) - Map failed
java.io.IOException: Map failed
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:907) ~[?:1.8.0_33]
        at com.infiniteautomation.tsdb.impl.ChecksumMappedByteBufferInputStream.<init>(ChecksumMappedByteBufferInputStream.java:41) ~[ias-tsdb-1.3.2.jar:?]
        at com.infiniteautomation.tsdb.impl.CorruptionScanner.openDataIn(CorruptionScanner.java:791) ~[ias-tsdb-1.3.2.jar:?]
        at com.infiniteautomation.tsdb.impl.CorruptionScanner.findCorruption(CorruptionScanner.java:498) [ias-tsdb-1.3.2.jar:?]
        at com.infiniteautomation.tsdb.impl.CorruptionScanner.checkFile(CorruptionScanner.java:467) [ias-tsdb-1.3.2.jar:?]
        at com.infiniteautomation.tsdb.impl.CorruptionScanner.checkShard(CorruptionScanner.java:434) [ias-tsdb-1.3.2.jar:?]
        at com.infiniteautomation.tsdb.impl.IasTsdbImpl.repairShard(IasTsdbImpl.java:1226) [ias-tsdb-1.3.2.jar:?]
        at com.infiniteautomation.tsdb.impl.IasTsdbImpl.multiQuery(IasTsdbImpl.java:529) [ias-tsdb-1.3.2.jar:?]
        at com.infiniteautomation.nosql.MangoNoSqlPointValueDao.getPointValuesBetween(MangoNoSqlPointValueDao.java:238) [mangoNoSqlDatabase-3.1.1.jar:?]
        at com.infiniteautomation.mango.excelreports.rt.ExcelReportWorkItem.execute(ExcelReportWorkItem.java:501) [excel-reports-3.1.2.jar:?]
        at com.serotonin.m2m2.rt.maint.BackgroundProcessing$RejectableWorkItemRunnable.run(BackgroundProcessing.java:556) [mango-3.1.1.jar:?]
        at com.serotonin.timer.Task.runTask(Task.java:179) [mango-3.1.1.jar:?]
        at com.serotonin.timer.TaskWrapper.run(TaskWrapper.java:23) [mango-3.1.1.jar:?]
        at com.serotonin.timer.OrderedThreadPoolExecutor$OrderedTaskCollection.run(OrderedThreadPoolExecutor.java:307) [mango-3.1.1.jar:?]

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_33]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_33]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_33]
Caused by: java.lang.OutOfMemoryError: Map failed
        at sun.nio.ch.FileChannelImpl.map0(Native Method) ~[?:1.8.0_33]
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:904) ~[?:1.8.0_33]
        ... 16 more
ERROR 2017-07-16T06:01:11,443 (com.infiniteautomation.tsdb.impl.CorruptionScanner.findCorruption:531) - Map failed
java.io.IOException: Map failed
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:907) ~[?:1.8.0_33]
        at com.infiniteautomation.tsdb.impl.ChecksumMappedByteBufferInputStream.<init>(ChecksumMappedByteBufferInputStream.java:41) ~[ias-tsdb-1.3.2.jar:?]
        at com.infiniteautomation.tsdb.impl.CorruptionScanner.openDataIn(CorruptionScanner.java:791) ~[ias-tsdb-1.3.2.jar:?]
        at com.infiniteautomation.tsdb.impl.CorruptionScanner.findCorruption(CorruptionScanner.java:498) [ias-tsdb-1.3.2.jar:?]
        at com.infiniteautomation.tsdb.impl.CorruptionScanner.checkFile(CorruptionScanner.java:467) [ias-tsdb-1.3.2.jar:?]
        at com.infiniteautomation.tsdb.impl.CorruptionScanner.checkShard(CorruptionScanner.java:434) [ias-tsdb-1.3.2.jar:?]
        at com.infiniteautomation.tsdb.impl.IasTsdbImpl.repairShard(IasTsdbImpl.java:1226) [ias-tsdb-1.3.2.jar:?]
        at com.infiniteautomation.tsdb.impl.IasTsdbImpl.multiQuery(IasTsdbImpl.java:529) [ias-tsdb-1.3.2.jar:?]
        at com.infiniteautomation.nosql.MangoNoSqlPointValueDao.getPointValuesBetween(MangoNoSqlPointValueDao.java:238) [mangoNoSqlDatabase-3.1.1.jar:?]
        at com.infiniteautomation.mango.excelreports.rt.ExcelReportWorkItem.execute(ExcelReportWorkItem.java:501) [excel-reports-3.1.2.jar:?]
        at com.serotonin.m2m2.rt.maint.BackgroundProcessing$RejectableWorkItemRunnable.run(BackgroundProcessing.java:556) [mango-3.1.1.jar:?]
        at com.serotonin.timer.Task.runTask(Task.java:179) [mango-3.1.1.jar:?]
        at com.serotonin.timer.TaskWrapper.run(TaskWrapper.java:23) [mango-3.1.1.jar:?]
        at com.serotonin.timer.OrderedThreadPoolExecutor$OrderedTaskCollection.run(OrderedThreadPoolExecutor.java:307) [mango-3.1.1.jar:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_33]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_33]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_33]
Caused by: java.lang.OutOfMemoryError: Map failed
        at sun.nio.ch.FileChannelImpl.map0(Native Method) ~[?:1.8.0_33]
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:904) ~[?:1.8.0_33]
        ... 16 more
INFO  2017-07-16T06:01:12,047 (com.infiniteautomation.tsdb.impl.IasTsdbImpl.repairShard:1237) - Corruption repair report available at: /opt/mango/logs/tsdb-serie                        s-68-shard-698-scan-report_2017-07-16_06-01-09.log
INFO  2017-07-16T06:01:12,047 (com.infiniteautomation.tsdb.impl.IasTsdbImpl.repairShard:1237) - Corruption repair report available at: /opt/mango/logs/tsdb-serie                        s-68-shard-698-scan-report_2017-07-16_06-01-09.log
WARN  2017-07-16T07:49:21,011 (com.serotonin.m2m2.util.timeout.TaskRejectionHandler.rejectedTask:77) - Rejected task: Interval logging: DP_299411 because Task Qu                        eue Full
WARN  2017-07-16T07:49:21,011 (com.serotonin.m2m2.util.timeout.TaskRejectionHandler.rejectedTask:77) - Rejected task: Interval logging: DP_299411 because Task Qu                        eue Full
WARN  2017-07-16T12:00:24,185 (com.serotonin.m2m2.util.timeout.TaskRejectionHandler.rejectedTask:77) - Rejected task: Interval logging: DP_452922 because Task Qu                        eue Full
WARN  2017-07-16T12:00:24,185 (com.serotonin.m2m2.util.timeout.TaskRejectionHandler.rejectedTask:77) - Rejected task: Interval logging: DP_452922 because Task Qu                        eue Full
WARN  2017-07-16T18:01:00,131 (com.serotonin.m2m2.util.timeout.TaskRejectionHandler.rejectedTask:77) - Rejected task: Generating report: Hourly Compliance becaus                        e Task Queue Full
WARN  2017-07-16T18:01:00,131 (com.serotonin.m2m2.util.timeout.TaskRejectionHandler.rejectedTask:77) - Rejected task: Generating report: Hourly Compliance becaus                        e Task Queue Full
WARN  2017-07-16T19:01:00,168 (com.serotonin.m2m2.util.timeout.TaskRejectionHandler.rejectedTask:77) - Rejected task: Generating report: Hourly Compliance becaus

            shutdown that won't complete: last line repeats forever:

            mango@mangoES2147:/opt/mango/bin$sudo ./ma.sh stop
mango@mangoES2147:/opt/mango/bin$INFO  2017-07-17T10:41:05,062 (com.serotonin.m2m2.Lifecycle.terminate:361) - Mango Lifecycle terminating...
INFO  2017-07-17T10:41:05,062 (com.serotonin.m2m2.Lifecycle.terminate:361) - Mango Lifecycle terminating...
INFO  2017-07-17T10:41:05,138 (com.serotonin.m2m2.rt.DataSourceGroupTerminator.terminate:72) - Terminating 8 NORMAL priority data sources in 8 threads.
INFO  2017-07-17T10:41:05,138 (com.serotonin.m2m2.rt.DataSourceGroupTerminator.terminate:72) - Terminating 8 NORMAL priority data sources in 8 threads.
INFO  2017-07-17T10:41:05,147 (com.serotonin.m2m2.rt.RuntimeManager.stopDataSourceShutdown:418) - Data source 'Cameras' stopped
INFO  2017-07-17T10:41:05,147 (com.serotonin.m2m2.rt.RuntimeManager.stopDataSourceShutdown:418) - Data source 'Cameras' stopped
INFO  2017-07-17T10:41:05,147 (com.serotonin.m2m2.rt.RuntimeManager.stopDataSourceShutdown:418) - Data source 'Mango Performance' stopped
INFO  2017-07-17T10:41:05,147 (com.serotonin.m2m2.rt.RuntimeManager.stopDataSourceShutdown:418) - Data source 'Mango Performance' stopped
INFO  2017-07-17T10:41:05,154 (com.serotonin.m2m2.rt.RuntimeManager.stopDataSourceShutdown:418) - Data source 'PLCi' stopped
INFO  2017-07-17T10:41:05,154 (com.serotonin.m2m2.rt.RuntimeManager.stopDataSourceShutdown:418) - Data source 'PLCi' stopped
INFO  2017-07-17T10:41:05,179 (com.serotonin.m2m2.rt.RuntimeManager.stopDataSourceShutdown:418) - Data source 'Mango Internal' stopped
INFO  2017-07-17T10:41:05,179 (com.serotonin.m2m2.rt.RuntimeManager.stopDataSourceShutdown:418) - Data source 'Mango Internal' stopped
INFO  2017-07-17T10:41:05,184 (com.serotonin.m2m2.rt.RuntimeManager.stopDataSourceShutdown:418) - Data source 'Unit 1' stopped
INFO  2017-07-17T10:41:05,184 (com.serotonin.m2m2.rt.RuntimeManager.stopDataSourceShutdown:418) - Data source 'Unit 1' stopped
INFO  2017-07-17T10:41:05,191 (com.serotonin.m2m2.rt.RuntimeManager.stopDataSourceShutdown:418) - Data source 'Unit 2' stopped
INFO  2017-07-17T10:41:05,191 (com.serotonin.m2m2.rt.RuntimeManager.stopDataSourceShutdown:418) - Data source 'Unit 2' stopped
INFO  2017-07-17T10:41:05,191 (com.serotonin.m2m2.rt.RuntimeManager.stopDataSourceShutdown:418) - Data source 'PLC0' stopped
INFO  2017-07-17T10:41:05,191 (com.serotonin.m2m2.rt.RuntimeManager.stopDataSourceShutdown:418) - Data source 'PLC0' stopped
INFO  2017-07-17T10:41:05,213 (com.serotonin.m2m2.rt.RuntimeManager.stopDataSourceShutdown:418) - Data source 'MangoES System' stopped
INFO  2017-07-17T10:41:05,213 (com.serotonin.m2m2.rt.RuntimeManager.stopDataSourceShutdown:418) - Data source 'MangoES System' stopped
INFO  2017-07-17T10:41:05,252 (com.serotonin.m2m2.rt.DataSourceGroupTerminator.terminate:102) - Termination of 8 NORMAL priority data sources took 114ms
INFO  2017-07-17T10:41:05,252 (com.serotonin.m2m2.rt.DataSourceGroupTerminator.terminate:102) - Termination of 8 NORMAL priority data sources took 114ms
INFO  2017-07-17T10:41:13,291 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:462) - BackgroundProcessing waiting 60 more seconds for 1 active and 1 queued low priority tasks to complete.
INFO  2017-07-17T10:41:13,291 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:462) - BackgroundProcessing waiting 60 more seconds for 1 active and 1 queued low priority tasks to complete.
INFO  2017-07-17T10:41:18,292 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:462) - BackgroundProcessing waiting 55 more seconds for 1 active and 1 queued low priority tasks to complete.
INFO  2017-07-17T10:41:18,292 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:462) - BackgroundProcessing waiting 55 more seconds for 1 active and 1 queued low priority tasks to complete.
INFO  2017-07-17T10:41:23,293 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:462) - BackgroundProcessing waiting 50 more seconds for 1 active and 1 queued low priority tasks to complete.
INFO  2017-07-17T10:41:23,293 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:462) - BackgroundProcessing waiting 50 more seconds for 1 active and 1 queued low priority tasks to complete.
INFO  2017-07-17T10:41:28,294 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:462) - BackgroundProcessing waiting 45 more seconds for 1 active and 1 queued low priority tasks to complete.
INFO  2017-07-17T10:41:28,294 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:462) - BackgroundProcessing waiting 45 more seconds for 1 active and 1 queued low priority tasks to complete.
INFO  2017-07-17T10:41:33,295 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:462) - BackgroundProcessing waiting 40 more seconds for 1 active and 1 queued low priority tasks to complete.
INFO  2017-07-17T10:41:33,295 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:462) - BackgroundProcessing waiting 40 more seconds for 1 active and 1 queued low priority tasks to complete.
INFO  2017-07-17T10:41:38,296 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:462) - BackgroundProcessing waiting 35 more seconds for 1 active and 1 queued low priority tasks to complete.
INFO  2017-07-17T10:41:38,296 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:462) - BackgroundProcessing waiting 35 more seconds for 1 active and 1 queued low priority tasks to complete.
INFO  2017-07-17T10:41:43,297 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:462) - BackgroundProcessing waiting 30 more seconds for 1 active and 1 queued low priority tasks to complete.
INFO  2017-07-17T10:41:43,297 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:462) - BackgroundProcessing waiting 30 more seconds for 1 active and 1 queued low priority tasks to complete.
INFO  2017-07-17T10:41:48,298 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:462) - BackgroundProcessing waiting 25 more seconds for 1 active and 1 queued low priority tasks to complete.
INFO  2017-07-17T10:41:48,298 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:462) - BackgroundProcessing waiting 25 more seconds for 1 active and 1 queued low priority tasks to complete.
INFO  2017-07-17T10:41:53,299 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:462) - BackgroundProcessing waiting 20 more seconds for 1 active and 1 queued low priority tasks to complete.
INFO  2017-07-17T10:41:53,299 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:462) - BackgroundProcessing waiting 20 more seconds for 1 active and 1 queued low priority tasks to complete.
INFO  2017-07-17T10:41:58,300 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:462) - BackgroundProcessing waiting 15 more seconds for 1 active and 1 queued low priority tasks to complete.
INFO  2017-07-17T10:41:58,300 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:462) - BackgroundProcessing waiting 15 more seconds for 1 active and 1 queued low priority tasks to complete.
INFO  2017-07-17T10:42:03,301 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:462) - BackgroundProcessing waiting 10 more seconds for 1 active and 1 queued low priority tasks to complete.
INFO  2017-07-17T10:42:03,301 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:462) - BackgroundProcessing waiting 10 more seconds for 1 active and 1 queued low priority tasks to complete.
INFO  2017-07-17T10:42:08,302 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:462) - BackgroundProcessing waiting 5 more seconds for 1 active and 1 queued low priority tasks to complete.
INFO  2017-07-17T10:42:08,302 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:462) - BackgroundProcessing waiting 5 more seconds for 1 active and 1 queued low priority tasks to complete.
INFO  2017-07-17T10:42:13,305 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 60 more seconds for 18 active and 0 queued high priority tasks to complete.
INFO  2017-07-17T10:42:13,305 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 60 more seconds for 18 active and 0 queued high priority tasks to complete.
INFO  2017-07-17T10:42:18,306 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 55 more seconds for 18 active and 0 queued high priority tasks to complete.
INFO  2017-07-17T10:42:18,306 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 55 more seconds for 18 active and 0 queued high priority tasks to complete.
INFO  2017-07-17T10:42:23,307 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 50 more seconds for 18 active and 0 queued high priority tasks to complete.
INFO  2017-07-17T10:42:23,307 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 50 more seconds for 18 active and 0 queued high priority tasks to complete.
INFO  2017-07-17T10:42:28,308 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 45 more seconds for 18 active and 0 queued high priority tasks to complete.
INFO  2017-07-17T10:42:28,308 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 45 more seconds for 18 active and 0 queued high priority tasks to complete.
INFO  2017-07-17T10:42:33,309 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 40 more seconds for 18 active and 0 queued high priority tasks to complete.
INFO  2017-07-17T10:42:33,309 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 40 more seconds for 18 active and 0 queued high priority tasks to complete.
INFO  2017-07-17T10:42:38,310 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 35 more seconds for 18 active and 0 queued high priority tasks to complete.
INFO  2017-07-17T10:42:38,310 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 35 more seconds for 18 active and 0 queued high priority tasks to complete.
INFO  2017-07-17T10:42:43,311 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 30 more seconds for 18 active and 0 queued high priority tasks to complete.
INFO  2017-07-17T10:42:43,311 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 30 more seconds for 18 active and 0 queued high priority tasks to complete.
INFO  2017-07-17T10:42:48,312 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 25 more seconds for 18 active and 0 queued high priority tasks to complete.
INFO  2017-07-17T10:42:48,312 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 25 more seconds for 18 active and 0 queued high priority tasks to complete.
INFO  2017-07-17T10:42:53,313 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 20 more seconds for 18 active and 0 queued high priority tasks to complete.
INFO  2017-07-17T10:42:53,313 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 20 more seconds for 18 active and 0 queued high priority tasks to complete.
INFO  2017-07-17T10:42:58,314 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 15 more seconds for 18 active and 0 queued high priority tasks to complete.
INFO  2017-07-17T10:42:58,314 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 15 more seconds for 18 active and 0 queued high priority tasks to complete.
INFO  2017-07-17T10:43:03,315 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 10 more seconds for 18 active and 0 queued high priority tasks to complete.
INFO  2017-07-17T10:43:03,315 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 10 more seconds for 18 active and 0 queued high priority tasks to complete.
INFO  2017-07-17T10:43:08,316 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 5 more seconds for 18 active and 0 queued high priority tasks to complete.
INFO  2017-07-17T10:43:08,316 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:473) - BackgroundProcessing waiting 5 more seconds for 18 active and 0 queued high priority tasks to complete.
INFO  2017-07-17T10:43:12,317 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:481) - All high priority tasks exited gracefully.
INFO  2017-07-17T10:43:12,317 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:481) - All high priority tasks exited gracefully.
INFO  2017-07-17T10:43:12,318 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:487) - All medium priority tasks exited gracefully.
INFO  2017-07-17T10:43:12,318 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:487) - All medium priority tasks exited gracefully.
INFO  2017-07-17T10:43:12,318 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:494) - 1 low priority tasks forcefully terminated.
INFO  2017-07-17T10:43:12,318 (com.serotonin.m2m2.rt.maint.BackgroundProcessing.joinTermination:494) - 1 low priority tasks forcefully terminated.
INFO  2017-07-17T10:43:12,763 (com.infiniteautomation.nosql.MangoNoSqlProxy.shutdown:115) - Terminating NoSQL Batch Write Manager.
INFO  2017-07-17T10:43:12,763 (com.infiniteautomation.nosql.MangoNoSqlProxy.shutdown:115) - Terminating NoSQL Batch Write Manager.
INFO  2017-07-17T10:43:12,764 (com.infiniteautomation.nosql.MangoNoSqlBatchWriteBehindManager.terminate:242) - Terminating NoSQL Point Value Mover.
INFO  2017-07-17T10:43:12,764 (com.infiniteautomation.nosql.MangoNoSqlBatchWriteBehindManager.terminate:242) - Terminating NoSQL Point Value Mover.
INFO  2017-07-17T10:43:12,764 (com.infiniteautomation.nosql.MangoNoSqlBatchWriteBehindManager.terminate:249) - Terminating 16 Batch Writer Tasks.
INFO  2017-07-17T10:43:12,764 (com.infiniteautomation.nosql.MangoNoSqlBatchWriteBehindManager.terminate:249) - Terminating 16 Batch Writer Tasks.
INFO  2017-07-17T10:43:12,767 (com.infiniteautomation.nosql.MangoNoSqlBatchWriteBehindManager.terminate:258) - 16 Batch Writer Tasks awaiting termination.
INFO  2017-07-17T10:43:12,767 (com.infiniteautomation.nosql.MangoNoSqlBatchWriteBehindManager.terminate:258) - 16 Batch Writer Tasks awaiting termination.
WARN  2017-07-17T10:43:22,767 (com.infiniteautomation.nosql.MangoNoSqlBatchWriteBehind.joinTermination:169) - Waiting for Batch Writer Task 0 to stop
WARN  2017-07-17T10:43:22,767 (com.infiniteautomation.nosql.MangoNoSqlBatchWriteBehind.joinTermination:169) - Waiting for Batch Writer Task 0 to stop
WARN  2017-07-17T10:43:32,768 (com.infiniteautomation.nosql.MangoNoSqlBatchWriteBehind.joinTermination:169) - Waiting for Batch Writer Task 0 to stop
WARN  2017-07-17T10:43:32,768 (com.infiniteautomation.nosql.MangoNoSqlBatchWriteBehind.joinTermination:169) - Waiting for Batch Writer Task 0 to stop
WARN  2017-07-17T10:43:42,769 (com.infiniteautomation.nosql.MangoNoSqlBatchWriteBehind.joinTermination:169) - Waiting for Batch Writer Task 0 to stop
            1 Reply Last reply Reply Quote 0
            • phildunlapP
              phildunlap
              last edited by phildunlap

              If this is Mango 2.8 there are some updates to Excel Reports and NoSQL that should solve these issues. If this is Mango 3 we'll have those updates out before the end of the day.

              1 Reply Last reply Reply Quote 0
              • C
                craig
                last edited by

                Hi Phil,

                This was on mango 3 at all times. I had upgraded to the latest mango 3 a week ago after the problem happened the first time.

                I have just upgraded the mangoNoSQL module. No update to the excel reports appeared in the list of newly released modules.

                We will make sure to upgrade all of the MangoES with version 2.8 to the latest 2.8 as well.

                Thanks for such a quick turn around!

                1 Reply Last reply Reply Quote 0
                • First post
                  Last post