Please Note This forum exists for community support for the Mango product family and the Radix IoT Platform. Although Radix IoT employees participate in this forum from time to time, there is no guarantee of a response to anything posted here, nor can Radix IoT, LLC guarantee the accuracy of any information expressed or conveyed. Specific project questions from customers with active support contracts are asked to send requests to support@radixiot.com.
MangoGT running out of memory?
-
Catching a few MangoGT units hanging, to find Out Of Memory errors in the logs:
Dec 06 01:00:43 mangoGT5176 start-mango.sh[529]: Exception in thread "high-pool-2-thread-5894" com.infiniteautomation.tsdb.IasTsdbException: java.io.IOException: Map failed Dec 06 01:00:43 mangoGT5176 start-mango.sh[529]: at com.infiniteautomation.tsdb.impl.IasTsdbImpl.count(IasTsdbImpl.java:1124) Dec 06 01:00:43 mangoGT5176 start-mango.sh[529]: at com.infiniteautomation.nosql.InternalGenericDao.rangeCount(InternalGenericDao.java:393) Dec 06 01:00:43 mangoGT5176 start-mango.sh[529]: at [snipped] Dec 06 01:02:22 mangoGT5176 start-mango.sh[529]: at com.serotonin.m2m2.persistent.pub.SyncHandler$PointSync.checkRangeImpl(SyncHandler.java:514) Dec 06 01:02:22 mangoGT5176 start-mango.sh[529]: at com.serotonin.m2m2.persistent.pub.SyncHandler$PointSync.checkRange(SyncHandler.java:345) Dec 06 01:02:22 mangoGT5176 start-mango.sh[529]: at com.serotonin.m2m2.persistent.pub.SyncHandler$PointSync.checkPoint(SyncHandler.java:326) Dec 06 01:02:22 mangoGT5176 start-mango.sh[529]: at com.serotonin.m2m2.persistent.pub.SyncHandler$PointSync.run(SyncHandler.java:276) Dec 06 01:02:22 mangoGT5176 start-mango.sh[529]: at com.serotonin.timer.sync.Synchronizer$TaskWrapper.run(Synchronizer.java:150) Dec 06 01:02:22 mangoGT5176 start-mango.sh[529]: at com.serotonin.timer.Task.runTask(Task.java:179) Dec 06 01:02:22 mangoGT5176 start-mango.sh[529]: at com.serotonin.timer.TaskWrapper.run(TaskWrapper.java:23) Dec 06 01:02:22 mangoGT5176 start-mango.sh[529]: at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) Dec 06 01:02:22 mangoGT5176 start-mango.sh[529]: at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) Dec 06 01:02:22 mangoGT5176 start-mango.sh[529]: at java.base/java.lang.Thread.run(Thread.java:830) Dec 06 01:02:22 mangoGT5176 start-mango.sh[529]: Caused by: java.io.IOException: Map failed Dec 06 01:02:22 mangoGT5176 start-mango.sh[529]: at java.base/sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:1017) Dec 06 01:02:22 mangoGT5176 start-mango.sh[529]: at com.infiniteautomation.tsdb.impl.ChecksumMappedByteBufferInputStream.<init>(ChecksumMappedByteBufferInputStream.java:41) Dec 06 01:02:22 mangoGT5176 start-mango.sh[529]: at com.infiniteautomation.tsdb.impl.ReversibleShard.openDataIn(ReversibleShard.java:250) Dec 06 01:02:22 mangoGT5176 start-mango.sh[529]: at com.infiniteautomation.tsdb.impl.Shard.query(Shard.java:473) Dec 06 01:02:22 mangoGT5176 start-mango.sh[529]: at com.infiniteautomation.tsdb.impl.CompressibleShard.query(CompressibleShard.java:391) Dec 06 01:02:22 mangoGT5176 start-mango.sh[529]: at com.infiniteautomation.tsdb.impl.Series.query(Series.java:532) Dec 06 01:02:22 mangoGT5176 start-mango.sh[529]: at com.infiniteautomation.tsdb.impl.IasTsdbImpl.count(IasTsdbImpl.java:1095) Dec 06 01:02:22 mangoGT5176 start-mango.sh[529]: ... 48 more Dec 06 01:02:22 mangoGT5176 start-mango.sh[529]: Caused by: java.lang.OutOfMemoryError: Map failed Dec 06 01:02:22 mangoGT5176 start-mango.sh[529]: at java.base/sun.nio.ch.FileChannelImpl.map0(Native Method) Dec 06 01:02:22 mangoGT5176 start-mango.sh[529]: at java.base/sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:1014) Dec 06 01:02:22 mangoGT5176 start-mango.sh[529]: ... 54 more
Suggestions? Adjust thread pools? Swap?
-
Was this ever fixed?
-
Either get more memory or try this env property:
#Set the number of files the database can have open at one time db.nosql.maxOpenFiles=500
It will close/unmap files at the expense of cpu use. If this problem persists then try another stream type:
#Query Performance Tuning, File Access Type:Available[INPUT_STREAM,FILE_CHANNEL,RANDOM_ACCESS_FILE,MAPPED_BYTE_BUFFER] db.nosql.shardStreamType=MAPPED_BYTE_BUFFER