Please Note This forum exists for community support for the Mango product family and the Radix IoT Platform. Although Radix IoT employees participate in this forum from time to time, there is no guarantee of a response to anything posted here, nor can Radix IoT, LLC guarantee the accuracy of any information expressed or conveyed. Specific project questions from customers with active support contracts are asked to send requests to support@radixiot.com.
disk full - mangoTSDB and reports directories take up all the space
-
Hi,
I have run out of disk space on mangoES. the mangoTSDB directory is 2.6 GB and the reports directory is 2.3 GB.
I have changed the purge settings down from purge all data after 45 days to purge all data after 35 days and ran the purge right away but haven't seen any increase in free disk space.
Report instances are set to purge after 2 days. I only have 1 report and it is sent by email so I don't think it should be stored on the mango server.
What is the reports directory and why is it so large even though I shouldn't have any stored report instances?
How can I free up some space? The opt/mango/databases directory needs to shrink a bit.(I deleted the apt-cache and jdk8 installer to get a bit of space to get mango working again so I could change the purge settings.)
-
Hi Craig,
What version of Mango is installed? I think in a older version there was an issue where the reports database was not purged during normal purging. If you don't need any of your report data you could manually delete the reports database under MA_HOME/databases/reports
Also note that data purging isn't exact. It is done by deleting shards in the database a shard can have up to approximately 11 days of data so the purge could be only happen once all the data in a shard is older than your purge time. This means you could have 11 days more data kept than expected.
If your Mango isn't up-to-date I would manually delete your reports database and then do the Mango upgrade.
-
Hi Joel,
the mango is 2.7.8. I changed the report purge settings down to 2 days and told it to purge now and the reports folder stayed at 2.3 GB. I clicked purge all reports and it reported it purged 1 report. reports folder is still 2.3 GB.
I have now deleted the contents of the reports directory except for db.properties and now have 2.5 GB free. Will see if this stays constant or gradually decreases over time. Thanks for the intel
craig -
I just did a simple test on my system by creating a report with 2 data points running it and I see data files in the reports database. Click on Purge all reports on the system setting page and all the .data files are deleted.
It might be worth making sure that the "Do not purge" option isn't used on the reports on the reports page. Also you can check your log file after trying to purge your reports to see if there are any errors. In my logs I see this:
INFO 2016-03-22 21:26:41,227 (com.serotonin.m2m2.reports.web.ReportsDwr.purgeAllNow:241) - Report purge ended, 1 report instances deleted
-
I believe there is some issue with deleting report data in mango 2.7.8 for reports either run by cron or sent by e-mail.
The report is run by cron at 6AM, noon, 5PM, 11PM and is sent by e-mail. The report instances that are executed by cron do not show up in the report queue which is consistent with how mango 1.12 works.
When I run the report from the reports page using the "run now" button the report instance shows up in the report queue. When I purge reports only reports that show up in the report queue are in fact purged.
I can see the timestamps on the folders in the mango/databases/reports directory are at 6AM, 5PM, noon, 11PM and this directory now has 40MB of data since yesterday morning, which is all the report instances that were executed by cron and sent by e-mail. The purge all reports now function does not purge them.
Should I file an issue on github?
craig
-
Thanks for bringing this to our attention Craig. I investigated, and created a github issue:
https://github.com/infiniteautomation/ma-core-public/issues/687Let me know if you need a cleanup script so that you can have data for the manually run reports still around (as opposed to clearing out the databases/reports directory completely). The only method I've come up with so far would be something like, make a copy of databases/reports, run a purge of all reports from the system settings page, list all files in databases/reports ending in .data, switch your copy and your purged databases with Mango off, move all non-purged reports or delete them.
something like...
find databases/reports | grep -e ".data$" > orphanData
==== switch databases to original ====
for line in $(cat orphanData); do rm "$line"; done
^ untested -
Hi Phil,
There are no report instances that we wish to save, they go out by e-mail and that is the end of them so for the time being i will have to remember to wipe the databases/reports directory every month.
It seems you have already identified a fix in the github issue so hopefully it gets some attention for a release in the next couple months.
thanks both for your time and assistance
