Please Note This forum exists for community support for the Mango product family and the Radix IoT Platform. Although Radix IoT employees participate in this forum from time to time, there is no guarantee of a response to anything posted here, nor can Radix IoT, LLC guarantee the accuracy of any information expressed or conveyed. Specific project questions from customers with active support contracts are asked to send requests to support@radixiot.com.
Derby database purging
-
Is there some way to purge old entries based on database size?
-
Size? No. How would it know what to purge?
-
Eg. A parameter sets the maximum size for the database starts purging lets say 30GB, when this size is reached the oldest entries can be erased, something like overwriting the oldest files.
-
Sure, but this would conflict with the explicit purging characteristics that can be set per point, i.e. one year on some points, one month on others. Even if all points were set the same, you'd still have to wonder how events and reports fold in.
How about this: a "big db" alarm could be created that triggers when the database reaches a given threshold. This can then be used to set the purging such that you don't get the alarm. Another thought: an automatic point could be created that reads the db size; say with an "internal monitoring" data source.
-
a "big db" alarm could be created that triggers when the database reaches a given threshold. This can then be used to set the purging such that you don't get the alarm. Another thought: an automatic point could be created that reads the db size; say with an "internal monitoring" data source.
Both of theese options sounds good for me.
I think some coding is needed, if not, could you please point me on how to do that in the current version? -
Yes, coding would be needed in both cases.
-
Nice, it will act as an automated task triggered by database size which will purge the database based on purging characteristics set for each point.
Should I add this feature to the wish list or do you plan to add this to the roadmap?
Thank you!
-
Um, i'm not sure about that one. The purging runs once a day anyway (cron trigger "0 5 3 * * ?"). The danger of triggering it from an event is that once the purging has already run subsequent executions may not have any effect. If the purge does not in fact cause the size of the database to drop below the threshold, you may have a situation where the event continuously runs an ineffective and expensive operation.
There are ways to detect this and prevent it, but it would be good to know about the characteristics of a system for which the daily purge is insufficient.
-
I undestand your point and it makes sense but don't you think it would be useful some kind of control based of database size?
Or at least some relation with the explicit purging characteristics and the database size.
We have this scenario: we wanna store data for 1 year and do logs at 15 seconds interval. That's ok but we don't know if the HD will have enough space to store the database.
Is it possible to calculate how much space do I need to do that, if not, how could we handle this?
-
With daily purging, that scenario will result in a very stable database size once all of the purging has kicked in. How about you set your system up, let it run for a month, look at the DB size then, and then multiply by 12?
I understand that it would be nice to have something automated - believe me i do. But there are a great number of things that are planned for this system. This one honestly has a strong case for remaining mostly manual.