DeadBanding data streams, ideas / thoughts / comments?

mrlucretius

Hello,

I am a new Mango user, on a path to semi-expertise hopefully down the road.

I am working on getting the source up and running in eclipse, so I think I may be able to architect this function myself after I understand Mango better.

I have had a request to be able to deadband data streams (to minimize storage usage when data is not changing or a system is "dead").

That is, when a new reading of a data point is made, it is compared to the last data point recorded, and then (according to some band width defined somewhere related to the data point) the new data reading is either ignored (not written to storage) or saved (written to storage) depending on whether the new data was within or outside of the dead band.

Forgive me if my terminology is not inline with Mango quite yet (data point / stream / source semantics)....

That is, I only record (store to db) a new voltage reading if it has changed by, say, +- 5%, or +-10 volts of the last recorded voltage.

However, if anyone has any pointers as to where / how I can implement this your thoughts are appreciated, Would this be easiest with a custom module / modules that wrap standard data sources maybe? Where would the band width / percentage and maybe some static setpoint be stored?

I hope to be able to answer these questions myself after some work... But if anyone jump start me / can point me in the right direction I would appreciate it.

Or, will this possibly break other functions within mango in a serious fashion? Ie, will data not on a "grid" (equal time spacing) somehow not play nice with charting / visualization / statistics? Might some major effort to re-grid / interpolate data be needed on some / a lot of current functions?

Thanks,
Alex

phildunlap

Hi Alex, welcome to the forum! You'll not need eclipse for this, unless you're engineering a new logging type and have reason not to use a meta point.

You have a lot of options in Mango for how your data is logged, but it sounds like you're looking for a point with an ON_CHANGE loggingType, which gives you the ability to define a "tolerance" that is a numeric deadband, where samples won't be recorded until they change by at least that amount.

Also, available in the latest versions of Mango, you can have your point record its data ON_CHANGE_INTERVAL which will record a sample, and then use that sample single "tolerance" +/- value to assess change, but if a change has not come within the interval, the latest sampled value (so, it can change within the tolerance).

Our data API enables you to get data with flatly interpolated values at the requested timestamps (so, the first value at the start time may be the value preceding the period and it is flagged as such, which is to ease some charting tasks. The last value in the period would also be presented at the end time, in that query). So, whatever logging type you've selected for your data, it will appear. A point with no values in the period will have the sample preceding the period, if there is one, at both the start and end times in a bookended query, marked as bookends. Or you can also not include these bookends.

As far as getting more exotic with logging types than the available options, such as having a percentage tolerance, I would direct you to a meta data point on a meta data source. You can add the origin point into the context and listen to the sample updates (even if you are logging the source point some way, you can choose the update event the meta point will respond to, if any), and then compute from the current value of the meta point and the current value of the context points (or their histories) if the meta point should log the new data. It can return UNCHANGED; if there is no new value. When minimizing storage, you can override the purge settings on the source point if it samples frequently.

phildunlap

Some more information: https://help.infiniteautomation.com/data-point-and-logging-settings/ but dated so far as not having the ON_CHANGE_INTERVAL setting at that time.