How to setup and event on rate of change?

[v8dave]

I'd like to be able to setup an alarm that sends an email if the value changes by a certain amount over a set time.

Eg, if the data point drops by more than 20 in 5 mins it should trigger an event. I am trying to detect if someone has started to empty a storage tank and trigger an alert based on this.

There is a change event for the data points but it doesn'y have any values or settings.

Is this possible to do?

[phildunlap]

Hi Dave,

This is done through a meta point.

Something like....

var changeThreshhold = 5 //5 units per minute, let's say
var data = p.past(MINUTE, 5)
if( data.count == 0 ) {
  //Handle no data in period condition, raise event?
  return true;
} else if ( data.count == 1 ) {
  //Handle only one piece of data in our range by getting another value
  var lv = p.lastValue(1);
  if ( lv == null ) //Only one data point, no cause for alarm. We may not need to check this
     return false
  return changeThreshhold > ( data.lastValue - lv.value )/((data.lastTime - lv.time)/60000);
} else {
  return changeThreshhold > ( data.lastValue - data.firstValue )/((data.lastTime - data.firstTime)/60000);
}

Then having a state detector on the meta point checking for a "true" state. This isn't its own detector because the shrewd observer may see data sets where this logic is deeply flawed (specifically, data that isn't monotonic). For situations where your data varies in both directions, you may be more interested in changing the logic to sum the differences as you iterate the list, and alarm if you cross the threshhold, more like....

var data = p.pointValuesSince(new Date().getTime() - 5*60*1000) //Five minutes in milliseconds
if( data.length <= 1 ) //we have the same edge conditions for our range
   return false
var sum = 0;
var changeThreshhold = 5; //so, this would mean that if any point in the period it is more than
for( var k = 0; k < data.length-1; k+=1 ) {
  sum += (data[k+1].value - data[k].value);
  if( sum < changeThreshhold || sum > changeThreshhold )
    return true;
}
return false;

So, what it means to have changed over time can be funky depending on your data set. That second version should be equivalent to seeing if any values in a range are outside the acceptable rate of change... like,

var data = p.pointValuesSince(new Date().getTime() - 5*60*1000) //Five minutes in milliseconds
if( data.length <= 1 ) //we have the same edge conditions for our range
   return false
var firstValue = data[0];
var changeThreshhold = 5; //so, this would mean that if any point in the period it is more than
for( var k = 1; k < data.length; k+=1 ) {
  if( firstValue.value - data[k].value > changeThreshhold || firstValue.value - data[k].value < changeThreshhold )
    return true;
}
return false;

But this could still be flawed. If data[0].value == 0, data[1].value == -3, and data[2].value == 3, then none of these detectors would alarm, even though we experienced a change of 6 at one point. One would expect the next run of the point will have data[0] == -3, but having uniform data in time is an assumption we shouldn't rest our logic on!

[v8dave]

Thanks Phil, once I figure out where the meta script is input I'll give this a go. I have the module loaded but can't see where to input this script. :)

[v8dave]

OK. I found how to create a meta script. It's actually a data source. I'll give your code a try out. Thanks again.

[phildunlap]

Data will be coming from it, won't it? :D

A solution some people use for noisy data, to avoid what I was talking about, is to just use averages from time periods to compare instead of raw data, so,

var period = 300000 //5*60*1000 Five minutes
var now = new Date();
var start = now.getTime() - period;
var middle = now.getTime() - period/2;
if( Math.abs( p.getStats(middle, now).average - p.getStats(start, middle).average ) > 5 ) //Threshhold
  return true;
return false;

Depending on your needs or thoughts on your data.

[v8dave]

Just trying this now and when I input the script and then click save, I get this error at the bottom.

invalid return
at line -1

I've selected a suitable data point and set its name to p for this test.

[phildunlap]

Are you using the scripts as posted or as they arrived in an email notification?

I am able to run all of these with a numeric point for p. That looks like a syntax error, which I think I would get regardless of the code path.

[v8dave]

Hi Phil, I am using a numerical data point. Here is the screen capture. I modified the script as I only need to detect a falling trend.!!

0_1469686311138_script.png

[phildunlap]

it looks like you are using the scripting data source, instead of a meta data source and putting these scripting in a meta data point.

Scripts are intended for a multiple-output control loop. Scripting data source scripts don't contain return statements, they set a point's value with p.set(value, time) or just p.set(value) for the current time.

This kind of thing, where we're producing a single answer from perhaps multiple inputs, should be a meta point.

[v8dave]

Ah yes, so I was. Just trying it now but there is an error about a null value on line 4 which seems strange because it would not get to this line unless there was data.

var data = bbl1.pointValuesSince(new Date().getTime() - 5*60*1000); //Five minutes in milliseconds
if( data.length <= 1 ) //we have the same edge conditions for our range
   return false;
var firstValue = data[0];
var changeThreshhold = -20; //so, this would mean that if any point in the period it is more than
for( var k = 1; k < data.length; k+=1 ) {
  if( firstValue.value - data[k].value < changeThreshhold)
    return true;
}
return false;

[phildunlap]

That is odd....
I would put print(data) at line 2 to investigate.

[v8dave]

I've added that but where do I see the output from this?

[phildunlap]

It would be below the script window, if it ran, where you saw the error. If it didn't, I wonder if you have a global script declared? The global scripts page is this icon:

[v8dave]

I don't have that icon. Is this a module I need to install or a setting somewhere?

[v8dave]

I found the module and installed it and the icon is now present. My scripts still don't seem the be running as there is no output from the print(data) statement.

[phildunlap]

No, I was curious if that's where the error was occurring, not telling you that you needed to install it.

Do you get the same error from a very simple script, perhaps just...

return false;

Are you sure your point is in the context as bbl1 (b - b - lower case L - numeral 1)?

[v8dave]

I am assuming that the variable is based on the area I circled?

[phildunlap]

Yeah, just checking. That script I posted surely runs for me, here's a version of yours with lots of print statements, maybe it will help find the issue?

print("bbl1:");
print(bbl1);
var data = bbl1.pointValuesSince(new Date().getTime() - 5*60*1000); //Five minutes in milliseconds
print("data:");
print(data);
if( data.length <= 1 ) //we have the same edge conditions for our range
   return false;
print("Passed the length condition...");
var firstValue = data[0];
var changeThreshhold = -20; //so, this would mean that if any point in the period it is more than
for( var k = 1; k < data.length; k+=1 ) {
  if( firstValue.value - data[k].value < changeThreshhold)
    return true;
}
return false;

If you see no output, I encourage you to 1) check for updates, 2) clear your browsers cache, 3) delete Mango/work/jsp and then try again.

[v8dave]

Hi Phil, it doesn't seem to run. Nothing appears from the debug. I also deleted the jsp directory and still the same.

[phildunlap]

Are you getting errors in the log, or just on the script's page? Is it still reporting null at line 4? Did you try a very simple script, just

return false;

And see if it reports success result=false?

Well, would it be possible for you to email me the JSON for your Mango and I'll try to find the problem locally? I'd like your whole configuration.