Calculating values on the fly

psysak

Hey guys, just wondering if it's possible to calculate values on the fly in order to graph and download? What I have is two streams of data, the first is hourly electrical consumption data and the second is hourly electricity price. Is it possible for me to keep the two separate and only calculate the cost, which is one hour of consumption data multiplied my that hours cost, when the user requests a graph showing the data? Can the mango platform perform that calculation only for the purpose of the graph? If so am I still able to let the user download all that data via the little download button which appears on graphs?

Thanks

CraigWeb

You didn't point it out but I assume you are talking about doing this logic in your frontend with the dashboard designer. You could use <ma-fn> , pass it the 2 arrays and create a new array with your required logic.

psysak

Yes on the front end. Thanks Craig, that's awesome!

So would the math be part of the "expression"?
<ma-fn expression="" fn="myFunction" ready="" arg-names=""></ma-fn>

Jared Wiltshire

@psysak Here's an example, with points from a watch list. The key is defining a function that multiplies together to two values and then applying it each element in the array - values="values.forEach(multiply); values"
I also pass a dummy point to the serial chart so it picks up the new value which is defined in the array and plots it.
You will need to use a rollup for this to work.

<ma-watch-list-get ng-model="designer.watchList" parameters="designer.parameters" on-points-change="designer.points = $points" id="b7bdf2f9-bf34-417b-af00-58410963e785" watch-list-xid="WL_9950dd8e-91a1-4cf3-8ab1-e6c2b74ccab0"></ma-watch-list-get>

<ma-calc input="designer.points | filter:{name:'Real power'}:true | maFirst" output="power"></ma-calc>
<ma-calc input="designer.points | filter:{name:'Power factor'}:true | maFirst" output="price"></ma-calc>
<ma-fn expression="$input.value_cost = $input['value_' + power.xid] * $input['value_' + price.xid]" fn="multiply" arg-names="['$input']"></ma-fn>

<div class="ma-designer-root" id="8ee80d8a-f0cc-4dd3-a8a4-34e96dbbbdb9" style="width: 1366px; height: 768px; position: relative;">
    <ma-watch-list-parameters id="2c9a323a-440f-4fc0-86c9-f07b6ab4f13b" ng-model="designer.parameters" watch-list="designer.watchList" style="position: absolute; width: 100%; left: 0px; top: 0px;"></ma-watch-list-parameters>
    <ma-point-values id="0be711f5-7aa2-4f1c-985d-f1009ccb7143" points="[power, price]" from="dateBar.from" to="dateBar.to" rollup="{{dateBar.rollupType}}" rollup-interval="{{dateBar.rollupIntervals + ' ' + dateBar.rollupIntervalPeriod}}" style="position: absolute; left: 0px; top: 700px;" values="values"></ma-point-values>
    <ma-serial-chart id="daeda3f3-3509-46c0-a63a-5877bbfb7946" style="position: absolute; width: 100%; height: 600px; left: 0px; top: 70px;" values="values.forEach(multiply); values" points="[power, price, {name: 'Cost', xid: 'cost', deviceName: 'Cost'}]" legend="true" export="true" series3-type="step"></ma-serial-chart>
</div>

psysak

Thank you guys! This is a ton of help.

psysak

Hey @Jared-Wiltshire one more question on this topic. Is it then possible to calculate and display the statistics for the calculated values?

I got it to work with that I needed by the way so thank you!

CraigWeb

hey psysak

Is there any reason you are not creating a meta-point. You can easily do statistics if you used a meta point on logged it. To do statistics on the front end you will need to make another function that loops over your new array with a stats formula

psysak

@craigweb said in Calculating values on the fly:

hey psysak

Is there any reason you are not creating a meta-point. You can easily do statistics if you used a meta point on logged it. To do statistics on the front end you will need to make another function that loops over your new array with a stats formula

I acknowledge that I probably should, it's just a bit complicated cause my two streams of data have different time stamps, and are not updated at the same interval. Basically I will have to write a rather complicated function to work on historical data, checking to see when things were last updated and only calculating my new values up to that point.

You're totally correct @CraigWeb I'm just up against a time deadline in this particular moment and as a proof of concept I was thinking that if it were easy to do on the front end that would be good enough for now.

Jared Wiltshire

@psysak said in Calculating values on the fly:

Is it then possible to calculate and display the statistics for the calculated values?

It is if you are willing to do the calculations yourself, use Array.prototype.reduce()

Here's an example calculating the max/avg/total.

<ma-watch-list-get ng-model="designer.watchList" parameters="designer.parameters" on-points-change="designer.points = $points" id="b7bdf2f9-bf34-417b-af00-58410963e785" watch-list-xid="WL_9950dd8e-91a1-4cf3-8ab1-e6c2b74ccab0"></ma-watch-list-get>

<ma-calc input="designer.points | filter:{name:'Real power'}:true | maFirst" output="power"></ma-calc>
<ma-calc input="designer.points | filter:{name:'Power factor'}:true | maFirst" output="price"></ma-calc>
<ma-fn expression="$input.value_cost = $input['value_' + power.xid] * $input['value_' + price.xid]" fn="multiply" arg-names="['$input']"></ma-fn>
<ma-fn expression="$item.value_cost > $max ? $item.value_cost : $max" fn="find_max" arg-names="['$max', '$item']"></ma-fn>
<ma-calc input="::(1).constructor.isFinite" output="isFinite"></ma-calc>
<ma-fn expression="isFinite($item.value_cost) ? $accum + $item.value_cost : $accum" fn="accumulate" arg-names="['$accum', '$item']"></ma-fn>

<div class="ma-designer-root" id="8ee80d8a-f0cc-4dd3-a8a4-34e96dbbbdb9" style="width: 1366px; height: 768px; position: relative;">
    <ma-watch-list-parameters id="2c9a323a-440f-4fc0-86c9-f07b6ab4f13b" ng-model="designer.parameters" watch-list="designer.watchList" style="position: absolute; width: 100%; left: 0px; top: 0px;"></ma-watch-list-parameters>
    <ma-point-values id="0be711f5-7aa2-4f1c-985d-f1009ccb7143" points="designer.points" from="dateBar.from" to="dateBar.to" rollup="{{dateBar.rollupType}}" rollup-interval="{{dateBar.rollupIntervals + ' ' + dateBar.rollupIntervalPeriod}}" style="position: absolute; left: 0px; top: 700px;" values="values"></ma-point-values>
    <ma-serial-chart id="daeda3f3-3509-46c0-a63a-5877bbfb7946" style="position: absolute; width: 100%; height: 600px; left: 0px; top: 70px;" values="values.forEach(multiply); values" points="[power, price, {name: 'Cost', xid: 'cost', deviceName: 'Cost'}]" legend="true" export="true" series3-type="step"></ma-serial-chart>

    <h1 id="59669906-8019-49fa-88f4-2a3d6c3d4b6c" style="position: absolute; left: 430px; top: 90px;">Max cost is {{values.reduce(find_max, 0) | currency}}</h1>
    <h1 id="b2cca2ae-d6a9-4265-a3db-ba4526bfd244" style="position: absolute; left: 430px; top: 130px;">Total cost is {{values.reduce(accumulate, 0) | currency}}</h1>
    <h1 id="b2cca2ae-d6a9-4265-a3db-ba4526bfd245" style="position: absolute; left: 430px; top: 170px;">Average cost is {{values.reduce(accumulate, 0) / values.length | currency}}</h1>
</div>

Screenshot for good measure:
0_1548361912789_9903c6ab-933a-414b-9fd5-bd27cffe4d69-image.png

psysak

@Jared-Wiltshire what language/syntax are the expressions written in in all this? Also out of curiosity what is maFirst?

Jared Wiltshire

@psysak said in Calculating values on the fly:

what language/syntax are the expressions written in in all this?

They are AngularJS expressions, essentially JavaScript

@psysak said in Calculating values on the fly:

what is maFirst

It is an AngularJS filter that simply takes the first value of an array.

psysak

@jared-wiltshire said in Calculating values on the fly:

@psysak Here's an example, with points from a watch list. The key is defining a function that multiplies together to two values and then applying it each element in the array - values="values.forEach(multiply); values"
I also pass a dummy point to the serial chart so it picks up the new value which is defined in the array and plots it.
You will need to use a rollup for this to work.

<ma-watch-list-get ng-model="designer.watchList" parameters="designer.parameters" on-points-change="designer.points = $points" id="b7bdf2f9-bf34-417b-af00-58410963e785" watch-list-xid="WL_9950dd8e-91a1-4cf3-8ab1-e6c2b74ccab0"></ma-watch-list-get>

<ma-calc input="designer.points | filter:{name:'Real power'}:true | maFirst" output="power"></ma-calc>
<ma-calc input="designer.points | filter:{name:'Power factor'}:true | maFirst" output="price"></ma-calc>
<ma-fn expression="$input.value_cost = $input['value_' + power.xid] * $input['value_' + price.xid]" fn="multiply" arg-names="['$input']"></ma-fn>

<div class="ma-designer-root" id="8ee80d8a-f0cc-4dd3-a8a4-34e96dbbbdb9" style="width: 1366px; height: 768px; position: relative;">
    <ma-watch-list-parameters id="2c9a323a-440f-4fc0-86c9-f07b6ab4f13b" ng-model="designer.parameters" watch-list="designer.watchList" style="position: absolute; width: 100%; left: 0px; top: 0px;"></ma-watch-list-parameters>
    <ma-point-values id="0be711f5-7aa2-4f1c-985d-f1009ccb7143" points="[power, price]" from="dateBar.from" to="dateBar.to" rollup="{{dateBar.rollupType}}" rollup-interval="{{dateBar.rollupIntervals + ' ' + dateBar.rollupIntervalPeriod}}" style="position: absolute; left: 0px; top: 700px;" values="values"></ma-point-values>
    <ma-serial-chart id="daeda3f3-3509-46c0-a63a-5877bbfb7946" style="position: absolute; width: 100%; height: 600px; left: 0px; top: 70px;" values="values.forEach(multiply); values" points="[power, price, {name: 'Cost', xid: 'cost', deviceName: 'Cost'}]" legend="true" export="true" series3-type="step"></ma-serial-chart>
</div>

Hey Jared, I've been trying to wrap my mind around the mechanisms involved in this, could you please confirm/explain a few things?

This expression

<ma-fn expression="$input.value_cost = $input['value_' + power.xid] * $input['value_' + price.xid]" fn="multiply" arg-names="['$input']"></ma-fn>

From messing around with this I've figured out that this expression is basically appending a "value_cost" property and value to whatever the array of data passed to it was correct? So in this case ma-point-values returned an array consisting of timestamp and two sets of values, one for power and one for price.

<ma-serial-chart id="daeda3f3-3509-46c0-a63a-5877bbfb7946" style="position: absolute; width: 100%; height: 600px; left: 0px; top: 70px;" values="values.forEach(multiply); values" points="[power, price, {name: 'Cost', xid: 'cost', deviceName: 'Cost'}]" legend="true" export="true" series3-type="step"></ma-serial-chart>

So ma-serial-chart gets an array of values from ma-point-values, passes that back to the function and the function transforms that array from something like this

[{timestamp: xxx, power: xxx, price: xxx}]

to

[{timestamp: xxx, power: xxx, price: xxx, value_cost: xxx}]

Is that about correct?

So then in ma-serial-chart what exactly is the relationship between "points" and "values"? Internally are they linked somehow? Cause "values" is the values array. Then for "points" do they have to be listed in the order that the values array has them in? You're passing [{timestamp: xxx, power: xxx, price: xxx, value_cost: xxx}] and you said that you added a dummy point so that it catches the new value. As I think about it it kind of makes sense but I just want to make sure I'm understanding this correctly. If the dummy point wasn't there would it just ignore the value_cost values? And what would be the minimum requirements for this dummy point? Name, xid and device name?

Jared Wiltshire

@psysak said in Calculating values on the fly:

Is that about correct?

Yes, you followed it through almost spot on. The property names for the values are actually more like -
[{timestamp: xxx, value_DP_power_xid: xxx, value_DP_price_xid: xxx, value_cost: xxx}]

@psysak said in Calculating values on the fly:

So then in ma-serial-chart what exactly is the relationship between "points" and "values"?

The serial chart directive will only plot a series for a point in the points array. It will take each point's XID and plot the property value_${XID} from the values array.

@psysak said in Calculating values on the fly:

Then for "points" do they have to be listed in the order that the values array has them in?

The order of the points is irrelevant, except maybe for the order they are displayed in the legend.

@psysak said in Calculating values on the fly:

you said that you added a dummy point so that it catches the new value

As above the serial chart directive needs a point to plot its series, I added a dummy point with XID cost so that it looks in the values array for the property value_cost and plots it.

@psysak said in Calculating values on the fly:

If the dummy point wasn't there would it just ignore the value_cost values?

Correct.

@psysak said in Calculating values on the fly:

And what would be the minimum requirements for this dummy point? Name, xid and device name?

Basically. That is all that is displayed in this configuration.

psysak

@jared-wiltshire said in Calculating values on the fly:

As above the serial chart directive needs a point to plot its series, I added a dummy point with XID cost so that it looks in the values array for the property value_cost and plots it.

Hey, thanks for the explanation, super helpful. One last bit of clarification if you don't mind. Is the

{xid: 'cost'}

an implied search term for xid which contains "cost"? It's not clear to me from this example and I want to understand that part. From that

{name: 'Cost', xid: 'cost', deviceName: 'Cost'}

It seems like you're assigning a name and deviceName to this dummy point but an xid already exists in the array so specifying xid: 'cost' is actually implying a search for it?

Jared Wiltshire

@psysak said in Calculating values on the fly:

Is the
{xid: 'cost'}

an implied search term for xid which contains "cost"?

No. This is object literal syntax - https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Grammar_and_types#Object_literals

By using points="[power, price, {name: 'Cost', xid: 'cost', deviceName: 'Cost'}]" I am creating an array containing the two points stored in the scope as power and price, and creating a third object in the array with its xid property set to the string cost.