Changing Data Source but Retaining Data Point History?

[MattFox]

I'll break it down for you. The code above is filtering out a single point from a watchlist using ma-calc, the first point with Flow as the point name.
I think for you, my focus was around the button itself. I'll throw together some markup for you here. Since it's 23 points we should be able to knock them out with an ng-repeat....

<!-- 1. Generate a watchlist with your desired points that you want data for -->
<ma-watch-list-get ng-model="designer.watchList" parameters="designer.parameters" on-points-change="designer.points = $points" watch-list-xid="[YOUR_WATCHLIST_XID_HERE]"></ma-watch-list-get>

<!-- 2. ng repeat the buttons for each point in the list -->
<md-button  ng-repeat="point in designer.points" id="{{point.xid}}_button" style="" class="md-raised" ng-href="/rest/v2/point-values/time-period/{{point.xid}}?fields=XID&fields=TIMESTAMP&fields=VALUE&fields=ANNOTATION&from={{dateBar.from.toISOString()}}&to={{dateBar.to.toISOString()}}&format=csv2" download="{{point.name}}.csv">Download {{point.name}} values</md-button>
</div>
<!-- 3. use the date bar at the top of the page to set your time span -->
<!-- 4. hit the download button! -->

Fox

[MattFox]

This post is deleted!

[mihairosu]

Ok I found the watch list for a test watchlist ID: WL_883061

{
         "xid":"WL_883061",
         "data":null,
         "query":null,
         "readPermission":null,
         "name":"Troubleshooting",
         "dataPoints":[
            "boosterPumpsFlowRate",
            "DHWBsmntReturnRate"
         ],
         "folderIds":null,
         "params":null,
         "type":"static",
         "user":"admin",
         "editPermission":null
      },

The page appears to work fine:

Data begins approximately Feb 7, 2017, so I set a test download from

Feb 7, 2017 to March 4, 2020 with 1 day rollup interval:

I was getting lost connectivity and restored notifications.

Anyway, after the server is finished doing whatever it is doing, with a high processor load, I get an error for the CSV download:

Any idea what the issue may be?

I tried a smaller time period for the download and that had no problems.

I assume it's some limit in some sub-system that I may have to adjust values for?

edit: I think I see what the issue is. ma/log shows this error:

ERROR 2020-03-04T12:36:28,716 (com.serotonin.m2m2.rt.maint.BackgroundProcessingImpl$RejectableWorkItemRunnable.run:632) - Error in work item
java.lang.OutOfMemoryError: GC overhead limit exceeded

[MattFox]

Looks like you may need to break it up. Secondly, check your point limit count under the ui settings.

[mihairosu]

Okay thanks for everything, I can certainly handle it from here! If anything, I'll have to split it up, like you said, though I would have hoped to be able to get more out of this method. I hope it's more than I can get out of the watchlist page, haha.

[MattFox]

Another idea: add &limit=10000 to the url. That will override system limits rather than setting a higher settings limit.

Fox

[mihairosu]

Okay awesome, I'll play around with stuff, see what works well.

[mihairosu]

@mattfox said in Changing Data Source but Retaining Data Point History?:

&limit=10000

Since the last time we worked on this, this '&limit=10000" does not allow the page to load.

Anyway, the mango install just can't handle downloading an entire data set for even a single XID. The whole mango service comes to a slow crawl, as it pegs all cores to 100%. All screens can't reach the mango service, and in general is not ideal. And in the end, it still can't download it.

Plus how am I supposed to re-upload it. This download format is different than what you get using the watch list page downloads. Tn addition, the timestamp column appears messed up =( There are also not enough rows in excel for some of this data, I suspect, lol.

I will have to continue this conversation with IA.

As far as I can tell, there's no decent way, for the lay-man such as myself, to manipulate large amounts of data and other critical information. I suspect a lot of back-end work would have to be done to allow for such functionality.

My uneducated guess is that the whole structure is a bit rigid.

I'll keep putting this work off until a good solution appears, or I can get help directly from IA.

[MattFox]

I'm really sorry man. I'd hoped you'd been able to crack it.
Perhaps I should write a user module for you...

Fox

[mihairosu]

No worries, much appreciated. =)