Move Data Points to new Data Source

[tomatopi]

I have around 100 data points currently spread across three HTTP Retriever data sources that are parsed using RegEx. There's been an update on the client end and I would now be able to pull all the data points from a single JSON request.

What is the best way to migrate all these data points while still retaining their point value history?

[terrypacker]

@tomatopi there is going to be 2 major parts to your solution:

A. Setting up the HTTP Retriever data source and points into a single request that gets the data into the desired points.

B. Retaining the history on the existing points so that new history is appended to it.

I'll focus on B here:

1. You could try modifying the points via the JSON import/export to move them onto a single data source.

• The general idea here is that if it is possible to reconfigure the points you would need to ensure that the seriesId in the database doesn't change. So use the XIDs to move the points and not create new ones.

2. You could create an entirely new set of points and migrate existing data into them.

• There are NoSQL tools in Mango for migrating data which you would need to explore.

3. You could create an entirely new set of points and then before you start them for the first time changing the series ID for them to point to the existing data.

• This would be done after you setup the new points by a Filestore Javascript.

All of this information hinges on the fact that you are on at least Mango 4. It can probably be done in Mango 3 but we didn't have the seriesId concept so the solution would be different. In Mango 3 the time series data is directly linked to the data point's id column in the database. In Mango 4 we added a column called seriesId to the dataPoints table to allows us to move the time series data between points if we needed to.

The caveat here is that you can only move data between points that have the same data type. For example Numeric time series can only be assigned to other Numeric points, not a Binary point.

[tomatopi]

That really helps.

Setting up the new data source isn't a problem.

The B part:

1. JSON import/export - that looks like the easiest way to go. In each data point, I'd just have to change dataSourceXid and the settings under pointLocator to reflect JSON fields instead of RegEx fields.

2. Sounds not too hard, but I find documentation is a bit lacking on the NoSQL tools.
   It would be useful is mango provided an easy point history import/export feature. I know in the watch-list I can export CSV, but it would be nice to have an "export all" for a datapoint somewhere that could be easily imported back in. Like an additional tab to import/export data points.

3. Sounds more complicated than the route I'd like to go down. Keeping the XID the same wouldn't break things elsewhere.

I don't know how often it happens, but maybe a future tool to move a data point to a new data source would be useful?