I believe it's this item, as seen here.

Use GUID as the default session cookie name

I don't have any proof as I have not collected any data but I can say the following:

Many months ago, out disk was getting full with a Purge data setting of 4-5 weeks.

Then the disk was starting to get full again, and I reduced that further to a 10 days.

We have not added any additional data points during this time, and the disk is getting full again.

I just realized that the purge settings probably only affect the NoSQL database, but it's the H2 database that is causing us issues.

Can we reset this database cleanly and start again, which I imagine will hold up well for another number of years without trouble. I have instructions for how to delete and restore the database, but that also removes the configuration settings. Is is possible to remove only the data history while keeping the configuration?

What is the difference between core-database and configuration backups anyway?

Here's how I did it.

 <!-- Get point values -->
 <ma-point-values point-xid="hallsFanInletAirTemp" point="inletAir"></ma-point-values>

 <!-- Create a way to manupulate point values to 12 previous hours and update.  In this example, I am getting previous 12 hours at a 1 minute average.  You only have to do this once -->

<ma-now update-interval="1 minutes" output="to"></ma-now>
<ma-calc input="to | moment:'subtract':12:'hours'" output="from"></ma-calc>

<!-- Manipulate the actual values of the point.  This code can be multiplied for as many points as you want -->

<ma-point-values point="inletAir" values="inletAirValues" from="from" to="to" rollup="AVERAGE" rollup-interval="1 minutes">
</ma-point-values>

 <!-- Display Chart -->
	<ma-serial-chart 
                style="height: 500px; width: 100%" 
                series-1-values="inletAirValues" 
                series-1-point="inletAir" 
                series-1-color="green" 
                series-1-axis="left"
                series-1-type="line"
                legend="true" 
                baloon="false"
                options="{
                    valueAxes:[
                        {
                            axisColor:'black', 
                            color:'black', 
                            title: 'Temperature °F', 
                            titleColor: 'black', 
                            minimum:-40, 
                            maximum:140,
                            fontSize:26,
                            titleFontSize:26,
                        }, 
                    ]
                }"
            >
            </ma-serial-chart>

Try this to read a point, like the Voltage from your meter. You can get here from the Data Sources page. See my picture below:

Oh, your modbus settings have to match the meter, of course, like the bit rate, parity, etc.

One more thing, you have to turn off the serial data source to do the test, or else i'll throw an error because it's busy.

Oh sorry we are running core core 3.3.4 - MangoES.

Which backup should I restore, NoSQL or SQL. I am guessing SQL, but I know restoring NoSQL could also work, or am I wrong?

I have not tried to restore yet, so there aren't any other problems.

For some reason, during this particular time period, the data is not syncing.

Data Source:
MangoES
core 3.34

Data Destination:
Mango Enterprise
core 3.34

Data is sent through Mango Persistent TCP Sync.

Source Prior To Sync:

Destination Prior to Sync:

Syncing:

Log from Source:

INFO  2018-04-20T09:12:02,021 (com.serotonin.m2m2.persistent.pub.PersistentSenderRT.raiseSyncCompletionEvent:301) - Points: 159, sent 773 requests, synced 383631 records, target overcounts: 9, response errors: 0, elapsed time: 1m15s

Source Post Sync (I added more points):

Destination Post Sync:

select count(*) from pointValues; = 0

mrosu@22historian:~$ du -sh /opt/mango/databases/*
3.2G /opt/mango/databases/mah2.h2.db
4.0K /opt/mango/databases/mah2.lock.db
12K /opt/mango/databases/mah2.trace.db
47M /opt/mango/databases/mangoTSDB
1.2M /opt/mango/databases/mangoTSDBAux
12K /opt/mango/databases/reports

Synchronize history prior to is set at 1 minute.

I've taken a representative sample of points, and checked again today. It looks like all points are affected.

Here's the data again as of today, but with a lot more points.. Well if we cannot resolve this by May 3, it will be purged from the source. It's not the end of the world, but if you are interested in investigating it, it might help you find issues. I don't know if this is a problem on our end either.

Still at source:

Still missing:

Thanks Phil, we're back to a very reasonable database size.

@jared-wiltshire

Yep, that did it, thanks!

@jared-wiltshire

Yep, that did it, thanks!

@jared-wiltshire

Hi Jared, thanks for the assistance. I have the following issues / questions:

• Regarding the Java version, I checked the system and it appears to be OpenJDK 13. This is what came with MangoGT.

openjdk version "13.0.1" 2019-10-15
OpenJDK Runtime Environment AdoptOpenJDK (build 13.0.1+9)
OpenJDK Server VM AdoptOpenJDK (build 13.0.1+9, mixed mode)

As far as I can tell, the armv7l CPUs on the MangoGTs are 32-bit.

When I tried to look for a newer Java version on Oracle's sites, I see that there's not even any 32-bit arm versions, so I'm confused. Also, on here it says this version has been superseded. I cannot find even the OpenJDK 11 package for arm 32-bit.

I cannot figure out which newer version of java I should be installing that is compatible with 32-bit arm, except for Java 8. Can you please give me a bit more guidance on this issues?

• I've updated the packages via apt just prior to doing the modules upgrade, as it is part of my monthly maintenance routine. I checked again and ca-certificates package is up to date.

• I've completed this step.

I never had a store.url parameter existing, but I did add this to /opt/mango/overrides/properties/env.properties

store.url=https://store.mango-os.com

I restarted the mango service and the whole OS and am still having this issue. I guess it may be because I haven't update OpenJDK.

Thanks!

The modules cannot be updated on both our mangoGT devices due to this error:

"Error checking for upgrades — Invalid CertificateVerify signature"

core 3.7.7

This issue happens with the old and new interface.

No worries, much appreciated. =)

@mattfox said in Changing Data Source but Retaining Data Point History?:

&limit=10000

Since the last time we worked on this, this '&limit=10000" does not allow the page to load.

Anyway, the mango install just can't handle downloading an entire data set for even a single XID. The whole mango service comes to a slow crawl, as it pegs all cores to 100%. All screens can't reach the mango service, and in general is not ideal. And in the end, it still can't download it.

Plus how am I supposed to re-upload it. This download format is different than what you get using the watch list page downloads. Tn addition, the timestamp column appears messed up =( There are also not enough rows in excel for some of this data, I suspect, lol.

I will have to continue this conversation with IA.

As far as I can tell, there's no decent way, for the lay-man such as myself, to manipulate large amounts of data and other critical information. I suspect a lot of back-end work would have to be done to allow for such functionality.

My uneducated guess is that the whole structure is a bit rigid.

I'll keep putting this work off until a good solution appears, or I can get help directly from IA.

Okay awesome, I'll play around with stuff, see what works well.

Okay thanks for everything, I can certainly handle it from here! If anything, I'll have to split it up, like you said, though I would have hoped to be able to get more out of this method. I hope it's more than I can get out of the watchlist page, haha.

Ok I found the watch list for a test watchlist ID: WL_883061

{
         "xid":"WL_883061",
         "data":null,
         "query":null,
         "readPermission":null,
         "name":"Troubleshooting",
         "dataPoints":[
            "boosterPumpsFlowRate",
            "DHWBsmntReturnRate"
         ],
         "folderIds":null,
         "params":null,
         "type":"static",
         "user":"admin",
         "editPermission":null
      },

The page appears to work fine:

Data begins approximately Feb 7, 2017, so I set a test download from

Feb 7, 2017 to March 4, 2020 with 1 day rollup interval:

I was getting lost connectivity and restored notifications.

Anyway, after the server is finished doing whatever it is doing, with a high processor load, I get an error for the CSV download:

Any idea what the issue may be?

I tried a smaller time period for the download and that had no problems.

I assume it's some limit in some sub-system that I may have to adjust values for?

edit: I think I see what the issue is. ma/log shows this error:

ERROR 2020-03-04T12:36:28,716 (com.serotonin.m2m2.rt.maint.BackgroundProcessingImpl$RejectableWorkItemRunnable.run:632) - Error in work item
java.lang.OutOfMemoryError: GC overhead limit exceeded

@mattfox

What parts of this script should I be changing to get the points I am looking for?

<ma-watch-list-get ng-model="designer.watchList" parameters="designer.parameters" on-points-change="designer.points = $points" id="29305bf3-cc2a-4bac-9650-3cbf64d738a9" watch-list-xid="WL_Flow"></ma-watch-list-get>
<div class="ma-designer-root" id="1a2a6980-f5bb-4acc-a41a-f8f69d5c12d2" style="width: 1366px; height: 768px; position: relative;">

    <ma-calc input="designer.points | filter:{name:'Flow'}:true | maFirst" output="point"></ma-calc>
    
    <md-button id="2585b1fa-a630-4255-8670-41884b8754e2" style="position: absolute; left: 260px; top: 70px;" class="md-raised" ng-href="/rest/v2/point-values/time-period/{{point.xid}}?fields=TIMESTAMP&fields=VALUE&fields=ANNOTATION&from={{dateBar.from.toISOString()}}&to={{dateBar.to.toISOString()}}&format=csv2" download="{{point.name}}.csv">Download values</md-button>
</div>

I'll re-create the data source because it's easy in bacnet, but I am still trying to figure out how to download the data easily.

I don't fully understand what XID I need to match to something in my own syste,

There's no point XID in there, as far as I can tell.

I am getting these errors when trying to import data point configuration for every point:

Data point with XID 'boosterPumpsOutputPress' does not already exist and references a data source that does not exist. Ignored.

I'm probably not following your instructions 100% somehow, I'll try to understand better.