Too much data for a datapoint

[till]

I get this exception

device_hub 'myDevice': Script error in point "myPoint": com.infiniteautomation.tsdb.IasTsdbException: com.infiniteautomation.tsdb.impl.BadRowException: Exception trying to serialize bytes: Data length can not exceed 65536

when trying to stuff too much into a virtual datapoint.

What other options do I have for storing a large amount of alphanumeric data. It's JSON, so I'm eying the JSON store but can't find any working examples of how to access that from a meta datapoint.

The code in https://forum.mango-os.com/topic/4495/calling-json-store-item-in-scripting-datasource/2?_=1630557115032 gives me another exception:

java.lang.RuntimeException: java.lang.ClassNotFoundException: com.serotonin.m2m2.Common.getBean

I need to publish these JSON objects to another mango instance eventually, which works fine with my datapoints.

[terrypacker]

@till there is a limit on the size of a value for Alphanumeric points in the NoSQL database, you won't be able to get around that. However we do have an auxiliary database for doing something like what you want. We call it the Mango data store. There aren't many examples around using it in a script but I could probably help you out.

There isn't a length restriction on the point data store AFAIK so this should be able to store much large JSON strings. The following is from a Scripting Data Source I just wrote that has a Virtual point in the external context with the key (variable name) of storage.

//get the point data dao (assuming Mango NoSQL module is installed)
var pointDataDao = com.serotonin.m2m2.Common.databaseProxy.noSQLProxy.createPointDataDao();
//get the common object mapper to convert Json 
var objectMapper = com.serotonin.m2m2.Common.getBean(
    com.fasterxml.jackson.databind.ObjectMapper.class, 'commonObjectMapper');

//Create empty object
var data = objectMapper.createObjectNode();
data.put("numberValue", 1.0);

//Get the time at which to store the data against:
var dataTime = com.serotonin.m2m2.Common.timer.currentTimeMillis();

//Create data to save (a PointDataTime object)
var dataToSave = new com.infiniteautomation.nosql.PointDataTime(
    dataTime, data);
    
//Save this to the data point at the given time
pointDataDao.savePointDataTime(storage.id, dataToSave);

//Get the data back out
print(pointDataDao.getPointDataTimeAt(storage.id, dataTime).getData());

The output of this script is:

{"numberValue":1.0}

As for publishing this data, it won't work using the Persistent TCP publisher/data source as that only works with point values, you would have the same problem with the JSON store. If you wanted to push this to another instance you could attempt to write a script using the HTTP Script utility to access the remote Mango's REST api using the endpoints at: /nosql/point-data

[till]

@terrypacker said in Too much data for a datapoint:

I shall give that a go, thanks.