datapoint value to file

sprokkie

Hi there,

i want to export a value of a datapoint to a textfile or csv file.
Each time there is a datapoint update there must be an update in that file as wel.

the filename must be the current dat so i have one file each day.
in the file there must be a time stamp and the value of the datapoint.

can some one help me with this

regards,
Sprokkie

phildunlap

I am not sure of exactly why you would have these requirements, but here are some thoughts:

I don't believe there is a perfectly direct way. Maybe it is finally time to give the ASCII File data source the ability to set points either in Regex Replace mode or in Append mode, or something. This would probably be ideal for getting the values out, but you can't do any of the file renaming or what have you on that avenue. You could have one file that is the current and some cron tab move that file to its new dated location.
You could have a second program specifically to address the needs of this application either query your pointValues table and transpose it into the file. It could poll, or you could use a datasource like TCP/IP or SSH to trigger it.
You could set up an SSH datasource to do its ssh'ing to localhost, then have a script handling which points to set to your ssh point. You could also use this to move the file, probably, through a script on a once daily cron.

I believe your SSH point's command would look something like "echo 'VALUE' >> /path/to/valueFile"
Being set from a script like... 0/1 * * * * ? cron

var pvt, data = p.past(SECOND)
for( var x in data) {
pvt = data[x];
sshFileAppender.set( String(pvt.time) + "\t" + String(pvt.value) )
}

And then another script like.... 0 0 * * * ?

sshFileMover.set(true);

... where sshFileMove is something like "mv /path/to/valueFile /path/to/valueFile-$(date +%F)"

I didn't test any of these, but I hope they are helpful!

phildunlap

Another possibility would be coding a custom application to receive published HTTP data. The HTTP publisher's format would be pretty straightforward to receive and parse in a second program.

sprokkie

Hi Phil,

thank you for your answers.

this is the reasson i want to have the values in a file:

I log my solarpanels in Mango, i have a different solution for publishing the graphs on the internet.
2 arrays of panels have their own inverter that produces a csv file, that is also used for the internet page.
The third array uses a inverter that doesnt have these file, only a kwh meter with modbus.
Mango is reading teh modbus, so i got the data in mango.

Now i want to publish that array with solar panels also on the webpage.

The example you wrote down is realy abracadabra for me...
where do i paste that script ?

phildunlap

To attempt the SSH solution, you'll have to generate an SSH key and add it into the authorized_keys of your system, then point the ssh data source to the private key on the filesystem. Here's a tutorial on generating keys: https://www.digitalocean.com/community/tutorials/how-to-set-up-ssh-keys--2

There are step by step instructions in the help document as well (>> is the concatenate operator, updating documentation....).

Those scripts would be new Scripting Data Sources. You would need a different SSH point for each file appender you want. You could set all those appenders from the same script. You could move all the files with the same point by giving the second script a set of files to move, i.e. once a day....

moveFiles = ["/path/to/file/1". "/path/to/file/2"]
for( var x in moveFiles)
sshFileMover.set(moveFiles[x]);

... and have your sshFileMover point have a set command like: mv "VALUE" "VALUE-$(date +%F)"

sprokkie

qoute:"There are step by step instructions in the help document as well (>> is the concatenate operator, updating documentation....)."

you mean the mango documentation ?

when i read you solution with ssh, you wrote that i have to make a ssh datapoint.
i want to have the value and time stamp out of mango in to a file not from a file in to mango.

phildunlap

i do mean the Mango documentation (the contextual help "?" buttons). That is also available on our wiki: http://infiniteautomation.com/wiki/doku.php?id=datasources:ssh&s[]=ssh

Yeah I understand that. It'd be using the SSH data source to append data points to a file. It's a little overkill when it's the same machine, but it'll work and you'd have a solution that could also keep the file published remotely.

Also I had a conversation with someone, and we appreciated the value of writing a File Publisher in the future.

sprokkie

Hi,
just reading the mango help.
the example is for a linux based system, my mango is running on windows 7.
is there an example for win 7 available ?

phildunlap

Most unfortunate! I am not very familiar with command prompt commands offhand, and all of my SSH example is geared toward Linux. It is possible to install an SSH server on Windows and run batch commands this way, but I haven't done it. Are you running on SQL? Option 2, querying the database directly is likely easier. Or you could create your own table and use the SQL data source to populate it. You could also have a program that subscribes to our API to get points and write them into the format you need.

sprokkie

i use the h2 database.
conecting to the api might be a solution, dont know how to write a program for that.
ill gues it will be a lot of reading on the interweb...

it is to bad the csv file from the report that can be emailed to me is not a file stored in the mango installation.

phildunlap

You can check out the endpoints available in your version by setting swagger.enabled=true in your env.properties, restarting and navigating to http://mangourl:port/swagger

You will need to make a login request to begin getting data, and if you update to later versions of the software you will need to record the XSRF token returned to you at login for subsequent requests.

sprokkie

If i navigate to /swagger i got a forbidden page.
i cannot see the endpoints :(

JoelHaggar

You need to enable swagger in the bottom of the env.properties file and then restart Mango.

You should upgrade to Mango 2.7 as 2.5 is not going to be supported any more. You mentioned something was slow on your test upgrade but didn't provide any details. Mango 2.7 is not somehow slower than 2.5 and has several performance improvements. You must have an issue that can be solved. It might be something simple like you need to clear your browser's cache.

sprokkie

Hi Joel,

As i rembered i gave you some details.
showing the graphical view was slower after the upgrade, that was really anoying

i have to test the new version first before i upgrade.
is it possble to have 2 mango runing on one computer ?

i enabled the swagger in the env file.
the env file is a mess, maybe that is the problem ?
i changed false in to true

phildunlap

Yes, you just need to modify the env.properties file to launch the second Mango on different web.port than the first.

sprokkie

i copied the mango folder in to the mango 2.7 folder
after the 2.5 is started i will perform an upgrade.
wil post the results here

sprokkie

Joel,

i am trying to upgrade for a while now.
is there a nice tutorial for upgrade ?

i copied my mango folder to a new folder started the mango and pressend update.
result all modules where upgraded exept for the core.
upgrading the core failed.

at the moment i am doing the same again, and try to upgrade the core first with the upgrade function

sprokkie

finaly i got it up and running, now i am missing values from one data point, ill gusess something went wrong while updating :(
the performance of the graphical view is better than the upgrade to 2.6.

now find out how to fix the missing value's and than renaming the folder to mango and it should work