ASCII File Reader How-To

torsten

I try to use the ASCII File Reader to parse a JSON structure (which is a single line in the file), but somehow I cannot recieve any data from it.
Can you please explain how to use the Point Identifier, Point Identifier Index, Value Index and Value RegEx correctly?

Data Source file:

{"result":{"overview":[{"meta":"GriPwr","name":"Power","unit":"W","value":"2994"},{"meta":"GriEgyTdy","name":"Day yield","unit":"kWh","value":"50.252"},{"meta":"GriEgyTot","name":"Total yield","unit":"kWh","value":"28441.116"},{"meta":"OpStt","name":"Condition","value":"Ok, Ok, Ok"},{"meta":"Msg","name":"Message","value":""}]},"format":"JSON","proc":"GetPlantOverview","version":"1.0","id":"1"}

If I want the Power, I tried the following with no result:
Point Identifier: {"result:"
Point Identifier Index: 1
Value Index: 1
Value RegEx: "meta":"GriPwr","name":"Power","unit":"W","value":"(\d+)"}

Thanks a lot!
Torsten

P.S. I just found the Wiki http://wiki.infiniteautomation.com by chance. If you want to use it in a serious way for documentation (which I strongly support), please link it from your homepage. Otherwise people cannot find it :wink:

phildunlap

Hi torsten,

This is because both the regex values need to match the whole line to be considered a match, and they can pull the values out of that match. So, we could make the point identifier regex {"(result)":.* which will match lines starting with {"result": and use the point identifier index on the point properties to match the point identifier regex with the point identifier. So your point wants an index of 1, which will pull 'result' out of the regex. Now that it's found that line, it will try to match the value regex against the entire line. If that regex matches, the value from the value index is made the value of the point, after it is internally cast to other data types if the point isn't alphanumeric.

For your regex, trying to get a repeated field 'value' the trick will be to use non-greedy * groups, which means using ?. To get result's value use the point identifier from above as the point identifier regex. Now, for the value index, set your value regex to {"result":.?(\d+).** with value regex 1.

You have to remember the regex must match the entire line.

Edit: this post is mistaken, see my next post as to how. I was still thinking about the serial data source, I guess.

torsten

First, about the above example: I believe the "{}" needs to be masked for Mango Regex as "{". What about the "[" ?

But somehow I cannot get the ASCII File Reader to work at all:
In the module section, it does not give a module description but only "???file.description(i18n_en)???". I tried removing the module, restarting, re-installing the module - no change.

I prepared a simple test text file po.txt (permissions -rw-rw-rw-):


sdfsfdf
power 1234 W
asdfasdfr 666

And I tried every RegEX and identifier variant I can image, but no data extracted for that point.
Debug-mode log files do not contain any information (except notification email about the new point created).

Is there a bug in the recent Mango/ASCII module, or am I a victim of too-close-to-the-screen blindness? :roll:

phildunlap

I apologize torsten, it seems I was still thinking of aspects of the serial data source in my last post. I just booted up the ASCII and took a closer look...

For that file, you can have a regex (power)(.), point identifier index 1, point identifier "power", value index 2 and get "1234 W " as the value of an alphanumeric file point. If you make it numeric, it won't work, because "1234 W " is not numeric. If you want to capture the 1234, move the " W" outside the second capture group. Also, the ASCII source is implementing a find, meaning it doesn't need to match the whole line. For instance, you could do regex: (12)(.) W point identifier: 12 point Identifier index: 1 value index: 2 and still capture values from the second line.

njbuch

Further along those lines, I am looking for an option to attach a timestamp to the point, like it is possible with the HTTP reader by putting in the "&__time=20131225073000" at the end of the request.

Is it possible to put an extra field somewhere in the ascii file that contains the timestamp, and will be attached to the data-point-value.

My problem is that I might have a serious delay on the upload of data.

phildunlap

Not in the present version of the module, but I agree that's an important feature. I will look into adding this and get back to you.

njbuch

@phildunlap said:

Not in the present version of the module, but I agree that's an important feature. I will look into adding this and get back to you.
Seriously quick service, too bad it was not possible. When you are at it, will you consider to support many datapoints in the same file?

phildunlap

You can have many datapoints in the same file, or do you mean recurrences of the same data point many times? What do you mean?

njbuch

@phildunlap said:

You can have many datapoints in the same file, or do you mean recurrences of the same data point many times? What do you mean?
Same point many times, with different data-stamps.