Time RegEx

[Ralf]

Hi,

I have a http-retriever-data-source that gives me a 1 hour weather forecast:

The Value-RegEx -> hour value="01:00"><temp value="(.*?)" works and give me the temp value 18 - all fine.
If I set the logging time to "When point value changes", Mango stored the value with the actual timestamp, but I need a timestamp with the forecast date and hour, in my example above -> day value="20190722" and -> hour value="01:00" --> 2019-07-22T01:00:00.000+2:00

If I try in the TimeRegEx:

only the date is correct but the time isn't 1:00.
Is there a way to put date and time together?

[phildunlap]

Hi Ralf,

This would not be easy to achieve in an HTTP Retriever. Have you considered using the HttpBuilder script utility? You would be able to put the whole data series into the same point, in that case, as you would have the text of the response available to run as much parsing on as you need.

[Ralf]

Hi phildunlap,

that's a good idea but I find no example in the forum/docu, Do you have one?

[phildunlap]

Certainly!

There are a few examples, this one is probably the most in-depth:
https://forum.infiniteautomation.com/topic/2999/intesis-integration
This is probably the most relevant:
https://forum.infiniteautomation.com/topic/3251/httpbuilder-not-defined
But there are others, and soon to be one more!
https://www.google.com/search?q=site%3A%2F%2Fforum.infiniteautomation.com+httpbuilder

For this, it might look like this...

//Untested, source URL not provided
function handleResponse(content) {
    //Why does [0-1] have a ?; does it present Jan 1 as 201911 ?
    var dayInformation = /day value="([0-9]{4})([0-1]?[0-9])([0-3]?[0-9])"/.exec(content);
    if(dayInformation === null)
        throw "Could not find date information in content";
        
    for(var k = 0; k < 24; k+=1) {
        var valueRegex = new RegExp('hour value="' + (k < 10 ? "0"+k : k) + ':00"><temp value="(.*?)"');
        var data = valueRegex.exec(content)
        if(data !== null) {
            //this timestamp should be in the server's timezone
            var timestamp = new Date(parseInt(dayInformation[1]), //year
                                     parseInt(dayInformation[2])-1, //month index
                                     parseInt(dayInformation[3]), k); //day and hours
            
            //This would be a scripting data point on the scripting data source:
            tempForecastPoint.set(parseFloat(data[1]), timestamp.getTime());
        }
    }
}

//Okay, let's pull in the data...
var headers = {} //Maybe you need an Accept header?
var parameters = {} //Perhaps an API token or date?
HttpBuilder.get("sptth://thisforecastwebsite.extension", headers, parameters).err(
    function(status, headers, content) {
        throw "Script failed with HTTP status: " + status;
}).resp(function(status, headers, content) {
    handleResponse(content);
}).execute();

Edit: And the documentation would be in the main Mango JavaScript help: https://help.infiniteautomation.com/about-mango-java-script

[Ralf]

Hi phildunlap,

thank you for the quick help and code!

If I run at 19:00 the script it validate with the list:

All fine, but the records end at 19:00 from 00:00 - 19:00. Data source saves historic is ON, the point logging is OFF. Are there specific settings for the logging till 24:00?

[phildunlap]

I would initially suspect the discard future dated values to be the culprit:

[Ralf]

I set 48h, no change. It only saves up to the current time: 9:00 here.

[phildunlap]

This is probably only a display issue. I would guess above that on the page it is showing the value for hour 23 as the current value, and that if you chart it for the day you will see the values in the chart. Thanks for bringing this to our attention!