Weather Forecast

---

Hi guys,

I've got the M2M core software running on my machine. So far everything looks great, still doing a lot of testing. I have been searching the forums and so far haven't found anything that has helped me yet so here goes. What would be the simplest way to pull data from a website like weather underground and create points for the forecast weather temperatures high and low? I have been trying to use the HTTP retriever and gave the data source the api key that I have for the site. When I press the green button to test to get a value, after setting up my first point, it pulls up 81.0 for Day 1 high temperature but after I finish creating the point it says that there is currently no data. The regex I'm using is extremely simple as I haven't had any experience with regex thus far and is as follows, <fahrenheit>(.*?)</fahrenheit> This pulls up the first point that uses this particular nomenclature but if I wanted to pull more days I would need something else to pull the data correctly for each day. I know this is maybe a lot to ask but I'm really interested in purchasing this software and just need to test to make sure that it can do the things that are 100% vital in my installs.

Here is an JSON export of a data source that uses the yahoo xml api.

Copy and past this into the Import/Export page. Let me know if this helps.

{
   "dataSources":[
      {
         "xid":"DS_734545",
         "name":"Weather",
         "enabled":true,
         "type":"HTTP_RETRIEVER",
         "alarmLevels":{
            "POLL_ABORTED":"URGENT",
            "PARSE_EXCEPTION":"URGENT",
            "DATA_RETRIEVAL_FAILURE":"URGENT",
            "SET_POINT_FAILURE":"URGENT"
         },
         "purgeType":"YEARS",
         "updatePeriodType":"SECONDS",
         "quantize":false,
         "retries":2,
         "setPointUrl":"",
         "timeoutSeconds":30,
         "updatePeriods":5,
         "url":"http:\/\/weather.yahooapis.com\/forecastrss?w=2400077&u=f",
         "purgeOverride":true,
         "purgePeriod":1
      }
   ],
   "dataPoints":[
      {
         "xid":"DP_343373",
         "name":"Image",
         "enabled":true,
         "loggingType":"ON_CHANGE",
         "intervalLoggingPeriodType":"MINUTES",
         "intervalLoggingType":"INSTANT",
         "purgeType":"YEARS",
         "pointLocator":{
            "dataType":"ALPHANUMERIC",
            "ignoreIfMissing":false,
            "setPointName":"",
            "settable":false,
            "timeFormat":"",
            "timeRegex":"",
            "valueFormat":"",
            "valueRegex":"img src=\"(.*?)\""
         },
         "eventDetectors":[
         ],
         "plotType":"STEP",
         "unit":"",
         "chartColour":null,
         "chartRenderer":null,
         "dataSourceXid":"DS_734545",
         "defaultCacheSize":1,
         "deviceName":"Weather",
         "discardExtremeValues":false,
         "discardHighLimit":0.0,
         "discardLowLimit":0.0,
         "intervalLoggingPeriod":15,
         "intervalLoggingSampleWindowSize":0,
         "overrideIntervalLoggingSamples":false,
         "purgeOverride":true,
         "purgePeriod":1,
         "textRenderer":{
            "type":"PLAIN",
            "useUnitAsSuffix":false,
            "unit":"",
            "renderedUnit":"",
            "suffix":""
         },
         "tolerance":0.0
      },
      {
         "xid":"DP_057256",
         "name":"Pressure",
         "enabled":true,
         "loggingType":"ON_CHANGE",
         "intervalLoggingPeriodType":"MINUTES",
         "intervalLoggingType":"INSTANT",
         "purgeType":"YEARS",
         "pointLocator":{
            "dataType":"NUMERIC",
            "ignoreIfMissing":false,
            "setPointName":"",
            "settable":false,
            "timeFormat":"",
            "timeRegex":"",
            "valueFormat":"",
            "valueRegex":"pressure=\"(\\d*\\.?\\d*)\""
         },
         "eventDetectors":[
         ],
         "plotType":"STEP",
         "unit":"",
         "chartColour":"",
         "chartRenderer":null,
         "dataSourceXid":"DS_734545",
         "defaultCacheSize":1,
         "deviceName":"Weather",
         "discardExtremeValues":false,
         "discardHighLimit":0.0,
         "discardLowLimit":0.0,
         "intervalLoggingPeriod":15,
         "intervalLoggingSampleWindowSize":0,
         "overrideIntervalLoggingSamples":false,
         "purgeOverride":true,
         "purgePeriod":1,
         "textRenderer":{
            "type":"ANALOG",
            "useUnitAsSuffix":false,
            "unit":"",
            "renderedUnit":"",
            "suffix":" mmHg",
            "format":"0"
         },
         "tolerance":0.0
      },
      {
         "xid":"DP_189378",
         "name":"Wind",
         "enabled":true,
         "loggingType":"NONE",
         "intervalLoggingPeriodType":"MINUTES",
         "intervalLoggingType":"INSTANT",
         "purgeType":"YEARS",
         "pointLocator":{
            "dataType":"NUMERIC",
            "ignoreIfMissing":false,
            "setPointName":"",
            "settable":false,
            "timeFormat":"",
            "timeRegex":"",
            "valueFormat":"",
            "valueRegex":"yweather:wind.*?speed=\"(\\d*\\.?\\d*)"
         },
         "eventDetectors":[
         ],
         "plotType":"STEP",
         "unit":"",
         "chartColour":"",
         "chartRenderer":null,
         "dataSourceXid":"DS_734545",
         "defaultCacheSize":1,
         "deviceName":"Weather",
         "discardExtremeValues":false,
         "discardHighLimit":0.0,
         "discardLowLimit":0.0,
         "intervalLoggingPeriod":15,
         "intervalLoggingSampleWindowSize":0,
         "overrideIntervalLoggingSamples":false,
         "purgeOverride":true,
         "purgePeriod":1,
         "textRenderer":{
            "type":"ANALOG",
            "useUnitAsSuffix":false,
            "unit":"",
            "renderedUnit":"",
            "suffix":" MPH",
            "format":"0"
         },
         "tolerance":0.0
      },
      {
         "xid":"DP_721519",
         "name":"Condition",
         "enabled":true,
         "loggingType":"ON_CHANGE",
         "intervalLoggingPeriodType":"MINUTES",
         "intervalLoggingType":"INSTANT",
         "purgeType":"YEARS",
         "pointLocator":{
            "dataType":"ALPHANUMERIC",
            "ignoreIfMissing":false,
            "setPointName":"",
            "settable":false,
            "timeFormat":"",
            "timeRegex":"",
            "valueFormat":"",
            "valueRegex":"yweather:condition.*?text=\"(.*?)\""
         },
         "eventDetectors":[
         ],
         "plotType":"STEP",
         "unit":"",
         "chartColour":null,
         "chartRenderer":null,
         "dataSourceXid":"DS_734545",
         "defaultCacheSize":1,
         "deviceName":"Weather",
         "discardExtremeValues":false,
         "discardHighLimit":0.0,
         "discardLowLimit":0.0,
         "intervalLoggingPeriod":15,
         "intervalLoggingSampleWindowSize":0,
         "overrideIntervalLoggingSamples":false,
         "purgeOverride":true,
         "purgePeriod":1,
         "textRenderer":{
            "type":"PLAIN",
            "useUnitAsSuffix":false,
            "unit":"",
            "renderedUnit":"",
            "suffix":""
         },
         "tolerance":0.0
      },
      {
         "xid":"DP_781473",
         "name":"Humidity",
         "enabled":true,
         "loggingType":"NONE",
         "intervalLoggingPeriodType":"MINUTES",
         "intervalLoggingType":"INSTANT",
         "purgeType":"YEARS",
         "pointLocator":{
            "dataType":"NUMERIC",
            "ignoreIfMissing":false,
            "setPointName":"",
            "settable":false,
            "timeFormat":"",
            "timeRegex":"",
            "valueFormat":"",
            "valueRegex":"yweather:atmosphere humidity=\"(\\d*\\.?\\d*)\""
         },
         "eventDetectors":[
         ],
         "plotType":"STEP",
         "unit":"",
         "chartColour":"",
         "chartRenderer":null,
         "dataSourceXid":"DS_734545",
         "defaultCacheSize":1,
         "deviceName":"Weather",
         "discardExtremeValues":false,
         "discardHighLimit":0.0,
         "discardLowLimit":0.0,
         "intervalLoggingPeriod":15,
         "intervalLoggingSampleWindowSize":0,
         "overrideIntervalLoggingSamples":false,
         "purgeOverride":true,
         "purgePeriod":1,
         "textRenderer":{
            "type":"ANALOG",
            "useUnitAsSuffix":false,
            "unit":"",
            "renderedUnit":"",
            "suffix":"%",
            "format":"0"
         },
         "tolerance":0.0
      },
      {
         "xid":"DP_324257",
         "name":"Visibility",
         "enabled":true,
         "loggingType":"ON_CHANGE",
         "intervalLoggingPeriodType":"MINUTES",
         "intervalLoggingType":"INSTANT",
         "purgeType":"YEARS",
         "pointLocator":{
            "dataType":"NUMERIC",
            "ignoreIfMissing":false,
            "setPointName":"",
            "settable":false,
            "timeFormat":"",
            "timeRegex":"",
            "valueFormat":"",
            "valueRegex":"yweather:atmosphere.*?visibility=\"(\\d*\\.?\\d*)"
         },
         "eventDetectors":[
         ],
         "plotType":"STEP",
         "unit":"",
         "chartColour":"",
         "chartRenderer":null,
         "dataSourceXid":"DS_734545",
         "defaultCacheSize":1,
         "deviceName":"Weather",
         "discardExtremeValues":false,
         "discardHighLimit":0.0,
         "discardLowLimit":0.0,
         "intervalLoggingPeriod":15,
         "intervalLoggingSampleWindowSize":0,
         "overrideIntervalLoggingSamples":false,
         "purgeOverride":true,
         "purgePeriod":1,
         "textRenderer":{
            "type":"ANALOG",
            "useUnitAsSuffix":true,
            "unit":"",
            "renderedUnit":"",
            "format":"0.0"
         },
         "tolerance":0.0
      },
      {
         "xid":"DP_442553",
         "name":"Temperature",
         "enabled":true,
         "loggingType":"INTERVAL",
         "intervalLoggingPeriodType":"MINUTES",
         "intervalLoggingType":"INSTANT",
         "purgeType":"YEARS",
         "pointLocator":{
            "dataType":"NUMERIC",
            "ignoreIfMissing":false,
            "setPointName":"",
            "settable":false,
            "timeFormat":"",
            "timeRegex":"",
            "valueFormat":"",
            "valueRegex":"yweather:condition.*?temp=\"(\\d*\\.?\\d*)"
         },
         "eventDetectors":[
         ],
         "plotType":"STEP",
         "unit":"",
         "chartColour":"",
         "chartRenderer":{
            "type":"IMAGE",
            "timePeriodType":"HOURS",
            "numberOfPeriods":24
         },
         "dataSourceXid":"DS_734545",
         "defaultCacheSize":1,
         "deviceName":"Weather",
         "discardExtremeValues":false,
         "discardHighLimit":0.0,
         "discardLowLimit":0.0,
         "intervalLoggingPeriod":15,
         "intervalLoggingSampleWindowSize":0,
         "overrideIntervalLoggingSamples":false,
         "purgeOverride":true,
         "purgePeriod":1,
         "textRenderer":{
            "type":"ANALOG",
            "useUnitAsSuffix":false,
            "unit":"",
            "renderedUnit":"",
            "suffix":"",
            "format":"0"
         },
         "tolerance":0.0
      }
   ]
}

I received the following errors when I tried to import the code.

Data source 'DS_734545': Invalid event code 'POLL_ABORTED'. Valid codes are [DATA_RETRIEVAL_FAILURE, PARSE_EXCEPTION, SET_POINT_FAILURE]
Data point with XID 'DP_343373' does not already exist and references a data source that does not exist. Ignored.
Data point with XID 'DP_057256' does not already exist and references a data source that does not exist. Ignored.
Data point with XID 'DP_189378' does not already exist and references a data source that does not exist. Ignored.
Data point with XID 'DP_721519' does not already exist and references a data source that does not exist. Ignored.
Data point with XID 'DP_781473' does not already exist and references a data source that does not exist. Ignored.
Data point with XID 'DP_324257' does not already exist and references a data source that does not exist. Ignored.
Data point with XID 'DP_442553' does not already exist and references a data source that does not exist. Ignored.
Import complete

So just looking at the code I added the data source that was at the beginning of your code, ran the script again and it added all of the data points. The points aren't actually getting any data though. Ideas?

I've gotten all of the points that you included working. Now my only question is how to poll forecast data. Because in the XML the forecast reads the same with the day changing every day obviously. So what would I write to be able to grab the day name, day high, and day low for the 5 day forecast?

This export was from Mango 2.5.0 Beta which has an added alarm. You should be able to just remove the: "POLL_ABORTED":"URGENT", and then the import would work.

I'm not the Regex guys so someone else will have to answer that or you can use the other example to figure out the forecast.