Save
Saving
  • G Gord Bond

    Ahh! That make sense. Thanks @MattFox !

    posted in User help read more
  • G Gord Bond

    Hey again,

    I'm working on setting up a POP3 data source and I've been running into issues getting data out of the emails. To be honest, I'm not sure if the issue is with data source not being configured properly causing it to fail to connect with the account (I've tried both gmail and outlook accounts). Or if it is an issue with the way I've set up the datapoint causing it to not properly read the data in the emails.

    I wanted to start simple with something I knew worked so I was using the configuration and data in the thread linked below. But no luck.
    https://forum.mango-os.com/topic/3950/extracting-data-from-pop3-email?_=1608212175632

    The feedback from the system is:

    Datapoint: No values

    Runtime status: Shows the polls are being aborted with no return (which probably explains the lack of values for the data point)

    Current Alarms: displays a message saying: 'emailtest2': EOF on socket

    My data source and data point is set up like this:

    {
      "dataSources":[
        {
          "xid":"DS_b6a7f349-69da-40ca-a948-7722447aba3b",
          "name":"emailtest2",
          "enabled":true,
          "type":"POP3",
          "alarmLevels":{
            "INBOX_EXCEPTION":"URGENT",
            "MESSAGE_READ_EXCEPTION":"URGENT",
            "POLL_ABORTED":"URGENT",
            "PARSE_EXCEPTION":"URGENT"
          },
          "purgeType":"YEARS",
          "updatePeriods":1,
          "updatePeriodType":"MINUTES",
          "password":"*******",//Hidden
          "pop3Server":"outlook.office365.com",
          "port":995,
          "username":"*******@outlook.com",//hidden
          "quantize":false,
          "useCron":false,
          "editPermission":"",
          "purgeOverride":false,
          "purgePeriod":1
        }
      ],
      "dataPoints":[
        {
          "xid":"DP_ea9dd5ff-10c0-4597-a895-205310281232",
          "name":"209",
          "enabled":true,
          "loggingType":"INTERVAL",
          "intervalLoggingPeriodType":"MINUTES",
          "intervalLoggingType":"INSTANT",
          "purgeType":"YEARS",
          "pointLocator":{
            "dataType":"NUMERIC",
            "findInSubject":false,
            "ignoreIfMissing":true,
            "timeFormat":"",
            "timeRegex":"",
            "useReceivedTime":true,
            "valueFormat":"#",
            "valueRegex":"209start (\\d+) end"
          },
          "eventDetectors":[
          ],
          "plotType":"SPLINE",
          "rollup":"NONE",
          "unit":"",
          "simplifyType":"NONE",
          "chartColour":"",
          "chartRenderer":{
            "type":"IMAGE",
            "timePeriodType":"DAYS",
            "numberOfPeriods":1
          },
          "dataSourceXid":"DS_b6a7f349-69da-40ca-a948-7722447aba3b",
          "defaultCacheSize":1,
          "deviceName":"emailtest2",
          "discardExtremeValues":false,
          "discardHighLimit":1.7976931348623157E308,
          "discardLowLimit":-1.7976931348623157E308,
          "intervalLoggingPeriod":1,
          "intervalLoggingSampleWindowSize":0,
          "overrideIntervalLoggingSamples":false,
          "preventSetExtremeValues":false,
          "purgeOverride":false,
          "purgePeriod":1,
          "readPermission":"",
          "setExtremeHighLimit":1.7976931348623157E308,
          "setExtremeLowLimit":-1.7976931348623157E308,
          "setPermission":"",
          "tags":{
          },
          "textRenderer":{
            "type":"ANALOG",
            "useUnitAsSuffix":false,
            "suffix":"",
            "format":"0.00"
          },
          "tolerance":0.0
        }
      ]
    }
    

    The email in my inbox is set up as follows:
    Screen Shot 2020-12-17 at 11.57.28 AM.png

    My goal here is to return the value in the email and set them as values for the datapoint. Any help here would be much appreciated.

    Thanks

    Gord

    posted in User help read more
  • G Gord Bond

    Fair enough, thanks Joel. Are there any weather API services you recommend? I've looked into Open Weather and heard good things. I

    posted in User help read more
  • G Gord Bond

    Hi Everyone,

    I was wondering if I could get some help with regards to an issue I'm having retrieving data from the Environment Canada data source. I'm working with a weather station in Muskrat Dam (station ID 50722). I can collect some data from the station but it stops whenever it hits a gap in the data. For instance, if I start collecting data on Jan. 1 2013 it stops on July 24th where there is missing data.

    Unfortunately, with this station there are a bunch of these data gaps. What I was wondering, is there a way to tell the parser to skip over these data gaps so that I can get the full data set? Ideally, I'd like to retrieve data starting on Jan 1, 2013 until present. @phildunlap I know you've done a bunch of work on this data source, I was hoping you might have some insight here.

    Environment Canada Muskrat Dam station (Data missing July 24th):
    https://climate.weather.gc.ca/climate_data/hourly_data_e.html?hlyRange=2012-11-13|2020-12-03&dlyRange=2012-11-15|2020-12-03&mlyRange=|&StationID=50722&Prov=ON&urlExtension=_e.html&searchType=stnName&optLimit=yearRange&StartYear=1840&EndYear=2020&selRowPerPage=25&Line=0&searchMethod=contains&txtStationName=muskrat+dam&timeframe=1&Year=2013&Month=7&Day=24

    Thanks,

    Gord

    posted in User help read more
  • G Gord Bond

    Ah ok, thank you guys. This helps quite a bit. I am currently using the free license at the moment to investigate whether or not Mango is the right for our project. So far it seems really great and I'm glad there is a solution for my duplicates problem. I appreciate all the help here in the forums.

    posted in User help read more
  • G Gord Bond

    Hi Craig,

    I think I need to clarify my issue a little further. I am hoping to be able to upload data as either a csv file with many values/timestamps or a single value/value and have it update in the database only if the value at that timestamp has changed. I've tried both the "When Point Timestamp Changes" and "When Point Value Changes" logging properties with no luck.

    With regards to the methods I am adding the value:
    I'm currently using a couple of methods to get data in via a "Data File" datasource but all return the same issue of duplicate data for a specific timestamp. The first method is by placing a csv in the filestore and having the data source configured to look for any csv files in the filestore directory. What happens here is the first csv file is uploaded, then if I place the same file back in the folder all the values are logged to the database again despite the time stamps being identical. What I am expecting is none of the values to be logged because the timestamp hasn't changed. Ideally, if I were to adjust one of those values and reuploaded the file with the single adjustment, only the adjustment would be logged. The second method is setting a single value via the /point-values end point in the API. When I hardcode an identical time stamp to an existing one it is logged to the database and the previous value remains. I have a solution in mind for setting a single value which would involve an sql update via the sql-console update/query end points but that feels like a bit of a hack. I'm sure I'm misunderstanding how the logging is supposed to work so any help pointing me in the right direction would be greatly appreciated.

    Thanks,

    Gord

    posted in User help read more
  • G Gord Bond

    To add further context. I am using the "When Point Timestamp Changes" logging property. When I add a new point with the same timestamp it still logs to the database (checked this using the sql console).

    posted in User help read more
  • G Gord Bond

    Good Morning!

    I was hoping to get a little help with something I am currently struggling with. Is it possible to limit data points to one value per timestamp? For example, say I had a data point, "Electricity Consumed", and it had a point value of 1500 for the time stamp, 1595822400000. If I add a new point value for the exact same timestamp, I want the new value to be logged and the previous value to be deleted. Essentially updating the value at a specific timestamp. At the moment, whenever I add a new point value for the same timestamp I add the new value but the previous value also remains in the system.

    Thanks,
    Gord

    posted in User help read more
  • G Gord Bond

    Amazing. Thank you very much.

    posted in How-To read more