@MattFox So if you are using a Mango for local development are you using any version control like github or bitbucket and pulling down from there for the production or are you personally using the cloud? I don't have any experience using Mango's cloud, so that is a bit of unfamiliar territory. If you have a moment I would really appreciate if you could further elaborate on your workflow.

Thanks @cwangv @MattFox I appreciate the feedback.

A quick follow up question:

Would a MangoGT make most sense as a dev environment?

Hey All,

I was curious to get the community's feedback on how you approach the separation of the development and production environments for your projects. Our project is still in a development stage but would like to set up a production environment relatively soon. Are there any best practices or suggestions how to approach this in the documentation I've missed?

Our current plan is to purchase a second license and mirror the two environments. Is this the best approach? The big question this raises is, if we want more than one person developing on the project do we have to purchase another license?

How do you guys approach transferring data between these environments?

Thanks,

Gord

Ahh! That make sense. Thanks @MattFox !

Hey again,

I'm working on setting up a POP3 data source and I've been running into issues getting data out of the emails. To be honest, I'm not sure if the issue is with data source not being configured properly causing it to fail to connect with the account (I've tried both gmail and outlook accounts). Or if it is an issue with the way I've set up the datapoint causing it to not properly read the data in the emails.

I wanted to start simple with something I knew worked so I was using the configuration and data in the thread linked below. But no luck.
https://forum.mango-os.com/topic/3950/extracting-data-from-pop3-email?_=1608212175632

The feedback from the system is:

Datapoint: No values

Runtime status: Shows the polls are being aborted with no return (which probably explains the lack of values for the data point)

Current Alarms: displays a message saying: 'emailtest2': EOF on socket

My data source and data point is set up like this:

{
  "dataSources":[
    {
      "xid":"DS_b6a7f349-69da-40ca-a948-7722447aba3b",
      "name":"emailtest2",
      "enabled":true,
      "type":"POP3",
      "alarmLevels":{
        "INBOX_EXCEPTION":"URGENT",
        "MESSAGE_READ_EXCEPTION":"URGENT",
        "POLL_ABORTED":"URGENT",
        "PARSE_EXCEPTION":"URGENT"
      },
      "purgeType":"YEARS",
      "updatePeriods":1,
      "updatePeriodType":"MINUTES",
      "password":"*******",//Hidden
      "pop3Server":"outlook.office365.com",
      "port":995,
      "username":"*******@outlook.com",//hidden
      "quantize":false,
      "useCron":false,
      "editPermission":"",
      "purgeOverride":false,
      "purgePeriod":1
    }
  ],
  "dataPoints":[
    {
      "xid":"DP_ea9dd5ff-10c0-4597-a895-205310281232",
      "name":"209",
      "enabled":true,
      "loggingType":"INTERVAL",
      "intervalLoggingPeriodType":"MINUTES",
      "intervalLoggingType":"INSTANT",
      "purgeType":"YEARS",
      "pointLocator":{
        "dataType":"NUMERIC",
        "findInSubject":false,
        "ignoreIfMissing":true,
        "timeFormat":"",
        "timeRegex":"",
        "useReceivedTime":true,
        "valueFormat":"#",
        "valueRegex":"209start (\\d+) end"
      },
      "eventDetectors":[
      ],
      "plotType":"SPLINE",
      "rollup":"NONE",
      "unit":"",
      "simplifyType":"NONE",
      "chartColour":"",
      "chartRenderer":{
        "type":"IMAGE",
        "timePeriodType":"DAYS",
        "numberOfPeriods":1
      },
      "dataSourceXid":"DS_b6a7f349-69da-40ca-a948-7722447aba3b",
      "defaultCacheSize":1,
      "deviceName":"emailtest2",
      "discardExtremeValues":false,
      "discardHighLimit":1.7976931348623157E308,
      "discardLowLimit":-1.7976931348623157E308,
      "intervalLoggingPeriod":1,
      "intervalLoggingSampleWindowSize":0,
      "overrideIntervalLoggingSamples":false,
      "preventSetExtremeValues":false,
      "purgeOverride":false,
      "purgePeriod":1,
      "readPermission":"",
      "setExtremeHighLimit":1.7976931348623157E308,
      "setExtremeLowLimit":-1.7976931348623157E308,
      "setPermission":"",
      "tags":{
      },
      "textRenderer":{
        "type":"ANALOG",
        "useUnitAsSuffix":false,
        "suffix":"",
        "format":"0.00"
      },
      "tolerance":0.0
    }
  ]
}

The email in my inbox is set up as follows:

My goal here is to return the value in the email and set them as values for the datapoint. Any help here would be much appreciated.

Thanks

Gord

Appreciate it @mattfox !

Fair enough, thanks Joel. Are there any weather API services you recommend? I've looked into Open Weather and heard good things. I

Hi Everyone,

I was wondering if I could get some help with regards to an issue I'm having retrieving data from the Environment Canada data source. I'm working with a weather station in Muskrat Dam (station ID 50722). I can collect some data from the station but it stops whenever it hits a gap in the data. For instance, if I start collecting data on Jan. 1 2013 it stops on July 24th where there is missing data.

Unfortunately, with this station there are a bunch of these data gaps. What I was wondering, is there a way to tell the parser to skip over these data gaps so that I can get the full data set? Ideally, I'd like to retrieve data starting on Jan 1, 2013 until present. @phildunlap I know you've done a bunch of work on this data source, I was hoping you might have some insight here.

Environment Canada Muskrat Dam station (Data missing July 24th):
https://climate.weather.gc.ca/climate_data/hourly_data_e.html?hlyRange=2012-11-13|2020-12-03&dlyRange=2012-11-15|2020-12-03&mlyRange=|&StationID=50722&Prov=ON&urlExtension=_e.html&searchType=stnName&optLimit=yearRange&StartYear=1840&EndYear=2020&selRowPerPage=25&Line=0&searchMethod=contains&txtStationName=muskrat+dam&timeframe=1&Year=2013&Month=7&Day=24

Thanks,

Gord

Ah ok, thank you guys. This helps quite a bit. I am currently using the free license at the moment to investigate whether or not Mango is the right for our project. So far it seems really great and I'm glad there is a solution for my duplicates problem. I appreciate all the help here in the forums.

Hi Craig,

I think I need to clarify my issue a little further. I am hoping to be able to upload data as either a csv file with many values/timestamps or a single value/value and have it update in the database only if the value at that timestamp has changed. I've tried both the "When Point Timestamp Changes" and "When Point Value Changes" logging properties with no luck.

With regards to the methods I am adding the value:
I'm currently using a couple of methods to get data in via a "Data File" datasource but all return the same issue of duplicate data for a specific timestamp. The first method is by placing a csv in the filestore and having the data source configured to look for any csv files in the filestore directory. What happens here is the first csv file is uploaded, then if I place the same file back in the folder all the values are logged to the database again despite the time stamps being identical. What I am expecting is none of the values to be logged because the timestamp hasn't changed. Ideally, if I were to adjust one of those values and reuploaded the file with the single adjustment, only the adjustment would be logged. The second method is setting a single value via the /point-values end point in the API. When I hardcode an identical time stamp to an existing one it is logged to the database and the previous value remains. I have a solution in mind for setting a single value which would involve an sql update via the sql-console update/query end points but that feels like a bit of a hack. I'm sure I'm misunderstanding how the logging is supposed to work so any help pointing me in the right direction would be greatly appreciated.

Thanks,

Gord

To add further context. I am using the "When Point Timestamp Changes" logging property. When I add a new point with the same timestamp it still logs to the database (checked this using the sql console).

Good Morning!

I was hoping to get a little help with something I am currently struggling with. Is it possible to limit data points to one value per timestamp? For example, say I had a data point, "Electricity Consumed", and it had a point value of 1500 for the time stamp, 1595822400000. If I add a new point value for the exact same timestamp, I want the new value to be logged and the previous value to be deleted. Essentially updating the value at a specific timestamp. At the moment, whenever I add a new point value for the same timestamp I add the new value but the previous value also remains in the system.

Thanks,
Gord

Amazing. Thank you very much.

Is it possible to access the tags for a data point when creating a script for a meta data point? I would like to use a switch case to do different calculations based on a point's tag. For instance, if a tag is kWh I would like to perform a different conversion calculation to GJ than a point with a Litres tag. I would then add all of the GJ values together and return this value.

Final Update: It took some fiddling but installing the 3.7.x branch worked. All modules are loaded can now but placed in the git/ma-core-public/Core/web/modules directory and loaded into the build. If anyone is interested, I created documentation for the process I used to set up Eclipse for version 3.7.x. Feel free to reach out and I can share it.

Alright, I'm stuck. I tried pulling the 3.7.x branch rather than the main branch and following the same process for setting up the dev environment in Eclipse but when I go to build the project I get to the log in page but won't let me log in. It tries to take me to /ui/agree-to-license but stays on the log in page and says "You are already logged in as admin. Go to the main UI."

In the logs it seems to be having trouble connecting to api.

I'm wondering if I am using the right branch?

I may have solved my own problem. I am running version 3.8.0 and the logs say the modules need a core version of 3.7.

I recently set up my development environment in Eclipse and wanted to include a bunch of modules including the dashboardDesigner and dataFile modules that weren't part of the repository. Unfortunately, I can't get them to load when I launch my instance of Mango. I included the files I downloaded from the store in the git/ma-core-public/Core/web/modules directory as @phildunlap mentioned to do in an older post but they don't appear when I launch the core. I was hoping someone might be able to provide some insight into what I might be missing.

Thanks,

Gord

@mattfox Thank you very much.