Please Note This forum exists for community support for the Mango product family and the Radix IoT Platform. Although Radix IoT employees participate in this forum from time to time, there is no guarantee of a response to anything posted here, nor can Radix IoT, LLC guarantee the accuracy of any information expressed or conveyed. Specific project questions from customers with active support contracts are asked to send requests to support@radixiot.com.
Environment Canada data source - Data gap issue
-
Hi Everyone,
I was wondering if I could get some help with regards to an issue I'm having retrieving data from the Environment Canada data source. I'm working with a weather station in Muskrat Dam (station ID 50722). I can collect some data from the station but it stops whenever it hits a gap in the data. For instance, if I start collecting data on Jan. 1 2013 it stops on July 24th where there is missing data.
Unfortunately, with this station there are a bunch of these data gaps. What I was wondering, is there a way to tell the parser to skip over these data gaps so that I can get the full data set? Ideally, I'd like to retrieve data starting on Jan 1, 2013 until present. @phildunlap I know you've done a bunch of work on this data source, I was hoping you might have some insight here.
Environment Canada Muskrat Dam station (Data missing July 24th):
https://climate.weather.gc.ca/climate_data/hourly_data_e.html?hlyRange=2012-11-13|2020-12-03&dlyRange=2012-11-15|2020-12-03&mlyRange=|&StationID=50722&Prov=ON&urlExtension=_e.html&searchType=stnName&optLimit=yearRange&StartYear=1840&EndYear=2020&selRowPerPage=25&Line=0&searchMethod=contains&txtStationName=muskrat+dam&timeframe=1&Year=2013&Month=7&Day=24Thanks,
Gord
-
Hi Gord, To be honest, I'm not sure this is a Data Source we are planning to continue to support into Mango v4. We have used the HTTP Data Source to get data from various weather API services which seems to be a bit more of a scalable method to use.
-
Fair enough, thanks Joel. Are there any weather API services you recommend? I've looked into Open Weather and heard good things. I
-
Used to use darkSky before Apple bought them out T-T ..
As for your environment canada, because it is a web based api, there's no reason why you can't use a scripted datasource or http retreiever if it's not too complex a query to pull all of the data in.Fox
-
Appreciate it @mattfox !
