Environment Canada data source - Data gap issue
-
Hi Everyone,
I was wondering if I could get some help with regards to an issue I'm having retrieving data from the Environment Canada data source. I'm working with a weather station in Muskrat Dam (station ID 50722). I can collect some data from the station but it stops whenever it hits a gap in the data. For instance, if I start collecting data on Jan. 1 2013 it stops on July 24th where there is missing data.
Unfortunately, with this station there are a bunch of these data gaps. What I was wondering, is there a way to tell the parser to skip over these data gaps so that I can get the full data set? Ideally, I'd like to retrieve data starting on Jan 1, 2013 until present. @phildunlap I know you've done a bunch of work on this data source, I was hoping you might have some insight here.
Environment Canada Muskrat Dam station (Data missing July 24th):
https://climate.weather.gc.ca/climate_data/hourly_data_e.html?hlyRange=2012-11-13|2020-12-03&dlyRange=2012-11-15|2020-12-03&mlyRange=|&StationID=50722&Prov=ON&urlExtension=_e.html&searchType=stnName&optLimit=yearRange&StartYear=1840&EndYear=2020&selRowPerPage=25&Line=0&searchMethod=contains&txtStationName=muskrat+dam&timeframe=1&Year=2013&Month=7&Day=24Thanks,
Gord
-
Hi Gord, To be honest, I'm not sure this is a Data Source we are planning to continue to support into Mango v4. We have used the HTTP Data Source to get data from various weather API services which seems to be a bit more of a scalable method to use.
-
Fair enough, thanks Joel. Are there any weather API services you recommend? I've looked into Open Weather and heard good things. I
-
Used to use darkSky before Apple bought them out T-T ..
As for your environment canada, because it is a web based api, there's no reason why you can't use a scripted datasource or http retreiever if it's not too complex a query to pull all of the data in.Fox
-
Appreciate it @mattfox !
