ECMWF data crawler only works for acquisition dates between 07/2012 and 06/2018 for the default variables.
I need to process some Landsat data acquired in 2008 in SICOR and I noticed that the ECMWF crawler does not download most of the variables for this year. I took a closer look into the code and I think I found the reason for it.
The ECMWF_download class downloads the individual variables (T2M, O3, SLP, TCWV, Ozone, AOT, ...) from different ECMWF datasets whereas each dataset only covers a specific time interval:
|dataset||dataset slug in the code||interval|
|ERA Interim, Daily||interim||01/1979 - 08/2019|
|CAMS Near-real-time||cams_nrealtime||07/2012 - present|
|MACC Near-real-time||macc_nrealtime||07/2012 - 06/2018|
Since only a few variables use the ERA interim dataset in SICOR, the download fails for most of them.
I already created a new branch and will try to fix that. My plan is to implement a check that takes the
date_to parameters that are passed to
download_variables() and automatically selects a suitable ECMWF dataset.
The question is only if we should prefer specific ECMWF datasets over others. As I am not familiar with these data I would just use the datasets with the newer time interval first if available. Or do you have a better idea?
Another issue is that the ERA interim dataset has phased out in 08/2019 and it is recommended to switch to ERA 5. However, this seems to use another API.