EnPT issueshttps://git.gfz-potsdam.de/EnMAP/GFZ_Tools_EnMAP_BOX/EnPT/-/issues2023-03-02T18:41:55+01:00https://git.gfz-potsdam.de/EnMAP/GFZ_Tools_EnMAP_BOX/EnPT/-/issues/97RuntimeError appears in combination with oversized DEM and gapfill options2023-03-02T18:41:55+01:00Maximilian BrellRuntimeError appears in combination with oversized DEM and gapfill optionsThe RuntimeError appears in combination with oversized DEM and gabfill options.
A possible solution could be to clip the DEM to the extent of a single tile before the. Coordinate normalization
```python
Traceback (most recent call last):...The RuntimeError appears in combination with oversized DEM and gabfill options.
A possible solution could be to clip the DEM to the extent of a single tile before the. Coordinate normalization
```python
Traceback (most recent call last):
File "/misc/hy14/Daten/EnMAP/EnPT/enpt/execution/controller.py", line 179, in run_all_processors
self.read_L1B_data()
File "/misc/hy14/Daten/EnMAP/EnPT/enpt/execution/controller.py", line 127, in read_L1B_data
self.L1_obj = RD.read_inputdata(root_dir_main=path_enmap_image, root_dir_ext=path_enmap_image_gapfill,
File "/misc/hy14/Daten/EnMAP/EnPT/enpt/io/reader.py", line 99, in read_inputdata
l1b_main_obj.meta.vnir.compute_geolayer_for_cube()
File "/misc/hy14/Daten/EnMAP/EnPT/enpt/model/metadata/metadata_sensorgeo.py", line 435, in compute_geolayer_for_cube
.compute_geolayer()
File "/misc/hy14/Daten/EnMAP/EnPT/enpt/processors/spatial_transform/spatial_transform.py", line 650, in compute_geolayer
lons_oneband, lats_oneband = self._compute_geolayer_for_unique_coeffgroup(kwargs_list[0])[:2]
File "/misc/hy14/Daten/EnMAP/EnPT/enpt/processors/spatial_transform/spatial_transform.py", line 626, in _compute_geolayer_for_unique_coeffgroup
).compute_geolayer()
File "/misc/hy14/Daten/EnMAP/EnPT/enpt/processors/spatial_transform/spatial_transform.py", line 512, in compute_geolayer
rows, cols = self.transform_LonLatHeight_to_RowCol(lon=lons, lat=lats, height=heights)
File "/misc/hy14/Daten/EnMAP/EnPT/enpt/processors/spatial_transform/spatial_transform.py", line 430, in transform_LonLatHeight_to_RowCol
self._normalize_map_coordinates(lon=lon, lat=lat, height=height)
File "/misc/hy14/Daten/EnMAP/EnPT/enpt/processors/spatial_transform/spatial_transform.py", line 345, in _normalize_map_coordinates
raise RuntimeError((llh.min(), llh.max()), msg % name)
RuntimeError: ((-1.3684170724625786, 1.034636410949972), 'Coordinate normalization yields significantly out-of-range values for longitudes. Check the coordinates and RPC coefficients.')
```https://git.gfz-potsdam.de/EnMAP/GFZ_Tools_EnMAP_BOX/EnPT/-/issues/104`auto_download_ecmwf` flag not working2023-12-21T18:44:26+01:00Daniel Scheffler`auto_download_ecmwf` flag not workingThe ECMWF download is executed although `auto_download_ecmwf` was set to False:
```python
Input parameters:
{ 'CPUs' : 32, 'auto_download_ecmwf' : False, 'average_elevation' : 0, 'blocksize' : 100, 'deadpix_P_algorithm' : 1, 'deadpix_P_...The ECMWF download is executed although `auto_download_ecmwf` was set to False:
```python
Input parameters:
{ 'CPUs' : 32, 'auto_download_ecmwf' : False, 'average_elevation' : 0, 'blocksize' : 100, 'deadpix_P_algorithm' : 1, 'deadpix_P_interp_spatial' : 0, 'deadpix_P_interp_spectral' : 0, 'disable_progress_bars' : True, 'drop_bad_bands' : True, 'enable_ac' : True, 'json_config' : '', 'mode_ac' : 2, 'n_lines_to_append' : None, 'ortho_resampAlg' : 2, 'output_dir' : '/home/gfz-fe/EnPT/Output/20230228__1', 'output_format' : 1, 'output_interleave' : 0, 'path_dem' : '/home/gfz-fe/scheffler/temp/EnPT/EnVAL_ID_134/geo_abs_elevation.tif', 'path_earthSunDist' : '', 'path_l1b_enmap_image' : '/home/gfz-fe/scheffler/temp/EnPT/EnVAL_ID_134/ENMAP01-____L1B-DT0000001867_20220724T104521Z_007_V010111_20230228T092319Z.ZIP', 'path_l1b_enmap_image_gapfill' : '', 'path_reference_image' : '/home/gfz-fe/scheffler/temp/EnPT/EnVAL_ID_134/geo_abs_ref.tif', 'path_solar_irr' : '', 'polymer_root' : '/home/gfz-fe/scheffler/python/polymer', 'run_deadpix_P' : False, 'scale_factor_boa_ref' : 10000, 'scale_factor_toa_ref' : 10000, 'target_epsg' : None, 'target_projection_type' : 0, 'threads' : -1, 'vswir_overlap_algorithm' : 3, 'working_dir' : '' }
```
```python
2023/02/28 23:19:52: Running polymer...
2023/02/28 23:19:52: Currently only tested with ERA5 data, i.e. set True for "auto_download_ecmwf"
Initializing ENMAP product of size (1024, 1000)
2023/02/28 23:19:52: Landmask is EnMAP L1B
Download 2022-07-24 10:00:00 -> /home/gfz-fe/scheffler/python/polymer/ANCILLARY/ERA5/2022/07/24/era5_20220724_100000.nc
2023-02-28 23:19:53,012 INFO Welcome to the CDS
2023-02-28 23:19:53,012 INFO Sending request to https://cds.climate.copernicus.eu/api/v2/resources/reanalysis-era5-single-levels
2023-02-28 23:19:53,266 INFO Request is queued
2023-02-28 23:19:54,310 INFO Request is running
2023-02-28 23:19:55,854 INFO Request is completed
2023-02-28 23:19:55,855 INFO Downloading https://download-0000-clone.copernicus-climate.eu/cache-compute-0000/cache/data6/adaptor.mars.internal-1677622794.747088-20076-11-6bbabe83-8d5a-4b0e-8963-a6bb1eeb68f9.nc to /home/gfz-fe/scheffler/python/polymer/ANCILLARY/ERA5/2022/07/24/era5_20220724_100000.nc.tmp (9.9M)
0%| | 0.00/9.91M [00:00<?, ?B/s] 1%| | 78.0k/9.91M [00:00<00:14, 726kB/s] 7%|▋ | 754k/9.91M [00:00<00:02, 3.84MB/s] 32%|███■| 3.14M/9.91M [00:00<00:00, 12.9MB/s] 62%|██████■| 6.15M/9.91M [00:00<00:00, 19.9MB/s] 87%|████████▋ | 8.58M/9.91M [00:00<00:00, 21.9MB/s] 2023-02-28 23:19:56,694 INFO Download rate 11.8M/s
Download 2022-07-24 11:00:00 -> /home/gfz-fe/scheffler/python/polymer/ANCILLARY/ERA5/2022/07/24/era5_20220724_110000.nc
2023-02-28 23:19:56,780 INFO Welcome to the CDS
2023-02-28 23:19:56,780 INFO Sending request to https://cds.climate.copernicus.eu/api/v2/resources/reanalysis-era5-single-levels
2023-02-28 23:19:56,897 INFO Request is queued
2023-02-28 23:19:57,942 INFO Request is running
2023-02-28 23:19:59,487 INFO Request is completed
2023-02-28 23:19:59,488 INFO Downloading https://download-0001-clone.copernicus-climate.eu/cache-compute-0001/cache/data2/adaptor.mars.internal-1677622797.8498647-11507-3-392d1195-e895-4600-b6db-3147f22fd704.nc to /home/gfz-fe/scheffler/python/polymer/ANCILLARY/ERA5/2022/07/24/era5_20220724_110000.nc.tmp (9.9M)
0%| | 0.00/9.91M [00:00<?, ?B/s] 1%| | 78.0k/9.91M [00:00<00:13, 740kB/s] 7%|▋ | 751k/9.91M [00:00<00:02, 4.04MB/s] 35%|███▌ | 3.48M/9.91M [00:00<00:00, 14.8MB/s] 70%|███████ | 6.95M/9.91M [00:00<00:00, 23.1MB/s] 2023-02-28 23:20:00,244 INFO Download rate 13.1M/s
Starting processing at 2023-02-28 23:20:00.527176
```
Also `auto_download_ecmwf` could be set to True by default.Daniel SchefflerDaniel Schefflerhttps://git.gfz-potsdam.de/EnMAP/GFZ_Tools_EnMAP_BOX/EnPT/-/issues/105ACwater shows progress bars although `disable_progress_bars` was set to True2024-01-02T19:59:28+01:00Daniel SchefflerACwater shows progress bars although `disable_progress_bars` was set to TrueACwater shows progress bars although `disable_progress_bars` was set to True:
```python
Input parameters:
{ 'CPUs' : 32, 'auto_download_ecmwf' : False, 'average_elevation' : 0, 'blocksize' : 100, 'deadpix_P_algorithm' : 1, 'deadpix_P_in...ACwater shows progress bars although `disable_progress_bars` was set to True:
```python
Input parameters:
{ 'CPUs' : 32, 'auto_download_ecmwf' : False, 'average_elevation' : 0, 'blocksize' : 100, 'deadpix_P_algorithm' : 1, 'deadpix_P_interp_spatial' : 0, 'deadpix_P_interp_spectral' : 0, 'disable_progress_bars' : True, 'drop_bad_bands' : True, 'enable_ac' : True, 'json_config' : '', 'mode_ac' : 2, 'n_lines_to_append' : None, 'ortho_resampAlg' : 2, 'output_dir' : '/home/gfz-fe/EnPT/Output/20230228__1', 'output_format' : 1, 'output_interleave' : 0, 'path_dem' : '/home/gfz-fe/scheffler/temp/EnPT/EnVAL_ID_134/geo_abs_elevation.tif', 'path_earthSunDist' : '', 'path_l1b_enmap_image' : '/home/gfz-fe/scheffler/temp/EnPT/EnVAL_ID_134/ENMAP01-____L1B-DT0000001867_20220724T104521Z_007_V010111_20230228T092319Z.ZIP', 'path_l1b_enmap_image_gapfill' : '', 'path_reference_image' : '/home/gfz-fe/scheffler/temp/EnPT/EnVAL_ID_134/geo_abs_ref.tif', 'path_solar_irr' : '', 'polymer_root' : '/home/gfz-fe/scheffler/python/polymer', 'run_deadpix_P' : False, 'scale_factor_boa_ref' : 10000, 'scale_factor_toa_ref' : 10000, 'target_epsg' : None, 'target_projection_type' : 0, 'threads' : -1, 'vswir_overlap_algorithm' : 3, 'working_dir' : '' }
```
```python
2023/02/28 23:19:52: Running polymer...
2023/02/28 23:19:52: Currently only tested with ERA5 data, i.e. set True for "auto_download_ecmwf"
Initializing ENMAP product of size (1024, 1000)
2023/02/28 23:19:52: Landmask is EnMAP L1B
Download 2022-07-24 10:00:00 -> /home/gfz-fe/scheffler/python/polymer/ANCILLARY/ERA5/2022/07/24/era5_20220724_100000.nc
2023-02-28 23:19:53,012 INFO Welcome to the CDS
2023-02-28 23:19:53,012 INFO Sending request to https://cds.climate.copernicus.eu/api/v2/resources/reanalysis-era5-single-levels
2023-02-28 23:19:53,266 INFO Request is queued
2023-02-28 23:19:54,310 INFO Request is running
2023-02-28 23:19:55,854 INFO Request is completed
2023-02-28 23:19:55,855 INFO Downloading https://download-0000-clone.copernicus-climate.eu/cache-compute-0000/cache/data6/adaptor.mars.internal-1677622794.747088-20076-11-6bbabe83-8d5a-4b0e-8963-a6bb1eeb68f9.nc to /home/gfz-fe/scheffler/python/polymer/ANCILLARY/ERA5/2022/07/24/era5_20220724_100000.nc.tmp (9.9M)
0%| | 0.00/9.91M [00:00<?, ?B/s] 1%| | 78.0k/9.91M [00:00<00:14, 726kB/s] 7%|▋ | 754k/9.91M [00:00<00:02, 3.84MB/s] 32%|███■| 3.14M/9.91M [00:00<00:00, 12.9MB/s] 62%|██████■| 6.15M/9.91M [00:00<00:00, 19.9MB/s] 87%|████████▋ | 8.58M/9.91M [00:00<00:00, 21.9MB/s] 2023-02-28 23:19:56,694 INFO Download rate 11.8M/s
Download 2022-07-24 11:00:00 -> /home/gfz-fe/scheffler/python/polymer/ANCILLARY/ERA5/2022/07/24/era5_20220724_110000.nc
2023-02-28 23:19:56,780 INFO Welcome to the CDS
2023-02-28 23:19:56,780 INFO Sending request to https://cds.climate.copernicus.eu/api/v2/resources/reanalysis-era5-single-levels
2023-02-28 23:19:56,897 INFO Request is queued
2023-02-28 23:19:57,942 INFO Request is running
2023-02-28 23:19:59,487 INFO Request is completed
2023-02-28 23:19:59,488 INFO Downloading https://download-0001-clone.copernicus-climate.eu/cache-compute-0001/cache/data2/adaptor.mars.internal-1677622797.8498647-11507-3-392d1195-e895-4600-b6db-3147f22fd704.nc to /home/gfz-fe/scheffler/python/polymer/ANCILLARY/ERA5/2022/07/24/era5_20220724_110000.nc.tmp (9.9M)
0%| | 0.00/9.91M [00:00<?, ?B/s] 1%| | 78.0k/9.91M [00:00<00:13, 740kB/s] 7%|▋ | 751k/9.91M [00:00<00:02, 4.04MB/s] 35%|███▌ | 3.48M/9.91M [00:00<00:00, 14.8MB/s] 70%|███████ | 6.95M/9.91M [00:00<00:00, 23.1MB/s] 2023-02-28 23:20:00,244 INFO Download rate 13.1M/s
Starting processing at 2023-02-28 23:20:00.527176
```Daniel SchefflerDaniel Schefflerhttps://git.gfz-potsdam.de/EnMAP/GFZ_Tools_EnMAP_BOX/EnPT/-/issues/109calc_solar_irradiance_CWL_FWHM_per_band delivers too smooth irradiances2023-04-04T14:07:55+02:00Maximilian Brellcalc_solar_irradiance_CWL_FWHM_per_band delivers too smooth irradiancesI do not understand what happens here (calc_solar_irradiance_CWL_FWHM_per_band(self)), but it does not look like it is convolved correctly to EnMAP CWL and FWHM...
` def calc_solar_irradiance_CWL_FWHM_per_band(self) -> np.array:
...I do not understand what happens here (calc_solar_irradiance_CWL_FWHM_per_band(self)), but it does not look like it is convolved correctly to EnMAP CWL and FWHM...
` def calc_solar_irradiance_CWL_FWHM_per_band(self) -> np.array:
from ...io.reader import Solar_Irradiance_reader
self.logger.debug('Calculating solar irradiance...')
sol_irr = Solar_Irradiance_reader(path_solar_irr_model=self.cfg.path_solar_irr, wvl_min_nm=350, wvl_max_nm=2500)
irr_bands = []
for band in self.srf.bands:
WVL_band = self.srf.srfs_wvl if self.srf.wvl_unit == 'nanometers' else self.srf.srfs_wvl * 1000
RSP_band = self.srf.srfs_norm01[band]
sol_irr_at_WVL = np.interp(WVL_band, sol_irr[:, 0], sol_irr[:, 1], left=0, right=0)
irr_bands.append(np.round(np.sum(sol_irr_at_WVL * RSP_band) / np.sum(RSP_band), 2))
return np.array(irr_bands)
`Daniel SchefflerDaniel Scheffler