EnPT issueshttps://git.gfz-potsdam.de/EnMAP/GFZ_Tools_EnMAP_BOX/EnPT/-/issues2019-01-21T15:17:34+01:00https://git.gfz-potsdam.de/EnMAP/GFZ_Tools_EnMAP_BOX/EnPT/-/issues/21Improve DEM processor2019-01-21T15:17:34+01:00Daniel SchefflerImprove DEM processor* [x] add overlap check
* [x] add geographic datum check
* [ ] add gap filling algorithm
* [ ] add computation of slopes
* [ ] add computation of aspects* [x] add overlap check
* [x] add geographic datum check
* [ ] add gap filling algorithm
* [ ] add computation of slopes
* [ ] add computation of aspectsProvide a working L1B -> L2A processing chain for official DLR test data.Daniel SchefflerDaniel Schefflerhttps://git.gfz-potsdam.de/EnMAP/GFZ_Tools_EnMAP_BOX/EnPT/-/issues/35Add smile correction2020-03-27T14:23:51+01:00Daniel SchefflerAdd smile correctionNeeded features:
* [ ] compute column-wise center wavelength based on smile coefficients read by L1B reader (based on detector_meta.smile attribute)
* [ ] implement spectral interpolation to correct for smileNeeded features:
* [ ] compute column-wise center wavelength based on smile coefficients read by L1B reader (based on detector_meta.smile attribute)
* [ ] implement spectral interpolation to correct for smileDaniel SchefflerDaniel Schefflerhttps://git.gfz-potsdam.de/EnMAP/GFZ_Tools_EnMAP_BOX/EnPT/-/issues/38Revise geometry adjustment config parameters.2023-12-12T13:14:59+01:00Daniel SchefflerRevise geometry adjustment config parameters.Replace the following options (already included in RPC coefficients):
--enable_keystone_correction
--enable_vnir_swir_coreg
by
--optimize_vnir_swir_coreg: "Optimize VNIR/SWIR co-registration using AROSICS Default: False / not implem...Replace the following options (already included in RPC coefficients):
--enable_keystone_correction
--enable_vnir_swir_coreg
by
--optimize_vnir_swir_coreg: "Optimize VNIR/SWIR co-registration using AROSICS Default: False / not implemented yet"Daniel SchefflerDaniel Schefflerhttps://git.gfz-potsdam.de/EnMAP/GFZ_Tools_EnMAP_BOX/EnPT/-/issues/46Handle smile, keystone and fringe patterns provided by the ground segment.2020-08-12T18:54:37+02:00Daniel SchefflerHandle smile, keystone and fringe patterns provided by the ground segment.The ground segment will provide smile, keystone and fringe patterns. They will be freely available but are not included in the L1B data. EnPT may implement appropriate correction algorithms but should at least add some information about ...The ground segment will provide smile, keystone and fringe patterns. They will be freely available but are not included in the L1B data. EnPT may implement appropriate correction algorithms but should at least add some information about these patterns in the L2A metadata.
Steps to do:
* [ ] download these patterns during EnPT runtime or include them in the repository
* [ ] handle them somehow (correction / additional L2A metadata)
@segl
@nbohnDaniel SchefflerNiklas BohnDaniel Schefflerhttps://git.gfz-potsdam.de/EnMAP/GFZ_Tools_EnMAP_BOX/EnPT/-/issues/51Include dead pixel map and quality test flags in L2A output.2021-12-16T20:11:55+01:00Daniel SchefflerInclude dead pixel map and quality test flags in L2A output.The dead pixel map is provided as a 3D cube (for each band), separately for VNIR and SWIR. To include it in the L2A data, the sensor geometry data need to be orthorectified using the (3D) geolayer. This might be a bit slow.
The quality ...The dead pixel map is provided as a 3D cube (for each band), separately for VNIR and SWIR. To include it in the L2A data, the sensor geometry data need to be orthorectified using the (3D) geolayer. This might be a bit slow.
The quality test flags layer is provided as one 2D array and could directly be included in the L2A output. However, it needs to be resampled in a way that (1) preserves the original pixel values and (2) ensures geometrical accuracy.Daniel SchefflerDaniel Schefflerhttps://git.gfz-potsdam.de/EnMAP/GFZ_Tools_EnMAP_BOX/EnPT/-/issues/57Convert options_default.json to options_default.yaml?2020-05-18T23:40:52+02:00Daniel SchefflerConvert options_default.json to options_default.yaml?YAML officially supports comments and is maybe a bit clearer:
```yaml
general_opts:
# number of CPU cores to be used for processing (default: "None" -> use all available)
CPUs: "None"
# the logging level to be used (choices: 'DEB...YAML officially supports comments and is maybe a bit clearer:
```yaml
general_opts:
# number of CPU cores to be used for processing (default: "None" -> use all available)
CPUs: "None"
# the logging level to be used (choices: 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL')
log_level: "INFO"
# whether to write all log messages to a file (within output the directory)
create_logfile: true
# input path of the EnMAP L1B image to be processed (zip-archive or root directory)
path_l1b_enmap_image: ""
# input path of an adjacent EnMAP L1B image to be used for gap-filling (zip-archive or root directory)
path_l1b_enmap_image_gapfill: ""
# input path of digital elevation model in map or sensor geometry; GDAL compatible file format
# (must cover the EnMAP L1B data completely if given in map geometry or must have the same
# pixel dimensions like the EnMAP L1B data if given in sensor geometry)
path_dem: ""
# Set to true in case of the preliminary, GFZ-internal dataformat as used for the
# Alpine test dataset. (default: False. Note: This will be removed in future.)
is_dummy_dataformat: false
# average elevation in meters above sea level; may be provided if no DEM is available;
# ignored if DEM is given
average_elevation: 0
# input path of the EnMAP SNR model
path_l1b_snr_model: "./resources/EnMAP_Sensor/EnMAP_Level_1B_SNR.zip"
# directory to be used for temporary files
working_dir: ""
# *Number of lines to be added to the main image [if None, use the whole path_l1b_enmap_image_gapfill].
# Requires path_l1b_enmap_image_gapfill to be set.
n_lines_to_append: "None"
# whether to disable all progress bars during processing
disable_progress_bars: false
output:
# output directory where processed data and log files are saved
output_dir: ""
# file format of all raster output files
# 'GTiff': GeoTIFF
# 'ENVI': ENVI BSQ*/
output_format: "GTiff"
# raster data interleaving type (default: 'pixel')
# - 'band': band-sequential (BSQ)
# - 'line': data interleaved-by-line (BIL; only usable for ENVI output format)
# - 'pixel' data interleaved-by-pixel (BIP)*/
output_interleave: "pixel"
```Daniel SchefflerDaniel Schefflerhttps://git.gfz-potsdam.de/EnMAP/GFZ_Tools_EnMAP_BOX/EnPT/-/issues/60Add functionality to crop the L2A output with a user-provided shapefile2020-08-12T18:42:32+02:00Daniel SchefflerAdd functionality to crop the L2A output with a user-provided shapefileIt might be useful for some users to have such a functionality.
ToDo:
- [ ] add a config parameter that allows to pass a shapefile with the user AOI
- [ ] validate that the shapefile has a spatial overlap
- [ ] crop the L2A outputIt might be useful for some users to have such a functionality.
ToDo:
- [ ] add a config parameter that allows to pass a shapefile with the user AOI
- [ ] validate that the shapefile has a spatial overlap
- [ ] crop the L2A outputDaniel SchefflerDaniel Schefflerhttps://git.gfz-potsdam.de/EnMAP/GFZ_Tools_EnMAP_BOX/EnPT/-/issues/62Add correction of spatial misregistrations based on user-provided GCPs.2020-09-21T13:22:46+02:00Daniel SchefflerAdd correction of spatial misregistrations based on user-provided GCPs.It might be helpful for some users to provide their own GCPs for spatial alignment. This would allow to orthorectify the dataset AND correct for spatial misregistration in only one resampling step.
Steps to do:
- [ ] define a suitable i...It might be helpful for some users to provide their own GCPs for spatial alignment. This would allow to orthorectify the dataset AND correct for spatial misregistration in only one resampling step.
Steps to do:
- [ ] define a suitable input format for these GCPs
- [ ] add a config parameter to pass them to EnPT
- [ ] respect them in the geometry processorDaniel SchefflerDaniel Schefflerhttps://git.gfz-potsdam.de/EnMAP/GFZ_Tools_EnMAP_BOX/EnPT/-/issues/63Add an uncertainty layer to the L2A product.2020-08-12T19:37:28+02:00Daniel SchefflerAdd an uncertainty layer to the L2A product.An additional layer containing reflectance uncertainty layer might be useful for some users.An additional layer containing reflectance uncertainty layer might be useful for some users.Daniel SchefflerDaniel Schefflerhttps://git.gfz-potsdam.de/EnMAP/GFZ_Tools_EnMAP_BOX/EnPT/-/issues/64Add functionality to create along-track mosaics of EnMAP tiles.2020-08-12T19:42:27+02:00Daniel SchefflerAdd functionality to create along-track mosaics of EnMAP tiles.This could be implemented as a simple gdal_merge call after separate EnPT calls. Note, that the number of adjacent EnMAP tiles the user could provide is theoretically not limited.This could be implemented as a simple gdal_merge call after separate EnPT calls. Note, that the number of adjacent EnMAP tiles the user could provide is theoretically not limited.Daniel SchefflerDaniel Schefflerhttps://git.gfz-potsdam.de/EnMAP/GFZ_Tools_EnMAP_BOX/EnPT/-/issues/65Add an AOT retrieval as an alternative to the 3-phases of water retrieval in ...2020-08-12T19:53:04+02:00Daniel SchefflerAdd an AOT retrieval as an alternative to the 3-phases of water retrieval in the AC over land.Currently, the ground segment provides a mean AOT value per EnMAP tile. This could be replaced by an AOT retrieval within SICOR. However, it must be implemented as an alternative to the 3-phases-of-water retrieval there.
Steps to do:
- ...Currently, the ground segment provides a mean AOT value per EnMAP tile. This could be replaced by an AOT retrieval within SICOR. However, it must be implemented as an alternative to the 3-phases-of-water retrieval there.
Steps to do:
- [ ] wait for a respective implementation in SICOR
- [ ] add a config parameter to choose between AOT retrieval and the 3-phases-of-water approachDaniel SchefflerDaniel Schefflerhttps://git.gfz-potsdam.de/EnMAP/GFZ_Tools_EnMAP_BOX/EnPT/-/issues/69Improve co-registration algorithm2020-09-21T13:24:53+02:00Daniel SchefflerImprove co-registration algorithmto be improved:
* [ ] make the EnMAP band to be used for co-registration configurable (currently band 40, hardcoded)
* [ ] post-process the mis-registration array, e.g., handle shifts exceeding the maximum allowed shift after interpolat...to be improved:
* [ ] make the EnMAP band to be used for co-registration configurable (currently band 40, hardcoded)
* [ ] post-process the mis-registration array, e.g., handle shifts exceeding the maximum allowed shift after interpolating
* [ ] better evaluate different interpolation algorithms to create the mis-registration array
* [ ] allow spatial reference images with less than 100% overlap
* [ ] improve the documentation of the co-registration featureAdd geometry processorsDaniel SchefflerDaniel Schefflerhttps://git.gfz-potsdam.de/EnMAP/GFZ_Tools_EnMAP_BOX/EnPT/-/issues/74Provide an option to flag bad bands with a band bands list instead of complet...2021-02-01T13:28:57+01:00Daniel SchefflerProvide an option to flag bad bands with a band bands list instead of completely dropping themThis was requested by HU. However, it has to work together with the options we discussed and implemented in https://git.gfz-potsdam.de/EnMAP/GFZ_Tools_EnMAP_BOX/EnPT/-/issues/61.This was requested by HU. However, it has to work together with the options we discussed and implemented in https://git.gfz-potsdam.de/EnMAP/GFZ_Tools_EnMAP_BOX/EnPT/-/issues/61.Daniel SchefflerDaniel Schefflerhttps://git.gfz-potsdam.de/EnMAP/GFZ_Tools_EnMAP_BOX/EnPT/-/issues/80Use 'new_kurucz' solar model when calling SICOR for simulated EnMAP data.2023-02-17T16:15:26+01:00Daniel SchefflerUse 'new_kurucz' solar model when calling SICOR for simulated EnMAP data.Since SICOR>0.16.4 will use the Fontenla solar model by default (compatible with real EnMAP data), EnPT should call SICOR with `options["retrieval"]["sol_model": "new_kurucz"]` in case of simulated EnMAP data.Since SICOR>0.16.4 will use the Fontenla solar model by default (compatible with real EnMAP data), EnPT should call SICOR with `options["retrieval"]["sol_model": "new_kurucz"]` in case of simulated EnMAP data.Daniel SchefflerDaniel Schefflerhttps://git.gfz-potsdam.de/EnMAP/GFZ_Tools_EnMAP_BOX/EnPT/-/issues/82Improve documentation of advanced parameters.2022-02-08T11:03:24+01:00Daniel SchefflerImprove documentation of advanced parameters.EnPT offers a long list of advanced configuration parameters but not all are also documented in the Sphinx documentation. This should be improved and a short documentation should also be added directly in the GUI.EnPT offers a long list of advanced configuration parameters but not all are also documented in the Sphinx documentation. This should be improved and a short documentation should also be added directly in the GUI.Daniel SchefflerDaniel Schefflerhttps://git.gfz-potsdam.de/EnMAP/GFZ_Tools_EnMAP_BOX/EnPT/-/issues/83Mask output files should always be saved with BSQ interleave2022-02-15T21:46:50+01:00Daniel SchefflerMask output files should always be saved with BSQ interleaveIf BIP is set, also the mask files are written with BIP interleave (at least in case of ENVI format).If BIP is set, also the mask files are written with BIP interleave (at least in case of ENVI format).Daniel SchefflerDaniel Schefflerhttps://git.gfz-potsdam.de/EnMAP/GFZ_Tools_EnMAP_BOX/EnPT/-/issues/86ValueError during SNR computation due to unexpected number of bands2022-08-11T14:53:11+02:00Daniel SchefflerValueError during SNR computation due to unexpected number of bands```python
../enpt/execution/controller.py:176: in run_all_processors
self.read_L1B_data()
../enpt/execution/controller.py:127: in read_L1B_data
self.L1_obj = RD.read_inputdata(root_dir_main=path_enmap_image, root_dir_ext=path_enm...```python
../enpt/execution/controller.py:176: in run_all_processors
self.read_L1B_data()
../enpt/execution/controller.py:127: in read_L1B_data
self.L1_obj = RD.read_inputdata(root_dir_main=path_enmap_image, root_dir_ext=path_enmap_image_gapfill,
../enpt/io/reader.py:94: in read_inputdata
l1b_main_obj.calc_snr_from_radiance()
../enpt/model/images/images_sensorgeo.py:739: in calc_snr_from_radiance
self.vnir.detector_meta.calc_snr_from_radiance(rad_data=self.vnir.data, dir_snr_models=tmpDir)
ValueError: operands could not be broadcast together with shapes (1000,91) (1000,88)
```Daniel SchefflerDaniel Schefflerhttps://git.gfz-potsdam.de/EnMAP/GFZ_Tools_EnMAP_BOX/EnPT/-/issues/88Adapt output filenames to clearly reflect EnPT as processing chain + EnPT ver...2023-03-01T01:25:18+01:00Daniel SchefflerAdapt output filenames to clearly reflect EnPT as processing chain + EnPT versionThe filename convention of the output files should be revised.
@brellThe filename convention of the output files should be revised.
@brellDaniel SchefflerDaniel Schefflerhttps://git.gfz-potsdam.de/EnMAP/GFZ_Tools_EnMAP_BOX/EnPT/-/issues/93Re-enable first guess water vapor retrieval2022-12-08T19:43:55+01:00Daniel SchefflerRe-enable first guess water vapor retrieval!69 disabled the first guess water vapor retrieval of SICOR due to #90 and #92. As soon as the actual reason of the current issues with the retrieval are found, it should be re-enabled in EnPT.!69 disabled the first guess water vapor retrieval of SICOR due to #90 and #92. As soon as the actual reason of the current issues with the retrieval are found, it should be re-enabled in EnPT.https://git.gfz-potsdam.de/EnMAP/GFZ_Tools_EnMAP_BOX/EnPT/-/issues/97RuntimeError appears in combination with oversized DEM and gapfill options2023-03-02T18:41:55+01:00Maximilian BrellRuntimeError appears in combination with oversized DEM and gapfill optionsThe RuntimeError appears in combination with oversized DEM and gabfill options.
A possible solution could be to clip the DEM to the extent of a single tile before the. Coordinate normalization
```python
Traceback (most recent call last):...The RuntimeError appears in combination with oversized DEM and gabfill options.
A possible solution could be to clip the DEM to the extent of a single tile before the. Coordinate normalization
```python
Traceback (most recent call last):
File "/misc/hy14/Daten/EnMAP/EnPT/enpt/execution/controller.py", line 179, in run_all_processors
self.read_L1B_data()
File "/misc/hy14/Daten/EnMAP/EnPT/enpt/execution/controller.py", line 127, in read_L1B_data
self.L1_obj = RD.read_inputdata(root_dir_main=path_enmap_image, root_dir_ext=path_enmap_image_gapfill,
File "/misc/hy14/Daten/EnMAP/EnPT/enpt/io/reader.py", line 99, in read_inputdata
l1b_main_obj.meta.vnir.compute_geolayer_for_cube()
File "/misc/hy14/Daten/EnMAP/EnPT/enpt/model/metadata/metadata_sensorgeo.py", line 435, in compute_geolayer_for_cube
.compute_geolayer()
File "/misc/hy14/Daten/EnMAP/EnPT/enpt/processors/spatial_transform/spatial_transform.py", line 650, in compute_geolayer
lons_oneband, lats_oneband = self._compute_geolayer_for_unique_coeffgroup(kwargs_list[0])[:2]
File "/misc/hy14/Daten/EnMAP/EnPT/enpt/processors/spatial_transform/spatial_transform.py", line 626, in _compute_geolayer_for_unique_coeffgroup
).compute_geolayer()
File "/misc/hy14/Daten/EnMAP/EnPT/enpt/processors/spatial_transform/spatial_transform.py", line 512, in compute_geolayer
rows, cols = self.transform_LonLatHeight_to_RowCol(lon=lons, lat=lats, height=heights)
File "/misc/hy14/Daten/EnMAP/EnPT/enpt/processors/spatial_transform/spatial_transform.py", line 430, in transform_LonLatHeight_to_RowCol
self._normalize_map_coordinates(lon=lon, lat=lat, height=height)
File "/misc/hy14/Daten/EnMAP/EnPT/enpt/processors/spatial_transform/spatial_transform.py", line 345, in _normalize_map_coordinates
raise RuntimeError((llh.min(), llh.max()), msg % name)
RuntimeError: ((-1.3684170724625786, 1.034636410949972), 'Coordinate normalization yields significantly out-of-range values for longitudes. Check the coordinates and RPC coefficients.')
```