Spatial artifacts in L2A output
For ENMAP01-____L1B-DT0000005210_20221108T071035Z_005_V010110_20221121T092646Z, the EnPT L2A output looks like this:
This is a desert scene, @nbohn might it be possible that the artifacts are caused by an incorrect retrieval result within SICOR?
Here is the log.
2022/11/25 14:34:16 INFO: Reading metadata for VNIR detector...
2022/11/25 14:34:16 DEBUG: Calculating solar irradiance...
2022/11/25 14:34:17 INFO: Reading metadata for SWIR detector...
2022/11/25 14:34:17 DEBUG: Calculating solar irradiance...
2022/11/25 14:34:18 INFO: Reading image masks in VNIR sensor geometry.
2022/11/25 14:34:21 INFO: Converting DN values to radiance [mW/m^2/sr/nm] for VNIR detector...
2022/11/25 14:34:24 INFO: Converting DN values to radiance [mW/m^2/sr/nm] for SWIR detector...
2022/11/25 14:34:27 INFO: Computing SNR from VNIR TOA radiance.
2022/11/25 14:34:36 INFO: Computing SNR from SWIR TOA radiance.
2022/11/25 14:34:40 INFO: Computing VNIR geolayer...
2022/11/25 14:34:43 INFO: Computing SWIR geolayer...
2022/11/25 14:35:04 INFO: Correcting dead pixels of VNIR detector...
Used algorithm: linear interpolation in the spectral domain
2022/11/25 14:35:07 INFO: Percentage of defective pixels: 0.15
2022/11/25 14:35:07 DEBUG: Percentage of pixels with a defect in any band: 12.60
2022/11/25 14:35:10 INFO: Correcting dead pixels of SWIR detector...
Used algorithm: linear interpolation in the spectral domain
2022/11/25 14:35:14 INFO: Percentage of defective pixels: 1.14
2022/11/25 14:35:14 DEBUG: Percentage of pixels with a defect in any band: 64.60
2022/11/25 14:35:23 INFO: No DEM for the VNIR detector provided. Falling back to an average elevation of 0 meters.
2022/11/25 14:35:23 INFO: No DEM for the SWIR detector provided. Falling back to an average elevation of 0 meters.
2022/11/25 14:35:29 INFO: Transforming the 'mask_landwater' attribute from VNIR to SWIR sensor geometry.
2022/11/25 14:35:33 INFO: Starting atmospheric correction for VNIR and SWIR detector in 'land' mode. Source radiometric unit code is 'TOARad'.
2022/11/25 14:35:34 DEBUG: SICOR AC configuration:
{'processing': {'Exception': None,
'Exception_type': '',
'clear_fraction': None,
'interface': {'args': (), 'kwargs': {}},
'status': 1,
'tIO': 0.0,
'tRT': 0.0,
'uncertainties': {}},
'retrieval': {'cpu': 32,
'default_aot_value': 0.279,
'disable_progressbars': False,
'fn_LUT': '',
'inversion': {'eps': 0.01,
'full': False,
'gnform': 'n',
'maxiter': 35},
'land_only': False,
'n_pca': 5,
'segmentation': True,
'segs': 200,
'sol_model': 'fontenla',
'state_vector': {'ice': {'ll': 0.001,
'prior_mean': 0.02,
'prior_sigma': 1000.0,
'ul': 0.499,
'use_prior_mean': False},
'intercept': {'ll': 0.001,
'prior_mean': 0.3,
'prior_sigma': 1000.0,
'ul': 0.999,
'use_prior_mean': False},
'liquid_water': {'ll': 0.001,
'prior_mean': 0.02,
'prior_sigma': 1000.0,
'ul': 0.499,
'use_prior_mean': False},
'slope': {'ll': -0.0004,
'prior_mean': 0.0002,
'prior_sigma': 1000.0,
'ul': 0.0004,
'use_prior_mean': False},
'water_vapor': {'ll': 0.001,
'prior_mean': 2.5,
'prior_sigma': 1000.0,
'ul': 4.999,
'use_prior_mean': False}},
'unknowns': {'ice_absorption_coefficients': {'sigma': 2.39e-07},
'liquid_water_absorption_coefficients': {'sigma': 2.43e-07},
'skyview': {'sigma': 0.1},
'water_vapor_absorption_coefficients': {'sigma': 0.01}}},
'sensor': {'fit': {'idx': [14,
15,
16,
17,
18,
19,
20,
21,
22,
23,
24,
25,
26,
27,
28,
29,
30,
31],
'snr': [322.94254006,
313.45752342,
313.32464722,
306.19455173,
281.35641378,
227.9759974,
157.22390595,
148.93620623,
154.40197388,
182.60431866,
232.89644996,
250.1244036,
252.2267179,
272.16592448,
299.71964816,
316.11909184,
326.33202741,
312.28461288]},
'name': 'EnMAP',
'resamp_alg': 'nearest'}}
2022/11/25 14:35:34 INFO: Setting up forward operator...
2022/11/25 14:35:36 INFO: SICOR is applied to land AND water pixels.
2022/11/25 14:35:36 INFO: Transforming VNIR data to SWIR sensor geometry to enable first guess retrievals for liquid water and ice...
2022/11/25 14:35:45 INFO: Getting observation metadata...
2022/11/25 14:35:45 INFO: Checking if observation metadata values are within LUT value ranges...
2022/11/25 14:35:45 INFO: Loading solar irradiance model...
2022/11/25 14:35:45 INFO: Loading RT LUT...
2022/11/25 14:35:45 INFO: LUT file was properly downloaded and is available for AC!
2022/11/25 14:35:46 INFO: Converting LUT to Fontenla solar model...
2022/11/25 14:35:47 INFO: Resampling LUT to EnMAP wavelengths...
2022/11/25 14:36:11 INFO: Reducing grid dimensionality of LUT to increase interpolation speed...
2022/11/25 14:36:11 INFO: Calculating absorption coefficients of liquid water and ice...
2022/11/25 14:36:13 INFO: Performing first guess water vapor retrieval based on a common band ratio using VNIR data...
2022/11/25 14:38:39 INFO: Transforming CWV first guess map to SWIR sensor geometry to enable segmentation and 3 phases of water retrieval...
2022/11/25 14:38:43 INFO: Performing first guess liquid water retrieval based on the NDWI...
2022/11/25 14:38:48 INFO: Performing first guess ice retrieval based on the NDSI...
2022/11/25 14:38:54 INFO: Calculating first guess for intercept and slope of absorption feature continuum...
2022/11/25 14:38:54 INFO: Segmenting SWIR L1B spectra to enhance processing speed...
2022/11/25 14:39:01 INFO: Preparing segmented SWIR L1B data cube...
2022/11/25 14:39:13 INFO: Performing 3 phases of water retrieval...
2022/11/25 14:39:13 INFO: Setting up multiprocessing...
2022/11/25 14:39:13 INFO: Multiprocessing on 32 cpu's
2022/11/25 14:39:13 INFO: Preparing optimal estimation input...
2022/11/25 14:39:13 INFO: Optimization...
2022/11/25 14:39:28 INFO: Done!
2022/11/25 14:39:28 INFO: Runtime: 15.11 s
2022/11/25 14:39:28 INFO: Extrapolating CWV retrieval map to full SWIR data cube...
2022/11/25 14:39:28 INFO: Transforming CWV retrieval map and segmentation labels to VNIR sensor geometry to enable VNIR segmentation...
2022/11/25 14:39:36 INFO: Segmenting VNIR data cube according to transformed SWIR segmentation labels...
2022/11/25 14:39:38 INFO: Starting surface reflectance retrieval...
2022/11/25 14:39:38 INFO: Calculating surface reflectance for vnir detector...
2022/11/25 14:39:38 INFO: Calculating surface reflectance for swir detector...
2022/11/25 14:39:38 INFO: Applying empirical line solution to extrapolate L2A data pixelwise...
2022/11/25 14:39:38 INFO: Extrapolating L2A spectra for vnir detector...
2022/11/25 14:39:49 INFO: Extrapolating L2A spectra for swir detector...
2022/11/25 14:40:00 INFO: Extrapolating liquid water and ice maps...
2022/11/25 14:40:02 WARNING: VNIR L2A data contain negative values indicating an overcorrection. Please check for errors in the input data, the options file, or the processing code.
2022/11/25 14:40:03 WARNING: VNIR L2A data contain values exceeding 1 indicating a saturation. Please check for errors in the input data, the options file, or the processing code.
2022/11/25 14:40:05 WARNING: SWIR L2A data contain negative values indicating an overcorrection. Please check for errors in the input data, the options file, or the processing code.
2022/11/25 14:40:06 WARNING: SWIR L2A data contain values exceeding 1 indicating a saturation. Please check for errors in the input data, the options file, or the processing code.
2022/11/25 14:40:06 INFO: SICOR atmospheric correction for EnMAP in land mode successfully finished!
2022/11/25 14:40:06 INFO: Joining results of atmospheric correction.
2022/11/25 14:40:08 INFO: Re-applying dead pixel correction to correct for spectral spikes due to fringe effect.
2022/11/25 14:40:08 INFO: Correcting dead pixels of VNIR detector...
Used algorithm: linear interpolation in the spectral domain
2022/11/25 14:40:11 INFO: Percentage of defective pixels: 0.15
2022/11/25 14:40:11 DEBUG: Percentage of pixels with a defect in any band: 12.60
2022/11/25 14:40:12 INFO: Correcting dead pixels of SWIR detector...
Used algorithm: linear interpolation in the spectral domain
2022/11/25 14:40:17 INFO: Percentage of defective pixels: 1.14
2022/11/25 14:40:17 DEBUG: Percentage of pixels with a defect in any band: 64.60
2022/11/25 14:40:21 INFO: Starting orthorectification...
2022/11/25 14:40:21 INFO: Computed common target extent of orthorectified image (xmin, ymin, xmax, ymax in EPSG 32641): (393930.0, 3208590.0, 430320.0, 3243030.0)
2022/11/25 14:40:21 INFO: Orthorectifying VNIR data using 'nearest' resampling algorithm...
2022/11/25 14:40:57 INFO: Orthorectifying SWIR data using 'nearest' resampling algorithm...
2022/11/25 14:41:41 INFO: Merging VNIR and SWIR data...
2022/11/25 14:41:41 INFO: Orthorectifying 'mask_landwater' attribute...
2022/11/25 14:41:43 INFO: Orthorectifying 'mask_clouds' attribute...
2022/11/25 14:41:43 INFO: Orthorectifying 'mask_cloudshadow' attribute...
2022/11/25 14:41:44 INFO: Orthorectifying 'mask_haze' attribute...
2022/11/25 14:41:45 INFO: Orthorectifying 'mask_snow' attribute...
2022/11/25 14:41:46 INFO: Orthorectifying 'mask_cirrus' attribute...
2022/11/25 14:41:47 INFO: Setting all pixels to nodata that have values in the VNIR or the SWIR only...
2022/11/25 14:41:52 INFO: Generating L2A metadata...
2022/11/25 14:42:05 INFO: Write product to: /home/gfz-fe/scheffler/python/EnPT/tests/data/test_outputs/ENMAP01-____L2A-DT0000005210_20221108T071035Z_005_V010110_20221121T092646Z
2022/11/25 14:42:10 WARNING: Currently, L2A dead pixel masks cannot be saved yet.
2022/11/25 14:42:10 WARNING: Currently, the L2A metadata XML file does not contain all relevant keys and contains not updated values!
2022/11/25 14:42:10 INFO: Writing metadata to /home/gfz-fe/scheffler/python/EnPT/tests/data/test_outputs/ENMAP01-____L2A-DT0000005210_20221108T071035Z_005_V010110_20221121T092646Z/ENMAP01-____L2A-DT0000005210_20221108T071035Z_005_V010110_20221121T092646Z-METADATA.XML
2022/11/25 14:42:10 INFO: L2A product successfully written!
2022/11/25 14:42:10 INFO: Total runtime of the processing chain: 0:07:59.480157Edited by Daniel Scheffler