sensormapgeo issueshttps://git.gfz-potsdam.de/EnMAP/sensormapgeo/-/issues2024-02-06T18:36:30+01:00https://git.gfz-potsdam.de/EnMAP/sensormapgeo/-/issues/11[pyresample backend] Missing data in case of bilinear resampling of Lon/Lat d...2024-02-06T18:36:30+01:00Daniel Scheffler[pyresample backend] Missing data in case of bilinear resampling of Lon/Lat data.Steps to reproduce:
* run Test_SensorMapGeometryTransformer.test_to_map_geometry_lonlat() with resampling algorithm 'blinear'
-> pyresample raises a numpy warning due to NaN values which are NOT in the input coming from sensormapgeo:
`...Steps to reproduce:
* run Test_SensorMapGeometryTransformer.test_to_map_geometry_lonlat() with resampling algorithm 'blinear'
-> pyresample raises a numpy warning due to NaN values which are NOT in the input coming from sensormapgeo:
```python
/root/mambaforge/envs/ci_env/lib/python3.10/site-packages/pyresample/bilinear/_base.py:388
invalid value encountered in true_divide
```
The result looks like this:
![image](/uploads/8a369ccfa252ed7e854ee3d9b163fe91/image.png)
Setting the radius_of_influence parameter to a very large value, reduces the missing pixels but does NOT remove them:
```python
result = \
NumpyBilinearResampler(source_geo_def, target_geo_def, **{'radius_of_influence': 30e300})\
.resample(data, **{'fill_value': -9999, 'nprocs': 32})\
.astype(data.dtype)
from geoarray import GeoArray
GeoArray(result).show()
```
Seems like this is a bug in pyresample.Daniel SchefflerDaniel Schefflerhttps://git.gfz-potsdam.de/EnMAP/sensormapgeo/-/issues/12[pyresample backend] Numpy warning "invalid value encountered in divide" when...2024-02-06T18:34:43+01:00Daniel Scheffler[pyresample backend] Numpy warning "invalid value encountered in divide" when running bilinear resampling```python
WARNING: /opt/conda/envs/ci_env/lib/python3.12/site-packages/pyresample/bilinear/_base.py:397
invalid value encountered in divide
``````python
WARNING: /opt/conda/envs/ci_env/lib/python3.12/site-packages/pyresample/bilinear/_base.py:397
invalid value encountered in divide
```Daniel SchefflerDaniel Schefflerhttps://git.gfz-potsdam.de/EnMAP/sensormapgeo/-/issues/8[pyresample backend] Take care of no-data values in the input datasets when r...2024-02-06T17:06:44+01:00Daniel Scheffler[pyresample backend] Take care of no-data values in the input datasets when resampling.pyresample supports masked arrays as inputs (see [here](https://pyresample.readthedocs.io/en/latest/grid.html#pyresample-image)). This may be helpful to take care of nodata values in the input datasets which might cause resampling artifa...pyresample supports masked arrays as inputs (see [here](https://pyresample.readthedocs.io/en/latest/grid.html#pyresample-image)). This may be helpful to take care of nodata values in the input datasets which might cause resampling artifacts at image edges.Daniel SchefflerDaniel Schefflerhttps://git.gfz-potsdam.de/EnMAP/sensormapgeo/-/issues/7AttributeError when using bilinear resampling in SensorMapGeometryTransformer...2021-11-26T00:24:58+01:00Daniel SchefflerAttributeError when using bilinear resampling in SensorMapGeometryTransformer.to_sensor_geometry().```python
1) ERROR: test_to_sensor_geometry (tests.test_sensormapgeo.Test_SensorMapGeometryTransformer)
----------------------------------------------------------------------
Traceback (most recent call last):
tests/test_sensormap...```python
1) ERROR: test_to_sensor_geometry (tests.test_sensormapgeo.Test_SensorMapGeometryTransformer)
----------------------------------------------------------------------
Traceback (most recent call last):
tests/test_sensormapgeo.py line 77 in test_to_sensor_geometry
dem_sensor_geo = SMGT.to_sensor_geometry(self.dem_map_geo,
sensormapgeo/transformer_2d.py line 398 in to_sensor_geometry
data_sensorgeo = self._resample(data, self.area_definition, self.swath_definition)
sensormapgeo/transformer_2d.py line 290 in _resample
result = resample_bilinear(data, source_geo_def, target_geo_def, **opts).astype(data.dtype)
/root/miniconda3/envs/ci_env/lib/python3.9/site-packages/pyresample/bilinear/_numpy_resampler.py line 103 in resample_bilinear
resampler.get_bil_info(kdtree_class=kdtree_class, nprocs=nprocs)
/root/miniconda3/envs/ci_env/lib/python3.9/site-packages/pyresample/bilinear/_base.py line 116 in get_bil_info
self._get_fractional_distances()
/root/miniconda3/envs/ci_env/lib/python3.9/site-packages/pyresample/bilinear/_base.py line 161 in _get_fractional_distances
out_x, out_y = self._get_output_xy()
/root/miniconda3/envs/ci_env/lib/python3.9/site-packages/pyresample/bilinear/_base.py line 171 in _get_output_xy
return _get_output_xy(self._target_geo_def)
/root/miniconda3/envs/ci_env/lib/python3.9/site-packages/pyresample/bilinear/_base.py line 266 in _get_output_xy
out_x, out_y = target_geo_def.get_proj_coords()
AttributeError: 'SwathDefinition' object has no attribute 'get_proj_coords'
```
This seems to be caused by an issue in pyresample: https://github.com/pytroll/pyresample/issues/325Daniel SchefflerDaniel Schefflerhttps://git.gfz-potsdam.de/EnMAP/sensormapgeo/-/issues/6Deadlock within SensorMapGeometryTransformer3D when running in multiprocessin...2020-10-02T17:57:09+02:00Daniel SchefflerDeadlock within SensorMapGeometryTransformer3D when running in multiprocessing for resampling algorithms 'near' and 'gauss'.Deadlock under Linux for 'near' and 'gauss' if mp_alg=='bands'.
This line hangs forever: https://gitext.gfz-potsdam.de/EnMAP/sensormapgeo/-/blob/master/sensormapgeo/transformer_3d.py#L228
When further traced down, the actual resampling...Deadlock under Linux for 'near' and 'gauss' if mp_alg=='bands'.
This line hangs forever: https://gitext.gfz-potsdam.de/EnMAP/sensormapgeo/-/blob/master/sensormapgeo/transformer_3d.py#L228
When further traced down, the actual resampling call within pyresample causes the deadlock: https://gitext.gfz-potsdam.de/EnMAP/sensormapgeo/-/blob/master/sensormapgeo/transformer_2d.py#L306
This seems to be caused by a GNU build of OpenMP used by pykdtree (dependency of pyresample).
Possible solution:
Force to use the LLVM build of OpenMP under Linux.Daniel SchefflerDaniel Schefflerhttps://git.gfz-potsdam.de/EnMAP/sensormapgeo/-/issues/5Reduce memory consumption of 3D transformation when running in multiprocessing.2020-08-07T18:35:56+02:00Daniel SchefflerReduce memory consumption of 3D transformation when running in multiprocessing.Transformation between sensor and map geometry currently needs a lot of memory for larger input data when running in multiprocessing. This needs to be revised.Transformation between sensor and map geometry currently needs a lot of memory for larger input data when running in multiprocessing. This needs to be revised.Daniel SchefflerDaniel Schefflerhttps://git.gfz-potsdam.de/EnMAP/sensormapgeo/-/issues/4VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences ...2020-08-07T12:39:14+02:00Daniel SchefflerVisibleDeprecationWarning: Creating an ndarray from ragged nested sequences ......\python\sensormapgeo\sensormapgeo\transformer_3d.py:229: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is depreca......\python\sensormapgeo\sensormapgeo\transformer_3d.py:229: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
band_inds = list(np.array(result)[:, -1])
https://gitext.gfz-potsdam.de/EnMAP/sensormapgeo/-/blob/master/sensormapgeo/transformer_3d.py#L229Daniel SchefflerDaniel Schefflerhttps://git.gfz-potsdam.de/EnMAP/sensormapgeo/-/issues/3NotADirectoryError due to race condition2020-08-07T12:22:49+02:00Daniel SchefflerNotADirectoryError due to race conditionNotADirectoryError at https://gitext.gfz-potsdam.de/EnMAP/sensormapgeo/-/blob/master/sensormapgeo/transformer_2d.py#L244
```
with TemporaryDirectory() as td:
[...]
ds_out = None
```
-> may raise a NotDirectoryError, possibly du...NotADirectoryError at https://gitext.gfz-potsdam.de/EnMAP/sensormapgeo/-/blob/master/sensormapgeo/transformer_2d.py#L244
```
with TemporaryDirectory() as td:
[...]
ds_out = None
```
-> may raise a NotDirectoryError, possibly due to a race condition between the tempfile directory deletion and the GDAL file deletionDaniel SchefflerDaniel Schefflerhttps://git.gfz-potsdam.de/EnMAP/sensormapgeo/-/issues/2Strip projection info2020-03-27T15:17:18+01:00Brenner SilvaStrip projection infoFollowing modification (".strip()" call) is suggested at **sensormapgeo.py, line 233**.
`projection=get_proj4info(proj=tgt_prj).strip(),`
Error: Without the above modification, srs.ExportToProj4() returns string with a blank character ...Following modification (".strip()" call) is suggested at **sensormapgeo.py, line 233**.
`projection=get_proj4info(proj=tgt_prj).strip(),`
Error: Without the above modification, srs.ExportToProj4() returns string with a blank character at the end,
which in turns leads to an error on parsing the projection string.