Commit c1097f32 authored by Daniel Scheffler's avatar Daniel Scheffler
Browse files

Replaced Python 2 compatible type hints by PEP 484 type hints. Revised...


Replaced Python 2 compatible type hints by PEP 484 type hints. Revised docstrings. Fixed a lot of Sphinx build warnings. Some code style improvements.

Signed-off-by: Daniel Scheffler's avatarDaniel Scheffler <danschef@gfz-potsdam.de>
parent 710a6870
Pipeline #20540 failed with stage
in 3 minutes and 44 seconds
This diff is collapsed.
This diff is collapsed.
...@@ -25,6 +25,7 @@ import collections ...@@ -25,6 +25,7 @@ import collections
import time import time
import warnings import warnings
import numpy as np import numpy as np
from typing import Union
# internal modules # internal modules
from geoarray import GeoArray from geoarray import GeoArray
...@@ -49,74 +50,78 @@ class DESHIFTER(object): ...@@ -49,74 +50,78 @@ class DESHIFTER(object):
See help(DESHIFTER) for documentation. See help(DESHIFTER) for documentation.
""" """
def __init__(self, im2shift, coreg_results, **kwargs): def __init__(self,
im2shift: Union[GeoArray, str],
coreg_results: dict,
**kwargs
) -> None:
"""Get an instance of DESHIFTER. """Get an instance of DESHIFTER.
:param (str | GeoArray) im2shift: :param im2shift:
path of an image to be de-shifted or alternatively a GeoArray object path of an image to be de-shifted or alternatively a GeoArray object
:param (dict) coreg_results : :param dict coreg_results:
the results of the co-registration as given by COREG.coreg_info or COREG_LOCAL.coreg_info the results of the co-registration as given by COREG.coreg_info or COREG_LOCAL.coreg_info
:key (int) path_out: :keyword int path_out:
/output/directory/filename for coregistered results /output/directory/filename for coregistered results
:key (str) fmt_out: :keyword str fmt_out:
raster file format for output file. ignored if path_out is None. can be any GDAL raster file format for output file. ignored if path_out is None. can be any GDAL
compatible raster file format (e.g. 'ENVI', 'GTIFF'; default: ENVI) compatible raster file format (e.g. 'ENVI', 'GTIFF'; default: ENVI)
:key (list) out_crea_options: :keyword list out_crea_options:
GDAL creation options for the output image, e.g., ["QUALITY=20", "REVERSIBLE=YES", "WRITE_METADATA=YES"] GDAL creation options for the output image, e.g., ["QUALITY=20", "REVERSIBLE=YES", "WRITE_METADATA=YES"]
:key (int) band2process: :keyword int band2process:
The index of the band to be processed within the given array (starts with 1), The index of the band to be processed within the given array (starts with 1),
default = None (all bands are processed) default = None (all bands are processed)
:key (int, float) nodata: :keyword float nodata:
no data value of the image to be de-shifted no data value of the image to be de-shifted
:key (float) out_gsd: :keyword float out_gsd:
output pixel size in units of the reference coordinate system (default = pixel size of the input array), output pixel size in units of the reference coordinate system (default = pixel size of the input array),
given values are overridden by match_gsd=True given values are overridden by match_gsd=True
:key (bool) align_grids: :keyword bool align_grids:
True: align the input coordinate grid to the reference (does not affect the output pixel size as long as True: align the input coordinate grid to the reference (does not affect the output pixel size as long as
input and output pixel sizes are compatible (5:30 or 10:30 but not 4:30), default = False input and output pixel sizes are compatible (5:30 or 10:30 but not 4:30), default = False
:key (bool) match_gsd: :keyword bool match_gsd:
True: match the input pixel size to the reference pixel size, default = False True: match the input pixel size to the reference pixel size, default = False
:key (list) target_xyGrid: :keyword list target_xyGrid:
a list with an x-grid and a y-grid like [[15,45], [15,45]]. a list with an x-grid and a y-grid like [[15,45], [15,45]].
This overrides 'out_gsd', 'align_grids' and 'match_gsd'. This overrides 'out_gsd', 'align_grids' and 'match_gsd'.
:key (int) min_points_local_corr: :keyword int min_points_local_corr:
number of valid tie points, below which a global shift correction is performed instead of a local number of valid tie points, below which a global shift correction is performed instead of a local
correction (global X/Y shift is then computed as the mean shift of the remaining points) correction (global X/Y shift is then computed as the mean shift of the remaining points)
(default: 5 tie points) (default: 5 tie points)
:key (str) resamp_alg: :keyword str resamp_alg:
the resampling algorithm to be used if neccessary the resampling algorithm to be used if neccessary
(valid algorithms: nearest, bilinear, cubic, cubic_spline, lanczos, average, mode, max, min, med, q1, q3) (valid algorithms: nearest, bilinear, cubic, cubic_spline, lanczos, average, mode, max, min, med, q1, q3)
:key (bool) cliptoextent: :keyword bool cliptoextent:
True: clip the input image to its actual bounds while deleting possible no data areas outside of the actual True: clip the input image to its actual bounds while deleting possible no data areas outside of the actual
bounds, default = False bounds, default = False
:key (list) clipextent: :keyword list clipextent:
xmin, ymin, xmax, ymax - if given the calculation of the actual bounds is skipped. xmin, ymin, xmax, ymax - if given the calculation of the actual bounds is skipped.
The given coordinates are automatically snapped to the output grid. The given coordinates are automatically snapped to the output grid.
:key (int) CPUs: :keyword int CPUs:
number of CPUs to use (default: None, which means 'all CPUs available') number of CPUs to use (default: None, which means 'all CPUs available')
:key (bool) progress: :keyword bool progress:
show progress bars (default: True) show progress bars (default: True)
:key (bool) v: :keyword bool v:
verbose mode (default: False) verbose mode (default: False)
:key (bool) q: :keyword bool q:
quiet mode (default: False) quiet mode (default: False)
""" """
# private attributes # private attributes
...@@ -305,9 +310,7 @@ class DESHIFTER(object): ...@@ -305,9 +310,7 @@ class DESHIFTER(object):
ymax = find_nearest(self.out_grid[1], ymax, roundAlg='off', extrapolate=True, tolerance=y_tol) ymax = find_nearest(self.out_grid[1], ymax, roundAlg='off', extrapolate=True, tolerance=y_tol)
return xmin, ymin, xmax, ymax return xmin, ymin, xmax, ymax
def correct_shifts(self): def correct_shifts(self) -> collections.OrderedDict:
# type: () -> collections.OrderedDict
if not self.q: if not self.q:
print('Correcting geometric shifts...') print('Correcting geometric shifts...')
...@@ -424,14 +427,18 @@ class DESHIFTER(object): ...@@ -424,14 +427,18 @@ class DESHIFTER(object):
return deshift_results return deshift_results
def deshift_image_using_coreg_info(im2shift, coreg_results, path_out=None, fmt_out='ENVI', q=False): def deshift_image_using_coreg_info(im2shift: Union[GeoArray, str],
coreg_results: dict,
path_out: str = None,
fmt_out: str = 'ENVI',
q: bool = False):
"""Correct a geometrically distorted image using previously calculated coregistration info. """Correct a geometrically distorted image using previously calculated coregistration info.
This function can be used for example to correct spatial shifts of mask files using the same transformation This function can be used for example to correct spatial shifts of mask files using the same transformation
parameters that have been used to correct their source images. parameters that have been used to correct their source images.
:param im2shift: <path,GeoArray> path of an image to be de-shifted or alternatively a GeoArray object :param im2shift: path of an image to be de-shifted or alternatively a GeoArray object
:param coreg_results: <dict> the results of the co-registration as given by COREG.coreg_info or :param coreg_results: the results of the co-registration as given by COREG.coreg_info or
COREG_LOCAL.coreg_info respectively COREG_LOCAL.coreg_info respectively
:param path_out: /output/directory/filename for coregistered results. If None, no output is written - only :param path_out: /output/directory/filename for coregistered results. If None, no output is written - only
the shift corrected results are returned. the shift corrected results are returned.
......
...@@ -26,6 +26,7 @@ import multiprocessing ...@@ -26,6 +26,7 @@ import multiprocessing
import os import os
import warnings import warnings
import time import time
from typing import Optional, Union
# custom # custom
from osgeo import gdal from osgeo import gdal
...@@ -47,8 +48,8 @@ from .CoReg import GeoArray_CoReg # noqa F401 # flake8 issue ...@@ -47,8 +48,8 @@ from .CoReg import GeoArray_CoReg # noqa F401 # flake8 issue
__author__ = 'Daniel Scheffler' __author__ = 'Daniel Scheffler'
global_shared_imref = None global_shared_imref: Optional[Union[GeoArray, str]] = None
global_shared_im2shift = None global_shared_im2shift: Optional[Union[GeoArray, str]] = None
def mp_initializer(imref, imtgt): def mp_initializer(imref, imtgt):
...@@ -74,44 +75,74 @@ class Tie_Point_Grid(object): ...@@ -74,44 +75,74 @@ class Tie_Point_Grid(object):
See help(Tie_Point_Grid) for documentation! See help(Tie_Point_Grid) for documentation!
""" """
def __init__(self, COREG_obj, grid_res, max_points=None, outFillVal=-9999, resamp_alg_calc='cubic', def __init__(self,
tieP_filter_level=3, outlDetect_settings=None, dir_out=None, CPUs=None, progress=True, v=False, COREG_obj: COREG,
q=False): grid_res: float,
max_points: int = None,
outFillVal: int = -9999,
resamp_alg_calc: str = 'cubic',
tieP_filter_level: int = 3,
outlDetect_settings: dict = None,
dir_out: str = None,
CPUs: int = None,
progress: bool = True,
v: bool = False,
q: bool = False):
"""Get an instance of the 'Tie_Point_Grid' class. """Get an instance of the 'Tie_Point_Grid' class.
:param COREG_obj(object): an instance of COREG class :param COREG_obj:
:param grid_res: grid resolution in pixels of the target image (x-direction) an instance of COREG class
:param max_points(int): maximum number of points used to find coregistration tie points
NOTE: Points are selected randomly from the given point grid (specified by :param grid_res:
'grid_res'). If the point does not provide enough points, all available points grid resolution in pixels of the target image (x-direction)
are chosen.
:param outFillVal(int): if given the generated tie points grid is filled with this value in case :param max_points:
no match could be found during co-registration (default: -9999) maximum number of points used to find coregistration tie points
:param resamp_alg_calc(str) the resampling algorithm to be used for all warping processes during calculation
of spatial shifts NOTE: Points are selected randomly from the given point grid (specified by 'grid_res'). If the point does
(valid algorithms: nearest, bilinear, cubic, cubic_spline, lanczos, average, not provide enough points, all available points are chosen.
mode, max, min, med, q1, q3)
:param outFillVal:
if given the generated tie points grid is filled with this value in case no match could be found during
co-registration (default: -9999)
:param resamp_alg_calc:
the resampling algorithm to be used for all warping processes during calculation of spatial shifts
(valid algorithms: nearest, bilinear, cubic, cubic_spline, lanczos, average, mode, max, min, med, q1, q3)
default: cubic (highly recommended) default: cubic (highly recommended)
:param tieP_filter_level(int): filter tie points used for shift correction in different levels (default: 3).
:param tieP_filter_level:
filter tie points used for shift correction in different levels (default: 3).
NOTE: lower levels are also included if a higher level is chosen NOTE: lower levels are also included if a higher level is chosen
- Level 0: no tie point filtering - Level 0: no tie point filtering
- Level 1: Reliablity filtering - filter all tie points out that have a low - Level 1: Reliablity filtering
reliability according to internal tests - filter all tie points out that have a low reliability according to internal tests
- Level 2: SSIM filtering - filters all tie points out where shift - Level 2: SSIM filtering
correction does not increase image similarity within matching window - filters all tie points out where shift correction does not increase image similarity within
(measured by mean structural similarity index) matching window (measured by mean structural similarity index)
- Level 3: RANSAC outlier detection - Level 3: RANSAC outlier detection
:param outlDetect_settings a dictionary with the settings to be passed to
arosics.TiePointGrid.Tie_Point_Refiner. Available keys: min_reliability, :param outlDetect_settings:
rs_max_outlier, rs_tolerance, rs_max_iter, rs_exclude_previous_outliers, a dictionary with the settings to be passed to arosics.TiePointGrid.Tie_Point_Refiner.
Available keys: min_reliability, rs_max_outlier, rs_tolerance, rs_max_iter, rs_exclude_previous_outliers,
rs_timeout, q. See documentation there. rs_timeout, q. See documentation there.
:param dir_out(str): output directory to be used for all outputs if nothing else is given
to the individual methods :param dir_out:
:param CPUs(int): number of CPUs to use during calculation of tie points grid output directory to be used for all outputs if nothing else is given to the individual methods
:param CPUs:
number of CPUs to use during calculation of tie points grid
(default: None, which means 'all CPUs available') (default: None, which means 'all CPUs available')
:param progress(bool): show progress bars (default: True)
:param v(bool): verbose mode (default: False) :param progress:
:param q(bool): quiet mode (default: False) show progress bars (default: True)
:param v:
verbose mode (default: False)
:param q:
quiet mode (default: False)
""" """
if not isinstance(COREG_obj, COREG): if not isinstance(COREG_obj, COREG):
raise ValueError("'COREG_obj' must be an instance of COREG class.") raise ValueError("'COREG_obj' must be an instance of COREG class.")
...@@ -387,7 +418,7 @@ class Tie_Point_Grid(object): ...@@ -387,7 +418,7 @@ class Tie_Point_Grid(object):
if not self.q: if not self.q:
print("Calculating tie point grid (%s points) on 1 CPU core..." % len(GDF)) print("Calculating tie point grid (%s points) on 1 CPU core..." % len(GDF))
results = np.empty((len(geomPoints), 14), np.object) results = np.empty((len(geomPoints), 14), object)
bar = ProgressBar(prefix='\tprogress:') bar = ProgressBar(prefix='\tprogress:')
for i, coreg_kwargs in enumerate(list_coreg_kwargs): for i, coreg_kwargs in enumerate(list_coreg_kwargs):
if self.progress: if self.progress:
...@@ -431,8 +462,7 @@ class Tie_Point_Grid(object): ...@@ -431,8 +462,7 @@ class Tie_Point_Grid(object):
return self.CoRegPoints_table return self.CoRegPoints_table
def calc_rmse(self, include_outliers=False): def calc_rmse(self, include_outliers: bool = False) -> float:
# type: (bool) -> float
"""Calculate root mean square error of absolute shifts from the tie point grid. """Calculate root mean square error of absolute shifts from the tie point grid.
:param include_outliers: whether to include tie points that have been marked as false-positives (if present) :param include_outliers: whether to include tie points that have been marked as false-positives (if present)
...@@ -448,8 +478,10 @@ class Tie_Point_Grid(object): ...@@ -448,8 +478,10 @@ class Tie_Point_Grid(object):
return np.sqrt(sum(shifts_sq) / len(shifts_sq)) return np.sqrt(sum(shifts_sq) / len(shifts_sq))
def calc_overall_ssim(self, include_outliers=False, after_correction=True): def calc_overall_ssim(self,
# type: (bool, bool) -> float include_outliers: bool = False,
after_correction: bool = True
) -> float:
"""Calculate the median value of all SSIM values contained in tie point grid. """Calculate the median value of all SSIM values contained in tie point grid.
:param include_outliers: whether to include tie points that have been marked as false-positives :param include_outliers: whether to include tie points that have been marked as false-positives
...@@ -466,9 +498,16 @@ class Tie_Point_Grid(object): ...@@ -466,9 +498,16 @@ class Tie_Point_Grid(object):
return float(np.median(ssim_col)) return float(np.median(ssim_col))
def plot_shift_distribution(self, include_outliers=True, unit='m', interactive=False, figsize=None, xlim=None, def plot_shift_distribution(self,
ylim=None, fontsize=12, title='shift distribution'): include_outliers: bool = True,
# type: (bool, str, bool, tuple, list, list, int, str) -> tuple unit: str = 'm',
interactive: bool = False,
figsize: tuple = None,
xlim: list = None,
ylim: list = None,
fontsize: int = 12,
title: str = 'shift distribution'
) -> tuple:
"""Create a 2D scatterplot containing the distribution of calculated X/Y-shifts. """Create a 2D scatterplot containing the distribution of calculated X/Y-shifts.
:param include_outliers: whether to include tie points that have been marked as false-positives :param include_outliers: whether to include tie points that have been marked as false-positives
...@@ -647,8 +686,11 @@ class Tie_Point_Grid(object): ...@@ -647,8 +686,11 @@ class Tie_Point_Grid(object):
lines[i, :] = self.CoRegPoints_table[self.CoRegPoints_table['POINT_ID'] == PID] lines[i, :] = self.CoRegPoints_table[self.CoRegPoints_table['POINT_ID'] == PID]
return lines return lines
def to_PointShapefile(self, path_out=None, skip_nodata=True, skip_nodata_col='ABS_SHIFT'): def to_PointShapefile(self,
# type: (str, bool, str) -> None path_out: str = None,
skip_nodata: bool = True,
skip_nodata_col: str = 'ABS_SHIFT'
) -> None:
"""Write the calculated tie points grid to a point shapefile (e.g., for visualization by a GIS software). """Write the calculated tie points grid to a point shapefile (e.g., for visualization by a GIS software).
NOTE: The shapefile uses Tie_Point_Grid.CoRegPoints_table as attribute table. NOTE: The shapefile uses Tie_Point_Grid.CoRegPoints_table as attribute table.
...@@ -709,17 +751,16 @@ class Tie_Point_Grid(object): ...@@ -709,17 +751,16 @@ class Tie_Point_Grid(object):
prj=self.COREG_obj.shift.prj, prj=self.COREG_obj.shift.prj,
attrDict=attr_dicts) attrDict=attr_dicts)
def to_vectorfield(self, path_out=None, fmt=None, mode='md'): def to_vectorfield(self, path_out: str = None, fmt: str = None, mode: str = 'md') -> GeoArray:
# type: (str, str, str) -> GeoArray
"""Save the calculated X-/Y-shifts to a 2-band raster file that can be used to visualize a vectorfield. """Save the calculated X-/Y-shifts to a 2-band raster file that can be used to visualize a vectorfield.
NOTE: For example ArcGIS is able to visualize such 2-band raster files as a vectorfield. NOTE: For example ArcGIS is able to visualize such 2-band raster files as a vectorfield.
:param path_out: <str> the output path. If not given, it is automatically defined. :param path_out: the output path. If not given, it is automatically defined.
:param fmt: <str> output raster format string :param fmt: output raster format string
:param mode: <str> The mode how the output is written ('uv' or 'md'; default: 'md') :param mode: The mode how the output is written ('uv' or 'md'; default: 'md')
'uv': outputs X-/Y shifts - 'uv': outputs X-/Y shifts
'md': outputs magnitude and direction - 'md': outputs magnitude and direction
""" """
assert mode in ['uv', 'md'], "'mode' must be either 'uv' (outputs X-/Y shifts) or 'md' " \ assert mode in ['uv', 'md'], "'mode' must be either 'uv' (outputs X-/Y shifts) or 'md' " \
"(outputs magnitude and direction)'. Got %s." % mode "(outputs magnitude and direction)'. Got %s." % mode
...@@ -828,23 +869,29 @@ class Tie_Point_Grid(object): ...@@ -828,23 +869,29 @@ class Tie_Point_Grid(object):
class Tie_Point_Refiner(object): class Tie_Point_Refiner(object):
"""A class for performing outlier detection.""" """A class for performing outlier detection."""
def __init__(self, GDF, min_reliability=60, rs_max_outlier=10, rs_tolerance=2.5, rs_max_iter=15, def __init__(self, GDF,
rs_exclude_previous_outliers=True, rs_timeout=20, q=False): min_reliability=60,
rs_max_outlier: float = 10,
rs_tolerance: float = 2.5,
rs_max_iter: int = 15,
rs_exclude_previous_outliers: bool = True,
rs_timeout: float = 20,
q: bool = False):
"""Get an instance of Tie_Point_Refiner. """Get an instance of Tie_Point_Refiner.
:param GDF: GeoDataFrame like TiePointGrid.CoRegPoints_table containing all tie :param GDF: GeoDataFrame like TiePointGrid.CoRegPoints_table containing all tie
points to be filtered and the corresponding metadata points to be filtered and the corresponding metadata
:param min_reliability: <float, int> minimum threshold for previously computed tie X/Y shift :param min_reliability: minimum threshold for previously computed tie X/Y shift
reliability (default: 60%) reliability (default: 60%)
:param rs_max_outlier: <float, int> RANSAC: maximum percentage of outliers to be detected :param rs_max_outlier: RANSAC: maximum percentage of outliers to be detected
(default: 10%) (default: 10%)
:param rs_tolerance: <float, int> RANSAC: percentage tolerance for max_outlier_percentage :param rs_tolerance: RANSAC: percentage tolerance for max_outlier_percentage
(default: 2.5%) (default: 2.5%)
:param rs_max_iter: <int> RANSAC: maximum iterations for finding the best RANSAC threshold :param rs_max_iter: RANSAC: maximum iterations for finding the best RANSAC threshold
(default: 15) (default: 15)
:param rs_exclude_previous_outliers: <bool> RANSAC: whether to exclude points that have been flagged as :param rs_exclude_previous_outliers: RANSAC: whether to exclude points that have been flagged as
outlier by earlier filtering (default:True) outlier by earlier filtering (default:True)
:param rs_timeout: <float, int> RANSAC: timeout for iteration loop in seconds (default: 20) :param rs_timeout: RANSAC: timeout for iteration loop in seconds (default: 20)
:param q: :param q:
""" """
...@@ -864,12 +911,13 @@ class Tie_Point_Refiner(object): ...@@ -864,12 +911,13 @@ class Tie_Point_Refiner(object):
:param level: tie point filter level (default: 3). :param level: tie point filter level (default: 3).
NOTE: lower levels are also included if a higher level is chosen NOTE: lower levels are also included if a higher level is chosen
- Level 0: no tie point filtering - Level 0: no tie point filtering
- Level 1: Reliablity filtering - filter all tie points out that have a low - Level 1: Reliablity filtering
reliability according to internal tests - filter all tie points out that have a low reliability according to internal tests
- Level 2: SSIM filtering - filters all tie points out where shift - Level 2: SSIM filtering
correction does not increase image similarity within matching window - filters all tie points out where shift correction does not increase image similarity within
(measured by mean structural similarity index) matching window (measured by mean structural similarity index)
- Level 3: RANSAC outlier detection - Level 3: RANSAC outlier detection
:return: :return:
......
...@@ -29,7 +29,8 @@ import time ...@@ -29,7 +29,8 @@ import time
import sys import sys
import warnings import warnings
import argparse import argparse
from matplotlib import pyplot as plt
plt.plot
from arosics import COREG, COREG_LOCAL, __version__ from arosics import COREG, COREG_LOCAL, __version__
__author__ = "Daniel Scheffler" __author__ = "Daniel Scheffler"
......
.. include:: ../README.rst
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment