README.rst 6.79 KB
Newer Older
Daniel Scheffler's avatar
Daniel Scheffler committed
1
2
3
=========================================================================================
gms_preprocessing - Spatial and spectral homogenization of satellite remote sensing data.
=========================================================================================
4

Daniel Scheffler's avatar
Daniel Scheffler committed
5
6
7
The goal of the gms_preprocessing Python library is to provide a fully automatic
pre-precessing pipeline for spatial and spectral fusion (i.e., homogenization)
of multispectral satellite image data. Currently it offers compatibility to
Daniel Scheffler's avatar
Daniel Scheffler committed
8
Landsat-5, Landsat-7, Landsat-8, Sentinel-2A and Sentinel-2B.
9

10
11
* Free software: GNU General Public License v3 or later (GPLv3+) (`license details <https://git.gfz-potsdam.de/geomultisens/gms_preprocessing/blob/master/LICENSE>`_)
* Documentation: https://geomultisens.git-pages.gfz-potsdam.de/gms_preprocessing/doc/
Daniel Scheffler's avatar
Daniel Scheffler committed
12
* Code history: Release notes for the current and earlier versions of gms_preprocessing can be found `here <./HISTORY.rst>`_.
Daniel Scheffler's avatar
Daniel Scheffler committed
13
* OS compatibility: Linux
Daniel Scheffler's avatar
Daniel Scheffler committed
14
15
16
17
18


Status
------

19
20
21
22
.. image:: https://git.gfz-potsdam.de/geomultisens/gms_preprocessing/badges/master/pipeline.svg
        :target: https://git.gfz-potsdam.de/geomultisens/gms_preprocessing/commits/master
.. image:: https://git.gfz-potsdam.de/geomultisens/gms_preprocessing/badges/master/coverage.svg
        :target: https://geomultisens.git-pages.gfz-potsdam.de/gms_preprocessing/coverage/
23
.. image:: https://img.shields.io/pypi/v/gms_preprocessing.svg
Daniel Scheffler's avatar
Daniel Scheffler committed
24
        :target: https://pypi.python.org/pypi/gms_preprocessing
25
26
27
.. image:: https://img.shields.io/conda/vn/conda-forge/gms_preprocessing.svg
        :target: https://anaconda.org/conda-forge/gms_preprocessing
.. image:: https://img.shields.io/pypi/l/gms_preprocessing.svg
28
        :target: https://git.gfz-potsdam.de/danschef/gms_preprocessing/blob/master/LICENSE
29
.. image:: https://img.shields.io/pypi/pyversions/gms_preprocessing.svg
Daniel Scheffler's avatar
Daniel Scheffler committed
30
        :target: https://img.shields.io/pypi/pyversions/gms_preprocessing.svg
31

32
See also the latest coverage_ report and the nosetests_ HTML report.
33
34
35
36

Features
--------

Daniel Scheffler's avatar
Daniel Scheffler committed
37
38
39
40
41
Level-1 processing:
^^^^^^^^^^^^^^^^^^^

* data import and  metadata homogenization (compatibility: Landsat-5/7/8, Sentinel-2A/2B)
* equalization of acquisition- and illumination geometry
42
43
* atmospheric correction (using `SICOR <https://git.gfz-potsdam.de/EnMAP/sicor>`_)
* correction of geometric errors (using `AROSICS <https://git.gfz-potsdam.de/danschef/arosics>`_)
Daniel Scheffler's avatar
Daniel Scheffler committed
44
45
46
47
48

Level-2 processing:
^^^^^^^^^^^^^^^^^^^

* spatial homogenization
49
* spectral homogenization (using `SpecHomo <https://git.gfz-potsdam.de/geomultisens/spechomo>`_)
Daniel Scheffler's avatar
Daniel Scheffler committed
50
51
52
53
54
55
56
57
58
59
60
* estimation of accuracy layers

=> application oriented analysis dataset


Getting started
---------------

Usage via WebApp
^^^^^^^^^^^^^^^^

Daniel Scheffler's avatar
Daniel Scheffler committed
61
62
63
64
The recommended way to use gms_preprocessing is to setup the WebApp (see the
gms-vis_ repository) providing a UI for GeoMultiSens. Using this UI, existing
satellite data can be explored, filtered and selected for processing. New data
homogenization jobs (using gms_preprocessing) can be defined and started. All
Daniel Scheffler's avatar
Daniel Scheffler committed
65
66
67
configuration parameters of gms_preprocessing are accessible in the UI.


68
.. image:: https://git.gfz-potsdam.de/geomultisens/gms_preprocessing/raw/master/docs/images/webapp_screenshot_900x497.png
Daniel Scheffler's avatar
Daniel Scheffler committed
69
70
71
    :width: 900 px
    :height: 497 px
    :scale: 100 %
72
    :alt: WebApp Screenshot
Daniel Scheffler's avatar
Daniel Scheffler committed
73
74
75
76
77


Usage via console interface
^^^^^^^^^^^^^^^^^^^^^^^^^^^

Daniel Scheffler's avatar
Daniel Scheffler committed
78
Homogenization jobs can also be created and started using the command line
79
interface. Documentation can be found `here <https://geomultisens.git-pages.gfz-potsdam.de/gms_preprocessing/doc/usage.html#gms-preprocessing-command-line-interface>`__.
Daniel Scheffler's avatar
Daniel Scheffler committed
80

Daniel Scheffler's avatar
Daniel Scheffler committed
81
82
Here is a small example:

83
.. code:: bash
Daniel Scheffler's avatar
Daniel Scheffler committed
84

Daniel Scheffler's avatar
Daniel Scheffler committed
85
86
    # start the job with the ID 123456 and override default configuration with the given one.
    >>> run_gms.py jobid 123456 --json_config /path/to/my/config.json
Daniel Scheffler's avatar
Daniel Scheffler committed
87

88
There is a default configuration file, called `options_default.json <https://git.gfz-potsdam.de/geomultisens/gms_preprocessing/blob/master/gms_preprocessing/options/options_default.json>`_.
Daniel Scheffler's avatar
Daniel Scheffler committed
89
This file contains the documentation for all the available configuration
Daniel Scheffler's avatar
Daniel Scheffler committed
90
91
92
93
94
95
parameters.


Usage via Python API
^^^^^^^^^^^^^^^^^^^^

Daniel Scheffler's avatar
Daniel Scheffler committed
96
There is also a Python API that allows to setup and start homogenization jobs
Daniel Scheffler's avatar
Daniel Scheffler committed
97
98
99
100
101
102
103
by a Python function call.

This is an example:

.. code:: python

    from gms_preprocessing import ProcessController
Daniel Scheffler's avatar
Daniel Scheffler committed
104

Daniel Scheffler's avatar
Daniel Scheffler committed
105
106
107
108
    configuration = dict(
        db_host='localhost',
        CPUs=20
        )
Daniel Scheffler's avatar
Daniel Scheffler committed
109

110
    PC = ProcessController(job_ID=123456, **configuration)
Daniel Scheffler's avatar
Daniel Scheffler committed
111
    PC.run_all_processors()
Daniel Scheffler's avatar
Daniel Scheffler committed
112

113
Possible configuration arguments can be found `here <https://git.gfz-potsdam.de/geomultisens/gms_preprocessing/blob/master/gms_preprocessing/options/options_schema.py>`__.
Daniel Scheffler's avatar
Daniel Scheffler committed
114

115

116
117
118
119
History / Changelog
-------------------

You can find the protocol of recent changes in the gms_preprocessing package
120
`here <https://git.gfz-potsdam.de/geomultisens/gms_preprocessing/-/blob/master/HISTORY.rst>`__.
121
122


Daniel Scheffler's avatar
Daniel Scheffler committed
123
124
License
-------
Daniel Scheffler's avatar
Daniel Scheffler committed
125

Daniel Scheffler's avatar
Daniel Scheffler committed
126
127
gms_preprocessing - Spatial and spectral homogenization of satellite remote sensing data.

128
Copyright 2020 Helmholtz Centre Potsdam GFZ German Research Centre for Geosciences, Potsdam, Germany
Daniel Scheffler's avatar
Daniel Scheffler committed
129

Daniel Scheffler's avatar
Daniel Scheffler committed
130
131
132
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
Daniel Scheffler's avatar
Daniel Scheffler committed
133
134
(at your option) any later version. This program is distributed in the hope
that it will be useful, but WITHOUT ANY WARRANTY; without even the implied
Daniel Scheffler's avatar
Daniel Scheffler committed
135
warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
Daniel Scheffler's avatar
Daniel Scheffler committed
136
137
GNU General Public License for more details. You should have received a copy
of the GNU General Public License along with this program.
Daniel Scheffler's avatar
Daniel Scheffler committed
138
If not, see <http://www.gnu.org/licenses/>.
139
140


Daniel Scheffler's avatar
Daniel Scheffler committed
141
142
143
144
145
Contact
-------

.. line-block::

Daniel Scheffler's avatar
Daniel Scheffler committed
146
147
    Daniel Scheffler
    eMail: daniel.scheffler@gfz-potsdam.de
Daniel Scheffler's avatar
Daniel Scheffler committed
148

Daniel Scheffler's avatar
Daniel Scheffler committed
149
150
151
152
153
    Helmholtz Centre Potsdam GFZ German Research Centre for Geoscienes
    Section 1.4 Remote Sensing
    Telegrafenberg
    14473 Potsdam
    Germany
Daniel Scheffler's avatar
Daniel Scheffler committed
154
155


156
Credits
157
-------
158

159
160
161
162
The development of the gms_preprocessing package was funded by the German Federal Ministry of Education and Research
(BMBF, project grant code: 01 IS 14 010 A-C).

The package was created with Cookiecutter_ and the `audreyr/cookiecutter-pypackage`_ project template.
163

164
Landsat-5/7/8 satellite data and SRTM/ASTER digital elevation models have been provided by the US Geological
Daniel Scheffler's avatar
Daniel Scheffler committed
165
166
Survey. Sentinel-2 data have been provided by ESA.

167
168
.. _Cookiecutter: https://github.com/audreyr/cookiecutter
.. _`audreyr/cookiecutter-pypackage`: https://github.com/audreyr/cookiecutter-pypackage
169
170
.. _coverage: https://geomultisens.git-pages.gfz-potsdam.de/gms_preprocessing/coverage/
.. _nosetests: https://geomultisens.git-pages.gfz-potsdam.de/gms_preprocessing/nosetests_reports/nosetests.html
171
.. _conda: https://conda.io/docs/
Daniel Scheffler's avatar
Daniel Scheffler committed
172
.. _redis-server: https://www.rosehosting.com/blog/how-to-install-configure-and-use-redis-on-ubuntu-16-04/
173
.. _gms-vis: https://git.gfz-potsdam.de/geomultisens/gms-vis