README.md 6.5 KB
Newer Older
Hannes Diedrich's avatar
Hannes Diedrich committed
1
# GTS2 Sentinel-2 downloader
Hannes Diedrich's avatar
Hannes Diedrich committed
2
3
Status
------
Hannes Diedrich's avatar
Hannes Diedrich committed
4
5
[![pipeline status](https://gitext.gfz-potsdam.de/gts2/gts2_client/badges/master/pipeline.svg)](https://gitext.gfz-potsdam.de/gts2/gts2_client/commits/master)
[![coverage report](https://gitext.gfz-potsdam.de/gts2/gts2_client/badges/master/coverage.svg)](https://gitext.gfz-potsdam.de/gts2/gts2_client/commits/master)
Hannes Diedrich's avatar
Hannes Diedrich committed
6
[![AUR](https://img.shields.io/aur/license/yaourt.svg)](https://gitext.gfz-potsdam.de/gts2/gts2_client/blob/master/LICENCE)
Hannes Diedrich's avatar
Hannes Diedrich committed
7
8


Hannes Diedrich's avatar
Hannes Diedrich committed
9
10
[Coverage report](http://gts2.gitext.gfz-potsdam.de/gts2_client/coverage/)

Hannes Diedrich's avatar
Hannes Diedrich committed
11
12
13
14
## Description
This program package downloads Sentinel-2 data from the GTS2
(GFZ Time Series System for Sentinel-2) cloud for a area of interest,
time of interest and wanted band combination and saves them to geotiff (.tiff)
Niklas Bohn's avatar
Niklas Bohn committed
15
 or alternatively as .json file or as netcdf (.nc) file.
Hannes Diedrich's avatar
Hannes Diedrich committed
16

Hannes Diedrich's avatar
Hannes Diedrich committed
17
## Requirements
Hannes Diedrich's avatar
Hannes Diedrich committed
18
1. Access to GTS2 API (username, password, port)
19
20
2. Python3
3. Python packages - check the requirements in [setup.py](./setup.py)
Hannes Diedrich's avatar
Hannes Diedrich committed
21

22
23
## Download package:
Clone the repository with:
Niklas Bohn's avatar
Niklas Bohn committed
24
<code>git clone https://gitext.gfz-potsdam.de/gts2/gts2_client.git
Hannes Diedrich's avatar
Hannes Diedrich committed
25
</code>
26

Hannes Diedrich's avatar
Hannes Diedrich committed
27
## Installation
Hannes Diedrich's avatar
Hannes Diedrich committed
28
29
### ... is easy (as long your python is compatible):
* Make sure your system meets all the requirements (see above)
Hannes Diedrich's avatar
Hannes Diedrich committed
30
* Install the package by running:
31
32
33
34
```
make install
make clean
```
Hannes Diedrich's avatar
Hannes Diedrich committed
35

Hannes Diedrich's avatar
Hannes Diedrich committed
36
### Install a compatible python:
Hannes Diedrich's avatar
Hannes Diedrich committed
37
The following instruction is only valid for Linux distributions but can be adapted on Windows machines.
Hannes Diedrich's avatar
Hannes Diedrich committed
38
39
40
Everything is based on bash shell.

#### EITHER: Use install script and follow instructions:
41
42
43
44
Run:
```
make install_conda_gts2_client
```
Hannes Diedrich's avatar
Hannes Diedrich committed
45

46
After providing the installation path, miniconda and all necessary packages as well as the gts2_client will be installed.
47
At the script will provide instructions which PATHs to add to your .bashrc.
Hannes Diedrich's avatar
Hannes Diedrich committed
48

Hannes Diedrich's avatar
Hannes Diedrich committed
49
#### OR: Per hand (expert modus):
Hannes Diedrich's avatar
Hannes Diedrich committed
50
* Download and install at least Miniconda with default settings:
Hannes Diedrich's avatar
Hannes Diedrich committed
51
52
```
wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
Hannes Diedrich's avatar
Hannes Diedrich committed
53
54
55
56
chmod 755 Miniconda3-latest-Linux-x86_64.sh
./Miniconda3-latest-Linux-x86_64.sh -b
rm -f Miniconda3-latest-Linux-x86_64.sh
```
Hannes Diedrich's avatar
Hannes Diedrich committed
57
* Install necessary packages for gts2_client using **gts2_client_conda_install.yml**:
Hannes Diedrich's avatar
Hannes Diedrich committed
58
59
60
`conda env update -f gts2_client_conda_install.yml`


Hannes Diedrich's avatar
Hannes Diedrich committed
61
### Finally:
Hannes Diedrich's avatar
Hannes Diedrich committed
62
63
Create a credentials file in your home directory **credentials_gts2_client**
that contains the following structure:
Hannes Diedrich's avatar
Hannes Diedrich committed
64

Hannes Diedrich's avatar
Hannes Diedrich committed
65
```bash
Hannes Diedrich's avatar
Hannes Diedrich committed
66
{"user": "",
67
"password": "",
Hannes Diedrich's avatar
Hannes Diedrich committed
68
"port": ""}
69
```
Hannes Diedrich's avatar
Hannes Diedrich committed
70
71

Please fill in your credentials (provided by GFZ Potsdam, please contact gts2@gfz-potsdam.de).
Hannes Diedrich's avatar
Hannes Diedrich committed
72

Hannes Diedrich's avatar
Hannes Diedrich committed
73
### If you want to use docker
Hannes Diedrich's avatar
Hannes Diedrich committed
74
75
76
77
78
79
80
81
**!! Attention: Expert mode !! root access to computer is needed.**
You only have to run **docker_gts2_client/build_run_gts2_client_docker.sh**
It creates an image that is based on a Centos:7, performs all needed installation steps and starts a container.
You end up with a shell where you can run `gts2_client`.
The building of the image can take some time (up to 30 Minutes), but once it is done,
**docker_gts2_client/build_run_gts2_client_docker.sh** skips the build and only starts the container.


Hannes Diedrich's avatar
Hannes Diedrich committed
82
83
## Usage

84
### As command line tool:
Hannes Diedrich's avatar
Hannes Diedrich committed
85
```bash
86
gts2_client required_arguments [optional_arguments]
Hannes Diedrich's avatar
Hannes Diedrich committed
87
88
89
```

The list of arguments including their default values can be called from the command line with:
Hannes Diedrich's avatar
Hannes Diedrich committed
90
`gts2_client --help`
Hannes Diedrich's avatar
Hannes Diedrich committed
91

92
93
#### Arguments:
##### Required:
Hannes Diedrich's avatar
Hannes Diedrich committed
94
95
  * -o OUT_DIR, --out_dir OUT_DIR
    output_directory
Hannes Diedrich's avatar
Hannes Diedrich committed
96
  * -l LAT_LL, --lat_ll LAT_LL
Hannes Diedrich's avatar
Hannes Diedrich committed
97
    latitude lower left corner
Hannes Diedrich's avatar
Hannes Diedrich committed
98
  * -k LON_LL, --lon_ll LON_LL
Hannes Diedrich's avatar
Hannes Diedrich committed
99
    longitude lower left corner
Hannes Diedrich's avatar
Hannes Diedrich committed
100
  * -i LAT_UR, --lat_ur LAT_UR
Hannes Diedrich's avatar
Hannes Diedrich committed
101
    latitude upper right corner
Hannes Diedrich's avatar
Hannes Diedrich committed
102
  * -j LON_UR, --lon_ur LON_UR
Hannes Diedrich's avatar
Hannes Diedrich committed
103
     longitude upper right corner
Hannes Diedrich's avatar
Hannes Diedrich committed
104
  * -s START_DATE, --start_date START_DATE
Hannes Diedrich's avatar
Hannes Diedrich committed
105
     Startdate (e.g. 20160701)
Hannes Diedrich's avatar
Hannes Diedrich committed
106
  * -e END_DATE, --end_date END_DATE
Hannes Diedrich's avatar
Hannes Diedrich committed
107
     Enddate (e.g. 20160701)
Hannes Diedrich's avatar
Hannes Diedrich committed
108

109
##### Optional
110
111
112
113
  * -r OUT_PREFIX, --out_prefix OUT_PREFIX
    output_prefix for naming the output files
  * -b BANDS, --bands BANDS
     list of Bands (e.g. -b B02_B03)
Hannes Diedrich's avatar
Hannes Diedrich committed
114
115
  * -m OUT_MODE, --out_mode OUT_MODE
    output_mode, use 'json' for json-file,'single' for
116
    single tiffs or 'stack' for tiff band stack, 'nc' for netcdf-file
Hannes Diedrich's avatar
Hannes Diedrich committed
117
118
119
120
121
  * -a SENSOR, --sensor SENSOR
     sensor name  (e.g. S2A)
  * -t LEVEL, --level LEVEL
     processing level (e.g. L2A)
  * -v VERSION, --version VERSION
122
     version of atmospheric correction (e.g. 0.14 - recommended version)
Hannes Diedrich's avatar
Hannes Diedrich committed
123
  * -c COREG, --coreg COREG
Hannes Diedrich's avatar
Hannes Diedrich committed
124
125
126
     get coregistrated data with corrected pixel shifts (e.g. True)
     [pixel shifts between time steps were eliminated by coregistrating
     images with a reference image (currently only avalaible for Germany)]
Hannes Diedrich's avatar
Hannes Diedrich committed
127
  * -z MAX_CLOUDY, --max_cloudy MAX_CLOUDY
Hannes Diedrich's avatar
Hannes Diedrich committed
128
     maximal percentage of cloudyness of requested scene (e.g. 0.2)
Hannes Diedrich's avatar
Hannes Diedrich committed
129
  * -f MINIMUM_FILL, --minimum_fill MINIMUM_FILL
Hannes Diedrich's avatar
Hannes Diedrich committed
130
     minimal percentage of data in scene (e.g. 0.5)
131
132
  * -d UTM_ZONE, --utm_zone UTM_ZONE
    only return data for specific utm-zone (MGRS-tile, e.g. 33UUV)
133
134
  * -g STACK_RESOLUTION, --stack_resolution STACK_RESOLUTION
    spatial sampling [in meters] of the output stack file,
135
                        choose from [10,20])
Hannes Diedrich's avatar
Hannes Diedrich committed
136
137
138
139
140
141
  * -n RGB_EXTENSION, --rgb_extension RGB_EXTENSION
                        file extension of rgb files e.g.[jpg, png], default:
                        jpg (default: jpg)
  * -q RGB_BANDS_SELECTION, --rgb_bands_selection RGB_BANDS_SELECTION
                        band selection for rgb production, choose from:
                        realistic, nice_looking, vegetation,
142
                        healthy_vegetation_urban, snow, agriculture
Hannes Diedrich's avatar
Hannes Diedrich committed
143
                        (default: realistic)
Hannes Diedrich's avatar
Hannes Diedrich committed
144
145
146
147
148
149
  * -w MERGE_TIFS, --merge_tifs MERGE_TIFS
                        Merge tifs and RGBs if area in two or more MGRS tiles
                        per time step (True or False). (default: False)
  * -x MERGE_TILE, --merge_tile MERGE_TILE
                        Choose MGRS tile into which the merge of files is
                        performed (e.g. 33UUV). (default: None)
150

Hannes Diedrich's avatar
Hannes Diedrich committed
151
152
153
154
155
156
157
158
159
160
161
162
**Legend of RGB band selection:**


| Name | band selection |
| :---------  | :----------- |
| realistic | B04, B03, B02 |
| nice_looking | B11, B08, B03 |
| vegetation | B8A, B04, B03 |
| healthy_vegetation_urban | B8A, B11, B02 |
| snow |  B08, B11, B04|
| agriculture | B11, B08, B02 |

163
### As python package
Hannes Diedrich's avatar
Hannes Diedrich committed
164

165
166
167
168
169
170
Set out_mode to "to_python" in order to include the results of the client (dictionary) into
the local namespace:
```
from gts2_client import gts2_client
result = gts2_client.client(out_mode="python", **kwargs)
```
Hannes Diedrich's avatar
Hannes Diedrich committed
171

172
## Limitations:
173
  * Bands with spatial resolution 60m (Band 1) are not stacked.
Hannes Diedrich's avatar
Hannes Diedrich committed
174
175
176
  * Requests with areas larger than 0.2°x0.2° will probably not be processed
     if you also request a large time range this threshold can also be smaller