README.md 6.76 KB
Newer Older
Cecilia Nievas's avatar
Cecilia Nievas committed
1
# gde-exporter
Cecilia Nievas's avatar
Cecilia Nievas committed
2

Cecilia Nievas's avatar
Cecilia Nievas committed
3
Exports the Global Dynamic Exposure (GDE) model onto desired output formats.
Cecilia Nievas's avatar
Cecilia Nievas committed
4
5
6

## Description

Cecilia Nievas's avatar
Cecilia Nievas committed
7
8
9
`gde-exporter` retrieves data from the Global Dynamic Exposure (GDE) model created with 
[gde-core](https://git.gfz-potsdam.de/dynamicexposure/globaldynamicexposure/gde-core) and 
exports it onto desired output formats.
Cecilia Nievas's avatar
Cecilia Nievas committed
10

11
12
13
14
15
16
17
Broadly speaking, the GDE model defines exposure in terms of zoom-level 18 quadtiles and
individual building footprints retrieved from OpenStreetMap (OSM) and processed under
OpenBuildingMap (OBM). The latter are usually referred to as "OBM buildings". Quadtiles are
assigned numbers of buildings from aggregated exposure models (referred to as "aggregated
buildings") and a number of "remainder buildings" is calculated when combining the former with
the OBM buildings. OBM and remainder buildings form the GDE model.

Cecilia Nievas's avatar
Cecilia Nievas committed
18
## Installing gde-exporter
Cecilia Nievas's avatar
Cecilia Nievas committed
19
20
21
22
23
24
25
26

### Software dependencies

- Python 3.7, 3.8 or 3.9

### Install

```bash
Cecilia Nievas's avatar
Cecilia Nievas committed
27
28
git clone https://git.gfz-potsdam.de/dynamicexposure/globaldynamicexposure/gde-exporter.git
cd gde-exporter
Cecilia Nievas's avatar
Cecilia Nievas committed
29
30
pip3 install -e .
```
31
32
33
34
35
36
37
38
## Preparation

### Configuration

Copy the file `config_example.yml` to your working directory as `config.yml` and provide the 
necessary parameters. Required parameters are:

- `model_name`: Name of the input aggregated exposure model to be processed.
39
40
41
42
- `output_path`: Path where export files will be stored (this path needs to exist for the
software to run). The software creates a sub-directory under this path with name
`exporter_YYYY_MM_DD_HH_MM_SS`, so as to avoid involuntary overwriting of previous output
results.
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
- `occupancies_to_run`: List of occupancies for which the code will be run, separated by ", " 
(comma and space). They need to exist for the `exposure format` of `model_name`. Currently 
supported values: residential, commercial, industrial.
- `exposure_entities_to_run`: List of names of exposure entities for which the code will be run. 
It is used even if `selection_mode` (see below) is not `exposure_entity`.
Currently supported options:
  - "all": The list of names associated with `model_name` will be retrieved from the 
  [GDE Tiles](https://git.gfz-potsdam.de/dynamicexposure/globaldynamicexposure/database-gdetiles) 
database.
  - A comma-space-separated list of entity names: This list of names will be used.
  - A full path to a .txt or .csv file:  The list of names will be retrieved from the indicated 
  .txt/.csv file.
- `exposure_entities_code`: Either "ISO3" or a nested structure with exposure entities names 
and 3-character codes. When running `model_name=esrm20`, "ISO3" is the preferred option.
- `geographic_selection`: Set of parameters that define the geographic area for which the output 
will be produced:
  - `selection_mode`: `exposure_entity`, `data_unit_id`, `quadkeys`, or `bounding_box`. In all 
  cases only data from the exposure entities specified in `exposure_entities_to_run` will be 
  considered, even if the geographic area includes other exposure entities (they will be 
  ignored). The meaning of the `selection_mode` options is as follows:
    - `exposure_entity`: The quadkeys associated with the exposure entities specified in 
	`exposure_entities_to_run` will be retrieved from `database_gde_tiles` (see below) and output.
	- `data_unit_id`: The quadkeys associated with the data units specified in `data_unit_ids` 
	(see below) will be retrieved from `database_gde_tiles` and output.
	- `quadkeys`: The quadkeys contained in a TXT file whose file path is specified in 
	`quadkeys_file` (see below) will be retrieved and output.
	- `bounding_box`: The quadkeys that contain the bounding box defined by the coordinates 
	`lon_w`, `lon_e`, `lat_s` and `lat_n`, under `bounding_box`(see below) will be retrieved and 
	output.
  - `data_unit_ids`: Required if `selection_mode = data_unit_id`. List of IDs of data units, 
  separated by a comma and a space.
  - `quadkeys_file`: Required if `selection_mode = quadkeys`. Full path to a TXT file 
  containing quadkeys (either one per row, comma-separared, or a mix).
  - `bounding_box`: Required if `selection_mode = bounding_box`. Coordinates `lon_w`, `lon_e`, 
  `lat_s` and `lat_n` of the bounding box.
78
79
80
81
82
- `output_format`: Format/s to which the GDE model will be exported. More than one format can be
listed separating them with ", " (comma and space). Currently supported values:
  - `OpenQuake_CSV`: Exposure CSV files compatible with the
  [OpenQuake Engine](https://github.com/gem/oq-engine), with complementary GeoPackage (.gpkg)
  files that contain the geometry of quadtiles and OBM buildings.
83
  - `GeoSummary_Tiles`: GeoPackage (GPKG) files showing summary values per tile.
84
85
86
87
88
89
90
91
92
- `buildings_to_export`: Types of buildings to export, separated by ", " (comma and space).
Currently supported values:
  - `OBM`: OBM buildings.
  - `remainder`: GDE remainder buildings.
  - `aggregated`: Buildings as in the input aggregated exposure model.
- `export_OBM_footprints`: True/False. If True, geometries of OBM buildings will be exported and
parameters associated with these buildings will be attached to their original OSM ID. If False,
the geometries will not be exported and anonymous IDs will be assigned to the parameters of the
OBM buildings.
93
94
95
- `database_gde_tiles`: Credentials for the 
[GDE Tiles](https://git.gfz-potsdam.de/dynamicexposure/globaldynamicexposure/database-gdetiles) 
database where information on the GDE tiles is stored.
96
- `number_cores`: Number of cores used for parallelisation.
Cecilia Nievas's avatar
Cecilia Nievas committed
97

Cecilia Nievas's avatar
Cecilia Nievas committed
98
## Running gde-exporter
Cecilia Nievas's avatar
Cecilia Nievas committed
99

Cecilia Nievas's avatar
Cecilia Nievas committed
100
From the working directory run the code by typing:
Cecilia Nievas's avatar
Cecilia Nievas committed
101
102

```
Cecilia Nievas's avatar
Cecilia Nievas committed
103
gdeexporter
Cecilia Nievas's avatar
Cecilia Nievas committed
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
```

## Copyright and copyleft

Copyright (C) 2022

* Helmholtz-Zentrum Potsdam Deutsches GeoForschungsZentrum GFZ

This program is free software: you can redistribute it and/or modify it
under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or (at
your option) any later version.

This program is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero
General Public License for more details.

You should have received a copy of the GNU Affero General Public License
along with this program. If not, see http://www.gnu.org/licenses/.

Also add information on how to contact you by electronic and paper mail.

If your software can interact with users remotely through a computer
network, you should also make sure that it provides a way for users to
get its source. For example, if your program is a web application, its
interface could display a "Source" link that leads users to an archive
of the code. There are many ways you could offer source, and different
solutions will be better for different programs; see section 13 for the
specific requirements.

See the [LICENSE](./LICENSE) for the full license text.