Commit c7f3419c authored by Felix Delattre's avatar Felix Delattre
Browse files

Updated information and instructions about datasources

parent 6fa8bc82
Pipeline #30190 passed with stage
in 2 minutes and 40 seconds
......@@ -4,7 +4,8 @@ Importer of aggregated exposure models to serve as input for the Global Dynamic
model.
## Installing obmgapanalysis
## Installing gde-importer
### Software dependencies
- Python >= 3.7
......@@ -17,30 +18,47 @@ model.
- `openpyxl`
### Install
```bash
git clone https://git.gfz-potsdam.de/dynamicexposure/globaldynamicexposure/gde-importer.git
cd gde-importer
pip3 install -e .
```
## Running gde-importer
## Preparation
To run the gde-importer for the exposure model of the European Seismic Risk Model 2020 (ESRM20):
### Create necessary databases
1. Clone the ESRM20 repository to a local path of your choice following these
[instructions](https://git.gfz-potsdam.de/dynamicexposure/datasources/-/tree/master/esrm20).
2. Copy the file `config-example.yml` to your working directory as `config.yml` and provide the
necessary parameters:
- exposure_format: esrm20
- metadata_filepath: /local/path/to/cloned/European_Exposure_Model_Data_Inputs_Sources.xlsx
3. From the working directory (where you placed `config.yml`), run the code by typing:
```
gdeimporter
```
1. If it does not already exist, create the GDE Tiles database as shown [here](https://git.gfz-potsdam.de/dynamicexposure/globaldynamicexposure/database-gdetiles).
2. If it does not already exist, create the OBM Tiles database as shown [here](https://git.gfz-potsdam.de/dynamicexposure/openbuildingmap/database-obmtiles) and populate the `obm_built_area_assessments` table (using, for example, [obmgapanalysis](https://git.gfz-potsdam.de/dynamicexposure/openbuildingmap/obmgapanalysis)).
### Obtain data for the European Seismic Risk Model 2020 (ESRM20)
Currently the `gde-importer` only supports the European Seismic Risk Model 2020 (ESRM20). The data for ESRM20 needs to be retrieved and placed in appropriate directories before running `gde-importer`.
1. Download main ESRM20 data.
You can do this in two alternative ways:
1. In your shell: $ `wget --recursive --no-parent https://datasources.dynamicexposure.org/ESRM20/data/`
2. Clone the ESRM20 repository to a local path of your choice following these
[instructions](https://git.gfz-potsdam.de/dynamicexposure/datasources/-/tree/master/ESRM20).
2. Obtain ESRM20-compatible boundaries (geodata files of administrative divisions):
$ `wget --http-user=USERNAME --http-password=PASSWORD --recursive --no-parent https://datasources.dynamicexposure.org/private/ESRM20_boundaries/data/`
*(Unfortunately we are not allowed to redistribute the data and you'll need a password to access the sources. [Read here](https://git.gfz-potsdam.de/dynamicexposure/datasources/-/tree/master/ESRM20_boundaries) for more information about the data being used.)*
3. Place the downloaded data into paths and directories of your preference.
### Configuration
## Configuration file (config.yml)
The file `config.yml` should contain the following parameters:
#### Quickstart:
Copy the file `config_example.yml` to your working directory as `config.yml` and provide the necessary parameters:
- `exposure_format: esrm20`
- `data_pathname: /path/to/downloaded/data/ESRM20`
- `boundaries_pathname: /path/to/downloaded/ESRM20_boundaries`
#### Configuration file
The following configuration options are available in the `config.yml`:
- `model_name`: Name of the input aggregated exposure model (only relevant for the user).
- `exposure_format`: Format of the input aggregated exposure model. Currently supported values: esrm20.
......@@ -51,6 +69,18 @@ The file `config.yml` should contain the following parameters:
- "all": The list of names will be retrieved from the metadata of the input aggregated exposure model.
- A comma-space-separated list of entity names: This list of names will be used.
- A full path to a .txt or .csv file: The list of names will be retrieved from the indicated .txt/.csv file.
- `exposure_entities_code`: Either "ISO3" or a nested structure with exposure entities names and 3-character codes. For running ESRM20, "ISO3" is the preferred option.
- `number_cores`: Number of cores used for parallelisation.
- `database_built_up`: Credentials for the [database](https://git.gfz-potsdam.de/dynamicexposure/openbuildingmap/database-obmtiles#obm_built_area_assessments-completeness-assessments-information) where the built-up areas per quadtile are stored.
- `database_gde_tiles`: Credentials for the [database](https://git.gfz-potsdam.de/dynamicexposure/globaldynamicexposure/database-gdetiles) where information on the GDE tiles is stored.
## Running gde-importer
From the working directory (where you placed `config.yml`), run the code by typing:
```
gdeimporter
```
## Copyright and copyleft
......
......@@ -3,10 +3,8 @@ exposure_format: esrm20 # Only supported value for now
data_pathname: path_to_directory_with_model_data
boundaries_pathname: path_to_directory_with_boundary_files
occupancies_to_run: residential, commercial # Need to exist for the indicated `exposure format`, industrial not supported
exposure_entities_to_run: all # Either "all", a comma-space-separated list of entity names, or a name of a .txt or .csv file
exposure_entities_code: # Either "ISO3" in this or a nested structure with exposure entities names and 3-character codes
Exposure Entity 1: EE1
Exposure Entity 2: XXX
exposure_entities_to_run: Luxembourg # Either "all", a comma-space-separated list of entity names, or a name of a .txt or .csv file
exposure_entities_code: ISO3 # Either "ISO3" in this or a nested structure with exposure entities names and 3-character codes
number_cores: 1 # Number of cores used for parallelisation
database_built_up: # Database where built-up areas per quadtile are stored
host: host_name
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment