Commit c7f3419c authored by Felix Delattre's avatar Felix Delattre
Browse files

Updated information and instructions about datasources

parent 6fa8bc82
Pipeline #30190 passed with stage
in 2 minutes and 40 seconds
...@@ -4,7 +4,8 @@ Importer of aggregated exposure models to serve as input for the Global Dynamic ...@@ -4,7 +4,8 @@ Importer of aggregated exposure models to serve as input for the Global Dynamic
model. model.
## Installing obmgapanalysis ## Installing gde-importer
### Software dependencies ### Software dependencies
- Python >= 3.7 - Python >= 3.7
...@@ -17,30 +18,47 @@ model. ...@@ -17,30 +18,47 @@ model.
- `openpyxl` - `openpyxl`
### Install ### Install
```bash ```bash
git clone https://git.gfz-potsdam.de/dynamicexposure/globaldynamicexposure/gde-importer.git git clone https://git.gfz-potsdam.de/dynamicexposure/globaldynamicexposure/gde-importer.git
cd gde-importer cd gde-importer
pip3 install -e . pip3 install -e .
``` ```
## Running gde-importer ## Preparation
To run the gde-importer for the exposure model of the European Seismic Risk Model 2020 (ESRM20): ### Create necessary databases
1. Clone the ESRM20 repository to a local path of your choice following these 1. If it does not already exist, create the GDE Tiles database as shown [here](https://git.gfz-potsdam.de/dynamicexposure/globaldynamicexposure/database-gdetiles).
[instructions](https://git.gfz-potsdam.de/dynamicexposure/datasources/-/tree/master/esrm20). 2. If it does not already exist, create the OBM Tiles database as shown [here](https://git.gfz-potsdam.de/dynamicexposure/openbuildingmap/database-obmtiles) and populate the `obm_built_area_assessments` table (using, for example, [obmgapanalysis](https://git.gfz-potsdam.de/dynamicexposure/openbuildingmap/obmgapanalysis)).
2. Copy the file `config-example.yml` to your working directory as `config.yml` and provide the
necessary parameters: ### Obtain data for the European Seismic Risk Model 2020 (ESRM20)
- exposure_format: esrm20
- metadata_filepath: /local/path/to/cloned/European_Exposure_Model_Data_Inputs_Sources.xlsx Currently the `gde-importer` only supports the European Seismic Risk Model 2020 (ESRM20). The data for ESRM20 needs to be retrieved and placed in appropriate directories before running `gde-importer`.
3. From the working directory (where you placed `config.yml`), run the code by typing:
``` 1. Download main ESRM20 data.
gdeimporter You can do this in two alternative ways:
``` 1. In your shell: $ `wget --recursive --no-parent https://datasources.dynamicexposure.org/ESRM20/data/`
2. Clone the ESRM20 repository to a local path of your choice following these
[instructions](https://git.gfz-potsdam.de/dynamicexposure/datasources/-/tree/master/ESRM20).
2. Obtain ESRM20-compatible boundaries (geodata files of administrative divisions):
$ `wget --http-user=USERNAME --http-password=PASSWORD --recursive --no-parent https://datasources.dynamicexposure.org/private/ESRM20_boundaries/data/`
*(Unfortunately we are not allowed to redistribute the data and you'll need a password to access the sources. [Read here](https://git.gfz-potsdam.de/dynamicexposure/datasources/-/tree/master/ESRM20_boundaries) for more information about the data being used.)*
3. Place the downloaded data into paths and directories of your preference.
### Configuration
## Configuration file (config.yml)
The file `config.yml` should contain the following parameters: #### Quickstart:
Copy the file `config_example.yml` to your working directory as `config.yml` and provide the necessary parameters:
- `exposure_format: esrm20`
- `data_pathname: /path/to/downloaded/data/ESRM20`
- `boundaries_pathname: /path/to/downloaded/ESRM20_boundaries`
#### Configuration file
The following configuration options are available in the `config.yml`:
- `model_name`: Name of the input aggregated exposure model (only relevant for the user). - `model_name`: Name of the input aggregated exposure model (only relevant for the user).
- `exposure_format`: Format of the input aggregated exposure model. Currently supported values: esrm20. - `exposure_format`: Format of the input aggregated exposure model. Currently supported values: esrm20.
...@@ -51,6 +69,18 @@ The file `config.yml` should contain the following parameters: ...@@ -51,6 +69,18 @@ The file `config.yml` should contain the following parameters:
- "all": The list of names will be retrieved from the metadata of the input aggregated exposure model. - "all": The list of names will be retrieved from the metadata of the input aggregated exposure model.
- A comma-space-separated list of entity names: This list of names will be used. - A comma-space-separated list of entity names: This list of names will be used.
- A full path to a .txt or .csv file: The list of names will be retrieved from the indicated .txt/.csv file. - A full path to a .txt or .csv file: The list of names will be retrieved from the indicated .txt/.csv file.
- `exposure_entities_code`: Either "ISO3" or a nested structure with exposure entities names and 3-character codes. For running ESRM20, "ISO3" is the preferred option.
- `number_cores`: Number of cores used for parallelisation.
- `database_built_up`: Credentials for the [database](https://git.gfz-potsdam.de/dynamicexposure/openbuildingmap/database-obmtiles#obm_built_area_assessments-completeness-assessments-information) where the built-up areas per quadtile are stored.
- `database_gde_tiles`: Credentials for the [database](https://git.gfz-potsdam.de/dynamicexposure/globaldynamicexposure/database-gdetiles) where information on the GDE tiles is stored.
## Running gde-importer
From the working directory (where you placed `config.yml`), run the code by typing:
```
gdeimporter
```
## Copyright and copyleft ## Copyright and copyleft
......
...@@ -3,10 +3,8 @@ exposure_format: esrm20 # Only supported value for now ...@@ -3,10 +3,8 @@ exposure_format: esrm20 # Only supported value for now
data_pathname: path_to_directory_with_model_data data_pathname: path_to_directory_with_model_data
boundaries_pathname: path_to_directory_with_boundary_files boundaries_pathname: path_to_directory_with_boundary_files
occupancies_to_run: residential, commercial # Need to exist for the indicated `exposure format`, industrial not supported occupancies_to_run: residential, commercial # Need to exist for the indicated `exposure format`, industrial not supported
exposure_entities_to_run: all # Either "all", a comma-space-separated list of entity names, or a name of a .txt or .csv file exposure_entities_to_run: Luxembourg # Either "all", a comma-space-separated list of entity names, or a name of a .txt or .csv file
exposure_entities_code: # Either "ISO3" in this or a nested structure with exposure entities names and 3-character codes exposure_entities_code: ISO3 # Either "ISO3" in this or a nested structure with exposure entities names and 3-character codes
Exposure Entity 1: EE1
Exposure Entity 2: XXX
number_cores: 1 # Number of cores used for parallelisation number_cores: 1 # Number of cores used for parallelisation
database_built_up: # Database where built-up areas per quadtile are stored database_built_up: # Database where built-up areas per quadtile are stored
host: host_name host: host_name
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment