Skip to content
Snippets Groups Projects
Select Git revision
  • master
  • 119-modify-the-vulnerability_model-py-to-use-iacolleti-s-csv-cumulative-vulnerability-functions
  • docs-patch
  • feature/110-compatibility-of-the-loss-calculator-with-aggregated-exposure-models
  • feature/implement_security
  • feature/105-read-in-the-inundation-map
  • 98-the-api-needs-to-have-user-restrictions
  • bug/where_statement
  • 82-correct-the-entry-point-in-the-losscalculator
  • feature/76-add-ground-motion-values-to-the-damage-database
  • feature/78-changing-damage_calculator-py-to-match-new-result-database-table-name-changes
  • first_release_losscalculator
  • feature/devel
  • feature/api-rewrite-fix-docker
  • feature/exposure-damage-api
  • feature/refactor-argument-parsing
  • bugfix/remove-deprecated-arguments
  • feature/api
  • bugfix/example
  • feature/9-create-simple-test
  • v1.0.1
  • v1.0.0
22 results

manual.md

Blame
    • user avatar
    Add API for interaction with shakemap service
    Marius Kriegerowski authored and Laurens committed
    5c051239
    History
    user avatar 5c051239
    History
    manual.md 15.47 KiB

    losscalculator

    This program computes the probabilities of occurrence (PoE) of damage states for real or scenario earthquakes, given a ground-motion field, an exposure model representing the assets in the region of interest and fragility functions that model the probability of exceeding defined damage states given a set of ground-motion values of a specific type.

    Terminology

    Asset:
    An asset is something of value, which can include buildings and population. For example, an asset can include an individual building at a given location, or a number of buildings that are grouped and co-located at a single location and are classified with the same taxonomy. Please note that, due to probabilistic assessment of building types, each building’s structural types can be represented by a different asset with the respective frequency.

    Exposure model:
    A set of assets grouped together, according to their geographical location, taxonomy and value. Each line of the exposure model file represents information for an asset.

    Fragility function:
    The probability of exceeding a set of damage states, given a set of ground-motion values of a specific type. These functions are defined in csv files.

    Ground-motion field:
    The geographic distribution of ground-motion values (of a real or scenario earthquake).

    Damage states:
    Damage states describe some predefined types and extent of damage which structures can experience after an earthquake. The damage states used in this program are defined by the fragility functions. Often, these are slight, moderate, extensive and complete. Please note that this program computes damage for only four damage states (probabilities of exceeding four damage states is calculated regardless of the damage states names in the fragility functions, but the result file header for the number of buildings of each damage states remain as: structural_no_damage, structural_slight, structural_moderate, structural_extensive and structural_complete)

    Taxonomy:
    Schema used to classify assets. For buildings, the classification schema considers a number of attributes including lateral-load resisting system and the building’s material, height and year of construction. Please see the GEM taxonomy for a good example.

    Definitions in this terminology section are taken from the Glossary section of the OpenQuake documentation.

    Usage

    To run the program please call:

    losscalculator damage [OPTIONS] -e filepath -f pathname (or -c pathname/searchpattern) -g filepath -t filepath -S string

    or

    python3 main.py damage [OPTIONS] -e filepath -f pathname (or -c pathname/searchpattern) -g filepath -t filepath -S string

    Please use

    losscalculator --help

    for information on how to use the program input options.

    Program input parameters

    Required parameters

    -e, --exposure=filepath
    Defines the filepath to the database exposure model.

    An exposure database can be created with the preprocessor.py using a CSV exposure model with the format given as the example below:

    "id","lon","lat","taxonomy","number","structural","night","occupancy","admin_name","admin_ID","tile_id"
"GDE_Ind_65386",23.6317973417,38.063296952,"CR/LFINF+CDM/H:2",0.224973135611,401840.352396,0.0,"Ind","Aspropyrgos","GR_3515002","tile_1111111111"
"GDE_Com_940036",23.6319444444,38.0625,"CR/LDUAL+CDM/H:1/8.0",12.5,61.8328463283,0.0,"Com","Aspropyrgos","GR_3515002","tile_1111111111"

    Exposure items definition:
    Please note that the assets of the region of interest can be presented differently in an exposure model. One way is to divide the region of interest into a grid and all the assets inside each tile will be represented by aggregated assets in the center of that tile. In the other way, all assets (buildings) will be given with their exact location (building centroid).

    id:  
Unique ID for each asset in the exposure model. It can consist of both letters and numbers. The ID relates each asset in the exposure model to the respective entries in the results. IDs can be freely defined as long as they contain numbers, letters and `_` only.  

lon:  
Longitude of the asset (centroid of either a building or a tile, depending on the exposure type) given in the WGS84 coordinate reference system (EPSG:4326).  

lat:  
Latitude of the asset (centroid of either a building or a tile, depending on the exposure type) given in the WGS84 coordinate reference system (EPSG:4326).  

taxonomy:  
Explained above in Section `Terminology`.

number:  
Number of buildings for the given asset (float). Due to the probabilistic assessment of building types, each building’s possible structural type is represented by a different asset with a respective frequency. Also, the distribution of aggregated exposure models to tiles can cause non-cardinal building numbers per tile.

structural:  
Reconstruction cost of the asset in an undefined currency. The user needs to make sure that only one currency is used. 

night:  
Number of people expected at night-time in the asset.  

occupancy:  
Occupancy type of the asset. Common types are `industrial`, `commercial`, and `residential`, but other types are possible.

admin_name:  
Name of the administrative unit of the asset location.  

admin_ID:  
ID of the administrative unit of the asset location.  

tile_id:
An ID with the prefix `tile_` (eg. tile_111111012) referring to the Quadtree tile in which the asset is located. For a building asset, the ID refers to the Quadtree tile in which the centroid of the buildings falls in.

    -f --fragility-file=filepath
    Path to the XML fragility function file. Please note that if this option is used, the use of CSV fragility function files (-c option) is not possible.

    -c, --csv-directory=filesearchpattern Search pattern for fragility functions as CSV files (e.g. pathname/*.csv). Please note that if this option is used, the use of XML fragility function file (-f option) is not possible.

    A CSV fragility-function file example extract:

    PGA,0.050000 ,0.052179 ,0.054454 ,0.056827 ,... ,3.271841 ,3.414451
slight,0.000000 ,0.000000 ,0.000000 ,0.000000 ,... ,0.873138 ,0.887882
moderate,0.000000 ,0.000000 ,0.000000 ,0.000000 ,... ,0.440287 ,0.469610
extensive,0.000000 ,0.000000 ,0.000000 ,0.000000 ,... ,0.222803 ,0.245484
complete,0.000000 ,0.000000 ,0.000000 ,0.000000 ,... ,0.121331 ,0.136897

    In a fragility function, sets of probabilities of exceedance (one row per damage state) are defined for a set of ground-motion values (first row). Each fragility function can be defined based on any ground-motion type such as PGA, SA(0.3), SA(0.6), SA(1.0) etc.

    -t --taxonomy-map=filepath
    Path to the file that maps taxonomies to their relevant fragility-function names.

    File example extract:

    taxonomy,fragility_function_name  
CR/LDUAL+CDM/HBET:6-/11.0,CR_LDUAL-DUM_H6  
CR/LDUAL+CDM/HBET:6-/5.0,CR_LDUAL-DUL_H6  
...  
CR/LDUAL+CDM/HBET:6-/SOS/11.0,CR_LDUAL-DUL_H6

    -g --ground-motion-field=filepath
    Path to the file that contains the ground-motion field.

    The ground-motion field needs to include values for all ground-motion types present in all fragility functions used for a computation.

    File example extract:

    lon,lat,gmv_PGA,gmv_SA(0.3),gmv_SA(0.6),gmv_SA(1.0)  
23.62917,38.06528,0.1291617,0.2559737,0.1626296,0.08834055  
...  
23.63472,38.05972,0.1246124,0.2471472,0.1570672,0.08533953

    The ground-motion values at asset locations are estimated through interpolation, thus the ground-motion field needs to extend further around the asset locations.

    Optional Parameters

    -h, --help
    Provides information on the inputs and terminates the program.

    -i, --interpolation-method={linear, nearest, cubic}
    Defines the method for interpolating ground-motion values. Default is linear.
    The ground-motion values are interpolated at the asset locations using the requested method.

    Output

    The output file is the same input database exposure model file given by the option -e, filled with the damage data. Thus the structure of the output database does not change during a damage assessment. Therefore, to see the tables and their fields for an output damage assessment database, please refer to the example extract of the preprocessor output at the end of this manual.

    Preprocessor

    This program imports an OpenQuake format CSV exposure model into a database for further use (damage calculation using the losscalculator). The losscalculator computes earthquake-related damage in a building-specific way. For a building-specific calculation, the exposure model needs to include both the geometries of tiles (As explained above in the section -e input, each asset in an exposure model can be represented either by their exact locations or in center of a tile.) and the buildings in addition to the data provided by an OpenQuake-compatible exposure model (An OpenQuake-compatible exposure model CSV file, with minimum information provided, has the headers: id,lon,lat,taxonomy,number,structural,night,occupancy). In fact, this program puts the geometries of both tiles and buildings together along with the OpenQuake-compatible exposure model information in a database format.
    In addition to the import command (which use is explained above), the preprocessor.py provides an export command for exporting the database exposure model to a CSV file.

    Usage

    To know further on how to import an input CSV exposure model file to a database, check out the import preprocessor example.
    To know further on how to export a database exposure model into CSV format, check out the export preprocessor example.

    Program input parameters

    -i, --import-search-pattern: str, optional
    Search pattern for the input CSV exposure model files (e.g. *.csv).
    Each input CSV exposure model file should follow the format below:
    id,lon,lat,taxonomy,number,structural,night,occupancy,admin_name,admin_ID,origin_id
    Where origin_id is a unique ID which enables us to trace an asset back to either the tile or the building it is originated from. Only required when using the import command.

    -d, --exposure-database: str, required
    Defines the path to an exposure model database file.
    When using the import command, it points to a file where the exposure database file is to be stored.
    When using the export command, it points to an (existing) exposure database file to be exported into CSV format. The exported exposure model will have the header as below:
    id,lon,lat,taxonomy,number,structural,night,occupancy,admin_name,admin_ID, tile_id, tile_geometry, building_id ,building_geometry

    -f, --field-name-osm-id: str, required Name of the field (of the table given as the -t input in which the data of the building GeoPackage file is stored) that contains the building IDs.

    -g, --building-geometries: str, optional
    Defines the path to the GeoPackage file that contains the building geometries along with their OSM IDs. Only required if there are building assets in the -i input file (the input OpenQuake-compatible CSV exposure model file).

    -o, --export-filepath: str, optional
    Defines the path to a file, to which the database exposure model is exported in CSV format. The exported CSV exposure model file follows the format below: id,lon,lat,taxonomy,number,structural,night,occupancy,admin_name,admin_ID, tile_id, tile_geometry, building_id ,building_geometry
    Only required when using the export module.

    -s, --spatialite-extension: str, optional
    Defines the filepath of the Spatialite extension. (default: mod_spatialite)

    -t, --table-name: str, required Name of the table with the building information in the building GeoPackage file.

    -w overwrite existing result file: optional
    To overwrite an existing database exposure model file.

    In order to use this program, please note that the input exposure model should be defined based on Quadtree tiles.

    Output

    Using import command

    The output using the import command is a database exposure model file with tables and their fields as below:

    table_name (fieldname1, fieldname2, ...) AssessmentSource (id, name, method, comment)
    Asset (entity_id, taxonomy_id, number, structural, night, occupancy) Building (osm_id, quadkey, occupancy, geom) Damage (assessment_id, damage_state_id, damage_probability) DamageAssessment (id, entity_id, assessment_source, date, comment)
    DamageState (id, damage_scale_name, damage_state_name) Entity (id, quadkey, osm_id, geom, status) GMField (entity_id, gm_data_row) Taxonomy (id, taxonomy_string) Tile (quadkey, built_Area_size, built_up_ratio, completeness, geom)

    Using export command

    The output using the export command is a CSV exposure model file (OQ-compatible) exported from a database exposure model with headers as below:
    id,lon,lat,taxonomy,number,structural,night,occupancy,admin_name,admin_ID,tile_id,tile_geometry,building_id,building_geometry
    You can find the definition of each column of the header above, in the -e input parameter of the losscalculator.

    API

    The loss-calculator comes with an http API based on fastAPI. The API requires the environment variables:

    • HOSTNAME_SHAKEMAP: URL of the shakemap service (default: https://shakemapi.openbuildingmap.org/)
    • LOSSCALCULATOR_DATA: Path where loss-calculator data is located. In this directory, the configuration file is located as well as all exposure models databases, taxonomy mappings CSV files and fragility XML files.
    • GDE_DATABASE: Location of the initial exposure database.

    Start the API on a console with:

    uvicorn losscalculator.api:app

    API workflow

    All functions referenced in this section can be found in examples/api-use/api-example.py.

    Creating a first scenario

    If an earthquake occurs, a QuakeML file should be created. This is used as input for the losscalculator. To run the loss-calculator, a scenario needs to be created with the post request create_scenario. Afterwards the scenario can be run in the losscalculator using run_scenario. They can also be combined using the function create_and_run_scenario. To retrieve the SpatiaLite database, get_scenario_database can be used.

    Updating a scenario

    If the earthquake turned out to have different parameters, a new QuakeML can be uploaded with a new assessment_source, in the same scenario. This time, the scenario is not created but only run using run_scenario.

    Assessment reports

    A damage assessment can be uploaded using the create_damage_assessment_report endpoint. The input is the scenario name, an assessment source name (e.g. 'damage assessment 23-06-2022') and the damage assessment itself in JSON format.

    Mission planning

    The API can also be used for mission planning. To update the mission planning of OSM buildings, update_mission_planning_status should be used. The input is a status ID and a list of OSM IDs. The status IDs are: 0 for buildings not visited; 1 for buildings being visited and 2 for buildings that are visited).

    Running the API in a docker container

    The API can be hosted using the shipped Dockerfile.