gde_calculations_prototype issueshttps://git.gfz-potsdam.de/globaldynamicexposure/libraries/gde_calculations_prototype/-/issues2022-03-07T08:44:23+01:00https://git.gfz-potsdam.de/globaldynamicexposure/libraries/gde_calculations_prototype/-/issues/15GeoPandas UserWarning when calling `overlay`2022-03-07T08:44:23+01:00Cecilia NievasGeoPandas UserWarning when calling `overlay`Need to explicitly specify parameter `keep_geom_type` when calling `geopandas.overlay` to avoid getting the following user warning:
```
/gde_calculations_prototype/GDE_TOOLS_create_industrial_cells.py:552: UserWarning: `keep_geom_type=T...Need to explicitly specify parameter `keep_geom_type` when calling `geopandas.overlay` to avoid getting the following user warning:
```
/gde_calculations_prototype/GDE_TOOLS_create_industrial_cells.py:552: UserWarning: `keep_geom_type=True` in overlay resulted in 6 dropped geometries of different geometry types than df1 has. Set `keep_geom_type=False` to retain all geometries
insct_i = gpd.overlay(aux_df_i, aux_df_not_i, how="intersection")
```https://git.gfz-potsdam.de/globaldynamicexposure/libraries/gde_calculations_prototype/-/issues/14Replace deprecated DataFrame.as_matrix() with DataFrame.to_numpy()2022-02-04T16:54:12+01:00Cecilia NievasReplace deprecated DataFrame.as_matrix() with DataFrame.to_numpy()DataFrame.as_matrix() is deprecated.DataFrame.as_matrix() is deprecated.https://git.gfz-potsdam.de/globaldynamicexposure/libraries/gde_calculations_prototype/-/issues/13Check in GDE_check_tiles_vs_visual_CSVs.py inconsistent with decision in GDE_...2021-11-09T08:20:57+01:00Cecilia NievasCheck in GDE_check_tiles_vs_visual_CSVs.py inconsistent with decision in GDE_gather_SERA_and_OBM.pyIn `GDE_gather_SERA_and_OBM.py`, when the completeness file says that the cell/tile is water (code 5) or empty (code 6) but the code finds OBM buildings in the cell/tile, the completeness value is changed to code 3 (incomplete) but a val...In `GDE_gather_SERA_and_OBM.py`, when the completeness file says that the cell/tile is water (code 5) or empty (code 6) but the code finds OBM buildings in the cell/tile, the completeness value is changed to code 3 (incomplete) but a value of 999 is assigned to the variable that is then used for the CSV visual output (https://git.gfz-potsdam.de/dynamicexposure/legacy/gde_calculations_prototype/-/blob/master/GDE_gather_SERA_and_OBM.py#L189). This 3 then gets written to the HDF5 GDE tiles, but a value of 999 is assigned to the same cell/tile in the visual CSV output by cell, with the purpose of indicating that there was an inconsistency. However, `GDE_check_tiles_vs_visual_CSVs.py` checks that the completeness values reported in the HDF5 GDE tiles match those reported in the visual CSV output by cell and raise an error (https://git.gfz-potsdam.de/dynamicexposure/legacy/gde_calculations_prototype/-/blob/master/GDE_check_tiles_vs_visual_CSVs.py#L88). This means that the check in `GDE_check_tiles_vs_visual_CSVs.py` is not consistent with what the code is doing (i.e. it is a badly designed check).https://git.gfz-potsdam.de/globaldynamicexposure/libraries/gde_calculations_prototype/-/issues/11Some checks in GDE_check_consistency.py can fail due to rounding of decimal p...2021-11-09T08:21:01+01:00Cecilia NievasSome checks in GDE_check_consistency.py can fail due to rounding of decimal places in output CSV filesProblem noted when comparing the output OQ input files for v002 and version Ind30 of the Attica case study. Residential and commercial exposure were meant to be "the same" in both versions, but a discrepancy in the total number of buildi...Problem noted when comparing the output OQ input files for v002 and version Ind30 of the Attica case study. Residential and commercial exposure were meant to be "the same" in both versions, but a discrepancy in the total number of buildings and size of the OQ input files was observed:
1) Res_number_OBM_with_classes: 256,375.03 in Ind30 vs 256,373 in v002.
2) Com_number_OBM_with_classes: 7,783.70 in Ind30 vs 7,784 in v002.
This discrepancy arises because the OQ input files are created using 5 decimal places now (v002 was created with str(), i.e. unspecified number of decimal places). This change was introduced in commit https://git.gfz-potsdam.de/dynamicexposure/legacy/gde_calculations_prototype/-/commit/0b89ddaf6c99fc1b45230e08e4ff400ef5004b9d.
This is a matter of:
1. How many decimal places is it reasonable to use for the OQ input files?
2. How to check the consistency of outputs when rounding is involved (very small differences per building add up when the total number of buildings is large). Note that checking building by building takes a very long time.https://git.gfz-potsdam.de/globaldynamicexposure/libraries/gde_calculations_prototype/-/issues/9Bug: "Greece" hard-coded in line 368 of GDE_check_consistency.py2021-11-09T08:20:48+01:00Cecilia NievasBug: "Greece" hard-coded in line 368 of GDE_check_consistency.pyhttps://git.gfz-potsdam.de/dynamicexposure/legacy/gde_calculations_prototype/-/blob/master/GDE_check_consistency.py#L367
Name of file written as:
`'Exposure_GDE_'+bdgs_part+'_'+crit+'_Greece_'+occu+'.csv'`
should be:
`'Exposure_GDE_'...https://git.gfz-potsdam.de/dynamicexposure/legacy/gde_calculations_prototype/-/blob/master/GDE_check_consistency.py#L367
Name of file written as:
`'Exposure_GDE_'+bdgs_part+'_'+crit+'_Greece_'+occu+'.csv'`
should be:
`'Exposure_GDE_'+bdgs_part+'_'+crit+'_'+location_var+'_'+occu+'.csv'`https://git.gfz-potsdam.de/globaldynamicexposure/libraries/gde_calculations_prototype/-/issues/8Check that files exist before tying to open them in GDE_check_consistency.py ...2021-11-09T08:20:52+01:00Cecilia NievasCheck that files exist before tying to open them in GDE_check_consistency.py and GDE_check_OQ_input_files.pyOpenQuake input files for left-over buildings in cells are not created if there are no left-over buildings. However, the code attempts to open these files anyway (e.g. https://git.gfz-potsdam.de/dynamicexposure/legacy/gde_calculations_pr...OpenQuake input files for left-over buildings in cells are not created if there are no left-over buildings. However, the code attempts to open these files anyway (e.g. https://git.gfz-potsdam.de/dynamicexposure/legacy/gde_calculations_prototype/-/blob/master/GDE_check_consistency.py#L367, https://git.gfz-potsdam.de/dynamicexposure/legacy/gde_calculations_prototype/-/blob/master/GDE_check_OQ_input_files.py#L64). The code should first check if the file exists.https://git.gfz-potsdam.de/globaldynamicexposure/libraries/gde_calculations_prototype/-/issues/7Make completeness file name configurable in GDE_gather_SERA_and_OBM.py2021-11-09T08:20:32+01:00Cecilia NievasMake completeness file name configurable in GDE_gather_SERA_and_OBM.pyThe name of the CSV file containing completeness data is hard-coded (https://git.gfz-potsdam.de/dynamicexposure/legacy/gde_calculations_prototype/-/blob/master/GDE_gather_SERA_and_OBM.py#L60). Make it a parameter of the configuration file.The name of the CSV file containing completeness data is hard-coded (https://git.gfz-potsdam.de/dynamicexposure/legacy/gde_calculations_prototype/-/blob/master/GDE_gather_SERA_and_OBM.py#L60). Make it a parameter of the configuration file.