Added feature to handle 30-arcsec industrial and subdivide fillers

README.md

@@ -53,10 +53,10 @@ Currently the `gde-importer` only supports the European Seismic Risk Model 2020
    $ `wget --http-user=USERNAME --http-password=PASSWORD --recursive --no-parent https://datasources.dynamicexposure.org/private/ESRM20_boundaries/data/`
    *(Unfortunately we are not allowed to redistribute the data and you'll need a password to access the sources. [Read here](https://git.gfz-potsdam.de/dynamicexposure/datasources/-/tree/master/ESRM20_boundaries) for more information about the data being used.)*
 3. Place the downloaded data into paths and directories of your preference.
+4. If you wish to run `gde-importer` for the industrial exposure of a country for which the geographical units used in ESRM20 are 30-arcsec cells, please read the [special preliminary steps for 30-arcsec industrial cells](#special-preliminary-steps-for-30-arcsec-industrial-cells) down below.
 
 ### Configuration
 
-
 #### Quickstart:
 
 Copy the file `config_example.yml` to your working directory as `config.yml` and provide the necessary parameters:
@@ -85,6 +85,27 @@ The following configuration options are available in the `config.yml`:
 - `data_units_surface_threshold`: Percentage difference (float between 0.0 and 100.0) of geographic areas to define the need to create data units to fill an exposure entity, if the data units defined in an aggregated exposure model do not fully cover the geographic extents of the exposure entity. Default: 1.0%.
 - `force_creation_data_units`: Create data units to fill an exposure entity irrespective of other conditions (e.g. irrespective of `data_units_surface_threshold`). Default: False.
 
+### Special preliminary steps for 30-arcsec industrial cells
+
+The following countries have their industrial exposure models defined in terms of 30-arcsec cells in the ESRM20 model (names and underscores as per naming in ESRM20): Albania, Austria, Belgium, Bosnia_and_Herzegovina, Bulgaria, Croatia, Cyprus, Czechia, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Moldova, Montenegro, Netherlands, North_Macedonia, Norway, Poland, Portugal, Romania, Serbia, Slovakia, Slovenia, Spain, Sweden, Switzerland, United_Kingdom. This list might change. This information can be retrieved from the [ESRM20 exposure model repository](https://gitlab.seismo.ethz.ch/efehr/esrm20_exposure), under the path `esrm20_exposure/sources/European_Exposure_Model_Data_Inputs_Sources.xlsx`.
+
+The ESRM20 model only provides the centroids of the 30-arcsec cells with a certain decimal precision. Some pre-processing of the input files is thus needed so as to create the geometry of the cells around these points export these cells to a geodata file, adjusting the geometries so as to avoid overlaps and gaps between neighbouring cells. It is planned that, in the future, `gde-importer` will be able to handle this peculiarity by itself (without need for pre-processing). However, the current version of `gde-importer` requires that the `SERA_creating_industrial_cells.py` script of the [GDE prototype code](https://git.gfz-potsdam.de/dynamicexposure/legacy/gde_calculations_prototype) be run first. The steps to follow (after steps 1 through 3 in [Obtain data for the European Seismic Risk Model 2020 (ESRM20)](#obtain-data-for-the-european-seismic-risk-model-2020-esrm20)) are:
+
+1. Create a configuration file as per the instructions contained [here](https://git.gfz-potsdam.de/dynamicexposure/legacy/gde_calculations_prototype/-/blob/master/docs/03_Config_File.md) and [this template](https://git.gfz-potsdam.de/dynamicexposure/legacy/gde_calculations_prototype/-/blob/master/GDE_config_file_TEMPLATE.ini). You can name it as you wish, e.g. `GDE_config_industrial_preprocessing.py`. The only two sections that are required are:
+  - "File Paths": indicate the location of the ESRM20 files (`sera_models_path`), the boundaries path (`sera_boundaries_path`) and type (`boundaries_type`) and the output path (`out_path`).
+  - "SERA_creating_industrial_cells": keep all parameters as in the [GDE_config_file_TEMPLATE.ini](https://git.gfz-potsdam.de/dynamicexposure/legacy/gde_calculations_prototype/-/blob/master/GDE_config_file_TEMPLATE.ini) file, except for `countries`, which can be all of the ones listed above or a reduced list of your choice.
+2. Make sure the output path indicated in the configuration file (`out_path`) exists, and that it contains a subfolder called `Ind_Cells`. The output of the code will be written to this path.
+3. Open an instance of `python3` to run `SERA_creating_industrial_cells.py`.
+4. Type `python3 SERA_creating_industrial_cells.py GDE_config_industrial_preprocessing.py` to start running the script.
+5. Once the script has run, the output can be found in `out_path/Ind_Cells` and consists of:
+  - Geodata files with names `Adm99_Country.shp`, which contain the geometries of the created cells.
+  - CSV files with names of the kind `Exposure_Model_Country_Ind.csv`. The contents of these CSV files are the same as those in the original ESRM20 CSV files for these countries, plus two columns named `ID_99` and `NAME_99`, which contain the IDs and names of the created cells' geometries.
+  - `log_consistency_checks.csv`, which indicates if any problem was found. If all cells are False, then no problem was encountered.
+  - `log_processing_times.csv`, which indicates the time it took to process each country, in seconds.
+6. Replace the original ESRM20 `Exposure_Model_Country_Ind.csv` files with the ones in `out_path/Ind_Cells` (in the folder of your choice defined as per the instructions above, where all other ESRM20 CSV files are located).
+7. Place the geodata files with names of the kind `Adm99_Country.shp` in the same directory where the rest of the ESRM20-compatible boundaries are located.
+
+Details on the algorithms of `SERA_creating_industrial_cells.py` can be found [here](https://git.gfz-potsdam.de/dynamicexposure/legacy/gde_calculations_prototype/-/blob/master/docs/08_Industrial_Cells.md).
 
 ## Running gde-importer
 

config_example.yml

@@ -22,3 +22,5 @@ database_gde_tiles:  # Database where info on the GDE tiles is stored
   password: password_of_username
 data_units_surface_threshold: 1.0  # Percentage difference of areas to define the need to create data units to fill an exposure entity
 force_creation_data_units: False  # Create data units to fill an exposure entity irrespective of other conditions; optional (default: False)
+data_units_min_admisible_area: 0.1  # (m2) Minimum surface area of created data units
+data_units_max_admisible_area: 3e9  # (m2) Maximum surface area of created data units

gdeimporter/aggregatedexposuremodel.py

@@ -809,32 +809,49 @@ class ExposureModelESRM20(AggregatedExposureModel):
                 occupancy_case,
                 geometries_table,
                 configuration.number_cores,
+                configuration.data_units_min_admisible_area,
+                configuration.data_units_max_admisible_area,
                 configuration.data_units_surface_threshold,
                 configuration.force_creation_data_units,
             )
             if filler_data_unit_geometry_table.shape[0] > 0:
-                filler_data_unit_id = filler_data_unit_geometry_table["ID_FILLER"].to_numpy()[0]
-                # Retrieve proportions and properties of building classes
-                # (weighted average of the whole exposure entity, as available from 'data_table'
+                # Retrieve proportions and properties of building classes (weighted average of
+                # the whole exposure entity, as available from 'data_table'), applicable to all
+                # filler data units created
                 (
                     building_classes_proportions_and_properties,
                     _,
                 ) = self._retrieve_building_classes(
                     data_table, exposure_entity_name, occupancy_case, True
                 )
-                # Create new DataUnit object
-                self.exposure_entities[exposure_entity_name].occupancy_cases[occupancy_case][
-                    "data_units"
-                ][filler_data_unit_id] = DataUnit(
-                    filler_data_unit_id,
-                    filler_data_unit_geometry_table,
-                    "ID_FILLER",
-                    0.0,  # buildings
-                    0.0,  # dwellings
-                    {"Census": 0.0, "Day": 0.0, "Night": 0.0, "Transit": 0.0, "Average": 0.0},
-                    {"Total": 0.0, "Structural": 0.0, "Non-Structural": 0.0, "Contents": 0.0},
-                    building_classes_proportions_and_properties,
-                )
+
+                for i in range(filler_data_unit_geometry_table.shape[0]):
+                    filler_data_unit_id = filler_data_unit_geometry_table["ID"].to_numpy()[i]
+
+                    # Create new DataUnit object
+                    self.exposure_entities[exposure_entity_name].occupancy_cases[
+                        occupancy_case
+                    ]["data_units"][filler_data_unit_id] = DataUnit(
+                        filler_data_unit_id,
+                        filler_data_unit_geometry_table,
+                        "ID",
+                        0.0,  # buildings
+                        0.0,  # dwellings
+                        {
+                            "Census": 0.0,
+                            "Day": 0.0,
+                            "Night": 0.0,
+                            "Transit": 0.0,
+                            "Average": 0.0,
+                        },
+                        {
+                            "Total": 0.0,
+                            "Structural": 0.0,
+                            "Non-Structural": 0.0,
+                            "Contents": 0.0,
+                        },
+                        building_classes_proportions_and_properties,
+                    )
 
         return
 
@@ -844,6 +861,8 @@ class ExposureModelESRM20(AggregatedExposureModel):
         occupancy_case,
         data_units_geometries_all,
         number_cores,
+        min_admisible_area,
+        max_admisible_area,
         surface_threshold=1.0,
         force_creation_data_units=False,
     ):
@@ -892,14 +911,20 @@ class ExposureModelESRM20(AggregatedExposureModel):
                 Number of CPU cores to be used to run
                 ExposureEntity.sum_surface_area_of_data_units(), which is called by this
                 function.
+            min_admisible_area (float):
+                Minimum surface area (in m2) of the data unit(s) created by this function.
+            max_admisible_area (float):
+                Maximum surface area (in m2) of the data unit(s) created by this function. If
+                the area to cover is larger than 'max_admisible_area', it gets successively
+                subdivided until complying with this requisite.
             surface_threshold (positive float):
                 Absolute threshold of the difference between the surface area of the exposure
                 entity and that of all of its data units that defines the actions to be taken by
                 this function (percentage equal to or larger than zero). Default: 1.0.
             force_creation_data_units (bool):
-                If True, a data unit covering the spatial extent of 'exposure_entity_name' that
-                is not covered by already existing data units will be created, irrespective of
-                all the built-in checks that make such a decision automatically.
+                If True, (a) data unit(s) covering the spatial extent of 'exposure_entity_name'
+                that is not covered by already existing data units will be created, irrespective
+                of all the built-in checks that make such a decision automatically.
 
         Returns:
             filler_data_unit_geometry_table (GeoPandas GeoDataFrame):
@@ -907,7 +932,7 @@ class ExposureModelESRM20(AggregatedExposureModel):
                 this function. If not empty, it contains the geometry and name of the data unit
                 created to cover the missing area of 'exposure_entity_name'. It contains the
                 following columns:
-                    ID_FILLER:
+                    ID:
                         ID of the created data unit.
                     geometry:
                         Geometry of the created data unit.
@@ -1001,25 +1026,41 @@ class ExposureModelESRM20(AggregatedExposureModel):
 
         # Create output GeoDataFrame
         if new_data_unit_geometry is not None:
+            new_data_unit_area = SpatialTools.get_area_albers_equal_area(new_data_unit_geometry)
+
+            # If the new data unit is too big, split it in smaller parts
+            if new_data_unit_area > max_admisible_area:
+                new_data_unit_geometries = (
+                    SpatialTools.subdivide_multipolygon_in_parts_by_max_area(
+                        new_data_unit_geometry, min_admisible_area, max_admisible_area
+                    )
+                )
+            else:
+                new_data_unit_geometries = [new_data_unit_geometry]
+
+            number_new_data_units = len(new_data_unit_geometries)
+            new_data_unit_ids = numpy.array(
+                [
+                    "%s_FILLER_%s" % (occupancy_case, count)
+                    for count in range(1, number_new_data_units + 1)
+                ]
+            )
+
             logger.info(
-                "A data unit has been created for Exposure Entity '%s', "
-                "occupance case '%s'."
+                "%s data unit(s) were created for Exposure Entity '%s', occupancy case '%s'."
                 % (
+                    number_new_data_units,
                     exposure_entity_name,
                     occupancy_case,
                 )
             )
 
-            filler_data_unit_id = "%s_FILLER" % (occupancy_case)
-
             filler_data_unit_geometry_table = geopandas.GeoDataFrame(
-                {"ID_FILLER": [filler_data_unit_id], "geometry": [new_data_unit_geometry]}
+                {"ID": new_data_unit_ids, "geometry": new_data_unit_geometries}
             )
 
         else:  # Create empty GeoDataFrame
-            filler_data_unit_geometry_table = geopandas.GeoDataFrame(
-                columns=["ID_FILLER", "geometry"]
-            )
+            filler_data_unit_geometry_table = geopandas.GeoDataFrame(columns=["ID", "geometry"])
 
         filler_data_unit_geometry_table.crs = pyproj.CRS("epsg:4326")
 
@@ -1066,7 +1107,7 @@ class ExposureModelESRM20(AggregatedExposureModel):
             data_units_definition = "NUTS"
         elif "30 arc seconds" in original_description:
             data_units_type = "cell"
-            data_units_level = "30arcsec_30arcsec"
+            data_units_level = "99"
             data_units_definition = "WGS84"
         else:
             data_units_type = "unknown"

gdeimporter/configuration.py

@@ -97,6 +97,17 @@ class Configuration:
         self.force_creation_data_units (bool):
             Create data units to fill an exposure entity irrespective of other conditions (e.g.
             irrespective of self.surface_threshold). Default: False.
+        self.data_units_min_admisible_area (float):
+            Minimum surface area (in m2) of data units created to fill an exposure entity if the
+            data units defined in an aggregated exposure model do not fully cover the geographic
+            extents of the exposure entity. Needs to be smaller than
+            'data_units_max_admisible_area'.
+        self.data_units_max_admisible_area (float):
+            Maximum surface area (in m2) of data units created to fill an exposure entity if the
+            data units defined in an aggregated exposure model do not fully cover the geographic
+            extents of the exposure entity. If the area to cover is larger than
+            'data_units_max_admisible_area', it gets successively subdivided until complying
+            with this requisite. Needs to be larger than 'data_units_min_admisible_area'.
     """
 
     REQUIRES = [
@@ -109,6 +120,8 @@ class Configuration:
         "database_built_up",
         "database_gde_tiles",
         "data_units_surface_threshold",
+        "data_units_min_admisible_area",
+        "data_units_max_admisible_area",
     ]
 
     def __init__(self, filepath, force_config_over_hierarchies=False):
@@ -178,6 +191,16 @@ class Configuration:
                 "configuration file and was automatically set to 'False'."
             )
             self.force_creation_data_units = False
+        self.data_units_min_admisible_area = self._assign_float_parameter(
+            config, "data_units_min_admisible_area", True, 1e-15, 30e12
+        )
+        self.data_units_max_admisible_area = self._assign_float_parameter(
+            config,
+            "data_units_max_admisible_area",
+            True,
+            self.data_units_min_admisible_area,
+            30e12,
+        )
 
         # Terminate if critical parameters are missing (not all parameters are critical)
         for key_parameter in self.REQUIRES:

gdeimporter/tools/spatial.py

@@ -334,10 +334,46 @@ class SpatialTools:
         if out_geometry_gdf.shape[0] > 1:
             out_geometry_gdf = out_geometry_gdf.dissolve()
 
+        out_geometry_gdf = SpatialTools.sanitise_invalid_geometries(out_geometry_gdf)
+
         out_geometry = out_geometry_gdf["geometry"].to_numpy()[0]
 
         return out_geometry
 
+    @staticmethod
+    def sanitise_invalid_geometries(geodataframe):
+        """This function checks whether 'geodataframe' contains any invalid geometries. If it
+        does, it fixes them; if not, it returns the same input 'geodataframe'.
+
+        Args:
+            geodataframe (GeoPandas GeoDataFrame):
+                GeoPandas GeoDataFrame whose geometries will be sanitised if found to be
+                invalid.
+
+        Returns:
+            geodataframe (GeoPandas GeoDataFrame):
+                Sanitised GeoPandas GeoDataFrame.
+        """
+
+        need_to_sanitise = False
+
+        for geometry in geodataframe["geometry"]:
+            if geometry.geom_type == "Polygon":
+                if not geometry.is_valid:
+                    need_to_sanitise = True
+                    break
+            elif geometry.geom_type == "MultiPolygon":
+                for subgeometry in geometry.geoms:
+                    if not subgeometry.is_valid:
+                        need_to_sanitise = True
+                        break
+
+        if need_to_sanitise:
+            sanitised_series = geodataframe.buffer(0)
+            geodataframe = geopandas.GeoDataFrame(geometry=sanitised_series)
+
+        return geodataframe
+
     @staticmethod
     def single_geometries_intersection(in_geometry_1, in_geometry_2, in_crs="epsg:4326"):
         """This function returns the intersection of 'in_geometry_1' and 'in_geometry_2',
@@ -495,3 +531,175 @@ class SpatialTools:
             geodataframe[column_name] = geodataframe[column_name].astype(datatypes[column_name])
 
         return geodataframe
+
+    @staticmethod
+    def subdivide_multipolygon_in_parts_by_max_area(
+        in_geometry, min_admisible_area, max_admisible_area, min_ew=2e-10, min_ns=2e-10
+    ):
+        """This function subdivides `in_geometry` successively until the largest piece has an
+        area (calculated with the Albers equal area projection) no larger than
+        'max_admisible_area'. It also checks that the individual pieces of `in_geometry` have a
+        minimum east-west length of 'min_ew' and a minimum north-south length of 'min_ns'. It is
+        assumed that 'in_geometry', 'min_ew' and 'min_ns' are provided in compatible units (e.g.
+        if 'in_geometry' is defined in ESPG:4326, then 'min_ew' and 'min_ns' should be expressed
+        in decimal degrees).
+
+        Args:
+            in_geometry (Shapely Polygon or MultiPolygon):
+                Geometry to be split.
+            min_admisible_area (float):
+                Lower limit of surface area that the output pieces of the subdivided
+                'in_geometry' should always exceed, in m2.
+            max_admisible_area (float):
+                Upper limit of surface area that the output pieces of the subdivided
+                'in_geometry' should not reach, in m2.
+            min_ew (float):
+                Minimum east-west dimension. Default: 2e-10 (two times the epsilon used by
+                mercantile as buffer for determining the tiles associated with a bounding box).
+            min_ns (float):
+                Minimum north-south dimension. Default: 2e-10 (two times the epsilon used by
+                mercantile as buffer for determining the tiles associated with a bounding box).
+
+        Returns:
+            out_geometries (list of Shapely Polygon or MultiPolygon):
+                List of Shapely Polygons and/or MultiPolygons, whose largest component has a
+                surface area no larger than 'max_admisible_area' and whose smallest component
+                has a surface area no smaller than 'min_admisible_area'.
+        """
+
+        out_geometries = [in_geometry]
+        areas = numpy.array([SpatialTools.get_area_albers_equal_area(in_geometry)])
+
+        while areas.max() > max_admisible_area:  # keep on subdividing
+            aux_out_geometries = []
+            aux_areas = []
+            for i, geometry in enumerate(out_geometries):
+                if areas[i] > max_admisible_area:
+                    subdivided_geometries = (
+                        SpatialTools.subdivide_multipolygon_in_four_quadrants(geometry)
+                    )
+                    subdivided_geometries = SpatialTools.discard_narrow_geometries(
+                        subdivided_geometries, min_ew, min_ns
+                    )
+                    aux_out_geometries = aux_out_geometries + subdivided_geometries
+                    aux_areas = aux_areas + [
+                        SpatialTools.get_area_albers_equal_area(subdivided_geometries[i])
+                        for i in range(len(subdivided_geometries))
+                    ]
+                else:
+                    if SpatialTools.bounding_box_is_larger_than_threshold(
+                        geometry, min_ew, min_ns
+                    ):
+                        aux_out_geometries.append(geometry)
+                        aux_areas.append(areas[i])
+            out_geometries = deepcopy(aux_out_geometries)
+            areas = numpy.array(aux_areas)
+
+        which_larger_than_min = numpy.where(areas > min_admisible_area)[0]
+        out_geometries = [out_geometries[i] for i in which_larger_than_min]
+
+        return out_geometries
+
+    @staticmethod
+    def subdivide_multipolygon_in_four_quadrants(in_geometry):
+        """This function subdivides `in_geometry` by cutting it vertically and horizontally as
+        per the vertical and horizontal lines that would divide its bounding box into four equal
+        parts.
+
+        The output is not necessarily four geometries, as it is possible that one of the four
+        quadrants of the bounding box of 'in_geometry' may not contain any piece of
+        'in_geometry', for example, or that there are more than four geometries because
+        'in_geometry' is a MultiPolygon.
+
+        Args:
+            in_geometry (Shapely Polygon or MultiPolygon):
+                Geometry to be split.
+
+        Returns:
+            out_geometries (list of Shapely Polygon or MultiPolygon):
+                List containing the output geometries, i.e. the sub-pieces of 'in_geometry'.
+        """
+
+        lon_w, lat_s, lon_e, lat_n = in_geometry.bounds
+
+        line_ew = shapely.geometry.LineString(
+            [(lon_w, (lat_s + lat_n) / 2.0), (lon_e, (lat_s + lat_n) / 2.0)]
+        )
+        line_ns = shapely.geometry.LineString(
+            [((lon_w + lon_e) / 2.0, lat_s), ((lon_w + lon_e) / 2.0, lat_n)]
+        )
+
+        # Divide into an eastern and western part by means of a north-south line
+        divided_ew = list(shapely.ops.split(in_geometry, line_ns).geoms)
+
+        out_geometries = []
+
+        for sub_geom in divided_ew:
+            # Divide into a northern and western part by means of an east-west line
+            divided_ns = list(shapely.ops.split(sub_geom, line_ew).geoms)
+            for sub_sub_geom in divided_ns:
+                out_geometries.append(sub_sub_geom)
+
+        return out_geometries
+
+    @staticmethod
+    def discard_narrow_geometries(in_geometries, min_ew, min_ns):
+        """This function verifies whether the bounding box of each element of 'in_geometries'
+        has a minimum east-west length of 'min_ew' and a minimum north-south length of
+        'min_ns', and discards those elements of 'in_geometries' that do not comply with this
+        criterion. It is assumed that all the elements of 'in_geometries', 'min_ew' and 'min_ns'
+        are provided in compatible units (e.g. if 'in_geometries' are defined in ESPG:4326, then
+        'min_ew' and 'min_ns' should be expressed in decimal degrees).
+
+        Args:
+            in_geometries (list of Shapely Polygons or MultiPolygons):
+                List of geometry to be analysed.
+            min_ew (float):
+                Minimum east-west dimension.
+            min_ns (float):
+                Minimum north-south dimension.
+
+        Returns:
+            out_geometries (list of Shapely Polygons or MultiPolygons):
+                List of elements of 'in_geometries' that complied with the selection criterion.
+        """
+
+        out_geometries = []
+        for geometry in in_geometries:
+            if SpatialTools.bounding_box_is_larger_than_threshold(geometry, min_ew, min_ns):
+                out_geometries.append(geometry)
+
+        return out_geometries
+
+    @staticmethod
+    def bounding_box_is_larger_than_threshold(in_geometry, min_ew, min_ns):
+        """This function verifies whether the bounding box of 'in_geometry' has a minimum
+        east-west length of 'min_ew' and a minimum north-south length of 'min_ns'. It returns
+        True if this is the case and False otherwise. It is assumed that 'in_geometry', 'min_ew'
+        and 'min_ns' are provided in compatible units (e.g. if 'in_geometry' is defined in
+        ESPG:4326, then 'min_ew' and 'min_ns' should be expressed in decimal degrees).
+
+        Args:
+            in_geometry (Shapely Polygon or MultiPolygon):
+                Geometry to be verified.
+            min_ew (float):
+                Minimum east-west dimension.
+            min_ns (float):
+                Minimum north-south dimension.
+
+        Returns (bool):
+            True if the bounding box of 'in_geometry' has the specified minimum dimensions,
+            False otherwise.
+        """
+
+        if SpatialTools.crosses_antimeridian(in_geometry):
+            raise OSError(
+                "verify_bounding_box_larger_than_threshold: geometry crosses the antimeridian"
+            )
+
+        lon_w, lat_s, lon_e, lat_n = in_geometry.bounds
+
+        if (lon_e - lon_w < min_ew) or (lat_n - lat_s < min_ns):
+            return False  # the bounding box is not larger than the threshold/s
+        else:
+            return True  # the bounding box is larger than the threshold/s

tests/data/ExposureModelESRM20_non_trivial/shapefiles/unit_data_geometries.csv

@@ -8,6 +8,11 @@ Entity_1,Unit_3,16.00,17.00,40.00,39.00
 Entity_2,Unit_A,21.00,22.00,45.00,44.00
 Entity_2,Unit_B,22.00,23.00,46.00,45.00
 Entity_2,Unit_C,23.00,24.00,47.00,46.00
-Entity_1,residential_FILLER,12.00,17.00,40.00,35.02
-Entity_1,commercial_FILLER,14.00,17.00,40.00,37.00
-Entity_2,commercial_FILLER,22.50,23.00,46.00,45.00
+Entity_1,residential_FILLER_1,12.00,17.00,40.00,35.02
+Entity_1,commercial_FILLER_1,14.00,15.00,38.00,37.00
+Entity_1,commercial_FILLER_2,15.00,15.50,38.50,38.00
+Entity_1,commercial_FILLER_3,15.00,15.50,39.00,38.50
+Entity_1,commercial_FILLER_4,15.50,16.00,39.00,38.50
+Entity_1,commercial_FILLER_5,15.50,16.00,38.50,38.00
+Entity_1,commercial_FILLER_6,16.00,17.00,40.00,39.00
+Entity_2,commercial_FILLER_1,22.50,23.00,46.00,45.00

tests/data/config_for_testing_cores_not_integer.yml

@@ -12,3 +12,5 @@ database_built_up:
   username: some_username
   password: some_password
 data_units_surface_threshold: 1.5
+data_units_min_admisible_area: 0.1
+data_units_max_admisible_area: 3e9

tests/data/config_for_testing_data_units_min_max_admisible_area_out_of_range.yml

+model_name: esrm20
+exposure_format: esrm20
+data_pathname: /some/path/to/directory
+boundaries_pathname: /some/path/to/directory
+occupancies_to_run: residential, commercial, industrial
+exposure_entities_to_run: all
+exposure_entities_code:
+  Entity_1: EN1
+  Entity_2: EN2
+  Entity_3: EN3
+  Entity_4: EN4
+  Entity_5: EN5
+  Entity_6: EN6
+  Entity_7: EN7
+  Entity_8: EN8
+number_cores: 4
+database_built_up:
+  host: host.somewhere.xx
+  dbname: some_database_name
+  username: some_username
+  password: some_password
+database_gde_tiles:
+  host: host.somewhere.xx
+  dbname: some_database_name
+  username: some_username
+  password: some_password
+data_units_surface_threshold: 1.5
+force_creation_data_units: False
+data_units_min_admisible_area: 3.1e9
+data_units_max_admisible_area: 3e9

tests/data/config_for_testing_data_units_surface_threshold_negative.yml

@@ -26,3 +26,5 @@ database_gde_tiles:
   password: some_password
 data_units_surface_threshold: -1E-16
 force_creation_data_units: False
+data_units_min_admisible_area: 0.1
+data_units_max_admisible_area: 3e9

tests/data/config_for_testing_data_units_surface_threshold_out_of_range.yml

@@ -26,3 +26,5 @@ database_gde_tiles:
   password: some_password
 data_units_surface_threshold: 123.5
 force_creation_data_units: False
+data_units_min_admisible_area: 0.1
+data_units_max_admisible_area: 3e9

tests/data/config_for_testing_db_built_up_not_dict.yml

@@ -8,3 +8,5 @@ exposure_entities_code: ISO3
 number_cores: 4
 database_built_up: some_database_name
 data_units_surface_threshold: 1.5
+data_units_min_admisible_area: 0.1
+data_units_max_admisible_area: 3e9

tests/data/config_for_testing_dummy.yml

@@ -20,3 +20,5 @@ database_gde_tiles:
   username: xxx
   password: xxx
 data_units_surface_threshold: 1.5
+data_units_min_admisible_area: 0.1
+data_units_max_admisible_area: 3e9

tests/data/config_for_testing_exposure_entities_code_dict_dict.yml

@@ -21,3 +21,5 @@ database_gde_tiles:
   username: some_username
   password: some_password
 data_units_surface_threshold: 1.5
+data_units_min_admisible_area: 0.1
+data_units_max_admisible_area: 3e9

tests/data/config_for_testing_exposure_entities_code_iso3.yml

@@ -17,3 +17,5 @@ database_gde_tiles:
   username: some_username
   password: some_password
 data_units_surface_threshold: 1.5
+data_units_min_admisible_area: 0.1
+data_units_max_admisible_area: 3e9

tests/data/config_for_testing_exposure_entities_code_wrong_dict.yml

@@ -19,3 +19,5 @@ database_gde_tiles:
   username: some_username
   password: some_password
 data_units_surface_threshold: 1.5
+data_units_min_admisible_area: 0.1
+data_units_max_admisible_area: 3e9

tests/data/config_for_testing_exposure_entities_code_wrong_string.yml

@@ -17,3 +17,5 @@ database_gde_tiles:
   username: some_username
   password: some_password
 data_units_surface_threshold: 1.5
+data_units_min_admisible_area: 0.1
+data_units_max_admisible_area: 3e9

tests/data/config_for_testing_good.yml

@@ -26,3 +26,5 @@ database_gde_tiles:
   password: some_password
 data_units_surface_threshold: 1.5
 force_creation_data_units: False
+data_units_min_admisible_area: 0.1
+data_units_max_admisible_area: 28e9

tests/data/config_for_testing_one_list_val.yml

@@ -17,3 +17,5 @@ database_gde_tiles:
   username: some_username
   password: some_password
 data_units_surface_threshold: 1.5
+data_units_min_admisible_area: 0.1