shakyground2 merge requestshttps://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests2021-02-02T15:44:11+01:00https://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/1setup rudimentary project structure2021-02-02T15:44:11+01:00Marius Kriegerowskimarius@gfz-potsdam.desetup rudimentary project structure@all please have a look at the rudimentary project structure.
The Makefile is a shortcut for linting. Just run
make check
to check your code or
make black
to let black format it. That's also documented in the README.md.
We...@all please have a look at the rudimentary project structure.
The Makefile is a shortcut for linting. Just run
make check
to check your code or
make black
to let black format it. That's also documented in the README.md.
We use a line length or 96 in other projects. But I'm open for anything between 80 and 120. Thoughts?https://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/2Add basic ci pipeline2021-02-03T17:36:51+01:00Marius Kriegerowskimarius@gfz-potsdam.deAdd basic ci pipelinehttps://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/3Add APGL license and default .gitignore2021-02-14T12:05:51+01:00Riccardo ZaccarelliAdd APGL license and default .gitignoreSee title
(Ps: created new branch "dev", if you prefer that we fork the repository for new Merge requests, let me know)See title
(Ps: created new branch "dev", if you prefer that we fork the repository for new Merge requests, let me know)Marius Kriegerowskimarius@gfz-potsdam.deMarius Kriegerowskimarius@gfz-potsdam.dehttps://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/4Implements Earthquake Object and Input validators2021-02-25T15:31:06+01:00Graeme WeatherillImplements Earthquake Object and Input validatorsMerge request implements initial set of features to describe an earthquake object, valid an input and generate a finite rupture surface from which source and distance properties can be calculated. Addresses: https://git.gfz-potsdam.de/sh...Merge request implements initial set of features to describe an earthquake object, valid an input and generate a finite rupture surface from which source and distance properties can be calculated. Addresses: https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/8
New features implemented are:
1. Earthquake object (`shakyground2.earthquake.Earthquake`) - contains attributes and initial functions for describing the earthquake source
2. Validation module (`shakyground2.valid`) - a set of validators to ensure input properties of the earthquake are physically correct with respect to the Earth and raise descriptive errors if not.
3. Dummy Magnitude Scaling relation (`shakyground2.earthquake.DummyScalingRelation`) - a simple magnitude to rupture dimension scaling relation based on the PEER Test definition. To be used as a placeholder until completion of https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/9, then a default thereafter.
Tests added for all modules.Graeme WeatherillGraeme Weatherillhttps://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/6Implements Feature/site model to handle the site configuration2021-02-25T15:36:06+01:00Graeme WeatherillImplements Feature/site model to handle the site configurationThis MR implements the `shakyground2.site_model.SiteModel` class to manage the construction and manipulation of the site properties needed for the shakemap, as specified in https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/12. Al...This MR implements the `shakyground2.site_model.SiteModel` class to manage the construction and manipulation of the site properties needed for the shakemap, as specified in https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/12. Allows construction of the site model from a bounding box (with constant site properties) or `pandas` dataframe, and handles default properties for cases where they are not specified for the user.
Note:
1. The site class itself mirrors some properties of the OpenQuake `SiteCollection` object, and indeed the site model will be exported to the OpenQuake `SiteCollection` object for running the shakemaps. This approach was adopted in favour of direct inheritance and extension of the `SiteCollection` object, which requires a stricter control on the inputs and manipulates the site model in a manner more specific to the OpenQuake calculation cases than the shakemap case considered here. The `SiteModel` here acts as an interface to handle the likelier range of inputs encountered for shakemap applications and to provide more control on how these are handled in order to build the site model. Both ways (inheritance and composition) were compared and the composition approach favoured.
2. This does not address the case in which the input Vs30 data is extracted from the USGS Global Vs30 database. That will be a critical workflow feature, and methods for handling that will be added to this class subsequently, but this requires further consideration. A new issue will be opened on this in due course.https://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/8Implements a more complete set of magnitude scaling relations2021-03-04T12:50:58+01:00Graeme WeatherillImplements a more complete set of magnitude scaling relationsThe construction of the rupture planes needed for the shakemap requires a magnitude scaling relation that calculates the expected finite dimensions of the rupture from the earthquake magnitude and related rupture properties. This MR remo...The construction of the rupture planes needed for the shakemap requires a magnitude scaling relation that calculates the expected finite dimensions of the rupture from the earthquake magnitude and related rupture properties. This MR removes the previous placeholder `DummyScalingRelation` and transfers its functionality to a more complete set of magnitude scaling relations including:
1. `PEERScalingRelation`: an idealised scaling relation adopted by the Pacific Earthquake Engineering Research Center (previously this relation was coded within the `DummyScalingRelation`)
2. `StrasserEtAl2010Interface` and `StrasserEtAl2010Inslab`: Relations for subduction environments by Strasser et al. (2010) (https://pubs.geoscienceworld.org/ssa/srl/article-abstract/81/6/941/143755/Scaling-of-the-Source-Dimensions-of-Interface-and?redirectedFrom=fulltext)
3. `Stafford2014`: A "hazard-consistent" scaling relation created by Peter Stafford (https://pubs.geoscienceworld.org/bssa/article-lookup/104/4/1620?casa_token=MN9JOoAHhvAAAAAA:eGlBEFnrLFw9i204_RqDDTSBboKbaMTKzXdhvCmv9yNVz_UfHvmagVMCTDVL3iwj5DWGXtoN)
An additional validation for correct scaling relation inputs is added to the `shakyground2.valid` module.Graeme WeatherillGraeme Weatherillhttps://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/9Adds Rupture Mechanism class2021-03-04T13:09:49+01:00Graeme WeatherillAdds Rupture Mechanism classThe generation of the 3D rupture planes for characterising the finite rupture distances depends on the rupture mechanism (strike, dip and rake). As this information may be absent or incomplete, the mechanism itself is defined as a probab...The generation of the 3D rupture planes for characterising the finite rupture distances depends on the rupture mechanism (strike, dip and rake). As this information may be absent or incomplete, the mechanism itself is defined as a probability mass function, with multiple possibles mechanisms and their corresponding probabilities, and this is subsequently used to generate the probability density of source-to-site distances within the shakemap calculation. This MR implements an object for rupture mechanism distribution from no data, incomplete data and complete data.https://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/10Bugfix/dependencies2021-03-04T15:34:09+01:00Marius Kriegerowskimarius@gfz-potsdam.deBugfix/dependencieshttps://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/11linting source code and tests2021-03-04T17:43:30+01:00Marius Kriegerowskimarius@gfz-potsdam.delinting source code and testsApplied `black` to auto-lint the source code.Applied `black` to auto-lint the source code.https://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/12Implements initial complete shakemap feature2021-03-11T16:42:57+01:00Graeme WeatherillImplements initial complete shakemap featureImplements the initial `Shakemap` class that combines the earthquake, site and ground motion models (here defined in terms of a dictionary containing a list of models and a list of their respective weights). This MR addresses two issues:...Implements the initial `Shakemap` class that combines the earthquake, site and ground motion models (here defined in terms of a dictionary containing a list of models and a list of their respective weights). This MR addresses two issues:
1. The migration of the main shakemap calculations from the original shakyground repository (https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/10)
2. The implementation of the synthetic rupture generator to handle the construction rupture surface needed for the ground motion calculations given the limited available information on the earthquake source (https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/9)
Additional methods have been added to the `Earthquake`, `SiteModel` and `RuptureMechanism` classes to handle the synthetic rupture generation and subsequent shakemaps more cleanly.https://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/13Implements classes to handle Regionalization and assignment of ground motion ...2021-04-28T11:54:16+02:00Graeme WeatherillImplements classes to handle Regionalization and assignment of ground motion modelsFor shakemap applications a set of ground motion models (and their corresponding weights) needs to be defined for each earthquake. Though these can be defined manually, it also desirable to define a `Regionalization`, i.e. a set of geogr...For shakemap applications a set of ground motion models (and their corresponding weights) needs to be defined for each earthquake. Though these can be defined manually, it also desirable to define a `Regionalization`, i.e. a set of geographical regions each with their own associated set of ground motion models. This is explained in https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/7
This MR implements two classes for handling regionalizations at any scale:
1. A `Regionalization` class, that loads and stores a given regionalisation and can, for a given earthquake, apply the geographical selection procedure to identify the region to which the earthquake is associated and its corresponding ground motion models.
2. A `RegionalizationSet` class, that manages multiple regionalizations, the order of which defines their own hierarchy. For example, a different regionalization may be required for `A`) a country, `B`) a continent (the spatial domain of which may include part or all of the country `A`), and `C`) a global scale regionalization. The order in which the `Regionalization` objects are input into the `RegionalizationSet` defines the hierarchy, e.g. if the three regionalizations are input in the order `A B C` then regionalization A will be applied if the earthquake falls within its domain, then `B` or `C` otherwise. If the order were reversed (`C B A`) then the global regionalization (`C`) would be applied in favour of the local regionalizations (`A` or `B`).
The geographical regions are managed within the `Regionalization` class as a GeoPandas GeoDataFrame, which must include the attributes `id`, `REGION`, `LOWER DEPTH`, `UPPER DEPTH` and geometry. The ground motion model mapping is manages as a dictionary, the structure of which reflects the json definition explained in https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/7.
A `classmethod` is include in both `Regionalization` and `RegionalizationSet` that allows the user to define the regionalization in the form of a `GeoJSON`, and the corresponding ground motion model mapping as a standard `json`. If a region type is found in the `GeoJSON` for which the ground motion model mapping is not defined in then an error is raised.
A set of files defining the "standard GFZ" ground motion model is also included, though further edits to these may be made in future merge requests.
For the geometry configurations tested so far, a reasonable amount of speed is gained by using `Rtree` for spatial indexing in order to determine the polygon in which the earthquake falls. This dependency is added to the `setup.py` file.https://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/15Adds exporters for shakemap raster formats (geotiff/esri ascii) and contours ...2021-04-21T15:30:14+02:00Graeme WeatherillAdds exporters for shakemap raster formats (geotiff/esri ascii) and contours (geojson)Addresses https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/16
A preliminary set of exporters is added to the `Shakemap` class to cover the initial needs of the EarthExplorer project:
1. Transformation of the shakemap to raster...Addresses https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/16
A preliminary set of exporters is added to the `Shakemap` class to cover the initial needs of the EarthExplorer project:
1. Transformation of the shakemap to raster and direct export to GeoTIFF or to ESRI ASCII file
2. Retrieval of contours from the shakemap and export these to GeoJSON
For the handling of the raster format I am currently using `rasterio` (now added to the dependencies). Future merge requests may add features that require more direct use of the GDAL libraries for handling rasters (e.g. `geopackage`), in which case the GeoTIFF exporter may be replaced with one based on GDAL and the `rasterio` dependency removed.https://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/14Import fdsnws2021-04-28T13:58:06+02:00Peter EvansImport fdsnwsAddresses #15Addresses #15https://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/18Implements single workflow function for GEOFON (Earth Explorer) use case2021-05-25T08:09:35+02:00Graeme WeatherillImplements single workflow function for GEOFON (Earth Explorer) use caseThough the shakyground2 library itself is intended to be modular, specific workflows that chain together the features in order to address a particular use case can make the library easier to call. A `workflows.py` module is added to begi...Though the shakyground2 library itself is intended to be modular, specific workflows that chain together the features in order to address a particular use case can make the library easier to call. A `workflows.py` module is added to begin to support such cases, and this MR in particular addresses the Earth Explorer (GEOFON) shakemap application. Here the workflow: i) creates the `Earthquake` class from the GEOFON FDSNWS (via the event ID) or local QuakeML, ii) builds the site model from bounding box centered on the event location, iii) retrieves the appropriate ground motion model from the regionalisation, iv) runs the shakemap calculations, v) exports the shakemap rasters to byte objects and the corresponding contours to GeoDataFrames
The entire workflow is run via the function: `geofon_workflow(event_id, imts)`, where `event_id` is the GEOFON event ID and `imts` a list of intensity measure types (e.g. `["PGA", "SA(0.2)", "SA(1.0)"]`).
There are several configurable options in the different steps of the workflow and these can be set via an optional `config` input. If these are not defined (or only partially defined) then a default set of configuration properties are included in the `GEOFON_CONFIGURATION` dictionary.https://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/17make gitlab ci jobs interruptible and silence install2021-05-25T09:05:38+02:00Marius Kriegerowskimarius@gfz-potsdam.demake gitlab ci jobs interruptible and silence installSilencing the installation of dependencies. This allows to focus more on the actual testing its' output.
Also, tests are marked is interruptible. This will cancel jobs if new changes were pushed to the same MR, making previous test runs...Silencing the installation of dependencies. This allows to focus more on the actual testing its' output.
Also, tests are marked is interruptible. This will cancel jobs if new changes were pushed to the same MR, making previous test runs obsolete. Speeds up the pipeline.
\approve @rizac @gweatherhttps://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/19Small edit to xml file2021-05-25T15:47:58+02:00Graeme WeatherillSmall edit to xml filehttps://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/16add pre-commit configuration, docs and gitlab-ci integration2021-05-25T16:09:31+02:00Marius Kriegerowskimarius@gfz-potsdam.deadd pre-commit configuration, docs and gitlab-ci integrationThis MR adds pre-commit for improved workflow.
\approve @gweather @rizac
\fyi @allThis MR adds pre-commit for improved workflow.
\approve @gweather @rizac
\fyi @allhttps://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/20Fixes datetime formatting and adds new error for FDSN failure2021-05-26T17:21:17+02:00Graeme WeatherillFixes datetime formatting and adds new error for FDSN failure1. Fixes the datetime formatting error mentioned here: https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/22
2. Raises an `HTTPException` when the FDSN web service is not available (or not found) for a certain event. As raised he...1. Fixes the datetime formatting error mentioned here: https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/22
2. Raises an `HTTPException` when the FDSN web service is not available (or not found) for a certain event. As raised here: https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/23https://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/21Another fix to the datetime isoformat2021-05-27T09:29:53+02:00Graeme WeatherillAnother fix to the datetime isoformatStill addressing: https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/22
Changes formatting of the "second" string in the datetime.from_isoformat function. Now adds a test of the `Earthquake.from_quakeml` function that inputs vario...Still addressing: https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/22
Changes formatting of the "second" string in the datetime.from_isoformat function. Now adds a test of the `Earthquake.from_quakeml` function that inputs various GEOFON event IDs and verifies that the `Earthquake` classes are built correctly.https://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/23Adds logging of shakemap statistics in shakemap_from_quakeml workflow2021-06-02T14:23:01+02:00Graeme WeatherillAdds logging of shakemap statistics in shakemap_from_quakeml workflowAddresses https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/20
Adds `statistics` to the `ShakemapWorkflowResult` class to log some summary statistics about the shakemaps themselves. This could make it easier in the future to, fo...Addresses https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/20
Adds `statistics` to the `ShakemapWorkflowResult` class to log some summary statistics about the shakemaps themselves. This could make it easier in the future to, for example, filter or sort some shakemap results by maximum ground motion or average ground motion. Initially only the maximum, minimum, geometric mean and log IQR of the ground motion fields are stored. Other information could be added in future.