shakyground2 merge requestshttps://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests2022-05-31T08:59:24+02:00https://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/36Fixes bug in quakeml reader, wrong magnitudes2022-05-31T08:59:24+02:00Graeme WeatherillFixes bug in quakeml reader, wrong magnitudesAddresses https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/29
The quakeml reader was returning the magnitude corresponding to the last in the QuakeML file rather than the preferred magnitude. This MR fixes this to ensure that t...Addresses https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/29
The quakeml reader was returning the magnitude corresponding to the last in the QuakeML file rather than the preferred magnitude. This MR fixes this to ensure that the preferred magnitude is returned.Graeme WeatherillGraeme Weatherillhttps://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/35Bumps black to 22.3.0, fixes OQ to 3.13.02022-05-05T11:11:48+02:00Graeme WeatherillBumps black to 22.3.0, fixes OQ to 3.13.0https://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/34Set explicit maximum distance for OQ ContextMaker2022-03-31T15:58:39+02:00Graeme WeatherillSet explicit maximum distance for OQ ContextMakerFixes problem of different number of sites input into GMM than expected (as mentioned in the thread here: https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/25)
Recent changes in OpenQuake meant that a default maximum distance of...Fixes problem of different number of sites input into GMM than expected (as mentioned in the thread here: https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/25)
Recent changes in OpenQuake meant that a default maximum distance of 1000 km for the ContextMaker calculations has been introduced when the `maximum_distances` is not specified in the OQ mock config dictionary. For some larger earthquakes the points in the `SiteModel` at the edge of the bounding box were being filtered out and only those within the maximum distance retained. As the pre-allocated array size for the shakemaps is based on the `SiteModel` size, this meant that ground motions were being calculated at fewer sites than expected; hence the array size mismatch.
This MR sets an explicit maximum distance of 10,000 km for the `ContextMaker`, which should avoid the problem for all conceivable earthquake magnitudes.
A couple of minor edits were added to remove some warnings regarding deprecated numpy datatype usage (e.g. np.float, np.bool etc), and a black fix.Graeme WeatherillGraeme Weatherillhttps://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/33Add workaround for point segments on extracting the contours2022-03-31T11:27:59+02:00Nils BrinckmannAdd workaround for point segments on extracting the contoursCloses #28Closes #28Nils BrinckmannNils Brinckmannhttps://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/32Bug/oq change dec20212022-02-07T09:01:17+01:00Graeme WeatherillBug/oq change dec2021More upstream changes to the `openquake.hazardlib.contexts.ContextMaker` class broke the Shakemap calculator (yet again!). Fundamentally now one cannot work with the three separate contexts (RuptureContext, SitesContext, DistancesContext...More upstream changes to the `openquake.hazardlib.contexts.ContextMaker` class broke the Shakemap calculator (yet again!). Fundamentally now one cannot work with the three separate contexts (RuptureContext, SitesContext, DistancesContext) but with the single context object. Trying to keep changes to a minimum I am still calling the GMPEs themselves with three inputs (now just the context object repeated), which is still supported by OpenQuake specifically for this backward compatibility, even if it is inelegant.Graeme WeatherillGraeme Weatherillhttps://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/31Compute IQR more manual2022-01-19T16:07:04+01:00Nils BrinckmannCompute IQR more manualIssue #27
This is my first idea on how to solve the IQR issue. It is not the most efficient thing as it needs to sort for the quantiles 2 times. But it would solve the problem from my point of view.Issue #27
This is my first idea on how to solve the IQR issue. It is not the most efficient thing as it needs to sort for the quantiles 2 times. But it would solve the problem from my point of view.Nils BrinckmannNils Brinckmannhttps://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/30Use orientation from upper left corner for all geotiffs2021-12-02T12:54:04+01:00Nils BrinckmannUse orientation from upper left corner for all geotiffsCloses #26
This MR uses a transformation that will always orient from the upper left corner.Closes #26
This MR uses a transformation that will always orient from the upper left corner.Nils BrinckmannNils Brinckmannhttps://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/29Adds the SHARE regionalisation to the regionalisation files and re-name files...2021-10-07T18:50:08+02:00Graeme WeatherillAdds the SHARE regionalisation to the regionalisation files and re-name files for better consistencyTo facilitate shakyground2 becoming a tool for eGSIM, the SHARE regionalisation is implemented according to the json/geojson structure adopted for other regionalisations. Note here that the SHARE regionalisation is not used in the global...To facilitate shakyground2 becoming a tool for eGSIM, the SHARE regionalisation is implemented according to the json/geojson structure adopted for other regionalisations. Note here that the SHARE regionalisation is not used in the global default regionalisation, where it is superseded by the ESHM20 regionalisation.
In the process, some file re-naming is done to apply a more consistent naming scheme. Now the files for region `xyz` should be `xyz.geojson`, which includes the geospatial information, and `xyz.json` for the GMM mapping.https://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/28Small edit to adapt to OQ changes, fixes broken quakeml test2021-10-05T08:17:34+02:00Graeme WeatherillSmall edit to adapt to OQ changes, fixes broken quakeml testIn the last few months OQ has made some API changes to its ground motion model library, as explained here: https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/25. This has caused some breakages in this library (and other libraries ...In the last few months OQ has made some API changes to its ground motion model library, as explained here: https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/25. This has caused some breakages in this library (and other libraries that use OQ's GMMs for this purpose). At present, shakyground2's shakemap calculator is using the `openquake.hazardlib.contexts.ContextMaker` class for preparing the inputs to the GMM, but calling the GMMs themselves according to the old style API. After some user feedback, OpenQuake's developers have ensured that OpenQuake's old style GMM remains backward compatible for the time being, so only one small change was needed to keep shakyground2 working with the 3.12.0 release of OpenQuake. This is implemented here along with some minor cleanup of formatting to address a `line too long` formatting error from Flake8.
Also, a small change in the precision of the magnitude and location of one of the GEOFON test events seemed to have occurred, so the test have been edited to fix this too.https://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/26Use a way stronger compression2021-10-04T11:59:16+02:00Nils BrinckmannUse a way stronger compressionThe deflate compression is the algorithm used for zip archives.
It is loss less and features some more tweaking options in order
to get smaller results.
With the usage of quite heavy compression options, we can reduce
the usual amount o...The deflate compression is the algorithm used for zip archives.
It is loss less and features some more tweaking options in order
to get smaller results.
With the usage of quite heavy compression options, we can reduce
the usual amount of memory of shakemap geotiff files to need only
around 60% of the original size using the lzw algorithm.
Of cause, read and write times will increase. But as the size of
a single shakemap is relativly small (our problem with the size is
the pure amount that we have for the earthquake explorer), it should
not be totally bad to work with those files later.https://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/25Added additional reprojection work for shakemaps that cross the dateline2021-07-21T15:21:50+02:00Nils BrinckmannAdded additional reprojection work for shakemaps that cross the datelineIt makes a reprojection so that the earthquake explorer geoserver
can display them properly.
It basically just change that the transformation starts on the upper left corner instead of the lower left corner.
For more details see https:...It makes a reprojection so that the earthquake explorer geoserver
can display them properly.
It basically just change that the transformation starts on the upper left corner instead of the lower left corner.
For more details see https://git.gfz-potsdam.de/earthquake-explorer/gfz-earthquake-explorer/-/issues/90https://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/24Implements Ground Motion Intensity Conversion Equation module2021-06-15T08:55:31+02:00Graeme WeatherillImplements Ground Motion Intensity Conversion Equation moduleA comprehensive shakemap suite may require the inclusion of shakemaps in terms of macroseismic intensity, as expressed here: https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/21
This MR implements a sub-module for ground motion ...A comprehensive shakemap suite may require the inclusion of shakemaps in terms of macroseismic intensity, as expressed here: https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/21
This MR implements a sub-module for ground motion intensity conversion equations (GMICEs), including two commonly used models (Atkinson & Kaka, 2007; Worden et al., 2012). These models are the most widely used and generally applicable throughout the world. Uncertainty in the ground motion is incorporated into the uncertainty of macroseismic intensity using standard error propagation. The application of the GMICE is incorporated into the `shakemap_from_quakeml` workflow for Earthquake Explorer, meaning that it is now a standard output of the tools for this purpose.
Although the exact relation between GMICE may be region-dependent, there are still few models suitable for a shakmap application such as this. Nevertheless, a simple regionalisation is currently implemented, which adopts Atkinson & Kaka (2007) for the `Global Stable` tectonic region type, and Worden et al. (2012) everywhere else. This may be refined in future as new models are added.
The GMICE implementations are verified against test tables of data constructed using the original USGS shakemap software (https://github.com/usgs/shakemap/tree/master/shakelib/gmice).
@rizac Please take a look when you can
@nils @eggi FYI: this feature now means that you will find macroseismic intensity (`MMI`) in the suite of outputs of the shakemaps (even if it is not specified in the input IMTs). However, in order to apply this conversion the selected IMTs for the shakemaps <i>must</i> include one or more of `PGA`, `SA(0.3)`, `SA(1.0)`, `SA(3.0)` or `PGV`. If none of these are provided in the `IMTs` then an error will be raised.https://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/23Adds logging of shakemap statistics in shakemap_from_quakeml workflow2021-06-02T14:23:01+02:00Graeme WeatherillAdds logging of shakemap statistics in shakemap_from_quakeml workflowAddresses https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/20
Adds `statistics` to the `ShakemapWorkflowResult` class to log some summary statistics about the shakemaps themselves. This could make it easier in the future to, fo...Addresses https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/20
Adds `statistics` to the `ShakemapWorkflowResult` class to log some summary statistics about the shakemaps themselves. This could make it easier in the future to, for example, filter or sort some shakemap results by maximum ground motion or average ground motion. Initially only the maximum, minimum, geometric mean and log IQR of the ground motion fields are stored. Other information could be added in future.https://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/22(Potentially) Fixes bug of shakemaps not working across antimeridian2021-06-02T14:22:36+02:00Graeme Weatherill(Potentially) Fixes bug of shakemaps not working across antimeridianAddresses https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/18
when the bounding box of the shakemap region crossed the antimeridian (not the "international date line" - I have been corrected on that terminology!) the shakemap c...Addresses https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/18
when the bounding box of the shakemap region crossed the antimeridian (not the "international date line" - I have been corrected on that terminology!) the shakemap could not be built. There were several steps in the calculation that prevented it, but it ultimately originates from the `ulon` of the bounding box being lower than the `llon` and thus no valid site model was built. This MR makes the necessary fixes in the shakemap code to ensure that the ground motion values are calculated at the sites correctly when the earthquake is near the antimeridian.
In the second modification, the generation of the raster and contours has been modified. For the contours these are re-centred such that longitudes in the western hemisphere (-180˚E to 0˚E) are shifted to an antimeridian centred system (i.e. 180˚E to 360˚E). Thus the contour can now contain longitudes greater than 180˚E. For the raster we use `rasterio`'s `reproject` functionality to transform the raster passing the `CENTER LONG = 180` keyword to the `GDAL` backend. This does not raise an error and seems to export complete rasters.
Here is an example of the shakemap in QGIS:
![Screen_Shot_2021-05-27_at_16.58.53](/uploads/c5a1c92f27aa186eb8373500fc065d7e/Screen_Shot_2021-05-27_at_16.58.53.png)
@nils @eggi Honestly, I have no idea whether this has worked or not. The results seem OK in QGIS when I set the basemap projection to EPSG:4326, but if I switch to another project (e.g. EPSG:3857) then it only shows me the portion of the raster in the eastern hemisphere. This could be just a QGIS issue, though, and it might be fine on the Earthquake Explorer server. I will keep issue https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/18 open after merging this MR until there is confirmation that this either i) works directly or ii) can be integrated into the pipeline.
An antimeridian crossing workflow has also been added to the workflow test cases.https://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/21Another fix to the datetime isoformat2021-05-27T09:29:53+02:00Graeme WeatherillAnother fix to the datetime isoformatStill addressing: https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/22
Changes formatting of the "second" string in the datetime.from_isoformat function. Now adds a test of the `Earthquake.from_quakeml` function that inputs vario...Still addressing: https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/22
Changes formatting of the "second" string in the datetime.from_isoformat function. Now adds a test of the `Earthquake.from_quakeml` function that inputs various GEOFON event IDs and verifies that the `Earthquake` classes are built correctly.https://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/20Fixes datetime formatting and adds new error for FDSN failure2021-05-26T17:21:17+02:00Graeme WeatherillFixes datetime formatting and adds new error for FDSN failure1. Fixes the datetime formatting error mentioned here: https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/22
2. Raises an `HTTPException` when the FDSN web service is not available (or not found) for a certain event. As raised he...1. Fixes the datetime formatting error mentioned here: https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/22
2. Raises an `HTTPException` when the FDSN web service is not available (or not found) for a certain event. As raised here: https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/23https://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/19Small edit to xml file2021-05-25T15:47:58+02:00Graeme WeatherillSmall edit to xml filehttps://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/18Implements single workflow function for GEOFON (Earth Explorer) use case2021-05-25T08:09:35+02:00Graeme WeatherillImplements single workflow function for GEOFON (Earth Explorer) use caseThough the shakyground2 library itself is intended to be modular, specific workflows that chain together the features in order to address a particular use case can make the library easier to call. A `workflows.py` module is added to begi...Though the shakyground2 library itself is intended to be modular, specific workflows that chain together the features in order to address a particular use case can make the library easier to call. A `workflows.py` module is added to begin to support such cases, and this MR in particular addresses the Earth Explorer (GEOFON) shakemap application. Here the workflow: i) creates the `Earthquake` class from the GEOFON FDSNWS (via the event ID) or local QuakeML, ii) builds the site model from bounding box centered on the event location, iii) retrieves the appropriate ground motion model from the regionalisation, iv) runs the shakemap calculations, v) exports the shakemap rasters to byte objects and the corresponding contours to GeoDataFrames
The entire workflow is run via the function: `geofon_workflow(event_id, imts)`, where `event_id` is the GEOFON event ID and `imts` a list of intensity measure types (e.g. `["PGA", "SA(0.2)", "SA(1.0)"]`).
There are several configurable options in the different steps of the workflow and these can be set via an optional `config` input. If these are not defined (or only partially defined) then a default set of configuration properties are included in the `GEOFON_CONFIGURATION` dictionary.https://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/17make gitlab ci jobs interruptible and silence install2021-05-25T09:05:38+02:00Marius Kriegerowskimarius@gfz-potsdam.demake gitlab ci jobs interruptible and silence installSilencing the installation of dependencies. This allows to focus more on the actual testing its' output.
Also, tests are marked is interruptible. This will cancel jobs if new changes were pushed to the same MR, making previous test runs...Silencing the installation of dependencies. This allows to focus more on the actual testing its' output.
Also, tests are marked is interruptible. This will cancel jobs if new changes were pushed to the same MR, making previous test runs obsolete. Speeds up the pipeline.
\approve @rizac @gweatherhttps://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/16add pre-commit configuration, docs and gitlab-ci integration2021-05-25T16:09:31+02:00Marius Kriegerowskimarius@gfz-potsdam.deadd pre-commit configuration, docs and gitlab-ci integrationThis MR adds pre-commit for improved workflow.
\approve @gweather @rizac
\fyi @allThis MR adds pre-commit for improved workflow.
\approve @gweather @rizac
\fyi @all