Shakemap issueshttps://git.gfz-potsdam.de/groups/shakemap/-/issues2022-09-30T22:57:57+02:00https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/30Validation of gitlab ci yml fails2022-09-30T22:57:57+02:00Nils BrinckmannValidation of gitlab ci yml failsWhen I currently made some tests about writing out the xml files that are alike the shakyground1 output that we use for RIESGOS, I realized that the CI has some problems currently.
I tried to check it with the functionality in gitlab he...When I currently made some tests about writing out the xml files that are alike the shakyground1 output that we use for RIESGOS, I realized that the CI has some problems currently.
I tried to check it with the functionality in gitlab here & there seems to be a problem with the following part:
```
artifacts:
reports:
cobertura: coverage.xml
```
Taking a look in the docs (https://docs.gitlab.com/ee/ci/yaml/artifacts_reports.html#artifactsreportscoverage_report) it seems to have changed slightly.
In my draft for the !37 MR, I switched over to:
```
artifacts:
reports:
coverage_report:
path: coverage.xml
coverage_format: cobertura
```
Maybe, this can be used in general to fix the current problem.https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/29Problem on processing gfz2022ihab2022-05-05T12:55:35+02:00Nils BrinckmannProblem on processing gfz2022ihabWhen I try to use the `shakemaps_from_quakeml` function with the event id `gfz2022ihab` an `imts = ["PGA", "SA(0.3)", "SA(1.0)", "MMI"]` I get a value error:
```python3
workflow_results = shakemaps_from_quakeml(
event_id="gfz2022i...When I try to use the `shakemaps_from_quakeml` function with the event id `gfz2022ihab` an `imts = ["PGA", "SA(0.3)", "SA(1.0)", "MMI"]` I get a value error:
```python3
workflow_results = shakemaps_from_quakeml(
event_id="gfz2022ihab", imts=["PGA", "SA(0.3)", "SA(1.0)", "MMI"], config=None
)
```
```
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/src/app/shakyground2/shakyground2/workflows.py", line 203, in shakemaps_from_quakeml
shakemap = Shakemap(
File "/usr/src/app/shakyground2/shakyground2/shakemap.py", line 120, in __init__
self._build_contexts()
File "/usr/src/app/shakyground2/shakyground2/shakemap.py", line 153, in _build_contexts
self.ctx = cmaker.get_ctxs(
File "/usr/local/lib/python3.8/dist-packages/openquake/hazardlib/contexts.py", line 606, in get_ctxs
r_sites, dctx = self.filter(sites, rup)
File "/usr/local/lib/python3.8/dist-packages/openquake/hazardlib/contexts.py", line 542, in filter
mdist = self.maximum_distance(rup.mag)
File "/usr/local/lib/python3.8/dist-packages/scipy/interpolate/_polyint.py", line 78, in __call__
y = self._evaluate(x)
File "/usr/local/lib/python3.8/dist-packages/scipy/interpolate/_interpolate.py", line 695, in _evaluate
below_bounds, above_bounds = self._check_bounds(x_new)
File "/usr/local/lib/python3.8/dist-packages/scipy/interpolate/_interpolate.py", line 724, in _check_bounds
raise ValueError("A value in x_new is below the interpolation "
ValueError: A value in x_new is below the interpolation range.
```
I would consider that as a bug, but I'm not sure at all.https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/25Revise Shakemap objects in line with OpenQuake GMM API changes2022-03-31T15:36:36+02:00Graeme WeatherillRevise Shakemap objects in line with OpenQuake GMM API changesFor the first time in a decade the OpenQuake Ground Motion Model library is undergoing a major refactoring (explanation here: https://github.com/gem/oq-engine/blob/master/doc/breaking-hazardlib.md). This impacts on how the ground motion ...For the first time in a decade the OpenQuake Ground Motion Model library is undergoing a major refactoring (explanation here: https://github.com/gem/oq-engine/blob/master/doc/breaking-hazardlib.md). This impacts on how the ground motion models are called inside of the Shakemap calculation, and though some effort has been made to maintain backward compatibility it has not been entirely possible to do so and it would be better to transition to the new GMM API sooner rather than later.
The preferred way to call the GMPEs is via the `.get_mean_stds` function of the `openquake.hazardlib.contexts.ContextMaker` class. This class is already used in `Shakemap` to retrieve the rupture, distance and site attributes that are needed for the GMMs themselves. The change would be to migrate the rest of the steps of the call to the GMPE into the same framework.Graeme WeatherillGraeme Weatherillhttps://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/24Incorporate subduction zones into regionalisation2021-10-04T15:03:49+02:00Graeme WeatherillIncorporate subduction zones into regionalisationThe first regionalisation includes all tectonic zones except subduction regions. To select the most suitable GMPEs for subduction regions we need to classify the earthquakes as interface, in-slab, outer rise or upper mantle - with interf...The first regionalisation includes all tectonic zones except subduction regions. To select the most suitable GMPEs for subduction regions we need to classify the earthquakes as interface, in-slab, outer rise or upper mantle - with interface and in-slab being the most important. In addition, for most events classified as subduction interface or inslab, the geometry of the slab itself should be used to disambiguate the fault plane from the auxilliary plane in the moment tensor.
For modelling the subduction zone geometry, we have available the comprehensive Slab 2.0 global model from the USGS and the recent Hellenic/Cypriot and Calabrian arc models from the 2020 European Seismic Hazard Model.
The regionalisation should work in the following steps:
1. Simple point in polygon to determine if the earthquake is within the surface projection (plus a buffer) of a specific subduction zone
2. If yes, then initial assignments can be made based on depth data (e.g. if depth > 50 km then in-slab)
3. For events in the 0 - 50 km depth range then further information is needed to determine if in-slab or interface. Use focal mechanism if available and proximity to the interface.
4. For interface (and some slab events) update the earthquake rupture plane to align it with the interface.
5. Return the GMPE selection and re-calculate the distances using the revised rupture plane
Further revisions to this system using a more probabilistic approach are also anticipated in the future.Graeme WeatherillGraeme Weatherillhttps://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/19Build SiteModel with Vs30 values from Global Vs30 database (and other site fe...2021-05-20T16:41:01+02:00Graeme WeatherillBuild SiteModel with Vs30 values from Global Vs30 database (and other site features)The standard global model to define shallow site properties (Vs30) comes from the USGS Global Vs30 model https://earthquake.usgs.gov/data/vs30/. Currently the site model assumes a fixed Vs30, but it would be critical that the global Vs30...The standard global model to define shallow site properties (Vs30) comes from the USGS Global Vs30 model https://earthquake.usgs.gov/data/vs30/. Currently the site model assumes a fixed Vs30, but it would be critical that the global Vs30 database can be used (as was the case for the previous shakyground library). However, no WEB API exists to query and download the Vs30 data, meaning that the data would need to be extracted from the global raster. This is problematic as the global raster is approximately 1 Gb in size, which would either need to be included in the code itself or can be retrieved easily from an online service.
Some points to consider:
1. The 1 Gb global raster covers the entire globe at 30 arc-seconds, including oceanic regions that effectively have no Vs30. If the raster were sliced into tiles and stored in an hdf5 file this could i) allow us to cut out about 2/3 of the data, ii) possibly make extracting the Vs30 values faster. But even in the best case this would still result in a data file of a couple of hundred Mb and extracting from tiles can be problematic for events on the edges of the tiles
2. An alternative would be to develop a lightweight API service that could allow users to access the site model information from GFZ's own files. This has the advantage that we could store all of the relevant site data (including other properties beyond just Vs30) in a single database or hdf5 and the user can download the site model information in one command. But the concerns here would be security, volume of usage etc.https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/17Proposal: Remove black from CI pipeline2021-04-29T12:18:57+02:00Graeme WeatherillProposal: Remove black from CI pipelineAfter working with `black` in the CI pipeline for a couple of months, I am increasingly skeptical of the value it is adding, and have encountered some situations where it has introduced unnecessary complications. My reasons are as follow...After working with `black` in the CI pipeline for a couple of months, I am increasingly skeptical of the value it is adding, and have encountered some situations where it has introduced unnecessary complications. My reasons are as follows:
1. The vast majority of the formatting changes that `black` makes are inconsequential and have no discernible impact on code functionality, quality or readability.
2. We have a pipeline fail here: https://git.gfz-potsdam.de/gweather/shakyground2/-/jobs/69384 because `black` and `flake8` are contradicting each other. `black` insists that the following line should be formatted as:
```python
tag = tag[tag.index("}") + 1 :]
```
while `flake8` insists that it must be:
```python
tag = tag[tag.index("}") + 1:]
```
This absolutely trivial issue is preventing a merge request from passing the CI pipeline - requiring time invested to find the solution.
I have also encountered problems myself using `black` with type hinting, which prefers to change the perfectly valid `Optional[x]` to `Union[x, None]`, which will introduce an error if `Union` has not been imported. Again, it is not an insurmountable problem, but it is an unnecessary one.
3. Other differences such as line-length requirements can be configured to avoid conflicts (as they are currently), but I can see circumstances where a contributor is unfamiliar with `black` and may run the standard command `black ...` with the defaults, only to have it modify the code in a way that is incompatible with the black configuration in the CI. You can see this particular error from me in the commits to this MR: https://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/13/commits, and though this might have been my own error it did cost time to understand and fix the problem, but for no gain in terms of code quality. For an outside contributor, annoyances like this can be a barrier to engaging with a project.
4. Quite simply, while the CI pipeline runs `flake8` then `black` is redundant for actually catching problems of formatting and readability. I am not suggesting to abandon the use of `black` entirely. As we have now begun to develop the code with one particular style I will continue to run `black` prior to submitting MRs and add this into the readme as an instruction to contributors to do the same. However, making `black` an integral part of the CI pipeline to the extent that trivial formatting issues causing it to break is adding complications for no gain.
@marius @rizac @ds I would not remove this without agreement (though I don't know how to get around the problem in point #2), but I would like to put forward my views and experiences on this, which may be relevant to you if you are using `black` in other projects.https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/14Add Example Jupyter Notebooks2021-03-24T13:27:21+01:00Graeme WeatherillAdd Example Jupyter NotebooksTo assist users in understanding how to use the library (also for teaching) some Jupyter notebooks will be added to illustrate the example workflows.To assist users in understanding how to use the library (also for teaching) some Jupyter notebooks will be added to illustrate the example workflows.Graeme WeatherillGraeme Weatherillhttps://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/13Follow-up from "linting source code and tests"2021-03-04T19:00:48+01:00Marius Kriegerowskimarius@gfz-potsdam.deFollow-up from "linting source code and tests"The following discussion from !11 should be addressed:
- [ ] @marius started a [discussion](https://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/11#note_52525):
> @gweather I need to disable pylints member check here ...The following discussion from !11 should be addressed:
- [ ] @marius started a [discussion](https://git.gfz-potsdam.de/shakemap/shakyground2/-/merge_requests/11#note_52525):
> @gweather I need to disable pylints member check here because `valid.mechanism` may return to different types: A `tuple` or a `NodalPlane` instance.
>
> It would be a little cleaner to not allow a mechanisms to be built with strike, dip or rake to be `None`, woudnt it? From a user perspective I would expect some exception if I create a mechanism from `strike=10`, `dip=10` and `rake=None`.https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/11Add documentation generator2021-02-11T14:55:14+01:00Marius Kriegerowskimarius@gfz-potsdam.deAdd documentation generatorAs discussed today, let's give `pdoc3` a shot.As discussed today, let's give `pdoc3` a shot.https://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/4Adding code2021-02-25T16:13:18+01:00Marius Kriegerowskimarius@gfz-potsdam.deAdding codeI think the project is all set. In our last call we discussed that it would be good to start with the scientific core. I will wrap an API around that after the core modules are there (hopefully simply recycling what I had done before).
...I think the project is all set. In our last call we discussed that it would be good to start with the scientific core. I will wrap an API around that after the core modules are there (hopefully simply recycling what I had done before).
@rizac and @gweather feel free to add your modules. What we experienced to work very well in the last few months is to add code in multiple steps to give everybody the chance to have a look and to understand what is going on under the hood.
You can leverage the Makefile to check and lint your code (`make check` and `make black`). I hope this helps a bit to get used to black :)
\fyi @dshttps://git.gfz-potsdam.de/shakemap/shakyground2/-/issues/1Project name2021-01-28T08:55:10+01:00Marius Kriegerowskimarius@gfz-potsdam.deProject nameIn todays discussion for decided to go with `shakyground2` as the project name for now. We could benefit from the `shakyground` project already in use and consider this project a next level shakyground.
I'm not big fan of versioned names...In todays discussion for decided to go with `shakyground2` as the project name for now. We could benefit from the `shakyground` project already in use and consider this project a next level shakyground.
I'm not big fan of versioned names especially since this is a completely unrelated design to shakyground and is not really related.
This issue serves as a thread to discuss other names. Simply comment if you have other ideas for names.
\cc @all