Newer
Older
The Early-Warning and Rapid Impact Assessment with real-time GNSS in the Mediterranean (EWRICA) is a federal Ministry of Education and Research funded project (funding period: 2020-2023) that aims to develop fast kinematic and point source inversion and modeling tools combining GNSS-based near field data with traditional broadband ground velocity and accelerometer data.
Fast and robust estimates of seismic source parameters are essential for reliable hazard estimates, e.g. in the frame of tsunami early warning. Hence, EWRICA aims for the development and testing of new real time seismic source inversion techniques based on local surface displacements. The resulting methods shall be applied for tsunami early warning purposes in the Mediterranean area.
In this framework, this repository is a suite of four packages that can be used and combined in different ways and are [ewricacore](https://git.gfz-potsdam.de/ewrica/ewricacore), [ewricasiria](https://git.gfz-potsdam.de/ewrica/ewricasiria), [ewricagm](https://git.gfz-potsdam.de/ewrica/ewricagm) and [ewricawebapp](https://git.gfz-potsdam.de/ewrica/ewricawebapp). These four packages can be deployed in a docker container (see instructions below) to demonstrate a possible output of Early-Warning and Rapid Impact Assessment.
The suite is composed of a probabilistic earthquake source inversion report (*ewricasiria*) and a Neural network-based Shake map (*ewricagm*) which are triggered upon event detection by *ewricacore*, a central unit of work that first fetches the waveform data via a configurable seedlink protocol and event data via event bus or FDSN web service, then collects and cuts waveforms segments according to a custom configuration, and eventually triggers custom processing (ewricasiria and ewricagm in the current workflow, but any processing can be implemented) whenever configurable conditions are met.
The final package in this suite, *ewricawebapp*, is a web-based graphical user interface that has been used mainly for testing purposes but that can be deployed also to visualize and check in your browser all output produced by workflow in form of HTML pges, images and data in various formats (e.g., JSON, log text files).
The whole workflow has been tested for two past earthquakes whose data (as of 2023) are continuously served by [GEOFON](https://geofon.gfz-potsdam.de/) at regualr intervals to produce and test a real case scenario. These two test cases can also be reproduced locally following the instructions below.
Zaccarelli, Riccardo; Metz, Malte; Lehman, Lukas; Heinloo, Andres (2023): EWRICA - Docker container for Early-Warning and Rapid Impact Assessment with real-time GNSS data. GFZ Data Services. https://doi.org/10.5880/GFZ.2.6.2023.003
|Jump to: | [Docker installation and connection](#docker-installation-and-connection) | [Docker services](#docker-services) | [Docker source code](#docker-source-code) | [Inspect generated processing files in Docker](#inspect-generated-processing-files-in-docker) | [Webapp (GUI) in Docker](#webapp-gui-in-docker) | [Git workflow (for developers)](#git-workflow) |
**NOTE** For any problem/question [open an issue](https://git.gfz-potsdam.de/ewrica/readme/-/issues)
## Docker installation and connection
### Installation
1. Make sure you have Docker installed, eg.:
```console
apt-get install docker.io
```
<details>
<summary>Get ewrica snapshot (outdated, click for details)</summary>
2. Get the EWRICA container snapshot from https://nextcloud.gfz-potsdam.de/s/ifJRJ3P9FoFdDj7
```console
docker load -i ewrica_snapshot_20220707.tar.gz
docker create --name ewrica --hostname ewrica ewrica_snapshot:20220707
```
```console
docker start ewrica
```
</details>
2. Download this repository and rename Dockerfile to `ewrica` (or copy it wherever you want). The latter is the dockerfile (or docker image) that you can use to start a container following the official documentation here: https://docs.docker.com/language/nodejs/run-containers/
docker inspect -f "{{ .NetworkSettings.IPAddress }}" ewrica
If the container started successfully, you can open http://172.17.0.2/ewrica-realtime/ or http://172.17.0.2/ewrica-playback/ in your browser.
#### Further useful commands:
Enter the container (eg., run a shell in container):
```console
docker exec -ti ewrica bash
```
Show latest logs (see also /var/log in container):
```console
docker logs ewrica 2>&1 | tail
```
Start and stop the container:
```console
docker start ewrica
docker stop ewrica
```
**NOTE on local deploys**: The docker container might be cut of the internet for any reason. This will result both in an unavailable Seedlink Connection `obspy.clients.seedlink.seedlinkexception.SeedLinkException: 'cannot connect to SeedLink server: [Errno 110] Connection timed out'` and/or not available remote git servers. Then logout of the docker, restart docker with
```console
service docker restart
```
or
```console
systemctl restart docker
```
The services in the docker container are simple Unix services that are setup to run ewrica source code continuously in
the background. This means that any time source code is changed, the service must be restarted to put changes into effect:
Or, in one command:
```terminal
sv restart <service>
* `ewricacore` background service acquiring real time data and triggering custom process
* `ewricacore-playback` same as above, but listening for played back Norcia event continuously
* `ewricawebapp` web service showing `ewricacore` data and processed output (mainly used as browser GUI)
* `ewricawebapp-playback` same as above, but shows `ewricacore-playback` data and processed output
* `nginx` web service for all static content (images and so on) of both `ewricawebapp` and `ewricawebapp-playback`
**IMPORTANT: the commands above DO NOT STOP all Python processes attached to the service, this can cause memory leaks in the long run. Therefore, after stopping a service, please run `ps aux` and `kill -9 pid1 pid2 ...`, where each `pid` are all process id related to `/root/ewrica/pyenv3.9.5` (ewricacore\* services) or `/root/ewrica/pyenv3.9.1` (ewricawebapp\* services)**
### Edit a service
`ll /etc/service/` will list all services. Inside each directory, the service command is implemented in the `run` file. So to edit the service `ewricacore` you can type:
```console
vim /etc/service/ewricacore/run
```
To edit any other service, just replace `ewricacore` with the service name, **with the exception of nginx**. In this case, edit the nginx config file:
```console
vim /etc/nginx/sites-available/ewricawebapp
```
(note that in this case you might need to ask site maintainers for changes to take effect, e.g., if you want to change the site URL).
The source code that the services run is located in the container inside `/root/ewrica`:
total 60
drwxr-xr-x 13 root root 4096 Oct 24 14:16 ./
drwxr-xr-x 1 root root 4096 Oct 25 10:54 ../
drwxr-xr-x 8 root root 4096 Oct 24 14:32 ewricacore/
drwxr-xr-x 4 root root 4096 Oct 19 17:35 ewrica-data/
drwxr-xr-x 5 root root 4096 Oct 19 17:35 ewrica-data-playback/
drwxr-xr-x 5 root root 4096 Oct 24 14:20 ewricagm/
drwxr-xr-x 8 root root 4096 Oct 20 13:46 ewricasiria/
drwxr-xr-x 5 root root 4096 Oct 24 14:41 ewricawebapp/
drwxr-xr-x 6 root root 4096 Oct 17 14:58 .old_stuff/
drwxr-xr-x 3 root root 4096 Sep 23 10:40 processing-data/
drwxr-xr-x 6 root root 4096 Mar 1 2022 pyenv3.9.1/
drwxr-xr-x 6 root root 4096 Oct 24 14:16 pyenv3.9.5/
drwxr-xr-x 16 root root 4096 Jan 16 14:31 pyrocko/
<!-- -rwxr-xr-x 1 root root 164 Oct 10 12:55 restart-services*
-rwxr-xr-x 1 root root 166 Oct 11 12:37 restart-web-services* -->
- `ewricacore`: real time data acquisition and custom process trigger
- `ewrica-data*`: the directories where `ewricacore` data is stored (each directory is relative to a different scenario. The directory paths are configured in `ewricacore/configs`)
- `ewricasiria`: source inversion code
- `ewricawebapp`: web app or GUI of ewricacore
- `grond`: A probabilistic earthquake source inversion framework required by `ewricasiria`
- `pyenv3.9.1`: the Python virtual environment of `ewricawebapp` (it has also some redundant packages becuase it was previously used to run also `ewricacore`. Note that the Python version is actually 3.9.5)
- `pyenv3.9.5`: the Python virtual environment of `ewricacore`
- `pyrocko`: needed for installing [pyrocko from source](https://pyrocko.org/docs/current/install/system/linux/index.html#user-installation-from-source-into-isolated-environment)
### Pyenv Installation notes (for docker maintainers only)
The packages installed in `pyenv3.9.5` (type `pip freeze` for details) were obtained by installing all required packages in the following order:
```
cd /root/ewrica
python3.9 -m venv ./pyenv3.9.5
source pyenv3.9.5/bin/activate
pip install --upgrade pip setuptools
(cd grond && pip install -e .)
(cd ewricasiria/ && pip install -r ./requirements.txt && pip install -e .)
(cd ewricagm && pip install -r ./requirements.txt && pip install -e .)
Note: as of January 2023, Pyrocko has to be (re)installed from source because of a small bugfix not available through `pip` (see last line above)
<!--
```
cd /root/ewrica
python3.9 -m venv ./pyenv3.9.5
source pyenv3.9.5/bin/activate
pip install --upgrade pip setuptools
cd grond
pip install -e .
cd ../ewricasiria/
pip install -r ./requirements.txt
pip install -e .
cd ../ewricagm
pip install -r ./requirements.txt
pip install -e .
```
-->
Once changes are performed (see [Git workflow](#git-workflow)), you `pull` the last changes:
```console
cd /root/ewrica/ewricasiria # ewricasiria is just an example, it can be any other git repo
git pull
sv ewricacore restart
... # and so on (see services list above)
```
Note: When updating only `ewricawebapp`, you can restart only the two services `ewricawebapp-*` and `nginx`
To change the background watcher and trigger service config:
```
vim /root/ewrica/ewricacore/configs/deploy-playback.yaml
```
Or, for real time (should not be the case for the moment)
```
Then push changes (see [Git workflow](#git-workflow)) and then restart **all** docker services
### Edit ewricacore processing function or function call:
```console
vim /root/ewrica/ewricacore/processing/functions.py
```
Note that if you change the name of the processing function, or implement a new one, you must set the new name in the config file(s) (see section above) in the YAML property 'function_name'.
When finished, restart the service(s)
Note: everything you put in the directory `/root/ewrica/ewricacore/processing` is ignored by `git` (with the exception of the example file `functions.example.py`), so you can add / remove wehatever you want inside with no risk of committing and pushing huge files, which would make the repository unmaintainable.
## Inspect generated processing files in Docker
The output files generated by the whole processing pipeline are under `/root/ewrica/ewrica-data-playback` (or `ewrica-data` for realtime).
Therein, there is a set of directories organizing events per year:
```console
root@ewrica2:/etc/service# ll /root/ewrica/ewrica-data-playback/
total 24
drwxr-xr-x 5 root root 4096 Jul 5 08:04 ./
drwxr-xr-x 10 root root 4096 Jul 5 12:21 ../
drwxr-xr-x 3 root root 4096 Jul 5 08:04 2022/
-rw-r--r-- 1 root root 1306 Jul 5 16:49 __current_running_process_info__.log
drwxr-xr-x 2 root root 4096 Jun 23 13:19 playedback-events/
drwxr-xr-x 2 root root 4096 Jun 25 14:42 stations/
```
(here in the playback we obviously have only one year, 2022). Inside each year you find the processed event(s) for that year. E.g., for the playback, we can inspect the only processed event `gfz2016vicq`:
```console
root@ewrica2:/etc/service# ll /root/ewrica/ewrica-data-playback/2022/gfz2016vicq/
total 24
drwxr-xr-x 4 root root 4096 Jul 5 08:15 ./
drwxr-xr-x 3 root root 4096 Jul 5 08:04 ../
-rw-r--r-- 1 root root 130 Jul 6 06:04 gfz2016vicq.json
-rw-r--r-- 1 root root 2245 Jul 6 06:04 gfz2016vicq.xml
drwxr-xr-x 3 root root 4096 Jul 6 05:57 process_execution_01/
drwxr-xr-x 3 root root 4096 Jul 6 06:01 process_execution_02/
```
we can see that there is the event file (XML and JSON), and several `process_execution_**` directories where we store the produced output and the input waveform data. For instance:
```console
root@ewrica2:/etc/service# ll /root/ewrica/ewrica-data-playback/2022/gfz2016vicq/process_execution_01
total 16
drwxr-xr-x 3 root root 4096 Jul 6 05:57 ./
drwxr-xr-x 4 root root 4096 Jul 5 08:15 ../
drwxr-xr-x 2 root root 4096 Jul 5 09:12 __input-data/
-rw-r--r-- 1 root root 127 Jul 6 05:57 __processing-info__.log
```
In the example above for instance, the execution finished successfully (there is no `__processing-error__.log` file) but it did not produce any output. The directory `__input-data` is where we store the miniSEED waveforms used by the process execution.
Similar to `ewricacore`, config files are located in `ewricawebapp/ewrica/configs`. Remeber that you need to restart only web related services after changing the config(s)
### Run GUI in local browser:
Execute `wsgi` or `wsgi-debug` inside `/root/ewrica/ewricawebapp` (`cd /root/ewrica/ewricawebapp` first):
```
export CONFIG_PATH="/root/ewrica/ewricawebapp/ewrica/configs/example-playback.yaml; /root/ewrica/pyenv3.9.1/bin/python ./wsgi-debug.py
```
(change `example-playback.yaml` with `example.yaml` if you want to test realtime, although there is nothing interesting to test that you cannot do with the playback)
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
## Git workflow
This section is intended for developers contributing to any of the Ewrica repositories. Remember: keep `main/master` branch **always production ready**. You can commit and push directly on the `main` branch only for small fixes (e.g. bugfixes) *and* if you have access to the remote docker (`git pull` from there). Otherwise, try whatever you want on separate branches:
1. create a new branch (let's call it `dev` as example, but you can call it whatever you want):
`git checkout -b dev`
Or move to an already existing branch:
`git checkout dev` (without `-b`). In this case, the branch might be behind `main` so update it: `git merge main`
2. Work on that branch as you always do: `git add`, `git commit` and `git push -u origin dev` (`-u` only the first time, to add the upstream branch, i.e. on `gitlab`).
3. When ready for merging, you can merge `dev` into `main/master`:
- When working with several contributors, one usually issues merge request ([full details here](https://docs.gitlab.com/ee/user/project/merge_requests/creating_merge_requests.html)), aka "pull requests" in `github`. **With merge requests other developers are notified and can review and/or deploy your code**:
I. In the browser go to the project `gitlab`'s page
II. Above the file list, on the right side, select `Create merge request`. A merge request is created. The default branch (`main`) is the target
III. Fill out the fields and select `Create merge request`
If you put the docker maintainer(s) in the list of reviewers, they will be notified, so that they can merge and deploy automativally without emails exchange
- Alternatively you can merge the "quick and dirty way" (but this should be done only for quick bugfixes and if you have docker remote access for deploy):
```console
git checkout main
git merge dev
git push
```
Then update (`git pull`) the remote docker.
### Merge conflicts
This should not be a big problem here, because we are not 100 developers working simultaneously and regularly on the same code. Howerver, sometimes merging is not possible due to conflicts. For instance, somebody changed or pushed to the `main` branch while you were working in `dev`, and the branches have diverged, which basically means that `git` cannot resolve conflicts.
You will need to fix your code before merging. You will find several instructions on how to do (e.g. [here](https://www.atlassian.com/git/tutorials/using-branches/merge-conflicts)). In case of merge requests, you are notified beforehand on the web page. You can also `git merge main` inside the `dev` regularly so that you can fix merge conflicts when still small and easily fixable.