Skip to content
GitLab
Menu
Projects
Groups
Snippets
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
id2
Software
services
FAIR
sqa_test
Commits
1646a640
Commit
1646a640
authored
Aug 27, 2020
by
Maximilian Dolling
Browse files
initialized sqa
parent
71341c65
Pipeline
#11918
passed with stage
in 42 seconds
Changes
7
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
.sqa/apply-license/config.yml
0 → 100755
View file @
1646a640
# SPDX-FileCopyrightText: 2020 Helmholtz Centre Potsdam - GFZ German Research Centre for Geosciences, Germany (https://www.gfz-potsdam.de/)
#
# SPDX-License-Identifier: CC0-1.0
# OPTIONAL, to determine for which files the license header should be with the file
comment in file
:
-
'
<PYTHON
REGEX>'
# MANDATORY, to add a description of the whole software to the header
description
:
'
<STRING>'
# OPTIONAL, to skip over files for applying the license header
# usually the things in .gitignore
ignore paths
:
-
'
<PYTHON
REGEX>'
# MANDATORY, to set scopes for multiple licenses in a project
licenses
:
# MANDATORY, all files, that do not go into another scope, go here
default
:
# MANDATORY, the copyright holder of the file that the header is applied to,
# additionally it is used for the entry in the README.md
copyright
:
'
<STRING>'
# MANDATORY, unique identifier used for licenses
spdx-identifier
:
'
<SPDX
IDENTIFIER,
see
https://spdx.org/licenses/>'
# OPTIONAL, as many scopes as you wish are possible
# their names must be unique
# the license of the first scope that matches (determined by 'files' below) will be applied
<SCOPE NAME>
:
# MANDATORY, the copyright holder of the file that the header is applied to
copyright
:
'
<STRING>'
# MANDATORY, unique identifier used for licenses
spdx-identifier
:
'
<SPDX
IDENTIFIER,
see
https://spdx.org/licenses/>'
# MANDATORY: regex to match to apply this scope
files
:
-
'
<PYTHON
REGEX>'
.sqa/apply-license/templates/header/default.jinja2
0 → 100755
View file @
1646a640
{% for copyright_line in copyright_lines %}
{{ copyright_line }}
{% endfor %}
{% for expression in spdx_expressions %}
SPDX-License-Identifier: {{ expression }}
{% endfor %}
\ No newline at end of file
.sqa/apply-license/templates/header/default.jinja2.license
0 → 100755
View file @
1646a640
software-quality-assurance
This Software provides services to check and improve the source code quality of software-projects
SPDX-FileCopyrightText: 2020 Helmholtz Centre Potsdam - GFZ German Research Centre for Geosciences, Germany (https://www.gfz-potsdam.de/)
SPDX-License-Identifier: GPL-3.0-or-later
Parts of this program <(especially the code for whatever)>
were developed within the context of the following publicly funded
projects or measures:
- Helmholtz Federated IT Services, Helmholtz Association of German Research Centres (https://software.hifis.net/)
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, Version 3 or later.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You can find a copy of the license in the LICENSES folder.
If not, see <http://www.gnu.org/licenses/>.
\ No newline at end of file
.sqa/apply-license/templates/license_entry_README.md
0 → 100755
View file @
1646a640
<!--
SPDX-FileCopyrightText: 2020 Helmholtz Centre Potsdam - GFZ German Research Centre for Geosciences, Germany (https://www.gfz-potsdam.de/)
SPDX-License-Identifier: CC0-1.0
-->
## License
Copyright © [
<YEAR>
] [
<COPYRIGHT>
]
This work is licensed under the following license(s):
[
<SCOPE>
]
Please see the individual files for more accurate information.
> **Hint:** We provided the copyright and license information in accordance to the [REUSE Specification 3.0](https://reuse.software/spec/).
\ No newline at end of file
.sqa/check-credentials/gitleaks_config.toml
0 → 100755
View file @
1646a640
# software-quality-assurance
# This Software provides services to check and improve the source code quality of software-projects
#
# SPDX-FileCopyrightText: 2020 Helmholtz Centre Potsdam - GFZ German Research Centre for Geosciences, Germany (https://www.gfz-potsdam.de/)
#
# SPDX-License-Identifier: GPL-3.0-or-later
#
# Parts of this program <(especially the code for whatever)>
# were developed within the context of the following publicly funded
# projects or measures:
# - Helmholtz Federated IT Services, Helmholtz Association of German Research Centres (https://software.hifis.net/)
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, Version 3 or later.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You can find a copy of the license in the LICENSES folder.
# If not, see <http://www.gnu.org/licenses/>.
title
=
"gitleaks config"
[[rules]]
description
=
"AWS Manager ID"
regex
=
'''(A3T[A-Z0-9]|AKIA|AGPA|AIDA|AROA|AIPA|ANPA|ANVA|ASIA)[A-Z0-9]{16}'''
tags
=
[
"key"
,
"AWS"
]
[[rules]]
description
=
"AWS Secret Key"
regex
=
'''(?i)aws(.{0,20})?(?-i)['\"][0-9a-zA-Z\/+]{40}['\"]'''
tags
=
[
"key"
,
"AWS"
]
[[rules]]
description
=
"AWS MWS key"
regex
=
'''amzn\.mws\.[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}'''
tags
=
[
"key"
,
"AWS"
,
"MWS"
]
[[rules]]
description
=
"Facebook Secret Key"
regex
=
'''(?i)(facebook|fb)(.{0,20})?(?-i)['\"][0-9a-f]{32}['\"]'''
tags
=
[
"key"
,
"Facebook"
]
[[rules]]
description
=
"Facebook Client ID"
regex
=
'''(?i)(facebook|fb)(.{0,20})?['\"][0-9]{13,17}['\"]'''
tags
=
[
"key"
,
"Facebook"
]
[[rules]]
description
=
"Twitter Secret Key"
regex
=
'''(?i)twitter(.{0,20})?[0-9a-z]{35,44}'''
tags
=
[
"key"
,
"Twitter"
]
[[rules]]
description
=
"Twitter Client ID"
regex
=
'''(?i)twitter(.{0,20})?[0-9a-z]{18,25}'''
tags
=
[
"client"
,
"Twitter"
]
[[rules]]
description
=
"Github"
regex
=
'''(?i)github(.{0,20})?(?-i)[0-9a-zA-Z]{35,40}'''
tags
=
[
"key"
,
"Github"
]
[[rules]]
description
=
"LinkedIn Client ID"
regex
=
'''(?i)linkedin(.{0,20})?(?-i)[0-9a-z]{12}'''
tags
=
[
"client"
,
"LinkedIn"
]
[[rules]]
description
=
"LinkedIn Secret Key"
regex
=
'''(?i)linkedin(.{0,20})?[0-9a-z]{16}'''
tags
=
[
"secret"
,
"LinkedIn"
]
[[rules]]
description
=
"Slack"
regex
=
'''xox[baprs]-([0-9a-zA-Z]{10,48})?'''
tags
=
[
"key"
,
"Slack"
]
[[rules]]
description
=
"Asymmetric Private Key"
regex
=
'''-----BEGIN ((EC|PGP|DSA|RSA|OPENSSH) )?PRIVATE KEY( BLOCK)?-----'''
tags
=
[
"key"
,
"AsymmetricPrivateKey"
]
[[rules]]
description
=
"Google API key"
regex
=
'''AIza[0-9A-Za-z\\-_]{35}'''
tags
=
[
"key"
,
"Google"
]
[[rules]]
description
=
"Google (GCP) Service Account"
regex
=
'''"type": "service_account"'''
tags
=
[
"key"
,
"Google"
]
[[rules]]
description
=
"Heroku API key"
regex
=
'''(?i)heroku(.{0,20})?[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}'''
tags
=
[
"key"
,
"Heroku"
]
[[rules]]
description
=
"MailChimp API key"
regex
=
'''(?i)(mailchimp|mc)(.{0,20})?[0-9a-f]{32}-us[0-9]{1,2}'''
tags
=
[
"key"
,
"Mailchimp"
]
[[rules]]
description
=
"Mailgun API key"
regex
=
'''((?i)(mailgun|mg)(.{0,20})?)?key-[0-9a-z]{32}'''
tags
=
[
"key"
,
"Mailgun"
]
[[rules]]
description
=
"PayPal Braintree access token"
regex
=
'''access_token\$production\$[0-9a-z]{16}\$[0-9a-f]{32}'''
tags
=
[
"key"
,
"Paypal"
]
[[rules]]
description
=
"Picatic API key"
regex
=
'''sk_live_[0-9a-z]{32}'''
tags
=
[
"key"
,
"Picatic"
]
[[rules]]
description
=
"SendGrid API Key"
regex
=
'''SG\.[\w_]{16,32}\.[\w_]{16,64}'''
tags
=
[
"key"
,
"SendGrid"
]
[[rules]]
description
=
"Slack Webhook"
regex
=
'''https://hooks.slack.com/services/T[a-zA-Z0-9_]{8}/B[a-zA-Z0-9_]{8}/[a-zA-Z0-9_]{24}'''
tags
=
[
"key"
,
"slack"
]
[[rules]]
description
=
"Stripe API key"
regex
=
'''(?i)stripe(.{0,20})?[sr]k_live_[0-9a-zA-Z]{24}'''
tags
=
[
"key"
,
"Stripe"
]
[[rules]]
description
=
"Square access token"
regex
=
'''sq0atp-[0-9A-Za-z\-_]{22}'''
tags
=
[
"key"
,
"square"
]
[[rules]]
description
=
"Square OAuth secret"
regex
=
'''sq0csp-[0-9A-Za-z\\-_]{43}'''
tags
=
[
"key"
,
"square"
]
[[rules]]
description
=
"Twilio API key"
regex
=
'''(?i)twilio(.{0,20})?SK[0-9a-f]{32}'''
tags
=
[
"key"
,
"twilio"
]
[[rules]]
description
=
"Generic Credential"
regex
=
'''(?i)(dbpasswd|dbuser|dbname|dbhost|api_key|apikey|secret|key|api|password|user|guid|hostname|pw|auth)(.{0,20})?['|"]([0-9a-zA-Z-_\/+!{}/=]{4,120})['|"]'''
tags
=
[
"key"
,
"API"
,
"generic"
]
[allowlist]
description
=
"Allowlisted files"
files
=
[
'''.*gitleaks_config.toml$'''
,
'''(.*?)(jpg|gif|doc|pdf|bin)$'''
,
'''(go.mod|go.sum)$'''
]
.sqa/list-used-licenses/config.yml
0 → 100755
View file @
1646a640
# SPDX-FileCopyrightText: 2020 Helmholtz Centre Potsdam - GFZ German Research Centre for Geosciences, Germany (https://www.gfz-potsdam.de/)
#
# SPDX-License-Identifier: CC0-1.0
# META
# valid names for programming languages
# ['Python',
# 'Ruby',
# 'Shell',
# 'Dockerfile',
# 'Java',
# 'HTML',
# 'C',
# 'PHP',
# 'Batchfile',
# 'JavaScript']
# OPTIONAL, files, paths to exclude from analyzing licenses
# non-programming language files should go here
exclude
:
-
'
<PYTHON
REGEX>'
manual dependency config
:
# MANDATORY, the name of the programming language the library is used in
<NAME OF PROGRAMMING LANGUAGE>
:
# MANDATORY, sometimes the name to import differs from the name to download via a pkg-manager
# therefore the matching can be done here, if it cannot be resolved by a pkg manager
-
import name
:
'
<IMPORT
NAME>'
pkg name
:
'
<DOWNLOAD
NAME>'
# OPTIONAL, if the dependency is a local import (part of your software) you should ignore it,
# because is has no license
ignore
:
'
<BOOLEAN>'
# OPTIONAL, it is possible to use two different versions of a dependency - you can set the version here
version
:
'
<VERSION
STRING>'
# OPTIONAL, it is common, that the pkg repositories do not know the license(s) for a package
# you can give assign it here
licenses
:
-
'
<SPDX-IDENTIFIER>'
README.md
View file @
1646a640
...
...
@@ -3,40 +3,11 @@
The script analyzes publicly available astronauts data from
[
Wikidata
](
https://www.wikidata.org/wiki/Wikidata:Main_Page
)
.
It generates a set of plots focusing on aspects such as time humans spent in space, the gender distribution as well as the age distribution.
##
Install
##
Steps on this Branch
The script comes with a predefined Python environment, which is managed by
[
pipenv
](
https://github.com/pypa/pipenv
)
.
The environment handles all dependencies.
> The script has been successfully tested on 5.7.8-arch1-1 with Python 3.8.3
Please clone this repository and install the
[
dependencies
](
requirements.txt
)
as follows:
```
bash
git clone ...
cd
astronaut-analysis
pip
install
-r
requirements.txt
```
## Usage
You can run the script as follows:
```
bash
python src/astronaut-analysis.py
```
shell script
docker run -v [/ABSOLUTE/PATH/TO/REPO]:/repo \
-u $(id -u ${USER}):$(id -g ${USER}) \
gitext.gfz-potsdam.de:5000/software/services/fair/software-quality-assurance/software-quality-assurance:latest \
--init
```
The script processes the
[
astronauts data set
](
data/astronauts.json
)
and stores the plots in the directory
`results`
.
The directory will be created by the script.
Existing result plots will be overwritten.
### Astronaut Data
The data set has been generated from the following SPARQL query [[1]] (retrieval date: 2018-10-25).
You can replace the data set as follows:
-
Run the SPARQL query
-
Download the resulting data formatted as JSON
-
Replace the file
`data/astronauts.json`
[
1
]:
https://query.wikidata.org/#%23Birthplaces%20of%20astronauts%0ASELECT%20DISTINCT%20%3Fastronaut%20%3FastronautLabel%20%3Fbirthdate%20%3FbirthplaceLabel%20%3Fsex_or_genderLabel%20%3Ftime_in_space%20%3Fdate_of_death%20WHERE%20%7B%0A%20%20%3Fastronaut%20%3Fx1%20wd%3AQ11631.%0A%20%20%3Fastronaut%20wdt%3AP569%20%3Fbirthdate.%0A%20%20%3Fastronaut%20wdt%3AP19%20%3Fbirthplace.%0A%20%20SERVICE%20wikibase%3Alabel%20%7B%20bd%3AserviceParam%20wikibase%3Alanguage%20%22en%22.%20%7D%0A%20%20OPTIONAL%20%7B%20%3Fastronaut%20wdt%3AP21%20%3Fsex_or_gender.%20%7D%0A%20%20OPTIONAL%20%7B%20%3Fastronaut%20wdt%3AP2873%20%3Ftime_in_space.%20%7D%0A%20%20OPTIONAL%20%7B%20%3Fastronaut%20wdt%3AP570%20%3Fdate_of_death.%20%7D%0A%7D%0AORDER%20BY%20DESC%28%3Ftime_in_space%29
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment