Skip to content
GitLab
Projects
Groups
Snippets
/
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
Daniel Scheffler
py_tools_ds
Commits
399cc577
Commit
399cc577
authored
Sep 02, 2020
by
Daniel Scheffler
Browse files
Merge branch 'bugfix/fix_ci_connection' into 'master'
Bugfix/fix ci connection See merge request
!22
parents
0117d611
c9e62798
Pipeline
#12222
canceled with stages
in 42 minutes and 26 seconds
Changes
4
Pipelines
2
Hide whitespace changes
Inline
Side-by-side
.gitlab-ci.yml
View file @
399cc577
...
@@ -20,11 +20,7 @@ stages:
...
@@ -20,11 +20,7 @@ stages:
test_py_tools_ds
:
test_py_tools_ds
:
stage
:
test
stage
:
test
script
:
script
:
-
source /root/miniconda3/bin/activate
-
source /root/miniconda3/bin/activate ci_env
-
source activate ci_env
-
export GDAL_DATA=/root/miniconda3/envs/ci_env/share/gdal
-
export PYTHONPATH=$PYTHONPATH:/root
# /root <- directory needed later
-
conda install -c conda-forge 'pyproj>=2.2'
# FIXME remove as soon as docker container is rebuilt
-
make nosetests
-
make nosetests
-
make docs
-
make docs
artifacts
:
artifacts
:
...
@@ -39,10 +35,7 @@ test_py_tools_ds:
...
@@ -39,10 +35,7 @@ test_py_tools_ds:
test_styles
:
test_styles
:
stage
:
test
stage
:
test
script
:
script
:
-
source /root/miniconda3/bin/activate
-
source /root/miniconda3/bin/activate ci_env
-
source activate ci_env
-
export GDAL_DATA=/root/miniconda3/envs/ci_env/share/gdal
-
export PYTHONPATH=$PYTHONPATH:/root
# /root <- directory needed later
-
make lint
-
make lint
artifacts
:
artifacts
:
paths
:
paths
:
...
@@ -61,10 +54,10 @@ test_py_tools_ds_install:
...
@@ -61,10 +54,10 @@ test_py_tools_ds_install:
# update conda and python
# update conda and python
-
conda update -n base -c defaults conda
-
conda update -n base -c defaults conda
#
- conda update -c conda-forge python
#
- conda update -c conda-forge python
# avoid package incompatibilities due to usage of wrong channels
# avoid package incompatibilities due to usage of wrong channels
#
- conda config --set channel_priority strict # otherwise gdal or libgdal may be installed from defaults channel
#
- conda config --set channel_priority strict # otherwise gdal or libgdal may be installed from defaults channel
# resolve some requirements with conda
# resolve some requirements with conda
-
conda install --yes -q -c conda-forge numpy gdal 'pyproj>=2.1.0' shapely geopandas
-
conda install --yes -q -c conda-forge numpy gdal 'pyproj>=2.1.0' shapely geopandas
...
@@ -93,12 +86,16 @@ pages: # this job must be called 'pages' to advise GitLab to upload content to
...
@@ -93,12 +86,16 @@ pages: # this job must be called 'pages' to advise GitLab to upload content to
-
mkdir -p public/doc
-
mkdir -p public/doc
-
mkdir -p public/coverage
-
mkdir -p public/coverage
-
mkdir -p public/nosetests_reports
-
mkdir -p public/nosetests_reports
# Copy over the docs
# Copy over the docs
-
cp -r docs/_build/html/* public/doc/
-
cp -r docs/_build/html/* public/doc/
# Copy over the coverage reports
# Copy over the coverage reports
-
cp -r htmlcov/* public/coverage/
-
cp -r htmlcov/* public/coverage/
# Copy over the nosetests reports
# Copy over the nosetests reports
-
cp nosetests.* public/nosetests_reports/
-
cp nosetests.* public/nosetests_reports/
# Check if everything is working great
# Check if everything is working great
-
ls -al public
-
ls -al public
-
ls -al public/doc
-
ls -al public/doc
...
@@ -117,8 +114,7 @@ deploy_pypi:
...
@@ -117,8 +114,7 @@ deploy_pypi:
dependencies
:
dependencies
:
-
test_py_tools_ds
-
test_py_tools_ds
script
:
script
:
-
source /root/miniconda3/bin/activate
-
source /root/miniconda3/bin/activate ci_env
-
source activate ci_env
-
pip install -U twine
-
pip install -U twine
-
python setup.py sdist
-
python setup.py sdist
-
twine upload dist/*
# requires creds as environment variables
-
twine upload dist/*
# requires creds as environment variables
...
@@ -133,8 +129,7 @@ deploy_anaconda:
...
@@ -133,8 +129,7 @@ deploy_anaconda:
dependencies
:
dependencies
:
-
test_py_tools_ds
-
test_py_tools_ds
script
:
script
:
-
source /root/miniconda3/bin/activate
-
source /root/miniconda3/bin/activate ci_env
-
source activate ci_env
-
conda install -y -q conda-build conda-build-all anaconda-client
-
conda install -y -q conda-build conda-build-all anaconda-client
-
yum install -y patch
# https://stackoverflow.com/questions/40392161/building-conda-skeleton-from-pypi-package-throws-error
-
yum install -y patch
# https://stackoverflow.com/questions/40392161/building-conda-skeleton-from-pypi-package-throws-error
# - conda skeleton pypi py_tools_ds --output-dir public/conda.recipe/py_tools_ds # create recipe
# - conda skeleton pypi py_tools_ds --output-dir public/conda.recipe/py_tools_ds # create recipe
...
...
HISTORY.rst
View file @
399cc577
...
@@ -2,6 +2,14 @@
...
@@ -2,6 +2,14 @@
History
History
=======
=======
0.15.5 (2020-09-02)
-------------------
* Revised build_testsuite_image.sh
* Added docker-pull-policy to register command.
* Added some comments to build_testsuite_image.sh and set concurrency to 4. Updated deprecated code in .gitlab_ci.yml.
0.15.4 (2020-09-01)
0.15.4 (2020-09-01)
-------------------
-------------------
...
...
py_tools_ds/version.py
View file @
399cc577
...
@@ -19,5 +19,5 @@
...
@@ -19,5 +19,5 @@
# You should have received a copy of the GNU Lesser General Public License along
# You should have received a copy of the GNU Lesser General Public License along
# with this program. If not, see <http://www.gnu.org/licenses/>.
# with this program. If not, see <http://www.gnu.org/licenses/>.
__version__
=
'0.15.
4
'
__version__
=
'0.15.
5
'
__versionalias__
=
'2020090
1
_0
2
'
__versionalias__
=
'2020090
2
_0
1
'
tests/CI_docker/build_testsuite_image.sh
View file @
399cc577
...
@@ -7,25 +7,46 @@ gitlab_runner="py_tools_ds_gitlab_CI_runner"
...
@@ -7,25 +7,46 @@ gitlab_runner="py_tools_ds_gitlab_CI_runner"
echo
"#### Build runner docker image"
echo
"#### Build runner docker image"
docker rmi
${
tag
}
docker rmi
${
tag
}
docker build
-f
${
context_dir
}
/
${
dockerfile
}
-m
20G
-t
${
tag
}
${
context_dir
}
docker build
${
context_dir
}
\
-f
${
context_dir
}
/
${
dockerfile
}
\
-m
20G
\
-t
${
tag
}
# create the gitlab-runner docker container for the current project
# NOTE: The 'gitlab-runner' and 'gitlab-ci-multi-runner' services will run within this container.
# The runner uses a 'config.toml' configuration file at /etc/gitlab-runner within the container which can be
# modified through additional parameters of the 'gitlab-runner register' command.
echo
"#### Create gitlab-runner (daemon) container with tag;
${
tag
}
"
echo
"#### Create gitlab-runner (daemon) container with tag;
${
tag
}
"
docker stop
${
gitlab_runner
}
docker stop
${
gitlab_runner
}
docker
rm
${
gitlab_runner
}
docker
rm
${
gitlab_runner
}
docker run
-d
--name
${
gitlab_runner
}
--restart
always
\
docker run
\
-v
/var/run/docker.sock:/var/run/docker.sock gitlab/gitlab-runner:latest
-d
\
--name
${
gitlab_runner
}
\
--restart
always
\
-v
/var/run/docker.sock:/var/run/docker.sock
\
gitlab/gitlab-runner:latest
# register the runner at the corresponding GitLab repository via a registration-token
echo
"#### Register container at gitlab, get token here https://gitext.gfz-potsdam.de/danschef/py_tools_ds/settings/ci_cd"
echo
"#### Register container at gitlab, get token here https://gitext.gfz-potsdam.de/danschef/py_tools_ds/settings/ci_cd"
read
-p
"Please enter gitlab token: "
token
read
-p
"Please enter gitlab token: "
token
echo
""
echo
""
read
-p
"Please enter gitlab runner name: "
runner_name
read
-p
"Please enter gitlab runner name: "
runner_name
echo
"New gitlab runner image will named
${
gitlab_runner
}
"
echo
"New gitlab runner image will named
${
gitlab_runner
}
"
docker
exec
-it
${
gitlab_runner
}
/bin/bash
-c
"export RUNNER_EXECUTOR=docker && gitlab-ci-multi-runner register -n
\
# NOTE: In case of locally stored images (like here), the docker pull policy 'never' must be used
# (see https://docs.gitlab.com/runner/executors/docker.html#how-pull-policies-work).
docker
exec
-it
${
gitlab_runner
}
/bin/bash
-c
"
\
export RUNNER_EXECUTOR=docker &&
\
gitlab-ci-multi-runner register
\
--non-interactive
\
--executor 'docker'
\
--docker-image '
${
tag
}
'
\
--url 'https://gitext.gfz-potsdam.de/ci'
\
--url 'https://gitext.gfz-potsdam.de/ci'
\
--registration-token '
${
token
}
'
\
--registration-token '
${
token
}
'
\
--run-untagged=true
\
--locked=true
\
--tag-list py_tools_ds_ci_client
\
--description '
${
runner_name
}
'
\
--description '
${
runner_name
}
'
\
--docker-image '
${
tag
}
' "
--tag-list py_tools_ds_ci_client
\
--run-untagged='true'
\
--locked='true'
\
--access-level='not_protected'
\
--docker-pull-policy='never'
\
--request-concurrency=4"
ls
ls
Write
Preview
Supports
Markdown
0%
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment