I am very new to Gitlab CI/CD and I have read its documentation very carefully about creating a new CI/CD process using .gitlab-ci.yml
file. As I have found out in order to have Continuous Deployment (also known as CD), it is needed to register a new gitlab-runner on my linux server.
Explanation
Here is my .gitlab-ci.yml
file:
stages:
- build
- deploy
docker-build:
image: docker:stable
services:
- docker:dind
stage: build
only:
refs:
- ci-test
before_script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
script:
- docker build --pull -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA .
- echo $CI_REGISTRY_IMAGE
- echo $CI_COMMIT_SHORT_SHA
- docker tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA $CI_REGISTRY_IMAGE:latest
- docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
- docker push $CI_REGISTRY_IMAGE:latest
deploy:
image: docker:stable
stage: deploy
services:
- docker:dind
only:
refs:
- ci-test
when: manual
except:
changes:
- "*.md"
script:
- docker build --pull -t $CI_REGISTRY_IMAGE$CI_COMMIT_SHORT_SHA .
- docker tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA $CI_REGISTRY_IMAGE:latest
- docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
- docker push $CI_REGISTRY_IMAGE:latest
and this is my Dockerfile:
version: '3'
services:
app:
# build: .
environment:
- DEFAULT_DATABASE_HOST=${DEFAULT_DATABASE_HOST}
- DEFAULT_DATABASE_NAME=${DEFAULT_DATABASE_NAME}
- DEFAULT_DATABASE_USER=${DEFAULT_DATABASE_USER}
- DEFAULT_DATABASE_PASSWORD=${DEFAULT_DATABASE_PASSWORD}
- DEBUG=${DEBUG}
- DJANGO_SETTINGS_MODULE=_base.settings
- CELERY_BROKER_URL=redis://redis:6379/0
# command: uwsgi --http :8000 --socket /socket/api.sock --chmod-socket=666 --module _base.wsgi --master --processes 5 --threads 2
command: uwsgi --http :8000 --module _base.wsgi --master --processes 5 --threads 2
# command: python manage.py runserver 0.0.0.0:8000
depends_on:
- redis
ports:
- 8000:8000
image: ${CLEARSIGHT_IMAGE_NAME}:${CLEARSIGHT_IMAGE_TAG}
restart: on-failure
network_mode: host
redis:
image: redis:6.0-alpine
volumes:
- /var/lib/redis/redis.dump:
ports:
- 6379:6379
#
celery-beat:
environment:
- DEFAULT_DATABASE_HOST=${DEFAULT_DATABASE_HOST}
- DEFAULT_DATABASE_NAME=${DEFAULT_DATABASE_NAME}
- DEFAULT_DATABASE_USER=${DEFAULT_DATABASE_USER}
- DEFAULT_DATABASE_PASSWORD=${DEFAULT_DATABASE_PASSWORD}
- DEBUG=${DEBUG}
- DJANGO_SETTINGS_MODULE=_base.settings
- CELERY_BROKER_URL=redis://redis:6379/0
command: celery -A _base beat -l info
depends_on:
- redis
image: ${CLEARSIGHT_IMAGE_NAME}:${CLEARSIGHT_IMAGE_TAG}
celery-worker-default:
environment:
- DEFAULT_DATABASE_HOST=172.17.0.1
- DEFAULT_DATABASE_NAME=${DEFAULT_DATABASE_NAME}
- DEFAULT_DATABASE_USER=${DEFAULT_DATABASE_USER}
- DEFAULT_DATABASE_PASSWORD=${DEFAULT_DATABASE_PASSWORD}
- DEBUG=${DEBUG}
- DJANGO_SETTINGS_MODULE=_base.settings
- CELERY_BROKER_URL=redis://redis:6379/0
command: celery -A _base worker -l INFO -Q clearsight-default --concurrency 1 -n clearsight-default
depends_on:
- redis
image: ${CLEARSIGHT_IMAGE_NAME}:${CLEARSIGHT_IMAGE_TAG}
celery-worker-aws:
environment:
- DEFAULT_DATABASE_HOST=172.17.0.1
- DEFAULT_DATABASE_NAME=${DEFAULT_DATABASE_NAME}
- DEFAULT_DATABASE_USER=${DEFAULT_DATABASE_USER}
- DEFAULT_DATABASE_PASSWORD=${DEFAULT_DATABASE_PASSWORD}
- DEBUG=${DEBUG}
- DJANGO_SETTINGS_MODULE=_base.settings
- CELERY_BROKER_URL=redis://redis:6379/0
command: celery -A _base worker -l INFO -Q clearsight-aws --concurrency 1 -n clearsight-default
depends_on:
- redis
image: ${CLEARSIGHT_IMAGE_NAME}:${CLEARSIGHT_IMAGE_TAG}
watchtower:
image: containrrr/watchtower:1.3.0
container_name: watchtower
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ~/.docker/config.json:/config.json
restart: always
command: --interval 60
Problem I have tried the CI pipeline and it works properly with shared runners:
So, I think there isn't any problem in CI side. Therefore, I registered a new gitlab-runner
(docker+machine) in the server and it can be seen in the repository CI/CD runners, but as it can be seen in the below picture, it is not connected!
Question How can I resolve the runner issue and make the runner connect to the jobs?
The problem was solved after running the gitlab-runner verify
command.
~ ➤ gitlab-runner -h # shows the help and other commands of gitlab-runner
NAME:
gitlab-runner - a GitLab Runner
USAGE:
gitlab-runner [global options] command [command options] [arguments...]
VERSION:
13.11.0 (7f7a4bb0)
AUTHOR:
GitLab Inc. <support@gitlab.com>
COMMANDS:
exec execute a build locally
list List all configured runners
run run multi runner service
register register a new runner
install install service
uninstall uninstall service
start start service
stop stop service
restart restart service
status get status of a service
run-single start single runner
unregister unregister specific runner
verify verify all registered runners
artifacts-downloader download and extract build artifacts (internal)
artifacts-uploader create and upload build artifacts (internal)
cache-archiver create and upload cache artifacts (internal)
cache-extractor download and extract cache artifacts (internal)
cache-init changed permissions for cache paths (internal)
health-check check health for a specific address
read-logs reads job logs from a file, used by kubernetes executor (internal)
help, h Shows a list of commands or help for one command
GLOBAL OPTIONS:
--cpuprofile value write cpu profile to file [$CPU_PROFILE]
--debug debug mode [$DEBUG]
--log-format value Choose log format (options: runner, text, json) [$LOG_FORMAT]
--log-level value, -l value Log level (options: debug, info, warn, error, fatal, panic) [$LOG_LEVEL]
--help, -h show help
--version, -v print the version
Now it works fine:
Thanks @WytrzymałyWiktor for his comment on this post. I didn't find anything helpful other than his comment.
P.S After performing the above steps, you may need to run gitlab-runner start
in order to resolve your problem!