I would like to create a multi-job GitHub Actions workflow to lint and test my python code, lint my Dockerfile, then build and push my docker image to my Docker Hub only if the pytest, pylint, and hadolint workflows finish successfully. I managed to do this in a single job using the following workflow:
name: Github Actions
on: [push]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python 3.10
uses: actions/setup-python@v5
with:
python-version: '3.10'
- name: Install dependencies
run: |
make install
- name: Lint with pylint
run: |
make lint
- name: Test with pytest
env:
DEVELOPER_KEY: ${{ secrets.DEVELOPER_KEY }}
run: |
make test
- name: Dockerfile linting
uses: hadolint/hadolint-action@v3.1.0
with:
dockerfile: Dockerfile
- name: Build and Push Docker Image
uses: mr-smithers-excellent/docker-build-push@v6
with:
image: stevenherrera/youtube-comment-microservice
tags: v1, latest
registry: docker.io
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
I created a separate dev
branch and attempted to convert this to a multi-job workflow to take advantage of parallelism using the following:
name: Dev CI
on:
push:
branches:
- dev
jobs:
lint_and_test:
name: Pylint and pyest microservice
uses: ./.github/workflows/job_lint_test_microservice.yml
hadolint:
name: Hadolint Dockerfile
uses: ./.github/workflows/job_hadolint_dockerfile.yml
build_and_push_img:
name: Docker Build and Push
needs: [lint_and_test, hadolint]
uses: ./.github/workflows/job_build_push_img.yml
However, the pytesting portion fails with the following error message:
google.auth.exceptions.DefaultCredentialsError: Your default credentials were not found.
Which I find odd since the first workflow is able to find my repository secrets and does not receive this error. I have provided a link to the relevant repository here so you can see more details. Please tell me how I can rewrite this workflow into a multi-job build that accomplishes what the first workflow does.
Many thanks.
After some tinkering I got the following to work:
name: Dev CI/CD Pipeline
on:
push:
branches:
- dev
jobs:
pylint:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python 3.10
uses: actions/setup-python@v5
with:
python-version: '3.10'
- name: Install Python dependencies
run: make install
- name: Lint Python code with pylint
run: make lint
pytest:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python 3.10
uses: actions/setup-python@v5
with:
python-version: '3.10'
- name: Install Python dependencies
run: make install
- name: Test Python code with pytest
env:
DEVELOPER_KEY: ${{ secrets.DEVELOPER_KEY }}
run: make test
hadolint:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Dockerfile linting with hadolint
uses: hadolint/hadolint-action@v3.1.0
with:
dockerfile: Dockerfile
build_and_push:
runs-on: ubuntu-latest
needs: [pylint, pytest, hadolint]
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Build and Push Docker Image
uses: mr-smithers-excellent/docker-build-push@v6
with:
image: ${{ secrets.DOCKER_USERNAME }}/youtube-comment-microservice
tags: v1, latest
registry: docker.io
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}