githubcontinuous-integrationgithub-actionscicdcontinuous-delivery

How can I create a multi-job GitHub Actions workflow with secrets?


I would like to create a multi-job GitHub Actions workflow to lint and test my python code, lint my Dockerfile, then build and push my docker image to my Docker Hub only if the pytest, pylint, and hadolint workflows finish successfully. I managed to do this in a single job using the following workflow:

name: Github Actions

on: [push]

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v4
    - name: Set up Python 3.10
      uses: actions/setup-python@v5
      with:
        python-version: '3.10'
    - name: Install dependencies
      run: |
        make install
    - name: Lint with pylint
      run: |
        make lint
    - name: Test with pytest
      env:
        DEVELOPER_KEY: ${{ secrets.DEVELOPER_KEY }}
      run: |
        make test

    - name: Dockerfile linting
      uses: hadolint/hadolint-action@v3.1.0
      with:
        dockerfile: Dockerfile

    - name: Build and Push Docker Image
      uses: mr-smithers-excellent/docker-build-push@v6
      with:
        image: stevenherrera/youtube-comment-microservice
        tags: v1, latest
        registry: docker.io
        username: ${{ secrets.DOCKER_USERNAME }}
        password: ${{ secrets.DOCKER_PASSWORD }}

I created a separate dev branch and attempted to convert this to a multi-job workflow to take advantage of parallelism using the following:

name: Dev CI

on:
  push:
    branches:
      - dev

jobs:
  lint_and_test:
    name: Pylint and pyest microservice
    uses: ./.github/workflows/job_lint_test_microservice.yml

  hadolint:
    name: Hadolint Dockerfile
    uses: ./.github/workflows/job_hadolint_dockerfile.yml

  build_and_push_img:
    name: Docker Build and Push
    needs: [lint_and_test, hadolint]
    uses: ./.github/workflows/job_build_push_img.yml

However, the pytesting portion fails with the following error message:

google.auth.exceptions.DefaultCredentialsError: Your default credentials were not found.

Which I find odd since the first workflow is able to find my repository secrets and does not receive this error. I have provided a link to the relevant repository here so you can see more details. Please tell me how I can rewrite this workflow into a multi-job build that accomplishes what the first workflow does.

Many thanks.


Solution

  • After some tinkering I got the following to work:

    name: Dev CI/CD Pipeline
    
    on:
      push:
        branches:
          - dev
    
    jobs:
      pylint:
        runs-on: ubuntu-latest
        steps:
        - name: Checkout code
          uses: actions/checkout@v4
    
        - name: Set up Python 3.10
          uses: actions/setup-python@v5
          with:
            python-version: '3.10'
    
        - name: Install Python dependencies
          run: make install
    
        - name: Lint Python code with pylint
          run: make lint
    
      pytest:
        runs-on: ubuntu-latest
        steps:
        - name: Checkout code
          uses: actions/checkout@v4
    
        - name: Set up Python 3.10
          uses: actions/setup-python@v5
          with:
            python-version: '3.10'
    
        - name: Install Python dependencies
          run: make install
    
        - name: Test Python code with pytest
          env:
            DEVELOPER_KEY: ${{ secrets.DEVELOPER_KEY }}
          run: make test
    
      hadolint:
        runs-on: ubuntu-latest
        steps:
        - name: Checkout code
          uses: actions/checkout@v4
    
        - name: Dockerfile linting with hadolint
          uses: hadolint/hadolint-action@v3.1.0
          with:
            dockerfile: Dockerfile
    
      build_and_push:
        runs-on: ubuntu-latest
        needs: [pylint, pytest, hadolint]
        steps:
        - name: Checkout code
          uses: actions/checkout@v4
    
        - name: Build and Push Docker Image
          uses: mr-smithers-excellent/docker-build-push@v6
          with:
            image: ${{ secrets.DOCKER_USERNAME }}/youtube-comment-microservice
            tags: v1, latest
            registry: docker.io
            username: ${{ secrets.DOCKER_USERNAME }}
            password: ${{ secrets.DOCKER_PASSWORD }}