With the matrix strategy and two jobs like those below, the first job runs a "matrix" of possible configurations in parallel, and when all of them succeed, it moves onto the second job.
jobs:
job1:
runs-on: ubuntu-latest
strategy:
matrix:
version: [10, 12, 14]
steps:
# etc
job2:
runs-on: ubuntu-latest
needs: job1
strategy:
matrix:
version: [10, 12, 14]
steps:
# etc
What would be nice is to essentially have a matrix over the whole workflow, where for each matrix value, the entire workflow of jobs is run unimpeded, in parallel. That means for a particular configuration job2
can proceed right after job1
completes for that configuration, regardless of whether job1
for another configuration is still in process or has failed.
In the following diagram, the first case is what will currently happen, and the second is the desired case:
I feel like this could be "emulated" by launching the workflow several times with a different context. But how could this be done, for example from a trigger? Somehow specify on: push
that 3 versions of the workflow must run simultaneously?
After some digging, I think "Reusable workflows" is the currently supported method.
The solution is to have two workflow files. One that has all the jobs, and another small workflow that has the matrix and calls the other workflow.
name: Common Workflow
on:
workflow_call:
inputs:
version: # the variable you can use in place of a matrix
required: true
type: number
jobs:
job1:
runs-on: ubuntu-latest
steps:
- run: echo 'job1 version ${{ inputs.version }}'
job2:
runs-on: ubuntu-latest
needs: job1
steps:
- run: echo 'job2 version ${{ inputs.version }}'
name: Master Workflow
on:
push:
jobs:
version-matrix:
strategy:
# super important if you want to see all results, even if one fails
# fail-fast is true by default
fail-fast: false
matrix:
version: [10, 12, 14]
uses: ./.github/workflows/common-workflow.yml # calls the one above ^
with:
version: ${{ matrix.version }}
secrets: inherit
Here's a real example of it running in the UI:
One thing you will run into once you run this though, is the UI lists the subjobs in alphabetical order, and not the order they are listed in the workflow. So you lose the sense of the dependencies between the jobs. This is true in February 2023, so maybe Github will improve the UI down the road. Until then, we got around this by naming our jobs like 1. First Job
, 2. Second Job
so alphabetical is equivalent to the order they run in. It's hacky but it works.
Another thing to note, especially for deployments is the fail-fast
attribute. This should be set to false
to make sure every workflow in the matrix gets to run to completion, even if one of them fails.