github-actions

Is it possible to share or reuse some job steps inside Github actions?


Giving the following sample Github actions workflow

name: My workflow

on: pull_request

jobs:
  foo:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
        with:
          fetch-depth: 0

      - name: Setup Go
        uses: actions/setup-go@v3
        with:
          go-version: 1.19

      - name: Foo
        run: echo "foo"

  bar:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
        with:
          fetch-depth: 0

      - name: Setup Go
        uses: actions/setup-go@v3
        with:
          go-version: 1.19

      - name: Bar
        run: echo "bar"

I want the jobs Foo and Bar to run in parallel. But as you can see they have some steps in common.

Is it possible to create a job that runs the checkout and setup step and provides itself to Foo and Bar so they only have to run their own commands? ( that would save some time, but I don't think that's possible because both jobs run in separate containers )

If that's not possible, is there a way to extract the "duplicate" lines and move them to a "step function" I can call in my jobs so I don't have to write those steps over and over again?


Solution

  • You may use caching to save some time: https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows and to consolidate the "duplicated lines in your YAML file" in every job, you may want to have a composite action, where you basically extract

    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
        with:
          fetch-depth: 0
    
      - name: Setup Go
        uses: actions/setup-go@v3
        with:
          go-version: 1.19
    

    To a new action that you may use in your workflow file to look like:

    name: My workflow
    
    on: pull_request
    
    jobs:
      foo:
        runs-on: ubuntu-latest
        steps:
          - name: My composite action
            uses: path/to/action
    
          - name: Foo
            run: echo "foo"
    
      bar:
        runs-on: ubuntu-latest
        steps:
          - name: My composite action
            uses: path/to/action
    
          - name: Bar
            run: echo "bar"
    

    Note that if you want to create this composite action in the same repository you will have to use actions/checkout@v3 before you call it using a relative URL.

    So it will be:

    name: My workflow
    
    on: pull_request
    
    jobs:
      foo:
        runs-on: ubuntu-latest
        steps:
          - name: Checkout repository
            uses: actions/checkout@v3
    
          - name: My composite action
            uses: ./.github/actions/my-action.yaml
    
          - name: Foo
            run: echo "foo"
    
      bar:
        runs-on: ubuntu-latest
        steps:
          - name: Checkout repository
            uses: actions/checkout@v3
    
          - name: My composite action
            uses: ./.github/actions/my-action.yaml
        
          - name: Bar
            run: echo "bar"
    

    And yes, if you only have only a few steps this approach may not bring much value to you. As you may only save few lines in the YAML file, and you can only cache your dependencies installation.

    And, this doesn't mean that your "shared/composite" actions will run only once, Github will re-run each of their steps for each job that's calling them (foo, bar in your case).


    One other approach, to consolidate some of the steps you run in your pipelines is by creating a Docker image in which your actions will run, this base docker image may have the necessary setup for you, for example: GoLang, and your necessary build and test modules installation.

    name: My workflow
    
    on: pull_request
    
    jobs:
      foo:
        runs-on: ubuntu-latest
        container: mydocker.image.uri/name:version
        steps:
          - name: Checkout repository
            uses: actions/checkout@v3
    
          - name: Foo
            run: echo "foo"
    
      bar:
        runs-on: ubuntu-latest
        container: mydocker.image.uri/name:version
        steps:
          - name: Checkout repository
            uses: actions/checkout@v3
    
          - name: Bar
            run: echo "bar"
    

    The gains with this approach are that you may cut some lines out of your workflow file, and you extracted some setup steps to the base docker image, in which you will run your actions.

    About the downsides, it might be a little bit challenging to build a stable base image with the necessary setups to run your steps. And also, you will need to maintain another part of your CI/CD pipelines.


    One other solution would be the usage of an execution matrix (with Caching dependency files and build outputs), which will run parallel jobs for each of your matrix values (they will be parallelized depending on the runner's availability or by your max-parallel value)

    name: My workflow
    
    on: pull_request
       
    jobs:
      foo:
        runs-on: ubuntu-latest
        strategy:
          matrix: 
            greeting: ["hello", "bonjour"]
        steps:
          - name: Checkout repository
            uses: actions/checkout@v3
          
          - name: Setup Go
            uses: actions/setup-go@v3
            with:
              go-version: 1.19
              cache: true
    
          - name: Saying ${{ matrix.greeting }}
            run: echo "${{ matrix.greeting }}!"
    

    steps running in parallel

    And as you see, all steps re-run again:

    step example

    which might not be interesting if you are doing more than GoLang setup but doing steps that may take a lot of time


    And the last option I have in my mind is to use dependant jobs, which may not work for this use case. But it might be a solution if you can redesign your workflow to produce an output, or a binary from a first step called baz then your workflow would have

      foo:
        runs-on: ubuntu-latest
        needs: baz
        steps:
         - name: Something
           run: echo "baz is saying: ${{ needs.baz.outputs.greeting }}"
    

    I hope this helps or gave you more ideas on how to optimize this workflow further!