Here's what I'm trying to achieve:

I have a project with a build job for a binary release. The binary takes a while to cross-compile for each platform, so I only want to release build to be done when a release is tagged, but I want the local-native version to build and tests to run for each checked-in version.

Based on the flight-school demo... so far, my pipeline configuration looks like this:

resources:
  - name: flight-school
    type: git
    source:
      uri: https://github.com/nbering/flight-school
      branch: master

  - name: flight-school-version
    type: semver
    source:
      driver: git
      uri: https://github.com/nbering/flight-school
      branch: master
      file: version

jobs:
  - name: test-app
    plan:
      - get: flight-school
        trigger: true
      - task: tests
        file: flight-school/build.yml

  - name: release-build
    plan:
      - aggregate:
        - get: flight-school-version
          trigger: true
        - get: flight-school
          passed: [test-app]
      - task: release-build
        file: flight-school/ci/release.yml

This produces a pipeline in the Web UI that looks like this:

The problem is that when I update the "release" file in the git repository, the semver resource, "flight-school-version" can check before the git resource "flight-school", causing the release build to be processed from the git version assigned to the previous check-in.

I'd like a way to work around this so that the release build appears as a separate task, but only triggers when the version is bumped.

Some things I've thought of so far

Create a separate git resource with a tag_filter set so that it only runs when a semver tag has been push to master

• Pro: Jobs only run when tag is pushed
• Con: Has the same disconnected-inheritance problem for tests as the semver-based example above

Add the conditional check for a semver tag (or change diff on a file) using the git history in the checkout as part of the build script

• Pro: Will do basically what I want without too much wrestling with Concourse
• Con: Can't see the difference in the UI without actually reading the build output
• Con: Difficult to compose with other tasks and resource types to do something with the binary release

Manually trigger release build

• Pro: Simple to set up
• Con: Requires manual intervention.

Use the API to trigger a paused build step on completion of tests when a version change is detected

• Con: Haven't seen any examples of others doing this, seems really complicated.

I haven't found a way to trigger a task when both the git resource and semver resource change.

I'm looking for either an answer to solve the concurrency problem in my above example, or an alternative pattern that would produce a similar release workflow.

Summary

Here's what I came up with for a solution, based on suggestions from the Concourse CI slack channel.

I added a parallel "release" track, which filters on tags resembling a semantic versioning versions. The two tracks share task configuration files and build scripts.

Tag Filtering

The git resource supports a tag_filter option. From the README:

tag_filter: Optional. If specified, the resource will only detect commits that have a tag matching the expression that have been made against the branch. Patterns are glob(7) compatible (as in, bash compatible).

I used a simple glob pattern to match my semver tags (like v0.0.1):

v[0-9]*

At first I tried an "extglob" pattern, matching semantic versions exactly, like this:

v+([0-9]).+([0-9]).+([0-9])?(\-+([-A-Za-z0-9.]))?(\++([-A-Za-z0-9.]))

That didn't work, because the git resource isn't using the extglob shell option.

The end result is a resource that looks like this:

resource:
  - name: flight-school-release
    type: git
    source:
      uri: https://github.com/nbering/flight-school
      branch: master
      tag_filter: 'v[0-9]*'

Re-Using Task Definitions

The next challenge I faced was avoiding re-writing my test definition file for the release track. I would have to do this because all the file paths use the resource name, and I now have a resource for release, and development. My solution is to override the resource with an option on the get task.

jobs:
  - name: test-app-release
    plan:
      - get: flight-school
        resource: flight-school-release
        trigger: true
      - task: tests
        file: flight-school/build.yml

Build.yml above is the standard example from the flight school tutorial.

Putting It All Together

My resulting pipeline looks like this:

My complete pipeline config looks like this:

resources:
  - name: flight-school-master
    type: git
    source:
      uri: https://github.com/nbering/flight-school
      branch: master

  - name: flight-school-release
    type: git
    source:
      uri: https://github.com/nbering/flight-school
      branch: master
      tag_filter: 'v[0-9]*'

jobs:
  - name: test-app-dev
    plan:
      - get: flight-school
        resource: flight-school-master
        trigger: true
      - task: tests
        file: flight-school/build.yml

  - name: test-app-release
    plan:
      - get: flight-school
        resource: flight-school-release
        trigger: true
      - task: tests
        file: flight-school/build.yml

  - name: build-release
    plan:
      - get: flight-school
        resource: flight-school-release
        trigger: true
        passed: [test-app-release]
      - task: release-build
        file: flight-school/ci/release.yml