Hi,

In our GitHub repository, we need to run e2e tests that we’d like to execute only when a pull request (PR) is ready for merging (e.g., has at least one approval). We don’t want to include these tests in the basic pull request checks.

Our e2e tests take approximately 10–15 minutes to complete. We’ve chosen GitHub Merge Queue and prepared a gate.yml workflow that runs on the merge_group event. Additionally, we’ve set up status checks for protected branches.

We kept the merge queue configuration mostly default:

• Merge method: Merge commit
• Build concurrency: 5
• Merge limits:
  • Min pull requests: 1 or after 5 mins (this seems problematic given our tests take 10 minutes)
  • Max pull requests to merge: 5
• Status check timeout: 118 minutes

The Problem:

When merging a single PR, everything works fine. However, issues arise when we queue PR A and PR B:

1. PR A is added to the queue, and the gate.yml workflow starts.
2. PR B is added to the queue, and the gate.yml workflow starts for it as well.
3. After 10 minutes, the gate.yml workflow for PR A completes successfully, and PR A is merged.
4. 5 minutes later, the gate.yml workflow for PR B also completes successfully, but PR B is not merged – instead, the gate.yml workflow for PR B is restarted.
5. PR B is eventually merged only after the second gate.yml run, which takes an additional 10 minutes.

The Outcome:

The gate.yml workflow is executed twice for PR B, which is redundant and introduces delays.

Questions:

1. How can we configure the merge queue so that the gate.yml workflow doesn’t run twice for a single PR?
2. Is GitHub Merge Queue the right solution for our use case? Or should we try to implement our custom workflow as a guard to run e2e only after pull-request was approved?

name: gate
on:
  merge_group:
    types:
      - checks_requested
jobs:
e2e-stage:

permissions:
id-token: write
contents: read
pull-requests: read
name: e2e-stage
uses: ./.github/workflows/rw-rush-build-e2e-tests.yml
secrets: inherit
ready-to-merge:
runs-on:
group: infra1-runners-arc
labels: runners-small
if: always()
needs:
- e2e-stage
steps:
- name: Check if needed jobs succeeded
uses: re-actors/alls-green@release/v1
with:
allowed-skips: ${{ toJSON(needs) }}
jobs: ${{ toJSON(needs) }}

Thanks very much for help and hints

The problem is mainly arising because of parallel runs. So, the simplest solution is to have only one workflow run at a time. Do not allow parallel runs.

Option 2: Use Caching

Implement caching mechanism for your gate workflow

- name: Cache E2E Test Results
  uses: actions/cache@v3
  with:
    path: ./e2e-test-results
    key: e2e-${{ github.event.pull_request.head.sha }}

And check the cache before running the tests

- name: Check Cache for Test Results
  id: cache-check
  uses: actions/cache@v3
  with:
    path: ./e2e-test-results
    key: e2e-${{ github.event.pull_request.head.sha }}

- name: Run E2E Tests
  if: steps.cache-check.outputs.cache-hit != 'true'
  run: ./run-e2e-tests.sh

Or Option 3 might be to trigger e2e tests only on approval.