Using github merge queues with self-hosted buildbot CI

We've been using bors-ng for our merge queues for some years and now we'd like to migrate to Github Merge Queues.

I find GitHub's documentation difficult to follow, but as far as I can see, I need to do the following:

1: Set up buildbot to build merge queue branches:

e.g.

c['schedulers'].append(schedulers.SingleBranchScheduler(
                       name="simple-ghmq-branches-scheduler",
                       change_filter=util.ChangeFilter(
                           branch_re=re.compile("^gh-readonly-queue/"),
                            repository=REPOS,
                        ),
                        builderNames=["buildbot-build-script"],
                        treeStableTimer=TREE_STABLE_TIMEOUT))

This is very similar to what we had for our working bors-ng setup (just branch_re=... was branch=['trying', 'staging'])

2: Turn on merge queues in GH

Add a branch protection rule for the main branch and turn on merge queues:

3: Add buildbot as a required check to the base branch

You can only add required checks that recently notified github. I just wrote a little python script to do this.

After that, in the main branch protection rules, set buildbot as the required check.

4: Add a workflow to the main branch.

As per these docs.

e.g. in .github/workflows/ci.yml:

on:
  pull_request:
  merge_group:

5: Raise a PR

Click the "merge when ready" button.

But the problem is, github never makes the branch for the CI to pick up, so it just sits there forever.

Can anyone see what the problem is?

EDIT: also tried removing the required status checks from main and adding them to a new branch protection rule for gh-readonly-queue/*. Under this setup, github did make a branch, and buildbot did build it. Buildbot reported failure, but github still merged the branch... I don't get it.

The problem here is that github currently requires the same set of CI checks to be run as the PR is raised as after the merge button is clicked.

We worked around this by having our CI system run for all "push" and "pull_request" events, but for the latter have the CI immediately return success without actually running any CI tests.

Contrary to the github docs, in this scenario you don't want to only run CI for gh-readonly-queue/* branches. If you do that, then CI won't be able to run as the PR is raised, and thus github will never receive the "OK" status from your CI, and thus it will never be deemed "ready to merge".

Instead, do the filtering inside your CI.

I really hope github improve this in the future.