I am using circleci to deploy an application, I deploy to both amd and arm architectures so my builds are multi-arch which I have been using docker buildx for. With the new arm support from circleci I was able to cut the time on this process down from sometimes 3 hours using quemu, to around 20 minutes by building both separately in their respective build environments (no need to use quemu when you build on the target arch). What I am running into is that when I run the buildx commands, one build will complete, push it's results to the repository and then the other completes and overwrites the previous. What I am trying to achieve is combining the built images into a single manifest to push together as if I built them at the same time. Is there a way to achieve what I am attempting without getting into direct modification of the manifest files? An example of the commands needed to achieve this would be extremely helpful!
Thanks in advance!
There are two options I know of.
First, you can have buildx run builds on multiple nodes, one for each platform, rather than using qemu. For that, you would use docker buildx create --append to add the additional nodes to the builder instance. The downside of this is you'll need the nodes accessible from the node running docker buildx which likely doesn't apply to ephemeral cloud build environments.
The second option is to use the experimental docker manifest command. Each builder would push a separate tag. And at the end of all those, you would use docker manifest create to build a manifest list and docker manifest push to push that to a registry. Since this is an experimental feature, you'll want to export DOCKER_CLI_EXPERIMENTAL=enabled to see it in the command line. (You can also modify ~/.docker/config.json to have an "experimental": "enabled" entry.)