I recently needed to create multi-arch images so that I could spin up containers on my arm-based laptop. Github Actions only runs on x64-86 VMs, which can make building images for other architectures tricky.
Regular old Docker Build
name: Docker
on:
push:
branches: ['main']
jobs:
docker:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v3
- name: Login to Docker Hub
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build and push
uses: docker/build-push-action@v3
with:
push: true
tags: user/app:latest
With the above Github Actions file, we could build and push to Docker Hub easily. Now let's check out how we can use buildx to build for multiple architectures.
QEMU and Buildx
QEMU is a generic and open source machine emulator and virtualizer. When used as a machine emulator, QEMU can run OSes and programs made for one machine (e.g. an ARM board) on a different machine (e.g. your own PC). By using dynamic translation, it achieves very good performance.
Docker buildx is a Docker plugin that extends the building ability of images by using the BuildKit builder. It allows us to build images for different platforms and architectures.
With a combination of QEMU and buildx, which are conveniently also available as Github Actions, we can easily get our images built on emulated OSes and push onto the docker registry with multi-platform support!
Add the following after the registry checkout on our Github Actions workflow file:
- name: Set up QEMU
uses: docker/setup-qemu-action@v2
with:
platforms: 'arm64,arm'
- name: Set up Docker buildx
uses: docker/setup-buildx-action@v2
Note: The QEMU setup step has to be put before the buildx step!
By default, setup-qemu-action will create many VMs, most of which you might not care about. Make sure to put in the platforms you actually want to build for in the platforms: field to limit this to only ones you actually need.
Arch-specific Dockerfile instructions
The docker-build-push-action will provide the --platform flag to docker buildx. This will set the following build ARGs automatically:
-
TARGETPLATFORM- platform of the build result. Eglinux/amd64,linux/arm/v7,windows/amd64. -
TARGETOS- OS component ofTARGETPLATFORM -
TARGETARCH- architecture component ofTARGETPLATFORM -
TARGETVARIANT- variant component ofTARGETPLATFORM -
BUILDPLATFORM- platform of the node performing the build. -
BUILDOS- OS component ofBUILDPLATFORM -
BUILDARCH- architecture component ofBUILDPLATFORM -
BUILDVARIANT- variant component ofBUILDPLATFORM
To use these ARGs in your Dockerfiles, you need to add them like the following example grabbed from Docker docs:
FROM alpine
ARG TARGETPLATFORM
RUN echo "I'm building for $TARGETPLATFORM"
End result
As a result, we can build and push multi-arch images by emulating other architectures on Github's x86_64 VMs using QEMU and building with docker buildx. We can even do arch-specific actions by using the ARGs automatically set by buildx.
Closing notes
Although this is convenient, no land without stones, or meat without bones.
There is one big and one relatively small problem this might cause for you.
1. Super long build times
Your builds might be taking very long after enabling multi-arch builds. This seems to be a known problem with emulating, but you could use some tricks to make less emulated code run while building.
In my case, I was building a Go application inside a build image, then copying that onto a target image in a two-step Dockerfile like so:
FROM golang:1.20-alpine as builder
WORKDIR /
RUN go env -w GOPRIVATE=github.com/myorganization
ARG GITHUB_TOKEN
RUN echo "machine github.com login github password $GITHUB_TOKEN" >> /root/.netrc
COPY . .
RUN go build "-ldflags=-s -w" -o /my-application cmd/main.go
FROM alpine:3.8
COPY --from=builder /my-application /app/my-app
WORKDIR /app
CMD /app/my-app
Instead of running the go build inside the Dockerfile, I switched to building directly inside the Github Actions using GOOS and GOARCH variables to specify building for a specific arch:
- name: Set up Go 1.20
uses: actions/setup-go@v1
with:
go-version: 1.20
- name: Setup netrc
run: |
echo "machine github.com login github password ${{ secrets.ACCESS_TOKEN }}" > ~/.netrc
- name: Build application
run: |
go env -w GOPRIVATE=github.com/myorganization
GOOS=linux GOARCH=amd64 go build "-ldflags=-s -w" -o build/my-app-amd64 cmd/main.go
GOOS=linux GOARCH=arm64 go build "-ldflags=-s -w" -o build/my-app-arm64 cmd/main.go
And then copying those builds into the container using the buildx ARGs:
FROM alpine:3.8
ARG TARGETARCH
COPY build/my-app-${TARGETARCH} /app/my-app
WORKDIR /app
CMD /app/my-app
In the end, this lowered the whole execution time from ~6m to ~1,5m. I've heard people lower theirs down from ~1h to ~15m as well.
Pushing to AWS ECR creates multiple images
If you're pushing to AWS ECR, you might realize one push is creating many untagged images of type Image and one image of type Image Index containing all the tags, which looks like this:
This is actually normal, since pushing a multi-arch image really is just pushing untagged images for each arch and using an Image Index to redirect the user to the correct architecture image when pulling. The arch-specific images are untagged so that they cannot be easily discovered and pulled, and instead the Image Index that handles this is reached.
It looks like AWS might update the UI to make this look less cluttered in the future.
Top comments (0)