Using Docker Buildx to Create Cross-Platform Docker Images for Seamless Compatibility

Docker is really a great tool and provides a convenient way to package and deploy applications.

Until my recent experience, I believed that Docker images were portable and could run on any machine regardless of CPU architecture. Basically, one of my colleagues built an image on his Mac and shipped it to me, and I was attempting to run it on Ubuntu then I realised I was mistaken.

The problem was that Mac's use ARM processors, whereas mine was AMD processor and whenever I attempted to run the container, it displayed a warning and the container was abruptly terminated.

WARNING: The requested image's platform (linux/arm64) does not match the detected host platform (linux/amd64) and no specific platform was requested

Later, after doing some research, I discovered something called docker buildx with --platform flag, which we can use while building an image.

Docker Buildx

Docker Buildx is a command-line tool that provides advanced capabilities for building and managing multi-platform Docker images. Basically, it is an extension to the regular docker build command.

Using docker buildx, we can create docker images that are compatible with multiple CPU architectures, such as AMD64, ARM64, etc. Honestly, after encountering the above issue, I think it is pretty useful when there is a requirement for deploying or running a container on different architectures.

A quick rundown of how it can be utilised

• Builder instance: For using docker buildx, there should be a builder instance up and running and for creating a builder instance fire docker buildx create --use. ```

$ docker buildx create --use --name buildx_instance
buildx_instance

- Once the builder instance is created and active, it can be used using `docker buildx build` command for building images.

- As well the status of the instance can be checked using `docker buildx ls` or `docker ps` command.

$ docker buildx ls
NAME/NODE DRIVER/ENDPOINT STATUS BUILDKIT PLATFORMS
buildx_instance * docker-container

buildx_instance0 unix:///var/run/docker.sock running v0.11.6 linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/386
default docker

default default running 20.10.22 linux/amd64, linux/386

$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e023022a7b0e moby/buildkit:buildx-stable-1 "buildkitd" 6 minutes ago Up 6 minutes buildx_buildkit_buildx_instance0

- `docker buildx build` command can be used for building images, the intresting option while building the image is `--platform` flag. In this command we are creating a image which is compatible with ARM as well as AMD processors. 

$ docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7 -t sunnybhambhani/multi_test:v2 .

- Note: The images won't be visible under `docker images` and these images will be in buildx cache only. Further `--push` flag can be used to directly push the images to dockerhub.

$ docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7 -t sunnybhambhani/multi_test:v2 --push .

- Here is how it will be visible on dockerhub:
![Here it is, a single image with many platforms, Screendump from Dockerhub.](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/av2ca6m6x2za2w4wt5r5.png)

Additionally, `docker buildx imagetools inspect` can be used for inspecting docker images.

$ docker buildx imagetools inspect sunnybhambhani/multi_test:v2
Name: docker.io/sunnybhambhani/multi_test:v2
MediaType: application/vnd.docker.distribution.manifest.list.v2+json
Digest: sha256:af1c773ab325739067687e2df19137de675b6c7b4d618e6fac0197b55d8136bc

Manifests:
Name: docker.io/sunnybhambhani/multi_test:v2@sha256:1b270f007c1645e69e43595f0b5cd7fd06951c2974901943bcc503647b4fb35f
MediaType: application/vnd.docker.distribution.manifest.v2+json
Platform: linux/amd64

Name: docker.io/sunnybhambhani/multi_test:v2@sha256:4520b7bb8af2c868205524856667aa88ce180d260330aef3a325f69a3f336acb
MediaType: application/vnd.docker.distribution.manifest.v2+json
Platform: linux/arm64

Name: docker.io/sunnybhambhani/multi_test:v2@sha256:1f5d06b9181d8e4b5b552f2c5f5f0978d9409aef5233059c3136b469871c5b7f
MediaType: application/vnd.docker.distribution.manifest.v2+json
Platform: linux/arm/v7

Other important `docker buildx` commands:

docker buildx du: Check the disk usage
docker buildx imagetools inspect: To inspect the created images
docker buildx inspect: To inspect current builder instance
docker buildx ls: To list builder instances
docker buildx prune: To remove build cache
docker buildx rm: To remove a builder instance
docker buildx stop: To stop builder instance

References and more details around this topic can be found on below URLs:
- https://docs.docker.com/build/building/multi-platform/
- https://github.com/docker/buildx
- `docker buildx --help`

Feel free to add your thoughts and correct me if I am wrong or have missed something, Happy learning :) 

Comments

[krlz]

This is really useful to know, I didn't ecounter the experience yet but I will keep it in mind

[Jason Andrews]

Readers may want to check Using Docker manifest to create multi-arch images for more multi-arch docker info.

[Andreas Winschu]

Unfortunatelly on Apple Silicon Macs those linux/amd64 images will run only in 32 bit CPU op-mode at the time of writing.

Especially if you try running some libs with native extension (as its often in python) those libs wont work.

so in case you came here as me, to find a way to test code locally under linux/amd64 emulation on a mac , this method wont work for most of things at the time of writing

[Deepak Devanand]

I think the "FROM" directive in Dockerfile chooses the right CPU architecture while building. If we are to transfer that image to a different CPU type/architecture PC, and create container, it will fail since the container now has to run as a process in the host OS, which is incompatible. This is similar to running a binary file on an incompatible CPU architecture, such as ARM-compiled binary on Intel PC.

Like we recompile the binary again to match the target CPU architecture, we'll have to "build" the docker image again. The "buildx" command here is equivalent to cross-compilation, and I'm afraid may increase image size considerably if we're including more than one architecture in the command.

When we say Docker is portable, it's within the context of what's inside the container - base os, application runtime and application. The docker image itself is OS-specific and require kernel facilities such as cgroups and namespaces, in order to instantiate container (process).

All these constraints when we think about it, it's because "docker image is immutable" and can't have runtime hooks changing its behaviour.