Building with Docker

Intro

Client engineers, embedded systems engineers, and other engineers who never worked with the backend can still benefit significantly from using containers. In this piece, I would like to share how I structure nearly all of my side projects so that they can be built on any machine that has make and docker installed.

Motivation

Environment guarantee

Multiple contributors and the continuous integration server (CI) can be confident they are building in exactly the same environment.

Environment versioning

Every team member does not have to build the builder images locally. Docker Registry can be used to store and version the images.

Easy workstation/CI build agent setup

As mentioned above, the only tools required to build such a project are make and docker.

Make is not used here as the project build system but rather as a convenient tool to wrap lengthy docker commands. For the scope of the article, we will assume a Rust project located at $(ROOT_DIR)/main using cargo build system.

Linux target

Let us start simple. Docker is always Linux, so the easiest platform to target is Linux since this does not imply cross-compilation. This also works perfect for unit testing if you are writing a Linux or platform-independent application.

The following Makefile utilizes an official rust:1.61.0 image.

Makefile

ROOT_DIR := $(shell dirname $(realpath $(firstword $(MAKEFILE_LIST))))
SOURCE_DIR := $(ROOT_DIR)/main

IMAGE_BUILDER_LINUX := rust:1.61.0

build_linux: builder_linux
    @docker run \
    --rm \
    -v $(SOURCE_DIR):/src \
    $(IMAGE_BUILDER_LINUX) \
    sh -c 'cd /src && cargo build'

test_linux: builder_linux
    @docker run \
    --rm \
    -v $(SOURCE_DIR):/src \
    $(IMAGE_TESTER_LINUX) \
    sh -c 'cd /src && cargo test'

builder_linux:
    @docker pull $(IMAGE_BUILDER_LINUX)

Now a simple make build_linux and make test_linux should:

1. Make sure the builder Docker image is pulled
2. Run build/test using the image

Custom builders

It is most likely that, eventually, you will need to customize the builder image. That would imply creating a Dockerfile and building an image using it instead of pulling the image as we did above.

Assuming the Dockerfile is located at $(ROOT_DIR)/image/x:

Makefile

ROOT_DIR := $(shell dirname $(realpath $(firstword $(MAKEFILE_LIST))))
SOURCE_DIR := $(ROOT_DIR)/main
PROJECT_NAME := $(shell basename $(ROOT_DIR))

IMAGE_BUILDER_X := $(PROJECT_NAME)-builder-x

builder_x:
    @docker build -t $(IMAGE_BUILDER_X) $(ROOT_DIR)/image/x

WebAssembly

Let me use this opportunity to share some scripts to build Rust code into WebAssembly and run the unit tests.

Makefile

ROOT_DIR := $(shell dirname $(realpath $(firstword $(MAKEFILE_LIST))))
SOURCE_DIR := $(ROOT_DIR)/main
PROJECT_NAME := $(shell basename $(ROOT_DIR))

RUST_TARGET_WEB := wasm32-unknown-unknown

IMAGE_BUILDER_WEB := $(PROJECT_NAME)-builder-web

build_web: builder_web
    @docker run \
    --rm \
    -v $(SOURCE_DIR):/src \
    $(IMAGE_BUILDER_WEB) \
    sh -c 'cd /src && cargo build --target $(RUST_TARGET_WEB)'

test_web: builder_web
    @docker run \
    --rm \
    -v $(SOURCE_DIR):/src \
    $(IMAGE_BUILDER_WEB) \
    sh -c 'cd /src && CARGO_TARGET_WASM32_UNKNOWN_UNKNOWN_RUNNER=wasmtime_test_runner.sh cargo test --target $(RUST_TARGET_WEB)'

builder_web:
    @docker build -t $(IMAGE_BUILDER_WEB) $(ROOT_DIR)/image/web

image/web/Dockerfile

FROM rust:1.61.0

RUN \
rustup toolchain install '1.61.0-x86_64-unknown-linux-gnu' \
--target 'wasm32-unknown-unknown' \
--component 'rust-std'

RUN curl https://wasmtime.dev/install.sh -sSf | bash

COPY wasmtime_test_runner.sh /usr/bin

VOLUME ["/usr/local/cargo/registry"]

image/web/wasmtime_test_runner.sh

#!/bin/sh

/root/.wasmtime/bin/wasmtime "$@" --invoke main 0 0

It is relatively easy with Rust because rustup has a wasm32 toolchain ready to download, but with a more complicated cross-compiler setup, being able to make a Dockerfile describing the setup sequence is a lifesaver.

Thoughts

The more complicated the required build setup is, the more I recommend looking at the Docker-based building. Setting up a workstation to build a project requiring multiple cross compilers, especially when using certain toolchains that might require some user-fixing, can take up to a few hundred bash commands. That could theoretically be done without containers by a bash script, but that how numerous caveats, including:

• State zero would have to be identical before running the script
• Updating the build setup while keeping it identical between workstations is a challenging task
• No environment guarantee
• No switching between build setup versions