Simplify Your Dockerfile

#docker #rust #taskfile #opensource

With rustlang gaining lots of popularity, I thought to give it a try. As cloud native application developer, the first thing I thought was to build was simple REST API that greets the user by name.

After building the application locally, the next immediate was to containerise it. Though we have an official rustlang image, I want to build a customised image that will allow me to build cross platform container images namely linux/arm64 and linux/amd64. I don't want to dwell into those details as that demands its own blog post ;).

So my Dockerfile with all rust specific tools and dependencies looks like:

	#syntax=docker/dockerfile:1.3-labs

	FROM --platform=$TARGETPLATFORM rust:1.67-alpine3.17 AS bins

	ARG TARGETPLATFORM

	RUN --mount=type=cache,target=/usr/local/cargo/registry \
	apk add -U --no-cache alpine-sdk gcompat go-task \
	&& cargo install cargo-zigbuild

	## The core builder that can be used to build rust applications
	FROM --platform=$TARGETPLATFORM alpine:3.17 as final

	ARG TARGETPLATFORM
	ARG rust_version=1.67.1
	ARG rustup_version=1.25.2
	ARG user_id=1001
	ARG user=builder

	ENV USER_ID=$user_id \
	USER=$user \
	RUST_VERSION=$rust_version \
	RUSTUP_VERSION=$rustup_version \
	RUSTUP_HOME=/usr/local/rustup \
	CARGO_HOME=/usr/local/cargo \
	PATH=/usr/local/cargo/bin:$PATH \
	RUST_VERSION=1.67.1

	COPY --from=bins /usr/bin/go-task /usr/local/bin/task

	COPY --from=bins /usr/local/cargo/bin/cargo-zigbuild /usr/local/cargo/bin/

	RUN apk add -U --no-cache ca-certificates sudo shadow bash curl curl-dev file musl musl-dev gcc \
	&& adduser --uid $USER_ID --ingroup root --system --shell /bin/bash --disabled-password $USER;
	&& echo "$USER ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/$USER;
	&& chmod 0440 /etc/sudoers.d/$USER \
	&& apkArch=$(apk --print-arch) \
	&& curl -L https://ziglang.org/download/$ZIG_VERSION/zig-linux-$apkArch-$ZIG_VERSION.tar.xz \| tar -J -x -C /usr/local \
	&& ln -s /usr/local/zig-linux-$apkArch-$ZIG_VERSION/zig /usr/local/bin/zig ;
	## these were copied from https://github.com/rust-lang/docker-rust/blob/master/1.67.1/alpine3.17/Dockerfile#L13-L25
	&& case "$apkArch" in \
	x86_64) rustArch='x86_64-unknown-linux-musl'; rustupSha256='241a99ff02accd2e8e0ef3a46aaa59f8d6934b1bb6e4fba158e1806ae028eb25' ;; \
	aarch64) rustArch='aarch64-unknown-linux-musl'; rustupSha256='6a2691ced61ef616ca196bab4b6ba7b0fc5a092923955106a0c8e0afa31dbce4' ;; \
	*) echo >&2 "unsupported architecture: $apkArch"; exit 1 ;; \
	esac; \
	url="https://static.rust-lang.org/rustup/archive/1.25.2/${rustArch}/rustup-init"; \
	wget "$url"; \
	echo "${rustupSha256} *rustup-init" \| sha256sum -c -; \
	chmod +x rustup-init; \
	./rustup-init -y --no-modify-path --profile minimal --default-toolchain $RUST_VERSION --default-host $rustArch --target x86_64-unknown-linux-gnu \
	x86_64-unknown-linux-musl \
	aarch64-unknown-linux-musl \
	aarch64-unknown-linux-gnu
	&& rm /usr/local/bin/rustup-init; \
	rm rustup-init; \
	chmod -R a+w $RUSTUP_HOME $CARGO_HOME;

view raw Dockefile.rustlang.zig hosted with ❤ by GitHub

The Dockerfile has two stages bins and final. bins is used to do all binary builds that are required by final stage. The final stage builds the final image that can be used with to build multi-arch container images out of our rust applications.

Though it is not a complicated Dockerfile, if you notice the last RUN instruction of the final stage, it is complex and hard to debug typically when one of the commands fails to run. It is also hard read and understand the RUN instruction. Doing multiple RUN instructions to split the commands, is not recommended as it creates new layers and your your final image will be bloated in size.

NOTE: rustlang builder images are usually bigger ~ 700mb(compressed) by virute of dependencies that it needs e.g gcc, cross compilation linkers etc.,

I was then thinking of ways to simplify this Dockerfile though not from size point of view but atleast making it simple to read and understand.

The target was to to have one RUN instruction but to split the commands into individual steps without compromising on the size.

I then stumbled upon Taskfile -- Task is a task runner / build tool -- which is similar to [GNU Make](https://www.gnu.org/software/make/) but way simpler. Taskfile helped me to make the Dockerfile simpler.

I kind of moved the whole set RUN instruction into a Taskfile:

	# https://taskfile.dev
	version: "3"

	tasks:
	default:
	desc: The default task that will be run when task is called without explicit task name
	aliases:
	- all
	cmds:
	- task: install_packages
	- task: add_and_config_user
	- task: install_rust
	- task: fix_permissions

	install_packages:
	desc: Installs required os packages
	silent: true
	cmds:
	- apk add -U --no-cache ca-certificates sudo shadow bash curl curl-dev file musl musl-dev gcc
	- task:install_zig

	install_zig:
	desc: install ziglang
	silent: true
	cmds:
	- "curl -L https://ziglang.org/download/${ZIG_VERSION}/zig-linux-{{.APK_ARCH}}-$ZIG_VERSION.tar.xz \| tar -J -x -C /usr/local"
	- "ln -s /usr/local/zig-linux-{{.APK_ARCH}}-$ZIG_VERSION/zig /usr/local/bin/zig"
	vars:
	# determine the current alpine architecture
	APK_ARCH:
	sh: apk --print-arch

	add_and_config_user:
	silent: true
	desc: add the builder user
	cmds:
	- adduser --uid $USER_ID --ingroup root --system --shell /bin/bash --disabled-password $USER
	- echo "$USER ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/$USER
	- chmod 0440 /etc/sudoers.d/$USER

	install_rust:
	desc: install rust and add targets
	silent: true
	preconditions:
	- sh: "[ '{{.APK_ARCH}}' = 'x86_64' ] \|\| [ '{{.APK_ARCH}}' = 'aarch64' ] "
	msg: "unsupported architecture {{.APK_ARCH}}"
	cmds:
	- task: download_rust_init
	vars:
	RUST_ARCH: "{{.RUST_ARCH}}"
	RUSTUP_SHA56: "{{.RUSTUP_SHA56}}"
	- task: rust_init
	vars:
	RUST_ARCH: "{{.RUST_ARCH}}"
	RUSTUP_SHA56: "{{.RUSTUP_SHA56}}"
	vars:
	# determine the current alpine architecture
	APK_ARCH:
	sh: apk --print-arch
	# the rust architecture to use
	RUST_ARCH:
	sh: \|
	if [ "{{.APK_ARCH}}" = 'x86_64' ];
	then
	echo 'x86_64-unknown-linux-musl'
	elif [ "{{.APK_ARCH}}" = 'aarch64' ];
	then
	echo 'aarch64-unknown-linux-musl'
	fi
	# the SHA256 Checksum for rustup
	RUSTUP_SHA56:
	sh: \|
	if [ "{{.APK_ARCH}}" = 'x86_64' ];
	then
	echo '241a99ff02accd2e8e0ef3a46aaa59f8d6934b1bb6e4fba158e1806ae028eb25'
	elif [ "{{.APK_ARCH}}" = 'aarch64' ];
	then
	echo '6a2691ced61ef616ca196bab4b6ba7b0fc5a092923955106a0c8e0afa31dbce4'
	fi

	fix_permissions:
	desc: set permissions on cargo home and rustup home to be accessible writable by all users
	silent: false
	cmds:
	- chmod -R a+w $RUSTUP_HOME $CARGO_HOME

	download_rust_init:
	silent: false
	cmds:
	- "wget https://static.rust-lang.org/rustup/archive/$RUSTUP_VERSION/{{.RUST_ARCH}}/rustup-init"
	- echo "{{.RUSTUP_SHA56}} *rustup-init" \| sha256sum -c -
	- chmod +x rustup-init
	- mv rustup-init /usr/local/bin

	rust_init:
	internal: true
	silent: false
	label: "rust_init_{{.RUST_ARCH}}"
	cmds:
	- >-
	/usr/local/bin/rustup-init -y --no-modify-path --default-toolchain $RUST_VERSION --default-host "{{.RUST_ARCH}}"
	--target x86_64-unknown-linux-gnu \
	x86_64-unknown-linux-musl \
	aarch64-unknown-linux-musl \
	aarch64-unknown-linux-gnu
	- rm /usr/local/bin/rustup-init

view raw Taskfile.yaml hosted with ❤ by GitHub

Though it is verbose, but helps in understanding commands we are running as part of the Docker build. With descriptions, comments, conditions it becomes more powerful and self documented in explaining what is being executed and when it will be executed.

Updating the Dockerfile file results in:

#syntax=docker/dockerfile:1.3-labs

FROM --platform=$TARGETPLATFORM rust:1.67-alpine3.17 AS bins

ARG TARGETPLATFORM

RUN --mount=type=cache,target=/usr/local/cargo/registry \
  apk add -U --no-cache alpine-sdk gcompat go-task \
  && cargo install cargo-zigbuild

## The core builder that can be used to build rust applications
FROM --platform=$TARGETPLATFORM alpine:3.17 as final

ARG TARGETPLATFORM
ARG rust_version=1.67.1
ARG rustup_version=1.25.2
ARG user_id=1001
ARG user=builder

ENV USER_ID=$user_id \
  USER=$user \
  RUST_VERSION=$rust_version \
  RUSTUP_VERSION=$rustup_version \
  RUSTUP_HOME=/usr/local/rustup \
  CARGO_HOME=/usr/local/cargo \
  PATH=/usr/local/cargo/bin:$PATH \
  RUST_VERSION=1.67.1

COPY --from=bins /usr/bin/go-task /usr/local/bin/task

COPY --from=bins /usr/local/cargo/bin/cargo-zigbuild /usr/local/cargo/bin/

COPY tasks/Taskfile.root.yaml ./Taskfile.yaml

RUN task

As we moved all our commands to Taskfile the RUN instruction now has to just run the task command, which will then run the default from the Taskfile.

To summarise we,

Wrote a multi stage Dockerfile to build multi arch rust app container
Moved all the instructions from RUN to Taskfile
Used the task command in RUN
Moving commands to Taskfile allows us to run/test the tasks separately before using them in Dockerfile. For more usage check the TaskFile documentation.