I'm making a static website in Rust. Last time I did this, I used Docker to automate the deployment. I was frustrated at how much bandwidth I was using shuffling around these massive build images, but the convenience was too hard to pass up and I wasn't rebuilding the image often, so just left it.
With this new method, my final production Docker image for the whole application is 6.85MB. I can live with that.
I'm using Askama for templating, which actually compiles your typechecked templates into your binary. The image assets I have are all SVG, which is really XML, so I can use
include_str!() for those along with things like
robots.txt and all CSS, which includes their entire file contents directly in my compiled binary as a
&'static str. As a result, I don't really need a full Rust build environment or even any asset files present to run the compiled output.
This time around, I did my homework and found this blog post by @alexbrand, which demonstrates this technique. Instead of just bundling up with all the build dependencies in place, you can use a multi-stage build to generate the compiled output first and then copy it into a minimal container for distribution. Here's my adaptation for this project:
# Build Stage FROM rust:1.40.0 AS builder WORKDIR /usr/src/ RUN rustup target add x86_64-unknown-linux-musl RUN USER=root cargo new deciduously-com WORKDIR /usr/src/deciduously-com COPY Cargo.toml Cargo.lock ./ RUN cargo build --release COPY src ./src COPY templates ./templates RUN cargo install --target x86_64-unknown-linux-musl --path . # Bundle Stage FROM scratch COPY --from=builder /usr/local/cargo/bin/deciduously-com . USER 1000 CMD ["./deciduously-com", "-a", "0.0.0.0", "-p", "8080"]
That's it! The top section labelled
builder uses the
rust:1.40.0 base image, which has everything needed to build my binary with rust. It targets
x86_64-unknown-linux-musl. The musl library is an alternative
libc designed for static linking as opposed to dynamic. Rust has top-notch support for this (apparently). This means the resulting binary is entirely self-contained - it has no environment requirements at all.
The second section, which defines the actual distribution, just starts from
scratch, not even
alpine or whatever other minimal Docker base image I'd otherwise use. You can use
COPY --from=builder to reference the previous Docker stage. This docker image has nothing at all in it. This means my image really just contains my binary, no Linux userland to be found! All with one invocation of
The middle part, with
cargo new, makes a dummy application leveraging the docker cache for dependencies. This means that while you're developing, subsequent runs of
docker build won't need to rebuild every dependency in your Rust application every time, it will only rebuild what's changed just like building locally. Marvelous!
I'm deploying on the DigitalOcean One-Click Docker app, which is an Ubuntu LTS image with docker pre-installed and some UFW settings preset. This was my whole deploy process:
$ docker build -t deciduously-com . $ docker tag SOMETAG83979287 deciduously0/deciduously-com:latest $ docker push deciduously0/deciduously-com:latest $ ssh root@SOME.IP.ADDR root@SOME.IP.ADDR# docker pull deciduously0/deciduously-com:latest root@SOME.IP.ADDR# docker run -dit -p 80:8080 deciduously0/deciduously-com:latest root@SOME.IP.ADDR# exit $
The remote server pulls down my whopping 6.85MB image and spins it up. I was immediately able to connect. This minuscule image just sips at disk space, memory, and CPU, so I'm going to be able to stretch my $5/month lowest-possible-tier DigitalOcean droplet as far as it can possibly go. The flashbacks I'm having from trying to do something similar with Clojure are terrifying...
Add in some scripts so you don't have to remember those commands, and my whole build and deploy process is distilled to a few keystrokes.
Why would I use anything else?
For those keeping score, yes, I've already scrapped Stencil in favor of Askama/Hyper. Within a day I had re-implemented all previous work in about a half of the code and a small fraction of the bundle size. Yes, there's a bigger post (and GitHub template) about it brewing, and no, I'm not even sorry. KISS and all...
Photo by Richard Sagredo on Unsplash
Top comments (22)
Have been looking for a working example of Rust multistage builds.
My rocket web app had an image size of 2.45GB which is now reduced to 9.19MB.
Update for Rust 2021,
In the build stage, replace this line:
RUN cargo build --release
RUN cargo install --target x86_64-unknown-linux-musl --path .
I've used staged builds before, but never used scratch for the last stage, only alpine. I wonder if there are tradeoffs to not using a Linux environment?
I'm curious to see if it holds me up when I inevitably do have other assets to include. When the only thing in the container is a single binary it's straightforward enough, at least.
All other concerns should be handled by the host server, though, I think - right?
Appreciate this article so much. So many concepts being added to my knowledge-base for future use.
One thing I find is that I'm managing container-container communications a lot. Not a problem normally. How is that with
scratch? Does Docker handle it all, or does the OS provide some abstraction that would need to be substituted within the binary?
EDIT: To be clear, I'm thinking of putting a binary at the end of an API call. If I could do that with a minimal image I would be so much happier. Just want to get the plumbing right 😉
I haven't tried yet, but my instinct tells me this is something Docker manages, not your containers. Like Moshe said, the Linux userland inside a container is explicitly for runtime needs of the container's internal commands, everything outside of that is handled by Docker itself.
scratchfeels like taking the training wheels off, but you have to do it at some point!
scratchisn't "nothing", it just downloads nothing.
There's still stuff from the runtime when it's alive.
Yeah, I never thought about it much, but the OS in the docker image is really only needed for runtime environment purposes. If all you have is a binary, you shouldn't need an environment aside from the host.
The scratch image description actually says the following:
Very timely for me. I've started using docker to wrangle some of the more complicated toolchains like cross-compiling (not unlike Rust+musl, really). And specifically multi-stage docker where I was using multiple Dockerfiles and had scripts "gluing" the results together.
Definitely interested in this. I've always used Gentoo/QEMU or Nix tooling to handle this sort of thing and never got to a point where it didn't feel clunky and brittle (gluing together stuff I don't understand). Docker sounds nice and clean.
I wrote about using docker and Qemu a while back which is pretty slick. But I’ve run into a few issues; dotnet core doesn’t work in Qemu, and some more ”obscure” platforms like ESP32 basically require cross-compiling since they lack native toolchain support.
Think I remember seeing another approach to dealing with dependencies and the docker cache, will have to try and find it. I know the bare minimum docker but really need to up my game.
scratchbuilds for some of the Go stuff I do. It definitely makes you feel a bit smug when your image is only a few MB.
Anytime you casually shave off an order of magnitude or two is cause for celebration, especially when it's a relatively trivial change.
Debugging can be tricky, but not impossible, especially with Kubernetes deployment, which tends to require privileged rights.
Good to keep in mind, thanks! Multi-stage builds in general or specifically
FROM scratchspecifically. See ahmet.im/blog/debugging-scratch/ for more details.
Wow I also learned about
scratchhere that's really cool! I'll have to try that on an image or two I have!
Why have an executable for a static site?
Is it not static, just server-rendered with dynamic data?
Mostly for educational purposes, but yes, it's more accurately server-rendered with potentially dynamic data. Right now nothing is dynamic, but I'm looking at this project as a sort of "playground", and this doesn't close any doors should I want to do something fancy in the future.
Ah, makes sense.
I just don't see a lot of advantages to server "rendering" vs an API.
The only thing that comes to mind is reduced complexity for very small projects.
But education is education!
What the purpose of the container if all you need is just a single binary?