DEV Community: Kevin Gimbel

How to distribute a Rust CLI tool as Docker image

Kevin Gimbel — Tue, 08 Dec 2020 12:44:15 +0000

I recently found a nice and clean way of building and distributing Rust CLI apps using docker. For my work I created a Rust app that wraps some AWS SDK functions to make my day-to-day work with AWS easier. This CLI is very focused on the way we work at Synoa and therefore unfortunately not open source. The tech does not matter much, as we can just create a tiny example "app" for this blog post.

Rust code

Below is the example Rust code we will use.

use std::env;

fn main() {
    let args: Vec<String> = env::args().collect();
    println!("{:?}", args);
}

This code only prints whatever arguments were passed to the script.

Building the binary - in docker

Next we will build the binary in docker using a "multi-stage build" setup. This way we do not need to manage our local Rust environment, for example we don't need to keep our Rust targets up-to-date or make sure other contributors have the same environment setup - the compiling is all done inside Docker.

We start by declaring a builder container. This container is only used for compiling the binary.

FROM clux/muslrust:1.45.0-stable as builder
WORKDIR /volume
COPY . .
RUN cargo build --release

These four lines do the following:

Create a container based on clux/muslrust
Give it a name of builder (so we can reference it later)
Declare the working directory to be /volume
Copy over all files from the current directory to /volume
Run the cargo build --release command which builds our Rust binary

Creating the docker image

Next, in the same Dockerfile, we declare the actual image. This is where we copy the compiled binary from the "builder" container.

FROM alpine
# Copy the compiled binary from the builder container
COPY --from=builder /volume/target/x86_64-unknown-linux-musl/release/docker-cli-sample .
# Pass all arguments etc to binary
ENTRYPOINT [ "/docker-cli-sample" ]

So what happens here?

First with FROM alpine we use the slim Alpine Linux as base image. Depending on what our binary is doing we could also use FROM scratch to not use a base image at all. For my case I chose Alpine because we needed to make HTTPS calls and they didn't work in a "scratch" image.
COPY copies the binary from the builder container and places it in the root directory of our container
ENTRYPOINT [ "/docker-cli-sample" ] means we execute the docker-cli-sample binary when we run the container

All together, the Dockerfile looks like this.

FROM clux/muslrust:1.45.0-stable as builder
WORKDIR /volume
COPY . .
RUN cargo build --release

FROM alpine
COPY --from=builder /volume/target/x86_64-unknown-linux-musl/release/docker-cli-sample .
ENTRYPOINT [ "/docker-cli-sample" ]

Building the image and running the container

We can build the image now with the docker build command. Open a terminal and type:

docker build -t kevingimbel/rust-docker-cli-sample:1.0 .

Then run the image in a container. --rm makes sure the container is removed after it is executed, as we do not need it anymore.

$ docker run --rm kevingimbel/rust-docker-cli-sample:1.0 -hello -world
["/docker-cli-sample", "-hello", "-world"]

Setting up the CLI and docker

To execute this container like a CLI script we add the following to ~/.bashrc (for Bash) or ~/.zshrc (for zsh).

alias docker-rust-cli='docker run --rm kevingimbel/rust-docker-cli-sample:1.0'

Source the file by running the following, then test the command.

# bash
source ~/.bashrc
# zsh
source ~/.zshrc

Now we can execute the command just like any other CLI. The arguments are all passed to the script, just as if it was a "normal" binary somewhere in our $PATH.

$ docker-rust-cli hello from docker
["/docker-cli-sample", "hello", "from", "docker"]

Advanced: volumes

We could end this post here, but there's one "advanced" topic I want to highlight: Volumes. If our script would create or download files we could not access them because they are only inside the container and the container. To prevent this we need to add a volume.

A volume can be added with -v in the docker command.

alias docker-rust-cli='docker run --rm -v $(pwd):/cmd-root-dir kevingimbel/rust-docker-cli-sample:1.0'

`WORKDIR`

Now we need to make sure that our cli app puts created files in the /cmd-root-dir directory. This can be done by specifying the WORKDIR in the Dockerfile. To do this we add a new line above ENTRYPOINT as shown below.

FROM clux/muslrust:1.45.0-stable as builder
WORKDIR /volume
COPY . .
RUN cargo build --release

FROM alpine
COPY --from=builder /volume/target/x86_64-unknown-linux-musl/release/docker-cli-sample .
WORKDIR /cmd-root-dir
ENTRYPOINT [ "/docker-cli-sample" ]

WORKDIR will create the directory if it doesn't exist. To test the changes, we can adjust the rust script to write to a file.

use std::env;
use std::fs;

fn main() -> std::io::Result<()> {
    let args: Vec<String> = env::args().collect();
    println!("{:?}", args);
    fs::write("docker-cli-sample.log", format!("Args: {:?}", args))?;
    Ok(())
}

This will write the arguments into the docker-cli-sample.log. Because we set WORKDIR this will execute in the /cmd-root-dir inside the container. To actually get the log, we can now mount the volume with -v in our alias.

alias docker-rust-cli='docker run --rm -v $(pwd):/cmd-root-dir kevingimbel/rust-docker-cli-sample:1.0'

$(pwd) always evaluates to the current directory. This only works if we use sigle-quotes (') in the alias!

So finally, running the command now will yield us the log in the current directory.

$ docker-rust-cli
["/docker-cli-sample", "hello", "world"]
$ cat docker-cli-sample.log
Args: ["/docker-cli-sample", "hello", "world"]

Advanced: managing versions

For a bit more comfort we can use a variable for the docker image tag so we can update easier. The .bashrc or .zshrc then looks like:

export MY_CLI_VERSION="1.0"
alias docker-rust-cli='docker run --rm -v $(pwd):/cmd-root-dir kevingimbel/rust-docker-cli-sample:$MY_CLI_VERSION'

And there's that! Everybody else with access to the docker image can now use our CLI by adding the alias and optionally version to their .bashrc or .zshrc.

Recap

So what did we learn?

We can use multi-stage builds to build our code using docker. This is done by creating a container with FROM image:tag as builder
Rust binaries can be run in a small image such as alpine or even in a blank image using FROM scratch
We can use an alias to comfortably run the long docker command
By using WORKDIR and volumes we can extract files from the container and save them in the current directory outside the container

The source code of the example Rust CLI can be found on GitHub at kevingimbel/docker-cli-sample. A working docker image can be found on Docker Hub at kevingimbel/rust-docker-cli-sample.

The docker sample can be run with

docker run --rm kevingimbel/rust-docker-cli-sample:1.0 hello from docker

This post was cross-posted from kevingimbel.de

Feel free to follow me on Twitter @KevinGimbel for cloud stuff and DevOps/SysOps content. I swear I won't rant every day!

Apply a patch from a GitHub PR

Kevin Gimbel — Mon, 07 Dec 2020 16:13:57 +0000

Ever found yourself needing that one fix from a PR on GitHub that hasn't been released yet? How can you get the code, for example a Magento fix, into your code base? Copy-paste it? That might work for small changes, but for large and complex changes copy-pasting is no solution. Luckily, we can use GitHub to generate a patch file, which can then be applied with either the git or patch command.

patch-file what?

A patch file is a text file containing instructions on how to apply a change to files. If you run git diff and see the changed files with +/- in front of lines that were added (+) or removed (-) you are looking at a patch file.

Example code

For illustration purpose I created a example repo: https://github.com/KevinGimbel/blog-patch-example.

You can clone the main branch and then apply the patch from the Pull Request to it, if you want to follow along.

Getting the patch file

First we will need to get the patch file. GitHub makes this easy, but for some reason hides it from us. You can append .patch to the URL of any pull request to get the patch file. So for the example above, open the URL https://github.com/KevinGimbel/blog-patch-example/pull/1.patch to see the plain text patch file.

From 6da12536ff4da0efdabdf1a4dd55ded127fa247c Mon Sep 17 00:00:00 2001
From: Kevin Gimbel <hallo@kevingimbel.com>
Date: Fri, 4 Dec 2020 16:31:30 +0100
Subject: [PATCH] docs: add actual URL

---
 README.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/README.md b/README.md
index 0fe5e72..7f6398a 100644
--- a/README.md
+++ b/README.md
@@ -2,4 +2,4 @@

 This repo is part of a Blog Post on how to get and apply patch files from GitHub.com

-You can read the full post here: ENTER_URL_HERE
\ No newline at end of file
+You can read the full post here: [https://kevingimbel.de/blog/2020/12/apply-patch-from-github-pr/](https://kevingimbel.de/blog/2020/12/apply-patch-from-github-pr/)
\ No newline at end of file

The patch file is both human and machine readable. Now to continue, get the patchfile!

Open a terminal and navigate to the repo you cloned earlier (github.com/KevinGimbel/blog-patch-example)
Download the patch file with wget:

$ wget "https://github.com/KevinGimbel/blog-patch-example/pull/1.patch"

This downloads a file named 1.patch into the current directory, which we can verify by running ls -l

ls -l
total 16
-rw-r--r--@ 1 kevingimbel  staff  733 Dec  4 16:33 1.patch  # 👈 there it is
-rw-r--r--  1 kevingimbel  staff  155 Dec  4 16:33 README.md

You can look at the file if you want to, it contains the same text as the example in this article.

Applying a patch with `git`

We can use git to apply the patch. For manual patching git has the apply command:
If we want to test the changes to see if they can be applied but don't want to change any files yet, we can use the --check and -v (verbose) flag:

$ git apply -v --check 1.patch
Checking patch README.md...

To really apply the patch, we remove the --check flag:

$ git apply -v 1.patch
Checking patch README.md...
Applied patch README.md cleanly.

Now check the README.md file and you'll see the new content!

$ cat README.md 
# blog-patch-example

This repo is part of a Blog Post on how to get and apply patch files from GitHub.com

You can read the full post here: [https://kevingimbel.de/blog/2020/12/apply-patch-from-github-pr/](https://kevingimbel.de/blog/2020/12/apply-patch-from-github-pr/)

Patching without git

Sometimes you may not have git available, especially when patching software running on some server that - for whatever reasons - has no deployment process (no judging here!) or version control.

Even without the git command the patch can still be applied by using the patch tool installed on most (all?) Linux systems.
To do this, follow these steps:

Get the patch from the Pull Request by appending .patch to the URL
Apply the patch with the patch command

$ patch -p 1 < filename.patch
# To try out changes first, use `--dry-run`
$ patch -p 1 --dry-run < filename.patch

And that's it for today! Feel free to create PRs in the repo https://github.com/KevinGimbel/blog-patch-example/ if you want to play around with patching.

Additional resources

This post was cross-posted from kevingimbel.de