Rust CI

Since Rust is a compiled language, it uses many techniques to help lower compile time. Most of these include storing files in the target/ folder of the project.

Looking at Veloren as a Case Study

Veloren is an open source game being developed with Rust. One of the key reasons we use Gitlab for Veloren is the integrated CI.

We have several Gitlab runners that are each used for all of the CI tasks that Veloren requires. Each runner can compile, test, lint, verify, and build executables.

The Docker image used on the runners contains:

• All of the dependencies for builds
• A specific version of Rust that is forced in dev as well
• Targets for cross-platform builds
• Rustup components for tools like Rustfmt and Clippy
• The cloned project for easier checkout during builds
• A target folder populated from cargo run

This means that the Docker image is quite large, around 6gb. This is unfortunate, however it greatly helps CI times. During CI, some of the benifits from this are:

• The entire ~150mb project doesn't need to be re-cloned each time
• The specific Rust version doesn't need to be installed
• Many of the dependancy crates won't need to be rebuilt, let alone redownloaded
• Many of the source files for Veloren are already compiled. Therefore, since many merges to master don't change all the files, they only have to compile the files they changed.

The runners have a cache that stores the target/ folder after each build. It uses a Docker anonymous volume for this. Here is what the target folder looks like after a week of a runner's work:

11G     target/debug
1.3G    target/release
2.5G    target/x86_64-pc-windows-gnu

The size difference make sense, since the initial compile and all tests are run with the linux target. This stores artifacts in target/debug. The target/release and target/x86_64-pc-windows-gnu only hold the final executables for their respective platforms.

However, the size of the folders are quite a problem. Since this runner has only been running for a week, a 15gb target folder is quite large. A runner that is new and contains only a single build of each type takes up the following space:

6.1G    target/debug
1.5G    target/release
2.4G    target/x86_64-pc-windows-gnu

Note, the reason that target/release is bigger is most likely that the runner didn't get asked to run that specific stage.

So, in the week that the runner was doing jobs, it accumilated around 5gb in just the target/debug folder. In fact, some of our runners have accumilated 100gb+ of target artifacts. This has caused us to sometimes run out of space on the runners, which we've then had to manually restart. Not ideal.

Let's examine some other difference between the target folders of these runners. Let's call the runner that has run for a while old runner, and the new one new runner.

First, here is some great information about what the target folder contains. Let's take a look at what the target folder of new runner contains:

drwxr-xr-x   8 root root      4096 Jul 21 20:29 .
drwxr-xr-x   5 root root      4096 Jul 21 19:43 ..
drwxr-xr-x 153 root root     12288 Jul 21 20:29 build
-rw-r--r--   1 root root         0 Jul  9 12:05 .cargo-lock
drwxr-xr-x   2 root root    131072 Jul 21 20:33 deps
drwxr-xr-x   2 root root      4096 Jul 21 20:29 examples
drwxr-xr-x 836 root root     45056 Jul 21 20:30 .fingerprint
drwxr-xr-x  41 root root      4096 Jul 21 20:33 incremental
drwxr-xr-x   2 root root      4096 Jul  9 12:05 native
-rwxr-xr-x   2 root root   4487824 Jul 21 20:29 veloren_chat_cli-94fcc09b2c5cad1e
-rw-r--r--   1 root root      2942 Jul 21 20:29 veloren_chat_cli-94fcc09b2c5cad1e.d
-rwxr-xr-x   2 root root   4487808 Jul 21 20:29 veloren_client-d0c1376a22680bbe
-rw-r--r--   1 root root      2896 Jul 21 20:29 veloren_client-d0c1376a22680bbe.d
-rwxr-xr-x   2 root root  11732848 Jul 21 20:29 veloren_common-0b3ae1d1ad5c1cb1
-rw-r--r--   1 root root      2812 Jul 21 20:29 veloren_common-0b3ae1d1ad5c1cb1.d
-rwxr-xr-x   2 root root  82117288 Jul 21 20:20 veloren-server-cli
-rw-r--r--   1 root root      3758 Jul 21 20:19 veloren-server-cli.d
-rwxr-xr-x   2 root root   4487840 Jul 21 20:29 veloren_server_cli-d029ad657c74936b
-rw-r--r--   1 root root      3775 Jul 21 20:29 veloren_server_cli-d029ad657c74936b.d
-rwxr-xr-x   2 root root   5311464 Jul 21 20:29 veloren_server-fc3a2f6366701d99
-rw-r--r--   1 root root      3725 Jul 21 20:29 veloren_server-fc3a2f6366701d99.d
-rwxr-xr-x   2 root root 227318048 Jul 21 19:34 veloren-voxygen
-rw-r--r--   1 root root      9061 Jul 21 19:29 veloren-voxygen.d
-rwxr-xr-x   2 root root  23485928 Jul 21 20:29 veloren_voxygen-fb154c7c142b5881
-rw-r--r--   1 root root      9078 Jul 21 20:29 veloren_voxygen-fb154c7c142b5881.d
-rwxr-xr-x   2 root root   4563864 Jul 21 20:29 veloren_world-1f1dbc7fde8cc98d
-rw-r--r--   1 root root      3466 Jul 21 20:29 veloren_world-1f1dbc7fde8cc98d.d