DEV Community

Cover image for Structure Testing for Docker Containers
Tomas Fernandez for Semaphore

Posted on • Originally published at semaphoreci.com

Structure Testing for Docker Containers

So you’ve set up continuous integration for your project. Everything looks good and now all you need is a container. Just build and run it, right? Not so fast!

Whether using containers to support development or for packaging an application, it’s easy to take them for granted. But many things can go wrong with them: moved files, incorrect permissions, a user is missing, the Dockerfile is incomplete, the list goes on…

Containers are as crucial as the code they support. In this tutorial, we’ll introduce a different way of testing them before deployment.

One of the many ways Google tests containers

Container Structure Tests (CST) is a container-testing tool developed by Google and open-sourced with the Apache 2.0 license. CST comes with a predefined set of tests for looking at what’s actually inside a container image.

For instance, CST can check if a file exists, run a command and validate its output, or check if the container exposes the correct ports. Almost every declarative keyword in the Dockerfile has a corresponding test.

One important note is that the project is not officially supported by Google, so it doesn’t show a lot of activity. But it’s popular enough to keep it active and it’s still accepting contributions.

More tests, less uncertainty

Any old container should do in order to try out CST. Though it will be easier if we build one from a known Dockerfile. So, if you want to follow along with me as I explore this tool, fork and clone our Ruby Kubernetes demo project:

GitHub logo semaphoreci-demos / semaphore-demo-ruby-kubernetes

A Semaphore demo CI/CD pipeline for Kubernetes.

Semaphore CI/CD demo for Kubernetes

Build Status

This is an example application and CI/CD pipeline showing how to build, test and deploy a microservice to Kubernetes using Semaphore 2.0.

Ingredients:

  • Ruby Sinatra as web framework
  • RSpec for tests
  • Packaged in a Docker container
  • Container pushed to Docker Hub registry
  • Deployed to Kubernetes

CI/CD on Semaphore

If you're new to Semaphore, feel free to fork this repository and use it to create a project.

The CI/CD pipeline is defined in .semaphore directory and looks like this:

CI/CD pipeline on Semaphore

Local application setup

To run the microservice:

bundle install --path vendor/bundle
bundle exec rackup

To run tests:

bundle exec rspec

To build and run Docker container:

docker build -t semaphore-demo-ruby-kubernetes
docker run -p 80:4567 semaphore-demo-ruby-kubernetes
curl localhost
> hello world :))

Additional documentation

License

Copyright (c) 2022 Rendered Text

Distributed under the MIT License…

It’s a “Hello, World” application written on Ruby. It comes with a Dockerfile and a complete CI/CD pipeline.

Image description

You’ll also need a:

  • A Docker installation.
  • A CST config file.
  • The CST executable.

Once you’ve cloned the demo, build the container image with:

$ docker build -t test-image .
Enter fullscreen mode Exit fullscreen mode

Install the CST tool using the installation instructions. For instance, on Linux:

$ curl -LO https://storage.googleapis.com/container-structure-test/latest/container-structure-test-linux-amd64 && chmod +x container-structure-test-linux-amd64 && sudo mv container-structure-test-linux-amd64 /usr/local/bin/container-structure-test
Enter fullscreen mode Exit fullscreen mode

If you’re running on a non-Intel architecture, you can build the project by yourself or try the CST image instead.

The command to run the test follows works like this:

$ container-structure-test test –config config-cst.yaml –image test-image:latest
Enter fullscreen mode Exit fullscreen mode

But of course, that won’t work until we configure the tests. We’ll do that next.

Setting up CST tests

CST supports three categories of tests:

  • Commands: starts the container, runs the command, and validates its result.
  • Filesystem: checks for file existence, owner, permissions, and contents.
  • Metadata: this category contains things such as environment variables, exposed ports, labels, among other image metadata.

Create a new file called config-cst.yaml (JSON also works) and add the following mandatory line:

schemaVersion: 2.0.0
Enter fullscreen mode Exit fullscreen mode

We’ll start with the command tests.

Command tests

Let’s try some command tests. We can use something like this to check that Ruby is installed. Of course, I cheated and looked where it’s actually located before trying.

commandTests:
  - name: "Ruby is installed"
    command: "which"
    args: ["ruby"]
    expectedOutput: ["/usr/local/bin/ruby"]
Enter fullscreen mode Exit fullscreen mode

Now we’ll check that ruby --version outputs the correct number:

  - name: "Ruby version is correct"
    command: "/usr/local/bin/ruby"
    args: ["--version"]
    expectedOutput: ["ruby 2.7.*"]
Enter fullscreen mode Exit fullscreen mode

Should we be in a really security-conscious mindset, we can checksum the Ruby binary for extra safety. We can run preparation commands before the test with setup.

  - name: "Ruby binary checksum"
    setup: [["apt-get", "update"], ["apt-get","install","-y","shatag"]]
    command: "sha512sum"
    args: ["/usr/local/bin/ruby"]
    expectedOutput: ["df2fb393261ab88e5f991b96d958363f5e5185b51f1af319375be0d8b9ed6c27097ac8bfab399798497909c3e9e2bcc6d715a1e514f13fe5b7365344af555c7e  /usr/local/bin/rub"]
Enter fullscreen mode Exit fullscreen mode

Now that we have some initial tests, we can actually run the tool for the first time:

$ container-structure-test test --config config.yaml --image test-image:latest

=== RUN: Command Test: Ruby is installed
--- PASS
duration: 391.76725ms
stdout: /usr/local/bin/ruby

=== RUN: Command Test: Ruby version is correct
--- PASS
duration: 319.6335ms
stdout: ruby 2.7.4p191 (2021-07-07 revision a21a3b7d23) [aarch64-linux]

=== RUN: Command Test: Ruby binary checksum
--- PASS
duration: 302.199834ms
stdout: df2fb393261ab88e5f991b96d958363f5e5185b51f1af319375be0d8b9ed6c27097ac8bfab399798497909c3e9e2bcc6d715a1e514f13fe5b7365344af555c7e  /usr/local/bin/ruby


===============================
=========== RESULTS ===========
===============================
Passes:      3
Failures:    0
Duration:    1.013600584s
Total tests: 3
Enter fullscreen mode Exit fullscreen mode

All command tests require a working Docker installation, as the tool must start a temporary container to run them. However, filesystem and metadata tests can be run without Docker appending the --driver tar option.

Filesystem tests

Filesystem tests inspect the contents of the image; they check if files exist, their permissions, their contents, owner, and group.

Here’s how we can test that the code was correctly copied in the image. If we look at our Dockerfile, it should live in the /app/ folder. So, we define a fileExistenceTests like this:

fileExistenceTests:
  - name: 'app.rb exists and has correct permissions'
    path: '/app/app.rb'
    shouldExist: true
    permissions: '-rw-rw-r--'
    uid: 0
    gid: 0
Enter fullscreen mode Exit fullscreen mode

We can also check the reverse: that a file is not present in the image. Setting shouldExist to false is a great way to avoid shipping sensitive files by mistake. For example, we don’t need the unit tests contained in spec for the final build.

  - name: 'spec/ directory should not exist'
    path: '/app/spec'
    shouldExist: false
Enter fullscreen mode Exit fullscreen mode

This time, CST should fail because the Docker image has the spec folder.

$ container-structure-test test --config config.yaml --image test-image:latest

=== RUN: Command Test: Ruby is installed
--- PASS
duration: 308.371875ms
stdout: /usr/local/bin/ruby

=== RUN: Command Test: Ruby version is correct
--- PASS
duration: 312.744792ms
stdout: ruby 2.7.4p191 (2021-07-07 revision a21a3b7d23) [aarch64-linux]

=== RUN: Command Test: Ruby binary checksum
--- PASS
duration: 286.408167ms
stdout: df2fb393261ab88e5f991b96d958363f5e5185b51f1af319375be0d8b9ed6c27097ac8bfab399798497909c3e9e2bcc6d715a1e514f13fe5b7365344af555c7e  /usr/local/bin/ruby

=== RUN: File Existence Test: app.rb exists and has correct permissions
--- PASS
duration: 0s
=== RUN: File Existence Test: spec/ directory should not exist
--- FAIL
duration: 0s
Error: File /app/spec should not exist but does

===============================
=========== RESULTS ===========
===============================
Passes:      4
Failures:    1
Duration:    907.524834ms
Total tests: 5
Enter fullscreen mode Exit fullscreen mode

We can fix the spec failure by adding the following line into the .dockerignore (you may also experience a permission error in /app/app.rb, which can be quickly fixed with chmod).

spec/
Enter fullscreen mode Exit fullscreen mode

After rebuilding the image, the test should pass.

We have tested that a file exists. But, what about its contents? For that, we should use fileContentTests. The following example shows how to test if the Ruby Gems have been installed from a safe repository.

fileContentTests:
  - name: 'Gemfile remote is rubygems.org'
    path: '/app/Gemfile.lock'
    expectedContents: ['remote: https://rubygems.org/']
Enter fullscreen mode Exit fullscreen mode

Both types of tests support regular expressions in the expected* fields for more flexibility.

Metadata tests

Unlike the others, you can only have one metadata test. But it may check several things simultaneously, as metadata tests include a whole range of standard Docker variables.

Let’s say we want to test environment variables. In that case, we use env.

metadataTest:
  env:
    - key: APP_HOME
      value: /app
    - key: RUBY_VERSION
      value: 2.7.4
Enter fullscreen mode Exit fullscreen mode

You can also check Docker declarations like WORKDIR, EXPOSE, VOLUME, or USER.

  exposedPorts: ["4567"]
  workdir: "/app"
  volumes: []
Enter fullscreen mode Exit fullscreen mode

And the always important CMD and ENTRYPOINT.

  cmd: ["bundle", "exec", "rackup", "--host", "0.0.0.0", "-p", "4567"]
  entrypoint: []
Enter fullscreen mode Exit fullscreen mode

Once satisfied with the tests, commit the config file into the repository. This is the final version I got after experimenting with some extra tests follows.

schemaVersion: 2.0.0
commandTests:
  - name: "Ruby is installed"
    command: "which"
    args: ["ruby"]
    expectedOutput: ["/usr/local/bin/ruby"]
  - name: "Ruby version is correct"
    command: "/usr/local/bin/ruby"
    args: ["--version"]
    expectedOutput: ["ruby 2.7.*"]
  - name: "Ruby binary checksum"
    setup: [["apt-get", "update"], ["apt-get","install","-y","shatag"]]
    command: "sha512sum"
    args: ["/usr/local/bin/ruby"]
    expectedOutput: ["df2fb393261ab88e5f991b96d958363f5e5185b51f1af319375be0d8b9ed6c27097ac8bfab399798497909c3e9e2bcc6d715a1e514f13fe5b7365344af555c7e  /usr/local/bin/rub"]
  - name: "Bundle is installed"
    command: "which"
    args: ["bundle"]
    expectedOutput: ["/usr/local/bin/bundle"]
  - name: "Bundler version is correct"
    command: "/usr/local/bin/bundle"
    args: ["--version"]
    expectedOutput: ["Bundler version 2.1.*"]
fileExistenceTests:
  - name: 'app.rb exists and has correct permissions'
    path: '/app/app.rb'
    shouldExist: true
    permissions: '-rw-rw-r--'
    uid: 0
    gid: 0
  - name: 'spec/ directory should not exist'
    path: '/app/spec'
    shouldExist: false
fileContentTests:
  - name: 'Gemfile remote is rubygems.org'
    path: '/app/Gemfile.lock'
    expectedContents: ['remote: https://rubygems.org/']
metadataTest:
  env:
    - key: APP_HOME
      value: /app
    - key: RUBY_VERSION
      value: 2.7.4
  exposedPorts: ["4567"]
  cmd: ["bundle", "exec", "rackup", "--host", "0.0.0.0", "-p", "4567"]
  workdir: "/app"
  entrypoint: []
  volumes: []
Enter fullscreen mode Exit fullscreen mode

Testing container structure in CI/CD

Of course, CST wouldn’t be a lot of help to us unless it’s part of a CI/CD pipeline. The logical place for structure tests is between container build and deployment.

Image description

Adding CST in your CI/CD pipeline is straightforward. To keep things simple, we’ll extend the one already included in the demo, but the steps should work with any pipeline.

First, ensure Semaphore has access to your repository. Follow the getting started guide to learn how to do this.

The demo pipeline builds and tests the Ruby app, then dockerizes it and finally deploys it to Kubernetes. We’ll add a structure test right between Docker build and Kubernetes deploy.

First, ensure you have stored your Docker Hub credentials as a Semaphore secret.

Image description

Next, open the workflow editor using the Edit Workflow button.

Image description

Expand the continuous delivery pipeline and add a block immediately after the Docker build step.

Image description

The CST block will have one job with five commands:

  1. Sign in with Docker Hub, where the container image is stored.
  2. Pull the image into the CI environment.
  3. Install the CST Linux binary.
  4. Clone the repository so we can access the CST config file.
  5. Run the tests. If the test fails, the process stops with an error.

The complete command sequence is:

echo "${DOCKER_PASSWORD}" | docker login -u "${DOCKER_USERNAME}" --password-stdin
docker pull "${DOCKER_USERNAME}"/semaphore-demo-ruby-kubernetes:$SEMAPHORE_WORKFLOW_ID
curl -LO https://storage.googleapis.com/container-structure-test/latest/container-structure-test-linux-amd64 && chmod +x container-structure-test-linux-amd64 && sudo mv container-structure-test-linux-amd64 /usr/local/bin/container-structure-test
checkout
container-structure-test test --config config-cst.yaml --image "${DOCKER_USERNAME}"/semaphore-demo-ruby-kubernetes:$SEMAPHORE_WORKFLOW_ID
Enter fullscreen mode Exit fullscreen mode

Image description

To finalize, enable the dockerhub secret on the block and give it a try by clicking on Run the workflow.

Image description

Once the CI pipeline is complete, an auto-promotion should kick off the continuous delivery pipeline.

Image description

The image is ready and tested for the next stage. Now you can deliver containers with more confidence.

Image description

Conclusion

The more you know about your container, the less surprises you’ll get. Container Structure Test may not be the most flexible tool, but it certainly is a quick and easy way of adding some confidence to the release process. So, it should be on the radar of anyone using containers for serious work.

Read next:

Top comments (0)