Creating an s2i builder for Go (and a runtime image)

This post is a continuation of:

Build an app image with s2i

Javier Romero ・ Jul 19 '20

#docker #devops #openshift #s2i

Objective

I will try to go through the process of creating an s2i builder for... go.

The goals of this builder are the following:

• The builder should compile go programs.
• The app image should not contain the source code.
• The app image should not contain the go compiler.
• The app image should not run as root.

---

---

App

First, we need an app to use as our test subject.

This is a simple http server app that uses mux for routing. This server could be written without the use of this library but I wanted to show how dependencies would be handled.

Let's look at the app contents...

$ tree test-app

test-app/
├── app.go    # app source
├── go.mod    # dependency declarations
└── go.sum    # dependency checksums

app.go:

package main

import (
    "fmt"
    "log"
    "net/http"
    "os"

    "github.com/gorilla/mux"
)

const port = "8080"

func main() {
    log.Println("Starting app on port:", port)
    r := mux.NewRouter()
    r.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        _, _ = fmt.Fprintln(w, "This is a test app!")
    })
    r.HandleFunc("/host", func(w http.ResponseWriter, r *http.Request) {
        var name, _ = os.Hostname()
        _, _ = fmt.Fprintf(w, "This request was processed by host: %s\n", name)
    })
    r.HandleFunc("/hello/{object}", func(w http.ResponseWriter, r *http.Request) {
        vars := mux.Vars(r)
        _, _ = fmt.Fprintf(w, "Hello %v!\n", vars["object"])
    })
    http.Handle("/", r)
    if err := http.ListenAndServe(":"+port, nil); err != nil {
        log.Fatal(err)
    }
    log.Println("Shutting down")
}

go.mod:

module github.com/jromero/learning-s2i/s2i-golang/test/test-app

go 1.14

require github.com/gorilla/mux v1.7.4

go.sum:

github.com/gorilla/mux v1.7.4 h1:VuZ8uybHlWmqV03+zRzdwKL4tUnIp1MAQtp1mIFE1bc=
github.com/gorilla/mux v1.7.4/go.mod h1:DVbg23sWSpFRCP0SfiEN6jmj59UnW/n46BH5rLB71So=

Builder

The builder as we learned previously is an image that knows how to build a specific type of application. In this case, we will write a builder for a go app.

In order to do this we'll need the following files:

$ tree builder/

builder/
├── Dockerfile               # docker instructions to create builder
└── s2i
    └── bin
        ├── assemble         # script to build application
        ├── run              # script to run application
        ├── save-artifacts   # script to package cached items
        └── usage            # script that displays usage

Let's go into more detail about each one...

s2i/bin/usage

The usage script is very straight forward. We will use it as a way to assist users when they try to run the builder using docker run.

Contents

#!/bin/bash -e
cat <<EOF
This is the s2i-golang S2I image:
To use it, install S2I: https://github.com/openshift/source-to-image

Sample invocation:

s2i build <source code path/URL> s2i-golang <application image>

You can then run the resulting image via:
docker run <application image>
EOF

s2i/bin/save-artifacts

The save-artifacts script is a special script that is called as part of the build process directly into a tar stream. We will use it to specify what data we want cached so that users can leverage --incremental builds.

In go, we care about caching the following directories:

• ${GOPATH}/src
• ${GOPATH}/pkg

Important Note: Due to the way this script is used, no output other than the cache contents should be present.

Contents

#!/bin/sh -e
pushd ${GOPATH} >/dev/null
if [ -d src ]; then
    chmod -R +w src
    tar cf - src
fi
if [ -d pkg ]; then
    chmod -R +w pkg
    tar cf - pkg
fi
popd >/dev/null

s2i/bin/assemble

The assemble script is responsible for the following:

1. restoring cache
2. compiling code
3. relocating contents

Contents

# If the 's2i-golang' assemble script is executed with the '-h' flag, print the usage.
if [[ "$1" == "-h" ]]; then
    exec /usr/libexec/s2i/usage
fi

# Restore artifacts from the previous build (if they exist).
echo
echo "---> Checking for cache..."
if [ "$(ls /tmp/artifacts 2>/dev/null)" ]; then
  pushd /tmp/artifacts >/dev/null
  echo "-----> Pulling cache..."
  shopt -s dotglob
  if [ -d src ]; then
    echo "Restoring cache ${GOPATH}/src/..."
    mv src ${GOPATH}/src
  fi
  if [ -d pkg ]; then
    echo "Restoring cache ${GOPATH}/pkg/..."
    mv pkg ${GOPATH}/pkg 
  fi
  shopt -u dotglob
  popd >/dev/null
fi

# Compile app to final location
echo
echo "---> Building application from source..."
pushd /tmp/src/ >/dev/null
go build -o ${APP_ROOT}/bin/app
popd >/dev/null

s2i/bin/run

The run script will be what is executed on the produced app image. In our case it will simply call the compiled executable.

Contents

#!/bin/sh -e
${APP_ROOT}/bin/app

Dockerfile

Finally, we'll put it all together by composing our builder image using a standard Dockerfile.

Contents

# s2i-golang
FROM openshift/base-centos7

      # the maintainer
LABEL maintainer="Javier Romero <[e-mail]>" \
      # specify where s2i scripts are located
      io.openshift.s2i.scripts-url="image:///usr/libexec/s2i/bin"

# configuration
ARG GO_VERSION=1.14
ARG GO_INSTALL_DIR=/usr/local/

# env vars
ENV APP_ROOT=/opt/app-root
ENV BUILDER_VERSION 1.0
ENV GOPATH ${APP_ROOT}/src/go
ENV PATH=${PATH}:${GOPATH}/bin:${GO_INSTALL_DIR}/go/bin

# build dependencies
RUN curl -sSL https://dl.google.com/go/go${GO_VERSION}.linux-amd64.tar.gz -o go${GO_VERSION}.linux-amd64.tar.gz && \
    tar -C ${GO_INSTALL_DIR} -xzf go${GO_VERSION}.linux-amd64.tar.gz && \
    rm -f go${GO_VERSION}.linux-amd64.tar.gz && \
    mkdir -p ${GOPATH}

# s2i scripts
COPY s2i /usr/libexec/s2i

# set permissions
RUN chown -R 1001:1001 ${APP_ROOT} && \
    chown -R 1001:1001 /usr/libexec/s2i

# default user
USER 1001

# default CMD for the image
CMD ["/usr/libexec/s2i/usage"]

Note: user 1001 and group 1001 have been created by the base image openshift/base-centos7.

Build!

Now that we've got all the pieces together let's test out our builder.

Build our builder image

Building the image using docker should be as easy as:

docker build -t s2i-golang ./builder

Build our app using the builder

To build our app we will use our builder:

$ s2i build --copy test-app/ s2i-golang my-go-app --incremental

---> Checking for cache...

---> Building application from source...
go: downloading github.com/gorilla/mux v1.7.4
Build completed successfully

Note: Because we are caching go dependencies, if we re-ran the same build command (with --incremental) we should see that mux is not downloaded again.

Run our app

With the app image built, we can run it using docker:

docker run --rm -it -p 8080:8080 my-go-app

... and verify that it works by going to http://localhost:8080

$ curl -s http://localhost:8080

This is a test app!

App Image

Let's take a look at what we've ended up with.

Checking what user we are running as yields default. 👍

$ docker run --rm my-go-app /bin/bash -c "whoami"

default

Our app ended up being 7.5MBs.

$ docker run --rm my-go-app /bin/bash -c "ls -lh /opt/app-root/bin/app"

-rwxr-xr-x 1 default root 7.5M Jul 22 15:53 /opt/app-root/bin/app

And our image... is 725MBs!!!

$ (docker images | head -1 && docker images | grep my-go-app) | awk '{print $1, $NF}'

REPOSITORY SIZE
my-go-app 725MB

If we dive into our app image we can see that this is because of a few things:

1. The base image of the app image is the builder image which in itself has a lot of stuff our app doesn't care about.
2. The go compiler and build tools are present in the final app image.
3. All the intermittent artifacts such as cache and source code are on the final app image.

In addition to the size, all this extra stuff has the additional negative side-effect of increasing our attack surface. 😈

If we review our goals, it's clear that we haven't satisfied them all.

• ✅ The builder should compile go programs.
• ❌ The app image should not contain the source code.
• ❌ The app image should not contain the go compiler.
• ✅ The app image should not run as root.

---

---

Runtime Image

We can try to optimize our app image by using a runtime image to meet our initial goals.

A runtime image is an image that will be used instead of the builder as the base for the app image.

In order to make this runtime image work with s2i we need the following:

$ tree runtime/

runtime/
├── Dockerfile                 # docker instructions to create image
└── s2i
    └── bin
        ├── assemble-runtime   # script to relocate assets
        └── run                # script to run our application

s2i/bin/run

Just like the run script for the builder this script will simply execute our binary. In this case we are executing it the current working directory since that's where we'll be placing it.

#!/bin/sh -e
./app

s2i/bin/assemble-runtime

The assemble-runtime is analogous to assemble but for the runtime image. In our case, there is nothing additional to do so we'll just echo. Unfortunately this script is required even if we don't actually need it.

#!/bin/sh -e
echo "Nothing special to do here..."

Dockerfile

Lastly, but most importantly, we'll define our image with the right permissions on the minimal base image we need. In this case we are going to go with the base cento7 image.

# s2i-golang-runtime
FROM centos:7

      # the maintainer
LABEL maintainer="Javier Romero <[e-mail]>" \
      # specify where s2i scripts are located
      io.openshift.s2i.scripts-url="image:///usr/libexec/s2i/bin"

# s2i scripts
COPY s2i /usr/libexec/s2i

# create user/group
RUN groupadd -g 1001 app && \
    adduser -u 1001 -g 1001 default

# default user
USER 1001

Build! (w Runtime Image)

Let's build our app again but this time providing a separate runtime image in an effort to improve the final app image.

Build our runtime image

Again, building the image using docker should be as easy as:

docker build -t s2i-golang-runtime ./runtime

Build our app using the builder and runtime

To build our app we will use our builder and newly build runtime image:

$ s2i build --copy test-app/ s2i-golang my-go-app --runtime-image s2i-golang-runtime --runtime-artifact /opt/app-root/bin/app

---> Checking for cache...

---> Building application from source...
go: downloading github.com/gorilla/mux v1.7.4
Build completed successfully

A few things to note:

1. We can no longer use --incremental 😞. See issue #824.
2. We must provide the location of what artifacts we want to "transfer" over to the runtime image. This requires the end user to know more than they should about the internals of the builder. 😥

Run our app

Now that we've built our app (again) we can run it:

docker run --rm -it -p 8080:8080 my-go-app

Cool it still works!

App Image (Redux)

Let's hope we have a better app image. 🤞

Checking the running user...

$ docker run --rm my-go-app /bin/bash -c "whoami"

default

Still good.

Image size?

$ (docker images | head -1 && docker images | grep my-go-app) | awk '{print $1, $NF}'

REPOSITORY SIZE
my-go-app 212MB

Nice! We've saved >500MBs!!! This is data that we don't have to store or transfer. 🙌

Plus, much less of an attack surface. 👿

A dive into the image:

This looks much better!

... and we've met all our original goals.

• ✅ The builder should compile go programs.
• ✅ The app image should not contain the source code.
• ✅ The app image should not contain the go compiler.
• ✅ The app image should not run as root.

---

---

Conclusion

So what did we learn?

1. Builders are fairly easy to construct and can be generally used with multiple applications (unlike pure Dockerfiles).
2. Built-in caching can be used to improve local development via --incremental.
3. --incremental does not currently work with --runtime-image.
4. To use --runtime-image you must have intimate knowledge about what artifacts will be produced.
5. We gain a performance and security boost from using a seperate runtime image.

If you'd like to look at the source used in this tutorial, you may find it here: https://github.com/jromero/learning-s2i

---

---

Additional Reading

• OpenShift "create s2i builder image" post
• Runtime Image docs