Introducing Trebuchet

One of the many hats my team wears is helping our developers build and publish their software. Something we strive to do is make it as easy as possible for new developers and teams to get up and running with our chosen platform. For most teams, one of the final steps in the CI/CD pipeline is publishing a docker container to some registry. For us, this happens to be Amazon ECR.

To simplify this for most teams, we've written a new tool we've called trebuchet:

HylandSoftware / trebuchet

Launch container images into Amazon ECR

Trebuchet - Launch container images into Amazon ECR

---

The purpose of Trebuchet is to improve the quality of life for pushing Docker images to Amazon Elastic Container Registry (ECR).

Usage

Trebuchet is shipped as a single binary (Linux/Windows) and as a Docker image. All images can be found here.

Commands

push:

Pushes a Docker image into ECR
Region
        Region is required to be set as a flag, as an AWS environment variable (AWS_DEFAULT_REGION), or in the AWS config
Profile:
        Profile may be set as a flag or an AWS environment variable. 

Amazon Resource Name (ARN):
        Passing in a valid ARN allows trebuchet to assume a role to perform actions within AWS. A typical use-case for this
        would be a service account to use in a software pipeline to push images to ECR.

Aliases:
        trebuchet push can also be used as 'treb launch' or 'treb fling' for a more

…

View on GitHub

To understand why we've written a tool for this, let's take a look at what all is involved with publishing images with the existing tooling from a CI System.

The Old Way

Before we can do anything, we have to install the AWS CLI and its dependencies. We utilize Jenkins and the Kubernetes plugin, so there are a few options here. We can either bake the CLI into one of the containers that our build pod uses, or we can install it at build time to keep the build container simple.

Installing the CLI

A common Jenkins pipeline step in most of our pipelines would look like this:

stage('install-tools') {
    steps {
        container('docker') {
            withCredentials([file(credentialsId: 'aws-credentials', variable: 'AWS_CREDENTIALS')]) {
                sh """
                    mkdir -p \${HOME}/.aws
                    cp "${AWS_CREDENTIALS}" \${HOME}/.aws/credentials
                    cp ./ci/aws-config \${HOME}/.aws/config
                    apk update && apk add py-pip
                    pip install awscli --upgrade --user
                    PATH="\${PATH}:\${HOME}/.local/bin"
                    which aws
                    aws --version
                """
            }
        }
    }
}

First, we copy over our CI credentials from a Jenkins secret. Then, we install pip, the python package manager, so we can install the AWS CLI package. We also update the PATH environment variable in case the pip shim path isn't there already.

Publishing Images

Now that we have the tooling installed, we are ready to push our docker image(s):

stage('publish') {
    steps {
        container('docker') {
            sh '''
                PATH="\${PATH}:\${HOME}/.local/bin"
                eval $(aws ecr get-login --no-include-email)
                REPO_NAME="$(aws ecr describe-repositories --repository-names ecr-demo --output text --query 'repositories[*].repositoryUri' || aws ecr create-repository --repository-name ecr-demo --output text --query 'repository.repositoryUri)"
                docker tag ecr-demo:${APP_VERSION} ${REPO_NAME}:${APP_VERSION}
                docker tag ecr-demo:${APP_VERSION} ${REPO_NAME}:latest
                docker push ${REPO_NAME}:${APP_VERSION}
                docker push ${REPO_NAME}:latest
                echo ${REPO_NAME} > repo_name.txt
            '''
        }
    }
}

Because Jenkins resets the PATH variable for each step invocation, we have to re-add the pip shim path so we can find the AWS CLI. Then, we use the CLI to obtain docker login credentials for our user.

ECR Requires that we create a repository before pushing new images. We can create one if it doesn't exist by asking AWS for the URI to push to, and if we get an error back, try to create one instead.

Finally, we can re-tag our images images with the full ECR Repo URI and docker push them.

Using Trebuchet

Instead of using a docker daemon in our Jenkins build pod, we use hylandsoftware/trebuchet. For example:

spec:
  containers:
  - name: jnlp
    image: jenkins/jnlp-slave
  - name: trebuchet
    image: hylandsoftware/trebuchet
    tty: true
    securityContext:
      privileged: true

This image runs a dind docker daemon as its entrypoint, so we can still docker build our images as we normally would. However, pushing to ECR is now a single command:

stage('Push Docker Image to ECR') {
    steps {
        withCredentials([[$class: 'AmazonWebServicesCredentialsBinding', credentialsId: 'aws_credentials_id']) {
            container('trebuchet') {
                sh 'treb push ecr-demo:${APP_VERSION}'
            }
        }
    }
}

Trebuchet will take care of creating the ECR repository for us if it needs to. It will also re-tag the images as needed and then remove the temporary ECR tags from the local docker daemon.

In Conclusion

Thanks to my co-worker Jason for leading the charge on this.

Trebuchet is released under the MIT License; we hope you'll find it useful. Be sure to open a GitHub issue or submit a Pull Request for any bugs or new features you'd like to see implemented!