Ephemeral CI environments using ttl.sh and Gitness

In the realm of software development, balancing continuous integration with maintaining quality is a significant challenge. ttl.sh and Gitness offer a solution. ttl.sh is an ephemeral Docker image registry that allows for the creation of temporary image tags with built-in expiry. Gitness, an open-source platform by Harness, simplifies the management of source code repositories and development pipelines. This blog post will explore how these tools can be used to create temporary CI environments to expedite development processes.

The Problem

Multiple developers working on different features can lead to merge conflicts and a bloated image registry. Traditional CI environments often retain Docker images from feature branches too long, which complicates image management and increases the likelihood of testing against outdated or incorrect images.

The Solution

The workflow in the diagram leverages ttl.sh and Gitness to manage these challenges:

1. Feature Branch Workflow: The creation of a pull request (PR) for a feature branch triggers a build in Gitness.

2. Image Creation: Gitness constructs a Docker image from the feature branch, using ttl.sh for tagging with a UUID and a time-to-live (TTL) limit. This process does not require credentials and maintains privacy.

3. Automated Testing: The image is put through automated tests. Should the tests fail, developers are notified; if they succeed, the process proceeds.

4. Image Promotion: After a PR passes all checks and is merged, the image is given a permanent tag and pushed to a central image registry, which at this point, requires proper credentials.

This approach ensures that only quality-assured images make it to the central registry, reducing the clutter and promoting a cleaner CI process.

Demo

Let's construct this setup step by step. Starting with pushing an image to ttl.sh through Gitness, I’ll guide you to resources for completing the remaining components of the system.

Begin with the Gitness documentation to set up your first project. You can start a new repository or link an existing one from GitHub or GitLab. The specific source code repository can be any; the essential requirement is the presence of a Dockerfile.

Navigate to "Pipelines'' within your repository and create a new pipeline. Gitness will suggest a sample pipeline. Replace it with the following YAML:

kind: pipeline
spec:
 stages:
   - name: build-and-push
     spec:
       platform:
         arch: amd64
         os: linux
       steps:
         - name: docker_build
           spec:
             inputs:
               repo: ttl.sh/xxxx-yyyy-nnnn-2a2222-4b44
               tags: 1h
             name: docker
           type: plugin
     type: ci
version: 1

Observe the image repo and tag. ttl.sh is the image repository name and the UUID (xxxx-yyyy-nnnn-2a2222-4b44) is the image name. You can replace the hard coded image name with a Gitness secret for a dynamic image name. Click on Secrets and add a new Gitness secret called random_image_name. Now you can update your pipeline YAML so that the image name is not fixed:

repo: ttl.sh/${{ secrets.get("random_image_name") }}

Save the pipeline and click on Run. After executing the pipeline, your output should resemble the following:

Next add the following steps yourself:

• Add a run step to execute tests on the container you just built and pushed.
• Add a trigger so that when a pull request is opened, Gitness can automatically trigger pipeline execution.
• Add a slack plugin step so that failed tests trigger slack webhook and notification.
• Add another run step so that when all tests pass, the image tag is retagged and pushed to a private image registry. You’ll need authentication at this step.

Security Considerations

While ttl.sh's no-credential, ephemeral approach offers flexibility and simplicity for CI environments, it introduces unique security considerations. The convenience of pushing images without credentials is counterbalanced by potential security risks — namely, the possibility of unauthorized image pulls. You can mitigate these risks through short-lived images and using UUIDs for image names:

Short-Lived Images: By design, images in ttl.sh are ephemeral. Setting a short expiration time for an image means it's available for a limited time window, reducing the risk exposure period.

Use of UUIDs: Incorporating UUIDs into image tags significantly lowers the risk of unauthorized access. The randomness and complexity of UUIDs make it exceedingly difficult for someone to guess the image name and pull it without authorization.

However, for production environments, organizations might consider deploying a private version of ttl.sh. This allows for more control over the security aspects, such as network isolation, access control, and auditing capabilities.

Takeaway

Integrating ttl.sh and Gitness creates an ephemeral CI environment that streamlines the build and test process. It helps to avoid the accumulation of outdated images and keeps the pipeline lean. This method is not just about speed; it's about maintaining a manageable and efficient development workflow. Adopting these tools can lead to more frequent and dependable software delivery for your engineering teams. Check out and follow Harness YouTube channel for