This guide serves as a direct follow-up to Building a Kubernetes Operator with the Operator Framework.
It meticulously walks you through the intricacies of packaging a Docker container and Helm Chart, subsequently uploading them to GitLab using GitLab CI, with a keen emphasis on generating semantic version tags. It's noteworthy that nearly all CI templates can be executed locally with minor adjustments to environment variables and utilizing Docker.
Prerequisites
Before commencing, ensure that you have the following tools installed:
-
yq:
brew install yq
-
docker:
brew install docker
-
kustomize:
brew install kustomize
-
helm:
brew install helm
-
helmify:
brew install arttor/tap/helmify
Release Strategy
Before delving into packaging the Docker Container and Helm Chart, let's discuss the strategy in detail. The aim is to create Semantic Releases on a Pipeline Schedule (weekly) to eliminate the need for manual releases. These releases will trigger a pipeline running on a tag, responsible for releasing the Helm Chart and Docker Image. Since pushing changes directly to production is not an option, a pipeline for testing new features and bug fixes will also be implemented.
The workflow unfolds as follows:
- Create a new feature branch (
feature/improvement
). - Make changes to the code.
- Submit a Merge Request to the Main Branch (
feat: Improve Code
). This triggers a pipeline where a Pre-Release is built. - Merge the changes to the Main branch after thorough testing.
- Wait for the weekly release to increment the version number (specified as feat, resulting in a Minor Release).
- The new tag will publish a Docker Image and Helm Chart to GitLab.
Build the Docker Image
The Operator SDK conveniently provides a Dockerfile, simplifying the building process. We will utilize Kaniko for its user-friendly approach and the fact that it doesn't require a Docker Socket or any form of Docker in Docker. This process ensures the Docker Image is uploaded to the GitLab Container Registry.
.docker:
stage: build
image:
name: gcr.io/kaniko-project/executor:v1.20.1-debug
entrypoint: [""]
script:
- echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_REGISTRY_PASSWORD\"}}}" > /kaniko/.docker/config.json # Authenticate against Gitlab
- /kaniko/executor --context $CI_PROJECT_DIR --dockerfile $CI_PROJECT_DIR/Dockerfile --destination $CI_REGISTRY_IMAGE:$TAG # Build and Push the Docker Image
Helm
Before delving into the process, it's essential to decide where to store Helm Charts within GitLab. Two viable options are:
- Package Registry
- GitLab Pages
While the Package Registry appears ideal, it comes with a drawback - a somewhat unconventional domain and Helm Chart path (https://gitlab.com/api/v4/projects/{projectId}/packages/helm/stable
). Alternatively, GitLab Pages introduces its own set of challenges. Since Pages primarily deals with static code, properly indexing every version becomes nearly impossible. This guide, however, will focus on indexing the latest version, providing a glimpse into this method.
Packaging the Helm Chart
The Operator SDK employs Kustomize instead of Helm, necessitating additional steps to arrive at our Helm Chart. The initial step involves generating the bundle containing our manifests.
.bundle:
stage: build
image:
name: quay.io/operator-framework/operator-sdk:v1.32
entrypoint: [""]
script:
- make bundle # Generate the Bundle directory dynamically
artifacts:
paths:
- bundle
Ensure to run the make bundle command at least once locally. This ensures the creation of necessary files for automation later on. Additionally, add the bundle/* directory to the .gitignore file. Also, remember to update the Docker Image of your Manager to the actual Docker Image (registry.gitlab.com/path/to/operator).
Generate and Upload the Helm Chart
Now that we have the necessary manifests, it's time to transform them into a proper Helm Chart. For this purpose, we will utilize Helmify and a custom Docker Image bundling useful tools for Helm Chart upload. After generating the Helm Chart, YQ will be used to set the version. As noted in the release strategy, there is no distinction between Chart and Docker Version. In other words, the Application Version and Helm Chart Version will always align. In this example, we will publish the Helm Chart to both GitLab Pages and the Package Registry.
.upload:
stage: helm
variables:
HELM_EXPERIMENTAL_OCI: 1
image: registry.gitlab.com/stammkneipe.dev/operator-packager:latest
before_script:
- git config --global --add safe.directory '*'
- VERSION=$(git describe --tags `git rev-list --tags --max-count=1` 2>/dev/null)
- if [ -z "$VERSION" ]; then VERSION="0.1.0"; fi
- if [ -z "$CI_COMMIT_TAG" ]; then VERSION="${VERSION}-${CI_PIPELINE_IID}"; fi
script:
- kustomize build config/default | helmify ${CI_PROJECT_NAME} # Generate the Helm Chart with Kustomize and Helmify
- yq e -P ".version = \"${VERSION}\"" -i ${CI_PROJECT_NAME}/Chart.yaml # Set the Version of the Helm Chart
- yq e -P ".appVersion = \"${VERSION}\"" -i ${CI_PROJECT_NAME}/Chart.yaml # Set the Version of the Application
- helm package ${CI_PROJECT_NAME} --destination ./public
- helm repo add --username gitlab-ci-token --password ${CI_JOB_TOKEN} ${CI_PROJECT_NAME} ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/helm/stable # Only for Package Registry
- helm plugin install https://github.com/chartmuseum/helm-push # Only for Package Registry
- helm cm-push ./public/${CI_PROJECT_NAME}-${VERSION}.tgz ${CI_PROJECT_NAME} # Only for Package Registry
- helm repo index --url https://${CI_PROJECT_NAMESPACE}.gitlab.io/${CI_PROJECT_NAME} . # Only for Gitlab Pages
- mv index.yaml ./public # Only for Gitlab Pages
artifacts:
paths:
- public # Gitlab Pages
Generating the Actual GitLab CI Jobs
Up until now, we've been creating templates for reuse in actual CI Jobs. Now it's time to generate the full [GitLab CI] configuration.
Stages
For this pipeline, we only need three stages. The build stages will handle building the Docker Image and the manifests for our Helm Chart, while the release stage will exclusively manage the Semantic Release.
stages:
- build
- helm
- release
Feature Branches
Feature or Renovate jobs will only run when a Merge Request is made to your Main Branch:
.feature_renovate:
variables:
TAG: $CI_COMMIT_REF_SLUG-$CI_PIPELINE_IID
rules:
- if: $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME =~ /^renovate/ && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == $CI_DEFAULT_BRANCH
- if: $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME =~ /^fix/ && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == $CI_DEFAULT_BRANCH
- if: $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME =~ /^feat/ && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == $CI_DEFAULT_BRANCH
- if: $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME =~ /^feature/ && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == $CI_DEFAULT_BRANCH
containerize_feature:
extends:
- .docker
- .feature_renovate
bundle_feature:
extends:
- .bundle
- .feature_renovate
upload_feature:
extends:
- .upload
- .feature_renovate
Main Branch
The Main Branch will serve two purposes. Firstly, we want to create a latest build, and on top of that, we want to generate semantic releases. This requires ensuring that semantic releases only run on a schedule, and other jobs do not run on schedule.
.latest:
variables:
TAG: latest
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH && $CI_PIPELINE_SOURCE != "schedule"
containerize_latest:
extends:
- .docker
- .latest
bundle_latest:
extends:
- .bundle
- .latest
upload_latest:
extends:
- .upload
- .latest
release_weekly:
stage: release
image: registry.gitlab.com/stammkneipe.dev/semantic-release:latest
script:
- git config --global --add safe.directory '*'
- npx -p @semantic-release/changelog -p @semantic-release/exec -p @semantic-release/git -p @semantic-release/gitlab semantic-release
rules:
- if: $CI_PIPELINE_SOURCE == "schedule"
Version Tag
The last jobs will be used for the actual versioned release and only run on a tag.
.tag:
variables:
FULL_IMAGE_NAME: $CI_COMMIT_TAG
rules:
- if: $CI_COMMIT_TAG
containerize_tag:
extends:
- .docker
- .tag
bundle_tag:
extends:
- .bundle
- .tag
upload_tag:
extends:
- .upload
- .tag
Creating the Schedule
The only thing left to do is to create a schedule to your liking. Keep in mind that Semantic Release will not create a release when there was no change in your code base. So there is no harm in using a short interval.
Usage
Now you can choose between your latest version inside Pages or use the Registry Package (https://gitlab.com/api/v4/projects/{projectId}/packages/helm/stable
) to install your Helm chart.
Conclusion
By following these steps, you've successfully set up a GitLab CI pipeline for packaging and deploying a Docker container and Helm Chart. The integration of semantic versioning ensures a structured release process for your application. This streamlined workflow enhances collaboration and facilitates the continuous delivery of your containerized applications.
Top comments (0)