Introduction
In this article, i will be discussing how I implemented a ci/cd pipeline from scratch to build a simple Golang application into a docker image and push said image to Docker hub
The stages of the said pipeline include checking out the source code repository, in this case, git, running analysis on the source code with sonar cube to check for vulnerabilities, building the source code into a docker image using multi-stage build to reduce the size of the image, and finally pushing the image into a docker hub repository
Prerequisites:
Because the installation and configuration of these tools are long and a subject of another topic, I won't be discussing them here today.
The reader should have the following installed and configured and have basic knowledge and understanding of these tools if they wish to follow along.
- Golang
- Docker
- IDE
- Jenkins
- Sonarcube
- Git
The reader should also have the following accounts signed in
- Docker hub
- Sonarcloud
- Jenkins
- GitHub
Setting Up Project Files
The Application
As mentioned earlier I’ll be using a simple application written in Golang, this application has three different routes that print three different messages to the browser.
func firstEndPointHandler(w http.ResponseWriter, r *http.Request) {
message := "this is the first endpoint"
_, err := w.Write([]byte(message))
if err != nil {
log.Fatal(err)
}
}
func secondEndPointHandler(w http.ResponseWriter, r *http.Request) {
message := "second endpoint"
_, err := w.Write([]byte(message))
if err != nil {
log.Fatal(err)
}
}
func thirdEndPointHandler(w http.ResponseWriter, r *http.Request) {
message := "second endpoint"
_, err := w.Write([]byte(message))
if err != nil {
log.Fatal(err)
}
}
func main() {
http.HandleFunc("/first", firstEndPointHandler)
http.HandleFunc("/second", secondEndPointHandler)
http.HandleFunc("/third", thirdEndPointHandler)
err := http.ListenAndServe(":8081", nil)
log.Fatal(err)
}
Dockerfile
As you all know to build an application into a docker image I’ll need to use a docker file, where I'll specify the build for the image
In this docker file, I’ll be using a multistage build. With multistage builds, you can drastically reduce the size of a docker image and optimize the image by using multiple FROM statements, and each of them begins a new stage of a build. Each FROM statement can use a different base image.
The essence of this is to be able to copy only the necessary artefacts from one stage to another and leave the ones you don't need.
FROM golang:1.22.4 AS builder
WORKDIR /app
COPY go.mod ./
RUN go mod download
COPY *.go ./
RUN CGO_ENABLED=0 GOOS=linux go build -o /test-app
FROM scratch
COPY --from=builder /test-app /test-app
EXPOSE 8081
CMD ["/test-app"]
The first stage uses the golang:1.22.4 as the base image and it is named builder
Sets the working directory to /app
Copies go.mod file and download all dependencies
Copies all files that end with the .go extension
Builds the go binary with CGO_ENABLED=0 and GOOS=linux and produces an output image /test-app
The second stage of the build:
Uses the base image: scratch which is an empty image
Copies the built /test-app binary from the builder stage
Exposes port 8081
Specifies the command to run the binary: CMD[“/test-app”]
Jenkinsfile
The Jenkinsfile defines the pipeline stages and the steps executed during the pipeline. It must be placed in the root directory of the project for Jenkins to discover and initiate the pipeline.
pipeline {
agent any
tools {
go 'golang'
}
environment {
DOCKERHUB_CREDENTIALS = credentials('dockerhub')
DOCKER_IMAGE = 'ephraimaudu/test-app'
GITHUB_CREDENTIALS = 'git-secret'
SONAR_TOKEN = credentials('SONAR_TOKEN')
}
stages{
stage('Checkout'){
steps{
echo "checking out repo"
git url: 'https://github.com/audu97/test-project', branch: 'master',
credentialsId: "${GITHUB_CREDENTIALS}"
}
}
stage('Run SonarQube Analysis') {
steps {
script {
echo 'starting analysis'
sh '/usr/local/sonar/bin/sonar-scanner -X -Dsonar.organization=eph-test-app -Dsonar.projectKey=eph-test-app-test-go-app -Dsonar.sources=. -Dsonar.host.url=https://sonarcloud.io'
}
}
}
stage('Run Docker Build'){
steps{
script{
echo "starting docker build"
sh "docker build build -t ${DOCKER_IMAGE}:${env.BUILD_ID} ."
echo "docker built successfully"
}
}
}
stage('push to docker hub'){
steps{
echo "pushing to docker hub"
script{
docker.withRegistry('https://index.docker.io/v1/', 'dockerhub'){
docker.image("${DOCKER_IMAGE}:${env.BUILD_ID}").push()
}
}
echo "done"
}
}
}
post {
always{
cleanWs()
}
}
}
NOTE: To use these credentials in the environment variables, you should add them in the “Credentials” section within the “Manage Jenkins” option of the Jenkins UI. This way, your Jenkins jobs can securely access the necessary credentials during their execution.
The agent specifies that the pipeline can run on any available executor in the Jenkins environment
Environment variables: defines various environment variables used in the pipeline execution. DOCKERHUB_CREDENTIALS -contains credentials for signing in to docker hub, GITHUB_CREDENTIALS -contains credentials to use to sign in to GitHub to check out the specified repository, SONAR_TOKEN- also serves as credentials for the Sonar cloud, where I can view the code analysis, DOCKER_IMAGE-specifies the name I want for the docker image.
Stages: the pipeline consists of several stages:
- Checkout: This stage checks out the code from the specified GitHub repository
- Run sonarqube analysis: executes sonar cube analysis on the code base. Sonar cube is a tool for static analysis on a code base by analyzing it statically. It detects bugs, vulnerabilities and code smells.
- Run docker build: builds a docker image using the specified dockerfile.
Push to docker hub: pushes the built docker image to docker hub
Post processing: the post section ensures that the workspace is cleaned up after the pipeline execution, even if the pipeline fails
Challenges
The biggest challenge I faced, which took considerable time to resolve, was that Jenkins could not locate Docker to execute the Docker build stage in my pipeline because I had installed both Jenkins and Docker using snaps. This resulted in repeated pipeline failures.
To overcome this issue, I uninstalled the snap versions of both Jenkins and Docker. Following that, I installed them following the instructions provided in their documentation. This approach solved my problem by allowing Jenkins to interact with Docker successfully.
Conclusion
This project has provided valuable insights into the importance and the need for CI/CD pipelines. Additionally, it emphasizes multistage Docker image builds and includes security considerations during the build stage (shifting security left) by using SonarQube.
The link to the repository containing the entire project can be located HERE
Top comments (0)