This short article aims to share some tips I learnt from optimizing our GitLab CI file (.gitlab-ci.yaml) for building multiple Docker images.
By implementing these techniques, you’ll not only make your GitLab CI files more readable but also set the stage for a more agile and adaptable CI/CD environment.
Let’s delve into the practical insights that can elevate your DevOps experience and contribute to a more efficient and maintainable codebase.
1- Use default runner tag and override it when needed:
In case you are using a standard or a generic runner for the majority of your jobs and a a different runner for a few jobs, consider defining a global runner tag instead of defining one for each job.
You can then override the global tag when necessary.
❌ Don’t do:
compile:
script:
- ...
tags:
- myruuner
test:
script:
- ...
tags:
- testrunner
deploy:
script:
- ...
tags:
- myrunner
✅ Do:
default:
tags:
- myrunner
compile:
script:
- ...
test:
script:
- ...
tags:
- testrunner
deploy:
script:
- ...
2- Use parallel:matrix
GitLab has a powerful CI feature to run a matrix of jobs in parallel.
In our case we had multiple jobs to test every customized Docker JDK image and ensure that it does not break a standard maven build.
Taking advantage of the keyword parallel:matrix, can save you multiple lines of code and make your CI files more readable and easier to maintain.
Here is an example of refactoring our test jobs:
❌ Instead of configuring multiple job with the same script part, for example:
test-mvn-jdk-11:
image: openjdk-11
stage: test-image
script:
- mvn -V compile ...
test-mvn-jdk-17:
image: openjdk-17
stage: test-image
script:
- mvn -V compile ...
test-mvn-jdk-21:
image: openjdk-21
stage: test-image
script:
- mvn -V compile ...
....
✅ You can easily replace them with:
test-mvn:
image: $JAVA_IMAGE
stage: test-image
script:
- mvn -V compile ...
parallel:
matrix:
- JAVA_IMAGE: [openjdk-11,openjdk-17,openjdk-21]
This will also make it easier to incorporate additional JDK versions in the future, requiring minimal adjustments and eliminating the need for extensive line changes or introducing new jobs.
3- Use rules to define variables:
GitLab rules provide a mechanism for specifying conditions to determine when CI/CD jobs run.
Additionally, the use of if statements within these rules allows for the dynamic definition of variables based on conditions such as branches or releases, offering flexibility in variable assignment.
For our repository's CI/CD workflow, we start by building docker images and test them if there's a Merge Request.
Only when everything checks out, we create a final version TAG for release.
Using rules reduced our build jobs by a half.
❌ Instead of defining two jobs, for example:
build-mr-jdk-11:
stage: build
script:
- docker build -t jdk-11:mr-validation
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
build-jdk-11:
image: openjdk-11
stage: build
script:
- docker build -t jdk-11:$CI_COMMIT_TAG
rules:
- if: $CI_COMMIT_TAG
✅ Use the same job and override the variables depending on the rules conditions:
build-jdk-11:
stage: build
script:
- docker build -t jdk-11:$TAG
rules:
- if: $CI_COMMIT_TAG
variables:
TAG: "$CI_COMMIT_TAG"
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
variables:
TAG: "mr-validation"
Conclusion: Lean back and let the magic happen 🧙♀
In summary, applying these tips reduced our GitLab CI file from 1176 to 667 lines, significantly improving efficiency, manageability, and overall workflow.
⭐⭐⭐ Enjoy your learning!!! ⭐⭐⭐
Top comments (3)
Thank you for sharing. One basic you did not mention are the job templates (hidden jobs) 😉
I'm amazed! You left a comment on my article, and I find it unbelievable because my article drew inspiration from yours GitLab CI: 10+ Best Practices to Avoid .
Regarding your remark, yes, templates is another effective method to optimize GitLab CI, I shared only the practices I implemented in that specific GitLab CI file. Next time, I'll share more ideas
Your comment is appreciated. Thank you!
You are too kind 😊 keep up the good work !