Gitlab CI/CD is the continuous integration and continuous delivery solution created by Gitlab, if you need help understanding the basics please refer to other sources like:
- Getting started with GitLab CI/CD - GitLab Unfiltered
- Gitlab CI pipeline tutorial for beginners - Valentin Despa
Is a feature that allows us to trigger pipelines that are hosted in other projects.
This is especially useful for microservices projects that need a way to deploy everything in the correct order, for projects that need to run E2E tests before deploying to production, or any other case that you can think of that needs integrated pipelines between projects.
Launching another's project pipeline was initially introduced to Gitlab in the form of API Triggers. Triggers are excellent when you need to launch a pipeline outside of Gitlab or for another project that you or your organization do not own. But the use of this type of triggers needs additional configurations like create a trigger token or use curl inside the job script.
Multi-Project Pipeline became free for all users in early 2020, but without the visualization of the pipeline triggered, like this:
In this case, we are going to explain some of the configurations in the Multi-Project Pipeline feature and from a free account ;)
Let's assume the following:
- We are developing an awesome app by our development team.
- The development team has already done some functionality of the awesome app and is already deployed to production.
- But we also have a QA team that needs to make E2E testing to our awesome app.
my-awesome-project: This is the project with the code we want to deploy to production.
e2e-testing-qa: This is the project we want to trigger before the deployment phase in
my-awesome-project, the main purpose of this project is to do E2E testing in a staging environment.
* This projects reside inside a group in Gitlab called my-awesome-group-trigger
In the current flow we have a stage to unit-testing, build our Docker image to later be deployed to production. We need to find a way to integrate this flow of the development team with the QA team and add some E2E tests.
So this is a problem because the code related to our E2E tests would be in the same project that our production code, and this involves the two teams. So probably the best approach, in this case, is to have separate projects, this means we need to use a strategy to launch and sync our deployment pipelines.
This would integrate the two projects and only when all the previous stages to deploy are successfully passed the code would be launched to production.
* I would explain later what this
Bridge job is.
image: alpine stages: - test - build - deploy test: stage: test script: - echo "This is the test job." build: stage: build script: - echo "This is the build job." deploy: stage: deploy script: - echo "This is the deploy job."
image: alpine stages: - deploy - e2e deploy: stage: deploy script: - echo "This is the deploy job." e2e: stage: e2e script: - echo "This is the E2E job."
As you can see we just need to make the QA integration, for this we are going to use the Multi-Project pipeline.
This is a job that has the special keyword
trigger and this automatically turns this job in a bridge job, this means that its only purpose is to launch a pipeline in another project or the same project (see parent-child-pipelines).
In this particular case we only need to modify the CI/CD file in
my-awesome-project, let's add the
bridge stage and job, between build and deploy to make our E2E testing:
# my-awesome-project/.gitlab-ci.yml image: alpine stages: - test - build - bridge - deploy test: stage: test script: - echo "This is the test job." build: stage: build script: - echo "This is the build job." bridge: stage: bridge trigger: my-awesome-group-trigger/e2e-testing-qa deploy: stage: deploy script: - echo "This is the deploy job."
Let's see how our pipelines are working together (for the following images the top pipeline is the project
my-awesome-project and the bottom is
- Passing pipeline
- Falling pipeline
Works! But it seems like the pipeline in the project
my-awesome-project it's not waiting for the
e2e-testing-qa to finish, and this is not our desired behavior for this case, remember we need to make the E2E tests before the production deployment. So we need to modify our
bridge job one more time to add the
# my-awesome-project/.gitlab-ci.yml ... bridge: stage: bridge trigger: my-awesome-group-trigger/e2e-testing-qa strategy: depend ...
- Passing pipeline
- Falling pipeline
Finally, when the E2E testing has passed in the
e2e-testing-qa our deploy to production can be made automatically, and when it fails will also be falling our main pipeline.