Handle Gitlab pipeline duplication

Problem

There is a known problem that in Gitlab it's not possible to avoid pipeline duplication in some scenarios.

Context

Let's assume you combine merge and branch pipelines. You can follow the Gitlab guideline or here is an improved one:

workflow:
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
    - if: '$CI_PIPELINE_SOURCE == "web"'
    - if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS'
      when: never
    - if: '$CI_COMMIT_BRANCH'

When you push and then you open a MR immediately it's not possible to avoid getting a second pipeline (though if you have one open then it's not an issue).

Solution with git options

• See Gitlab CI push options

git push -o merge_request.create

This solution with the script below

Here is a script that supposed to be run after the push to cancel the duplication.

How to use?

1. Create an alias in your bash profile / .zshrc file, e.g.: alias cancel-branch-pipeline="sh ~/gitlab/gitlab-cancel-branch-pipeline.sh"
2. Set you Gitlab personal access token with scopes api and read_api as GITLAB_TOKEN environment variable (you can set it in your bash profile file to have in every session: export GITLAB_TOKEN=<your-token>)
3. Call it after with the alias anytime you push, and you want to open a MR: $ cancel-branch-pipeline

Comments

[Benoit COUETIL 💫]

My friend, the two modes, branch pipeline and MR pipelines, were meant to be triggered on every occasion, depending on variables you use.

By default, there are only branch pipelines, and if you use some variables present in both cases, you may trigger also MR pipeline.

workflow:rules/rules are not meant to be that complicated. When reading yours, it is very difficult to understand in which case it is triggered.

Be more straightforward, even if more verbose, it does not hurt.

Let's say you want :

• a pipeline on develop and main branches
• a pipeline on MR targeting develop
• not on MR targeting main, because it is from develop and there is already a pipeline for each

The workflow rules would be :

workflow:
  rules:
    # OK on MR to develop
    - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "develop"
    # OK on main branches
    - if: $CI_COMMIT_BRANCH == "develop"
    - if: $CI_COMMIT_BRANCH == "main"

[Artem Ptushkin]

Hey, it's deeper, I'm sorry, I know all the basics thanks. There is a known problem or pipeline duplication when you push and then immediately, that's what I wrote

When you push and then you open a MR immediately it's not possible to avoid getting a second pipeline (though if you have one open then it's not an issue).

[Artem Ptushkin]

I leverage the switch between branch and MR pipelines, it's supported by Gitlab here, having this you can find the root reason of the issue

[Benoit COUETIL 💫]

Sorry I read your article too quicky, I should have taken the time needed.

What do we gain, switching from one type of pipeline to the other for the same branch ?

Seems interesting but a use case would help me grasp it.

[Artem Ptushkin]

Np, I didn't pay attention to the reasons either. Thanks for asking.

The switching is for developers who prefer working on a branch and opening an MR when he's really ready, example flow:

1. you work, commit, and push
2. you get a pipeline
3. you make sure it's green
4. you open the MR when you're ready to show it and you get pipelines there now aka public

The problem happens between 3 and 4 if they happen one after another - simple one-commit merge requests are always ready for review immediately.

[Benoit COUETIL 💫]

OK, so it is interesting if the pipeline is different ; else keeping the branch pipeline type all the way would suffice (no MR pipeline type, but still branch pipelines from now on ; the default behavior).

Do you have examples of differences in your pipelines, before and after the MR creation ? E2E tests added maybe ?

[Artem Ptushkin]

No, no difference I have. You mostly need a merge requests pipeline to run pipelines from the merge request web page

You're right branch one is linked by default to each MR but you can not run that button. That's how we ended up combining solutions in one pipeline code, we provide one common pipeline for all.

[Benoit COUETIL 💫]

So much for a shortcut button 😅.

But it has opened me possibilities.

Thanks !