How to debug artifact passing on Gitlab easily

#devops #gitlab #tooling

How to debug artifact passing on Gitlab

Lately, I've been working a lot with GitLab pipeline files and I have now a bit more complicated setup than usual. In my case, the middle step is included from another project as a template. I had a nasty bug that from the first step artifacts were never arriving at the last step. However as per Gitlab's docs that should be working by default.

A simplified version of my gitlab-ci.yml. Inspiration from StackOverflow:

image: ubuntu:18.04

stages:
  - test
  - build
  - deploy

test:
  stage: test
  script:
    - echo "test result..." >> test_result.txt
  artifacts:
    paths:
    - "test_result.txt"
    expire_in: 1 day

# runs in stage build
include:
 - project: 'another-project'
   ref: 'master'
   file:
     - '/templates/build.yml' # this creates build_result.txt

# trying to debug if included step can see the artifact from before
included-build:
  script:
    # This is wrong! Overriding the included template's script section
    - cat test_result.txt

# test_result didn't arrive here at all
deploy:
  stage: deploy
  script:
    - cat test_result.txt
    - cat build_result.txt

The Problem

So I wanted to debug why test_result.txt from the first step never arrived at the last stage.

My first thought was I can debug this by adding a script to every step in between to cat test_result.txt right? Nope!

Because the included template also introduces a script section, if I add a script section to included-build the included scripts will be overridden. Thus losing the original behavior.

🔥 Hot tip, you can always validate this by looking through the (View Merged YAML)[https://docs.gitlab.com/ee/ci/pipeline_editor/#view-expanded-configuration] tab. This way you make sure you see everything merged from templates and won't be surprised :).

The solution

I discovered that every job can have a before_script. In my case, this was not used in any steps, so I could use it with confidence. This way I can be sure I don't break any scripts already included.

# trying to debug if included step can see the artifact from before
included-build:
  when: on_success
  before_script:
    # right! Won't change the original behavior of this job
    - cat test_result.txt

So I could add this before_script section to every job in between the first and last step, and this way found the bug easily. Hopefully, this small tip helps you as well.

🔥🔥 Extra hot tip, use the Gitlab Workflow VS Code Extension, it works with on-prem servers as well (not just official gitlab.com projects)!