DEV Community

Mintu Ghosh
Mintu Ghosh

Posted on Originally published at linkedin.com

A green pipeline that deployed nothing

The worst CI outcome is not a failing build. It is a passing build that did
nothing and looked correct.

Here is one way to get there, and it takes a single missing line of YAML.

The setup

A pipeline that deploys only what changed. It diffs against the parent commit,
maps changed paths onto folders, and conditions each deployment stage on the
result.

files = run_git(["diff", "--name-only", base, head], repo_root)
Enter fullscreen mode Exit fullscreen mode

The failure

Azure DevOps can check out with shallow fetch. A shallow clone has no parent
commit.

So git diff HEAD~1 HEAD does not error. It returns nothing.

Nothing changed, therefore nothing needs deploying, therefore every stage skips,
therefore the pipeline is green. Nobody is alerted, because from the outside
this is exactly what a run looks like when someone edits a README.

You find out later, when something that was supposed to be in production is not.

The fix is two parts

Make the pipeline correct:

- checkout: self
  fetchDepth: 0
Enter fullscreen mode Exit fullscreen mode

And make the script refuse to guess:

if run_git(["rev-parse", "--is-shallow-repository"], repo_root) == "true":
    raise GitError(
        "This is a shallow clone, so there is no history to diff against.\n"
        "Set 'fetchDepth: 0' on the checkout step, or disable shallow fetch "
        "in the pipeline settings."
    )
Enter fullscreen mode Exit fullscreen mode

The second part matters more than the first. fetchDepth: 0 is one line in a
YAML file that someone will eventually remove while tidying up, or that will be
absent when a colleague copies the pipeline into a new project. The check turns
that into a red build with an instruction attached.

And a test, because the line is deletable

def test_detection_stage_checks_out_full_history(self):
    checkout = detect_stage["jobs"][0]["steps"][0]
    self.assertEqual(checkout.get("checkout"), "self")
    self.assertEqual(
        checkout.get("fetchDepth"), 0,
        "change detection needs full history; fetchDepth must be 0",
    )
Enter fullscreen mode Exit fullscreen mode

It parses the pipeline YAML and asserts the line is still there. Slightly
unusual to test your own CI config, but this is a config value with a silent
failure mode, which is exactly the kind worth pinning.

The general shape

Ask of any conditional deployment: what does this do when its input is
missing rather than wrong?

Wrong input usually errors. Missing input often produces an empty result that
looks like a legitimate negative. Empty and "nothing to do" are the same value,
and only one of them is safe.

Anywhere those two are indistinguishable, make the code refuse rather than
assume.


Full context: [https://www.linkedin.com/pulse/enterprise-microsoft-fabric-cicd-selective-one-azure-devops-ghosh-uxzsf]
Code: [https://github.com/VEDAFORGE/fabric-cicd-reference/tree/v2.0.0]

Top comments (0)