Treat Every `.gitlab-ci.yml` Edit as a Graph Change, Not a Text Diff

A dangerous GitLab CI change rarely looks dangerous in a merge request. The YAML diff is small: one rules: block becomes if: $CI_COMMIT_BRANCH, a job gains needs: [], somebody edits a cache key because a blog post said it would be faster. Reviewers read it like prose, approve it, and then discover the real effect later: a job silently stopped running on tags, artifacts now expire before downstream jobs can fetch them, or the pipeline became fully parallel in a way that makes main nondeterministic.

The problem is that .gitlab-ci.yml is not documentation. It is a little compiler input. The meaningful unit is not the edited line; it is the effective job graph after include, extends, rules, needs, dependencies, cache, and artifacts have interacted. I stopped reviewing CI YAML as text and started forcing every proposal through a tiny “effective graph” preview. The preview does not need to perfectly reimplement GitLab. It needs to make semantic movement visible before merge.

Below is a reproducible local version. It is intentionally conservative: where GitLab semantics are complicated, it emits unknown instead of pretending to be right.

The artifact: compile CI YAML into an effective graph

This Python example expands local include entries, merges extends, then emits one JSON record per job with the fields reviewers actually argue about: when it may run, what it needs, what it consumes, what it publishes, and which cache namespace it touches.

#!/usr/bin/env python3
"""ci_graph.py — simplified effective-graph preview for GitLab CI YAML.
Not a full GitLab parser. Emits 'unknown' where semantics are ambiguous.
"""
import json, sys, pathlib, yaml

ROOT = pathlib.Path(sys.argv[1] if len(sys.argv) > 1 else ".")
MAIN = ROOT / ".gitlab-ci.yml"

def load(path):
    return yaml.safe_load(path.read_text()) or {}

def includes(doc):
    inc = doc.get("include", [])
    if isinstance(inc, str):
        inc = [{"local": inc}]
    out = {}
    for item in inc:
        if isinstance(item, dict) and "local" in item:
            out.update(load(ROOT / item["local"]))
        else:
            out[f"__external_include_{len(out)}__"] = {"_unsupported": item}
    return out

def merge(a, b):
    # b extends a; scalars in b win, mappings merge shallowly, lists replace.
    out = dict(a)
    for k, v in b.items():
        if isinstance(v, dict) and isinstance(out.get(k), dict):
            inner = dict(out[k]); inner.update(v); out[k] = inner
        else:
            out[k] = v
    return out

def resolve_extends(jobs, name, seen=()):
    if name in seen:
        return {"_error": f"extends cycle at {name}"}
    job = jobs.get(name, {})
    parents = job.get("extends", [])
    if isinstance(parents, str):
        parents = [parents]
    acc = {}
    for p in parents:
        acc = merge(acc, resolve_extends(jobs, p, seen + (name,)))
    return merge(acc, {k: v for k, v in job.items() if k != "extends"})

def rule_summary(rules):
    if rules is None:
        return {"mode": "default", "risk": "unknown"}
    text = json.dumps(rules, sort_keys=True)
    flags = []
    if "$CI_COMMIT_BRANCH" in text and "changes" in text:
        flags.append("branch+changes rule combination")
    if "when: manual" in text and "allow_failure" not in text:
        flags.append("manual gate without failure policy")
    if "when: always" in text and "changes" in text:
        flags.append("always can defeat path filtering")
    return {"mode": "rules", "flags": flags, "raw": rules}

def main():
    base = includes(load(MAIN))
    main_doc = load(MAIN)
    jobs = {k: v for k, v in {**base, **main_doc}.items()
            if isinstance(v, dict) and not k.startswith(".") and k not in
            {"stages", "workflow", "default", "include", "variables"}}
    graph = {}
    for name, raw in jobs.items():
        eff = resolve_extends({**base, **main_doc}, name)
        graph[name] = {
            "stage": eff.get("stage", "unknown"),
            "rules": rule_summary(eff.get("rules")),
            "needs": eff.get("needs", "default-stage-order"),
            "dependencies": eff.get("dependencies", "default-all-prior"),
            "artifacts": eff.get("artifacts", {}).get("paths", []),
            "cache_key": (eff.get("cache", {}) or {}).get("key", "default"),
            "tags": eff.get("tags", []),
        }
    print(json.dumps(graph, indent=2, sort_keys=True))

main()

Run it before and after a branch, then diff the JSON, not the YAML:

python3 ci_graph.py ~/repo/main > before.json
python3 ci_graph.py ~/repo/feature > after.json
diff -u before.json after.json | less

That sounds almost too simple, which is why it works. Reviewers are bad at mentally simulating YAML inheritance and rule precedence. They are much better at reacting to a line that says: package moved from needs: default-stage-order to needs: [], while deploy still has dependencies: default-all-prior. That is a reviewable fact.

What I flag first

I use a short decision table so the preview stays opinionated but explainable. It is not a correctness proof; it is a severity ladder.

Effective-graph movement	Why I care	Default action
Job disappears from some ref patterns	The worst failures are silent non-runs	Require a pipeline evidence link before merge
needs becomes narrower than artifact producers	Parallelism can starve consumers	Mark high risk even if lint passes
cache_key changes while job names stay stable	Old caches poison or miss in confusing ways	Ask for invalidation note and rollback key
artifacts.expire_in shortened	Downstream jobs may fetch after expiry	Block unless consumers are enumerated
rules.flags contains always can defeat path filtering	Intended optimization becomes no filtering	Rewrite rule or add explicit negative case
External include appears	Local preview cannot expand it	Require pinned ref and owner approval

The important discipline is that the table is data, not vibes. If a reviewer disagrees, the disagreement is attached to a concrete movement in before.json vs after.json.

Where a model helps, and where I do not let it decide

The deterministic compiler finds movement; it does not write a good merge-request comment by itself. For that last mile I sometimes use free model access from MonkeyCode to turn three JSON diffs into a short reviewer note: what changed, which consumer is affected, and what evidence would satisfy the table. I host the scheduled preview on the free server option there because the job is small, stateless, and easy to move. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The boundary is strict: the model can phrase a comment, but it cannot approve, label risk, or override the table. If the compiler says unknown, the comment must say unknown. If the table blocks, the bot blocks. This keeps the language model away from the only decision that matters: whether a semantic pipeline change is acceptable.

A minimal prompt contract looks like this:

Input: three effective-graph diffs as JSON.
Output: Markdown with exactly these sections: Change, Consumer at risk, Evidence requested.
Never use the words safe, unsafe, approved, or blocked.
If any field is "unknown", repeat it verbatim as UNKNOWN.
Maximum 120 words.

Banning words like safe is deliberate. Once a generated comment can say “looks safe,” reviewers outsource judgment to tone.

Rollout that does not annoy everyone

I would not start by gating all CI edits. Start with a comment-only bot on repositories that already have frequent pipeline changes. For two weeks, collect false positives: which flagged movements were intentional, which table rows fired too often, which jobs everyone ignores. Then gate only two categories first: disappearing jobs and artifact expiry shortening. Those are the changes that create the longest debugging sessions because the pipeline can be green while still wrong.

Keep the bot outside the critical path if possible. A preview that posts five minutes after push is still useful; a required check that times out becomes a reason people bypass process. If the preview fails open, say so in the comment. If it fails closed, page a human owner rather than letting authors retry blindly.

Limitations and who should skip this

This is not a replacement for GitLab’s own validation, and the simplified parser will misread complex setups: nested includes, remote templates, extends across many files, dynamic child pipelines, and workflow rules can all exceed what a local preview should claim to understand. Treat external includes as an approval boundary, not as text to expand casually.

Skip this approach if your pipelines are small, rarely edited, and owned by one person who already simulates them mentally. Also skip it if your organization cannot agree on owners for CI semantics; a bot that flags risk without an accountable decider becomes noise. And do not build anything essential around the assumption that free hosting or free model access will remain available, identical, or fast; keep the compiler portable and the comment generator replaceable.

The durable piece is the habit: when CI YAML changes, demand an effective-graph diff. Everything else—the bot, the phrasing, the hosting—is packaging.

If you maintain shared GitLab pipelines, steal the compiler idea and tell me which semantic change burned you hardest: disappearing jobs, cache-key churn, or artifact expiry.