Which GitLab Job Went Green? Scratch-Box Myths

A teammate pastes a green agent log into the MR. The comment says the tests ran. Are we merging on that screenshot?

Which GitLab job produced that log, exactly? Was it test, lint, or something unnamed?

I keep seeing this mix-up in reviews. A free remote box is not your runner. A free model is not .gitlab-ci.yml.

This FAQ kills five claims I still hear. Then it gives a parity checklist you can run.

Why the mix-up keeps landing in MRs

Agents now draft job YAML. They also execute code on a spare server.

That pair looks like CI from a distance. It is not CI.

GitLab still owns the job graph. rules, needs, workflow, and protected variables still pick the real path.

Chat cannot evaluate those edges. A nameless green log cannot either.

Scope, and one product note

I use a scratch pad for drafts. I do not use it as a merge gate.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode offers free model access and a free server option. I treat both as a pad beside GitLab, not a replacement for gitlab-runner.

Strip that name out. The checklist below still stands.

FAQ: five claims that do not survive a job name

Myth 1: "The model read the YAML, so the pipeline is understood."

Did it evaluate the graph? Or did it summarize a file?

.gitlab-ci.yml is not a README. It is a directed graph with hidden edges.

Those edges are easy to miss in a chat window:

• rules:if on $CI_COMMIT_BRANCH or $CI_PIPELINE_SOURCE
• workflow:rules that skip the whole pipeline
• needs that drop a job on tags
• include: from another project or a template
• extends and hidden keys that start with .

A model can paraphrase script:. That is reading. That is not evaluating rules.

Corrected model: YAML literacy is not graph execution.

Ask one question in review. Which rules clause fired for this SHA?

If nobody can answer, the "understanding" is fan fiction.

Myth 2: "The free server ran it, so GitLab would too."

Same image digest? Same runner tags? Same before_script?

Probably not. Guessing is not a contract.

A GitLab job inherits a real contract:

• runner tags and the executor behind them
• image plus entrypoint
• services that the job actually linked
• CI_JOB_TOKEN scoped to that job
• protected variables that never leave protected refs

A scratch box is a generic shell. It has whatever you installed today.

That is a laptop with extra latency. It is not gitlab-runner.

Corrected model: execution without the job contract is a demo.

Demos are useful. Demos do not sign merge trains.

Myth 3: "The env keys matched, so secrets are fine."

Did someone print values into the chat? Stop. Rotate them.

Did you only compare names? Better. Still incomplete.

Protected variables are absent on unprotected refs. Child pipelines get a different CI_JOB_TOKEN. Masked variables vanish from logs. File-type variables land as paths, not strings.

Name overlap is not value overlap. Value overlap is not scope overlap.

Corrected model: compare provenance, not os.environ.keys().

If the scratch box cannot receive a protected variable, that cell stays n/a. Do not invent a yes.

Myth 4: "Green pytest on the box means the merge pipeline is green."

Which stage failed last week? test? build? a tiny lint job?

Agents love the happy path. They run pytest from the repo root. They skip needs: ["compile"]. They skip coverage regexes. They skip artifacts:reports:junit.

GitLab can still fail after that green pytest:

• the next job has no artifacts to download
• a resource_group is locked
• an environment stop job ran on the wrong ref
• allow_failure: false on a job nobody mentioned
• a merge train replays the graph and hits a flake

Corrected model: a unit-test process is one node. The pipeline is the graph.

Name the node. Then name its edges.

Myth 5: "The advice was cheap, so it is cheap enough to trust."

Cheap inference is not a checked pipeline.

A free model can draft a job. A free server can run a subset. Neither emits a GitLab job ID.

I want a receipt GitLab already knows how to emit. Pipeline ID. Job ID. SHA. Job name.

Corrected model: cost of a draft is not cost of being wrong on main.

If the MR cannot link a pipeline, we do not have a receipt. We have a story.

Artifact: a CI parity card you can fill today

Do not trust vibes. Build a contract card.

This workflow is proposed. It is unexecuted on your runners until you run it. Never print secret values.

Step 1 — Name the job, name the SHA

# Proposed local commands. Pin the commit you think was tested.
git rev-parse HEAD
git status --porcelain
test -f .gitlab-ci.yml && wc -l .gitlab-ci.yml

Pick one job name from the YAML. Write it down.

No "the tests." A real key: unit-py. Or lint. Or typecheck.

If you cannot name it, stop here. There is nothing to compare.

Step 2 — Dump the contract, never the secrets

Save this as ci_parity_check.py. It prints names and shapes only.

#!/usr/bin/env python3
"""Proposed helper. Run it locally against .gitlab-ci.yml.

Prints job contract fields. Never prints variable values.
Does not expand include:, and does not evaluate rules.
"""
from __future__ import annotations

import sys
from pathlib import Path

try:
    import yaml
except ImportError:
    sys.stderr.write("pip install pyyaml\n")
    raise SystemExit(1)

RESERVED = {
    "stages",
    "variables",
    "include",
    "workflow",
    "default",
    "image",
    "services",
    "cache",
    "before_script",
    "after_script",
}

CONTRACT_KEYS = (
    "image",
    "tags",
    "stage",
    "rules",
    "needs",
    "before_script",
    "script",
    "services",
    "cache",
    "artifacts",
    "environment",
    "resource_group",
    "allow_failure",
    "coverage",
)


def load_doc(path: Path) -> dict:
    data = yaml.safe_load(path.read_text())
    if not isinstance(data, dict):
        raise SystemExit("expected a mapping in .gitlab-ci.yml")
    return data


def job_names(doc: dict) -> list[str]:
    names = []
    for key in doc:
        if key in RESERVED or str(key).startswith("."):
            continue
        if isinstance(doc[key], dict):
            names.append(str(key))
    return names


def contract_for(job: dict, defaults: dict) -> dict:
    out = {}
    for key in CONTRACT_KEYS:
        if key in job:
            out[key] = job[key] if key != "script" else f"{len(job[key])} lines"
        elif key in defaults:
            out[key] = defaults[key] if key != "script" else f"{len(defaults[key])} lines"
        else:
            out[key] = None
    if isinstance(job.get("variables"), dict):
        out["var_names"] = sorted(job["variables"].keys())
    else:
        out["var_names"] = []
    out["has_rules"] = "rules" in job
    out["has_artifacts"] = "artifacts" in job
    return out


def main() -> None:
    path = Path(sys.argv[1] if len(sys.argv) > 1 else ".gitlab-ci.yml")
    want = sys.argv[2] if len(sys.argv) > 2 else None
    doc = load_doc(path)
    defaults = doc.get("default") if isinstance(doc.get("default"), dict) else {}
    names = job_names(doc)
    if want is None:
        print("jobs:")
        for name in names:
            print(f"  - {name}")
        raise SystemExit(0)
    if want not in names:
        raise SystemExit(f"unknown job {want!r}. known: {names}")
    card = contract_for(doc[want], defaults)
    print(f"job: {want}")
    for key, value in card.items():
        print(f"{key}: {value}")


if __name__ == "__main__":
    main()

Run it like this:

python3 ci_parity_check.py .gitlab-ci.yml
python3 ci_parity_check.py .gitlab-ci.yml unit-py

You now have a contract card. Image. Tags. Needs. Artifact flag. Script length.

Still missing include: expansion? Open CI Lint in GitLab and view the rendered config. The helper is a flashlight. The UI is the source of truth.

Step 3 — Fill three columns. Use yes, no, or n/a.

Check	Local shell	Scratch box	GitLab job log
Same git SHA
Same job name
Same image digest
Runner tags match
rules evaluated
needs artifacts present
Protected var names present
CI_JOB_TOKEN usable for the API you need
JUnit or coverage report uploaded
Pipeline ID recorded

If a scratch-box cell is n/a, that is the finding. Do not upgrade it to yes.

The pad is allowed to be empty. Your merge gate is not.

Step 4 — Write a receipt the MR can link

Commit nothing secret. Keep the file in the MR description if you want.

# Proposed receipt. Copy IDs from GitLab, not from chat.
cat > ci_receipt.txt <<'EOF'
sha:
pipeline_id:
job_id:
job_name:
image:
runner_description:
junit_uploaded: yes/no
protected_ref: yes/no
EOF

Need the IDs from a real job log? They are already in the UI. You can also echo names, not values:

# Inside a real GitLab job only. Names, never secret values.
echo "CI_PIPELINE_ID=$CI_PIPELINE_ID"
echo "CI_JOB_ID=$CI_JOB_ID"
echo "CI_JOB_NAME=$CI_JOB_NAME"
echo "CI_COMMIT_SHA=$CI_COMMIT_SHA"
echo "CI_JOB_IMAGE=$CI_JOB_IMAGE"

Chat logs are not receipts. Pipeline IDs are.

A pad is allowed. A gate is not.

Need a scratch pad for the table? Fine. Draft the empty rows there.

Then stop. Open the real job log. Copy the pipeline ID. Paste only that ID into the MR.

I will not claim timings. I will not name models. I will not invent quotas or hardware.

The pad is optional. The receipt is not.

Limitations

This helper does not expand include:. It does not evaluate rules.

It does not call the GitLab API. Parent-child pipelines will look like missing jobs.

It never proves a job is safe to deploy. It only makes missing contract fields loud.

YAML merge keys and some anchors can still surprise safe_load. When the card looks wrong, trust the rendered CI in GitLab over the script.

Image names without digests are weak. Two tags can point at different bytes a day later.

Who should not use this approach

Do not send .gitlab-ci.yml to any external model if policy forbids it. Redact runner tags and internal image names first.

Do not use a scratch box for protected-ref jobs. You will not get those variables. You might think you did.

Do not replace gitlab-runner on release branches with a demo shell. Compliance artifacts need GitLab's job, not a paste.

If you cannot name the job, this method will not save you. Go read the YAML.

What I want in the next review

Three lines. That is the whole bar.

1. Job name copied from .gitlab-ci.yml
2. Pipeline ID for this SHA
3. The parity table with no empty GitLab column

Skip the chat screenshot. I cannot replay a chat. Can you name the job?

If not, we are not merging. Name it, then we talk.