FAQ: A Suggested Job Is Not a Pipeline Contract

I keep seeing one shortcut in merge requests.
Someone pastes a red job into chat.
The reply looks like a finished YAML file.

Did the model actually see your runner config?
Did it see any protected variables at all?
I want a contract, not a polished vibe.

The shortcut I keep rejecting

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

A teammate asks a free model to repair CI.
They run one command on a free server.
The shell exits zero, so they merge.

Free model access can draft a patch quickly.
A free server can run one smoke command.
Neither option is your actual GitLab project.

I am not claiming a quota or a hardware spec.
I have not timed any run for this note.
Treat these checks as a method, not a scoreboard.

Myth 1: The model understood the job graph

The claim

People say the model fully understood the pipeline.
It saw only the snippet you pasted in.
A snippet is not the resolved job graph.

The evidence

GitLab expands includes before those jobs are scheduled.
A chat window does not perform that merge.
Those missing files become a silent wrong config.

A needs key can skip stages you forgot.
A rules block can hide the job you need.
A parent pipeline can pass variables you never pasted.

The correction

So what did the model actually verify here?
Text shape, mostly, and one confident tone.
That is a proposal, not a resolved graph.

Correct that mental model before you hit merge.
A chat rewrite is not project CI lint.
Project CI lint still has to run before merge.

# Proposal only. Not a verified pipeline.
lint-contract:
  stage: test
  script:
    - python3 ci_contract_check.py .gitlab-ci.yml
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"

That sketch shows where a check could live.
Your real image may still be a floating tag.
Pin a digest before you trust the job twice.

Myth 2: A free-server zero proves the script

The claim

I hear this after a quick remote shell.
It compiled there, so the job is green.
Does that zero match the job script list?

The evidence

Each script item is a command the job will run.
A single pasted blob is not that command list.
A pipe can hide a failing test status.

Your poke may use different shell flags entirely.
Bash fail-fast is not a promise on every shell.
Exit zero can still skip the artifact path.

The correction

A zero exit proves that command, there, once.
It does not prove an artifact upload happened.
It does not prove the next job can download.

Did you compare script lines, not one blob?
Did the failure mode match the job shell?
If not, you proved a demo, not the job.

Myth 3: The patch kept rules and variables

The claim

Free models love to clean up noisy YAML.
They drop rules when the sample looks messy.
They invent variable names your project never set.

The evidence

Protected variables are not sitting in the chat.
Protection and scope live in project settings, not prompts.
A free server will not inject them for you.

Masked values are not a safe teaching example.
Environment scope can hide a variable from you.
A name in YAML is not a value in the vault.

The correction

Would you paste a masked token just to check?
I would not, and you should not either.
Ask for assumptions, then verify them in GitLab.

# Local static poke. It does not call GitLab.
python3 ci_contract_check.py .gitlab-ci.yml

If the checker warns, read the warning first.
Do not ask the model to silence the line.
Silence is how a weak check reaches main.

Myth 4: One green test covers needs

The claim

You ran the test suite on the free server.
The job that publishes coverage never actually ran.
The downstream job never received those files either.

The evidence

Those artifact paths form a contract between jobs.
Needs and dependencies decide who receives the files.
A single shell has no artifact coordinator at all.

An expiry setting can drop files before the next stage.
An empty dependencies list can block a download.
None of that runs inside one borrowed shell.

The correction

Can one green command stand in for the graph?
No, you should split command success from graph success.
Smoke on a free server, then lint on GitLab.

Myth 5: Free inputs stay fixed next week

The claim

People treat today's free box as a stable pin.
They paste the same prompt again next week.
They expect the same image, model, and result.

The evidence

Free access can change without a notice you saw.
A free server image can drift overnight too.
I will not pretend either option is permanent.

A tag can move to a new manifest tomorrow.
A digest names one manifest you can record.
Your lockfile is the other pin you own.

The correction

Use the free path to draft and to poke.
Record the file, the command, and open assumptions.
Re-run the contract on GitLab before merge.

A chat transcript is not a pin you own.
Store the patch in git, not in scrollback.
Scrollback disappears, and the config should not.

A fixture that should fail

I like a tiny fixture before I touch a real repo.
Two jobs, one missing artifact, one eager need.
The checker should nag, not bless this file.

stages: [test, report]
unit:
  stage: test
  image: golang:1.22
  script:
    - go test ./...
report:
  stage: report
  needs:
    - unit
  script:
    - echo $COVERAGE_FILE

Do you see the hole in that fixture?
The unit job never publishes an artifact path.
The report job still expects a coverage file.

A free server can run go test and exit zero.
That result never creates the missing coverage file.
The graph fails later, after you already smiled.

Add an artifacts block, then rerun the checker.
You still need project CI lint after that poke.
Those strings are a hint, and lint is a gate.

The contract poke

This script is a proposal, not a scored lab run.
I am not claiming it parsed your include tree.
Copy it, then point it at a fixture you control.

#!/usr/bin/env python3
"""Static hints for a GitLab CI file. Not CI lint."""
import sys
from pathlib import Path

MARKERS = ("script:", "rules:", "artifacts:")

def main() -> int:
    path = Path(sys.argv[1] if len(sys.argv) > 1 else ".gitlab-ci.yml")
    text = path.read_text(encoding="utf-8")
    problems = []
    for marker in MARKERS:
        if marker not in text:
            problems.append("missing " + marker)
    if "image:" in text and "@sha256:" not in text:
        problems.append("image present but no digest pin")
    watch = {"allow_failure: true", "when: manual"}
    for line in text.splitlines():
        stripped = line.strip()
        if stripped in watch:
            problems.append("review " + stripped)
    if problems:
        print("contract poke failed:")
        for item in problems:
            print("- " + item)
        return 1
    print("markers present; job graph still unverified")
    return 0

if __name__ == "__main__":
    raise SystemExit(main())

What does a clean print actually prove here?
It only shows that those marker strings exist.
It does not prove GitLab accepted the file.

The required markers are a house heuristic, not GitLab law.
A valid job can omit rules and still run.
Use the flags as questions, not as doctrine.

A missing digest line is a review flag.
It is not a full registry policy decision.
Pin the digest in the job you intend to rerun.

What I will grant

I keep a small table beside every suggested patch.
The middle column is the only claim I grant.
The right column stays open until GitLab runs.

Claim you heard	What I will grant	What stays open
The model fixed CI	It proposed text	Includes, rules, CI lint
Free server went green	That command exited zero	Shell flags, artifacts, needs
Variables looked fine	Names appeared in the snippet	Protection, scope, and mask
The image was current	A tag string was written	Digest pin and registry policy
Artifacts will flow	A path was typed in YAML	Upload, expiry, and needs

Read the middle column before you start celebrating.
The right column is the real merge bar.
Skip that column, and you shipped a story.

Questions before I trust a patch

1. Which include files were absent from the prompt?
2. Which rules changed, and who reviewed that diff?
3. Which image tag is still floating this week?
4. Which variables are protected, masked, or environment scoped?
5. Which jobs upload files the next stage needs?
6. Which command ran, and on which exact box?
7. Which assumption did the model admit it guessed?

If you cannot answer those, you lack a contract.
You have a suggestion with a confident tone.
Suggestions are fine, but unreviewed merges are not.

How I use the free options

I paste the failing job, not the secret file.
I ask for a patch plus a list of assumptions.
I run the checker as a syntax poke only.

Then I stop treating the chat as evidence.
Project CI lint is the next real step.
A merge request pipeline has to come after that.

Would I paste a job token into the chat?
Would I mount production env files on that box?
A free box is a scratch pad, not a vault.

Who should skip this method

Skip this if includes pull untrusted remote files.
Skip this if protected variables boot the job.
Skip this if a bad artifact can ship code.

This method fits draft review, not release evidence.
It is a bad fit for secret-bearing jobs.
It will not replace your runner contract either.

Limits I will not blur

The checker is string matching, not a YAML parser.
A file can hold the strings and still fail.
Extends and YAML anchors can hide the real script.

Comments can trip a marker and look present.
A false pass here is still an unverified graph.
Do not wire this script in as your only gate.

I do not know your runner tags at all.
I do not know your include graph either.
Do not treat these examples as your config.

Free model text can be fluent and false.
Free server logs can be green and local.
Fluent output is still not a verified pipeline.

Need a second look at the draft text?
A free MonkeyCode model pass can mark those assumptions.
Keep the merge on GitLab lint and a real pipeline.