FAQ: A Model Patch Is Not a GitLab Review

I keep reading merge requests that tell a bigger story than the diff. Someone asked a model to repair a red job. The patch looks tidy, so the review stops there.

Would you approve a comment that never touched the repo? I would not treat chat prose as a merge. GitLab will not treat it that way either.

This FAQ is the mental model I want before anyone hits merge. Each myth is a claim developers still repeat in reviews. Each fix points at a file or a project setting.

What this FAQ refuses to pretend

I will not invent a model name, a quota, or a hardware bill. I will not invent a session length or a permanence promise. I was not given those facts, so I will not fake them.

The operator supplied two availability facts about MonkeyCode, and only those. Free model access is available, and a free server option is available. Those facts do not make a reviewed pipeline by themselves.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. I would use that access to draft checks, not to approve merges. I would use the free server option as scratch space only.

Myth 1: A tidy YAML patch is a reviewed pipeline

People say the model read the log, so the YAML is reviewed. A fluent diff is still just text sitting in a chat pane. Did anyone open the pipeline editor and compare the stages?

GitLab runs the file in the repository, not the paragraph above it. If the patch never lands on a branch, the project did not change. I want the branch commit, not a screenshot of the suggestion.

Corrected model: a review starts when the YAML is committed and linted. Chat is a draft tray, not the project contract you merge. The committed file is the contract your runners will execute.

Lint the file, not the vibe

I treat schema lint as the first gate, not the model tone. GitLab documents CI Lint for pipeline files in the project. You can also call the lint API with a real project token.

I keep the payload in a file so secrets stay out of history. The host and project id should come from the environment. If lint fails, the model draft is still only a draft.

Read the current CI Lint docs before you trust this curl shape. Endpoints move, and I will not freeze a stale path as law. The snippet below is a proposal, not a measured run.

# Proposal only. Do not paste a real token into any chat.
# Confirm the path in GitLab's current CI Lint API docs.
curl --silent --show-error --fail \
  --header "Content-Type: application/json" \
  --data @ci-lint-payload.json \
  "$GITLAB_HOST/api/v4/projects/$PROJECT_ID/ci/lint"

I inspect the cached diff before I push the scratch branch. A chat paste can hide a line the diff will show. If the diff echoes a token, I stop and rotate it.

git checkout -b ci-evidence-scratch
git add .gitlab-ci.yml
git diff --cached -- .gitlab-ci.yml

Myth 2: "Tests passed" in prose is a test report

I still see job logs pasted under a heading called evidence. A log line can be edited, truncated, or never uploaded. Would you bet a release decision on a single sentence?

GitLab's YAML reference separates job logs from report artifacts. The junit report key is how test summaries reach the merge request. A green echo does not create that report file.

Corrected model: evidence is a file path GitLab can ingest. Prose is narration, and narration can sound finished. The XML can still be missing while the story sounds done.

I set when always so a failed job still uploads the report. I set expire_in myself instead of trusting an unspoken default. I do not cite a default duration, because defaults can move.

# Unexecuted example. Confirm keys in the current CI YAML reference:
# https://docs.gitlab.com/ci/yaml/
unit_tests:
  stage: test
  script:
    - mkdir -p reports
    - python -m pytest --junitxml=reports/junit.xml
  artifacts:
    when: always
    expire_in: 7 days
    paths:
      - reports/junit.xml
    reports:
      junit: reports/junit.xml

I read the job artifacts docs before I trust an upload story. I read the YAML reference before I copy keys from chat. I treat pytest junitxml as a runner detail, not a GitLab feature.

Myth 3: Free model access is a safe vault for CI variables

Someone says the prompt is private, so the token is fine. Is a convenience box really a vault for credentials? I do not grant a chat box that title.

GitLab documents protected variables for protected branches and tags. Confirm the current variables page before you rely on that limit. Masking is a log filter, not a license to print values.

Corrected model: secrets live in GitLab or a real secret store. A prompt is a copy, and copies have a habit of multiplying. I will not trade a red job for a leaked token.

I do not paste CI_JOB_TOKEN into a free model prompt. I do not paste masked variables just to help a debug session. I do not paste customer data so a sample patch looks real.

The free server option does not weaken that copying rule. A scratch shell is still a machine you do not administer. If I cannot name its retention, I will not feed it secrets.

A dotenv file can smuggle what the prompt should not

Models love to emit KEY equals VALUE and call it config. GitLab can import dotenv reports into later jobs. Would you let a later job inherit a guessed secret?

I want dotenv keys to be build metadata, never credentials. A bad dotenv line is not a convenience. It is a second copy with a job scope.

Myth 4: A free server transcript is a merge request artifact

The transcript says the script exited zero on a free server. People attach that transcript and call the pipeline finished. Where is the artifact entry in the job sidebar?

GitLab artifacts are files uploaded by a job the project ran. A foreign shell transcript is not an artifact record in GitLab. Cache is not an artifact, and a cache hit can miss later.

Corrected model: no widget, no report, no argument from me. A transcript is a story about a run you did elsewhere. Stories are not downloads your reviewer can open later.

Myth 5: Model confidence replaces rules, needs, and approvals

I hear that the model checked every branch, so rules are noise. Confidence is not a rules block in your pipeline file. Confidence is not needs, and it is not an approval rule.

allow_failure true can keep a pipeline green while a job failed. needs can start a job without waiting for every earlier stage. Approval rules live in project settings, not in a chat score.

Corrected model: control flow is YAML plus project settings. Tone is not control flow, no matter how certain it sounds. I read the policy files before I trust the adjective.

The order I trust more than a fluent patch

1. Commit the YAML on a branch you can throw away.
2. Run CI Lint against that committed file, not the chat.
3. Produce a junit file from the test command you really run.
4. Run the local evidence check against those files on disk.
5. Push and confirm the merge request shows the test report.

A local check on files, not on vibes

This script is a proposal you can run on a local folder. I have not published timings, pass rates, or hardware notes. It checks files on disk, not a live shared runner.

A pass means the files exist and look roughly shaped right. A pass does not mean GitLab accepted the report upload. I still want CI Lint, then a pipeline on a branch I control.

#!/usr/bin/env bash
# ci-evidence-check.sh - proposal, not a published benchmark.
set -euo pipefail

root="${1:-.}"
yaml="$root/.gitlab-ci.yml"
junit="$root/reports/junit.xml"
dotenv="$root/reports/build.env"

fail() { printf 'FAIL: %s\n' "$1" >&2; exit 1; }

[[ -f "$yaml" ]] || fail "missing .gitlab-ci.yml"
[[ -f "$junit" ]] || fail "missing reports/junit.xml"

# -q so a match does not print a secret into the terminal.
if grep -E -q 'CI_JOB_TOKEN|echo .*(SECRET|TOKEN|PASSWORD)' "$yaml"; then
  fail "yaml looks ready to print a secret; open the file locally"
fi

grep -q 'reports:' "$yaml" || fail "yaml has no artifacts reports block"
grep -q 'junit:' "$yaml" || fail "yaml does not point at a junit report"

# Minimal XML sniff. This is not a full JUnit schema validation.
grep -q '<testsuite' "$junit" || fail "junit xml has no testsuite element"
grep -E -q '</testsuites>|</testsuite>' "$junit" || fail "junit xml looks truncated"

if [[ -f "$dotenv" ]]; then
  while IFS= read -r line; do
    [[ -z "$line" || "$line" == '#'* ]] && continue
    [[ "$line" =~ ^[A-Z_][A-Z0-9_]*= ]] || fail "dotenv line is not KEY=VALUE form"
    case "$line" in
      *TOKEN*|*SECRET*|*PASSWORD*|*PRIVATE*) fail "dotenv looks like a secret" ;;
    esac
  done < "$dotenv"
fi

printf 'PASS: files look like evidence, not like prose\n'

That invocation is a usage sketch, not a recorded lab result. If the script fails, fix the files before you debate tone. If it passes, you still owe GitLab a real pipeline.

mkdir -p reports
bash ci-evidence-check.sh .

The table I keep beside the editor

I keep this table beside the editor when a patch arrives as prose. Each row names the claim, the demand, and the refusal. If a row fails, I do not argue from the model's tone.

Claim I hear	What I demand instead	Why I refuse the shortcut
The model rewrote the job	Committed YAML plus CI Lint	Chat text is not the repo file
It said tests passed	reports/junit.xml on the job	Logs are not report artifacts
The prompt can hold the token	Variables in project settings	A prompt is a copy, not a vault
The free server exited zero	Artifact uploaded by the project job	Foreign transcripts are not downloads
The model felt sure about rules	rules, needs, approval settings	Tone is not control flow
A generated dotenv is harmless	Metadata keys only, no credentials	Later jobs can inherit the file

Who should skip this, and what it cannot prove

Skip this approach if your pipeline has no tests to report. Skip it if you cannot keep secrets out of prompts and shells. Skip it if you need a signed release attestation, not a FAQ.

This check does not validate a full JUnit schema. This check does not prove runner tags, images, or digest pins. Those are separate gates, and I am not repeating them here.

The greps are heuristics, and comments can fool them. Treat a pass as a smell test, not as a security audit. A green local pass is still not a merge approval.

Who should use it: reviewers staring at a fluent CI patch. Use it when the author cites a model patch as the proof. Do not use it as a compliance program or a quota claim.

What I would ask for, and nothing more

I would ask a free model for a checklist and a hostile reading. I would ask it to flag allow_failure and secret echoes. I would not ask it to bless a merge request.

A free server can hold the script while I poke sample files. I would not point that shell at production runners or secrets. I would not store release artifacts on a scratch box.

If scratch space would help, MonkeyCode is the option I would open. Then close that tab and prove the files inside GitLab. I would not let the draft close the review by itself.

So ask the rude question before you click merge. Which file will GitLab ingest when the chat tab is closed? If you cannot name that file, you do not have evidence.