A Green Agent Session Is Not a Green Pipeline

Did CI pass, or did an agent merely narrate success?
I still watch teams treat those greens as twins.
This FAQ is the wedge I keep between them.

Cheap inference invites extra retries without shame.
A free server invites extra experiments without tickets.
Neither one mints a release badge on its own.

Why a FAQ, not another lucky transcript

I do not need another diary of a passing chat.
I need a promotion rule I can run twice.
Myths survive because the box felt productive.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode provides free model access and a free server option.
I treat both as exploration, never as a pipeline.
If those product lines vanished, this gate would still matter.

Myth 1: The model said tests passed, so CI would pass

The claim. That closing paragraph is already a status check.

What to inspect. Open the workspace. Find the report CI publishes.
Hunt for JUnit XML, TAP output, or a pytest log on disk.
If you only have adjectives, you only have a story.

Corrected model. Narration is not a publisher.
CI consumes files, exit codes, and immutable logs.
Ask for the artifact path before you trust the adjective.

# proposed gate — fail closed when no CI-shaped report exists
set -euo pipefail
report=""
for cand in \
  test-results.xml \
  junit.xml \
  reports/junit.xml \
  .pytest_cache/v/cache/lastfailed; do
  if [ -f "$cand" ]; then
    report="$cand"
    break
  fi
done
if [ -z "$report" ]; then
  echo "no CI-shaped report on disk"
  exit 1
fi
echo "report=$report"

Would your current agent session survive that snippet?
If not, you do not have a green pipeline.
You have a green sentence.

Myth 2: The free server is a pinned baseline

The claim. Whatever ran remotely is the environment of record.

What to inspect. Print the runtime, the lockfile digest, and the kernel.
Compare those three values to whatever CI already pins.
Silent drift here is common, and it is boring until it ships.

# proposed snapshot — do not trust the agent's memory of versions
{
  echo "python=$(python --version 2>&1)"
  echo "node=$(node --version 2>/dev/null || echo missing)"
  command -v git >/dev/null && {
    [ -f package-lock.json ] && echo "npm_lock=$(git hash-object package-lock.json)"
    [ -f pnpm-lock.yaml ] && echo "pnpm_lock=$(git hash-object pnpm-lock.yaml)"
    [ -f poetry.lock ] && echo "poetry_lock=$(git hash-object poetry.lock)"
    [ -f uv.lock ] && echo "uv_lock=$(git hash-object uv.lock)"
  }
  echo "uname=$(uname -s -r)"
} | tee /tmp/env-snapshot.txt

Corrected model. A sandbox is a sample, not a pin.
Pins live in lockfiles, images, and pipeline YAML.
Free boxes may be dirty. Pins may not.

Myth 3: Throwaway boxes do not need lockfiles

The claim. It is scratch compute, so floating versions are fine.

What to inspect. Resolve one dependency graph twice, then diff it.
If the tree moves, your pass was a lottery ticket.
Lottery tickets do not belong on main.

# proposed: refuse floating installs before the agent touches deps
fail=0
if [ -f package.json ]; then
  if [ ! -f package-lock.json ] && [ ! -f pnpm-lock.yaml ] && [ ! -f yarn.lock ]; then
    echo "JS project without a lockfile"
    fail=1
  fi
fi
if [ -f pyproject.toml ]; then
  if [ ! -f poetry.lock ] && [ ! -f uv.lock ] && [ ! -f requirements.txt ]; then
    echo "Python project without a freeze file"
    fail=1
  fi
fi
exit "$fail"

Corrected model. Throwaway compute still needs frozen inputs.
You can delete the box. You cannot un-roll the version dice.
Lockfiles protect you, not the invoice.

Myth 4: One lucky path is a regression suite

The claim. The agent walked the happy path, so coverage exists.

What to inspect. Count tests collected. Count files touched. Count assertions.
A single path through a fat diff is a demo.
Demos are allowed. Suites are earned.

# proposed: quantify the diff before you quote "tests passed"
echo "files_touched=$(git diff --name-only origin/main...HEAD 2>/dev/null | wc -l | tr -d ' ')"
echo "tests_collected=$(python -m pytest --collect-only -q 2>/dev/null | tail -n 1)"
# Still run your real suite. This print is not the suite.

Corrected model. Exploration can be one path.
Promotion needs the suite your repo already agreed on.
Do not let a free model redefine "enough" inside a paragraph.

Myth 5: Destroying the free box leaves the chat as archive

The claim. The transcript is a complete record of the run.

What to inspect. After teardown, what still exists off-box?
Exit codes, env snapshots, and binary logs rarely survive chat export.
If the VM is gone, prose cannot reconstruct the report.

# proposed: copy promotion evidence before you kill the session
mkdir -p ./agent-evidence
cp /tmp/env-snapshot.txt ./agent-evidence/ 2>/dev/null || true
find . -name 'junit.xml' -o -name 'test-results.xml' | while read -r f; do
  cp "$f" ./agent-evidence/
done
git rev-parse HEAD > ./agent-evidence/head.txt
tar -czf agent-evidence.tgz ./agent-evidence
echo "bundle=agent-evidence.tgz"

Corrected model. Ephemeral compute is a feature.
Ephemeral evidence is a bug.
Copy artifacts first. Quote chat second.

Artifact: a closed promotion gate

I do not promote a session. I promote a bundle.
The table below is the rule I want in writing.
It is a proposal, not a measured SLA.

Question	If yes	If no
Is there a CI-shaped report on disk?	Continue	Stop. Chat is not a publisher.
Does a lockfile exist for this ecosystem?	Continue	Stop. You ran a lottery.
Does the env snapshot match CI pins?	Continue	Stop. You sampled a drift box.
Did the agreed suite run, not one path?	Continue	Stop. You have a demo.
Did you copy evidence off the box?	Continue	Stop. Teardown will erase the proof.

Glue those rows into one script. Fail closed.

# proposed evidence_gate.py — unexecuted until you run it locally
from __future__ import annotations

import hashlib
import subprocess
from pathlib import Path

ROOT = Path(".")
REPORT_NAMES = ("test-results.xml", "junit.xml", "reports/junit.xml")
LOCK_NAMES = (
    "package-lock.json",
    "pnpm-lock.yaml",
    "yarn.lock",
    "poetry.lock",
    "uv.lock",
    "requirements.txt",
    "Cargo.lock",
    "go.sum",
)


def sha256(path: Path) -> str:
    return hashlib.sha256(path.read_bytes()).hexdigest()[:16]


def first_existing(names: tuple[str, ...]) -> Path | None:
    for name in names:
        p = ROOT / name
        if p.is_file():
            return p
    return None


def porcelain() -> str:
    r = subprocess.run(
        ["git", "status", "--porcelain"],
        check=False,
        capture_output=True,
        text=True,
    )
    return r.stdout.strip()


def main() -> int:
    errors: list[str] = []
    report = first_existing(REPORT_NAMES)
    lock = first_existing(LOCK_NAMES)
    if report is None:
        errors.append("missing CI-shaped test report")
    if lock is None:
        errors.append("missing lockfile or freeze file")
    dirty = porcelain()
    if dirty:
        errors.append("dirty worktree; reset before quoting a result")
    print(f"report={report}")
    print(f"lock={lock} digest={sha256(lock) if lock else 'n/a'}")
    print(f"dirty={bool(dirty)}")
    for e in errors:
        print(f"FAIL {e}")
    return 1 if errors else 0


if __name__ == "__main__":
    raise SystemExit(main())

Run it after the agent stops talking.
Do not run it instead of CI.
The script only answers, "may I even quote this session?"

python evidence_gate.py
echo "gate_exit=$?"

How I actually use a free box

I let a free model draft the change on a free server.
Then I pull the diff and the evidence bundle locally.
Then CI, the real one, is the only promotion voter.

That split is the whole point.
Exploration can be cheap.
Promotion must stay boring.

1. Freeze inputs with a lockfile before any install.
2. Let the agent work in a throwaway session.
3. Write reports to known paths, not into chat.
4. Snapshot runtime and lockfile digests.
5. Copy the bundle off the box.
6. Reset or destroy the session.
7. Run evidence_gate.py on the copy.
8. Only then open a pipeline on the same commit.

Which step do people skip first?
Step five. They kill the box and keep the paragraph.
That is how a false green becomes folklore.

Limitations, said plainly

This gate does not replace GitHub Actions, GitLab CI, or Jenkins.
It does not pin OS images, secrets, or network policy.
It does not prove tests are good. It proves evidence exists.

The snippets are labeled proposals.
I am not publishing latency numbers, quotas, or hardware claims.
If your repo has no tests, the script will correctly refuse you.

Skip this approach when any of these are true:

• You must place production secrets on a shared free box.
• You have no suite, only a manual click path.
• You need a signed provenance chain for every install.
• You are debugging kernel, GPU, or licensed compilers.
• You want the chat window to be the archive of record.

A free model can still be useful there as a rubber duck.
A free server can still be useful as scratch paper.
Neither should vote on main.

The mental model I want stuck on the wall

A session is a sample.
A pipeline is a contract.
A transcript is a comment.

Comments do not ship.
Contracts do.
Keep cheap retries in the sample lane.

You can run the same gate after a MonkeyCode free-server session; the files either exist or they do not.