The Model Said Done: A Completion-Evidence Glossary, a Four-Leaf Tree, and a Worked Example at Every Leaf

Friday, 16:40. A Python invoice service. An assistant prints All set — rounding is fixed and a 40-line diff. No test command. No log. The pull request title is already fix tax rounding.

That sentence is a claim, not a gate. Teams lose hours when they treat the claim as the gate. The rest of this article is a small vocabulary, a four-leaf tree, and one worked example at every leaf so a session cannot end on tone.

The workflow below stays useful without any product. A free-model coding assistant and a free server option only appear where they change the method: drafting the missing reproducer, or running a job the laptop cannot finish cleanly.

Glossary (pin these before the next “done”)

Claim of done. The model’s natural-language assertion that work is finished. It is not evidence. Treat it as a pointer to files, not as a merge bit.

Behavioral diff. A change that can alter outputs, persisted state, network calls, or process exit codes. Comment-only and README-only edits are not behavioral diffs. Mixed diffs that include both comments and runtime code are behavioral diffs.

Reproducer. A command a second person can run without the chat transcript. It must name the working directory, the interpreter or package manager, and the exact test selector. pytest -q with no path is not a reproducer.

Red/green pair. Two runs of the same reproducer: failing on the parent revision, passing on the candidate revision. One green run on the candidate alone does not prove the test can fail.

Local gate. A red/green pair that finishes on a developer laptop with deterministic fixtures. No extra hosts. No long queues.

Off-laptop gate. A red/green pair that needs a longer job, a shared fixture, or an isolated machine. The laptop may still author the job. It does not count as the gate until a log comes back.

Session report. A tiny JSON document the human (or a script) writes when the assistant stops. It records leaf, command, exit code, and log path. If the report is missing, the session is unfinished by definition.

The four-leaf tree

Walk the questions in order. Do not skip to a favorite leaf because the diff looks small.

1. Does the candidate contain a behavioral diff?
2. If yes: is there a reproducer with a red/green pair?
3. If yes: can that pair finish as a local gate?

Leaf	Path	Merge bit
A	No behavioral diff	Allowed after a human read of the full diff
B	Behavioral diff, no red/green pair	Blocked
C	Behavioral diff, local red/green pair	Allowed after the log is stored
D	Behavioral diff, off-laptop red/green pair	Allowed after the remote log is stored

Leaf B is the common failure mode of vibe-shaped sessions. The model is fluent. The repository is not safer.

Artifact: a done-gate classifier you can run

The script below is a method, not a production metric. Save it as done_gate.py. It reads a session report and prints a leaf plus a merge bit. It does not execute tests. It only refuses to call a missing field “done.”

#!/usr/bin/env python3
"""Classify a coding-assistant session report. Unexecuted until you feed it JSON."""
from __future__ import annotations

import json
import sys
from pathlib import Path

REQUIRED = ("behavioral_diff", "reproducer", "red_on_parent", "green_on_candidate", "local_gate")


def classify(report: dict) -> tuple[str, bool, str]:
    missing = [k for k in REQUIRED if k not in report]
    if missing:
        return "INVALID", False, f"missing fields: {missing}"

    if not report["behavioral_diff"]:
        return "A", True, "docs/comments only; human diff-read is the gate"

    has_pair = bool(report["reproducer"]) and report["red_on_parent"] and report["green_on_candidate"]
    if not has_pair:
        return "B", False, "behavioral diff without a red/green reproducer"

    if report["local_gate"]:
        return "C", True, "local red/green pair; keep the log next to the PR"
    return "D", True, "off-laptop red/green pair; keep the remote log next to the PR"


def main() -> int:
    if len(sys.argv) != 2:
        print("usage: python done_gate.py session_report.json", file=sys.stderr)
        return 2
    report = json.loads(Path(sys.argv[1]).read_text())
    leaf, merge, reason = classify(report)
    print(json.dumps({"leaf": leaf, "merge": merge, "reason": reason}, indent=2))
    return 0 if merge or leaf == "B" else 1


if __name__ == "__main__":
    raise SystemExit(main())

A Leaf B fixture looks like this. Save it as session_b.json and run the classifier before you argue about the patch.

{
  "behavioral_diff": true,
  "reproducer": "",
  "red_on_parent": false,
  "green_on_candidate": false,
  "local_gate": false
}

python done_gate.py session_b.json

Expected stdout: leaf is B, merge is false. That is the whole point of the gate. Fluency in the chat window does not flip the bit.

Worked example at every leaf

The same fictional unit is used on all four leaves: tax.py in an invoice service. Rounding of a 10% tax on cents. No company names. No claimed production outage.

# tax.py — starting point for every leaf
def tax_cents(amount_cents: int, rate_bp: int = 1000) -> int:
    """rate_bp is basis points; 1000 == 10%."""
    return (amount_cents * rate_bp) // 10_000

Leaf A — no behavioral diff

The assistant rewrites the docstring and adds a comment about basis points. tax_cents is byte-identical. The session report sets behavioral_diff to false.

Human gate: read the full diff, including files the model did not mention. Merge is allowed. Do not spend a server job on comments. Do not ask a model to “verify” a docstring with a generated essay. The artifact is the diff itself.

Leaf B — behavioral diff, no red/green pair

The assistant changes integer division to a float round because “money should round half up.” It prints “done.” There is no test file. There is no command.

# proposed change — do not merge on this claim
def tax_cents(amount_cents: int, rate_bp: int = 1000) -> int:
    return round(amount_cents * rate_bp / 10_000)

This is Leaf B. Stop. The missing object is the reproducer, not another prompt that says “please be careful.”

A free-model session is useful here as a drafting tool for the test, not as a witness. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode is an open-source coding assistant with free model access and a free server option; those two availability facts are the only product claims used below. No model names, quotas, or hardware are assumed.

Ask the free model for a test that fails on the parent and would pass only if the new rounding is intended. Then you run it. The model does not get to mark red_on_parent.

# test_tax.py — proposed oracle, still unproven until you run it
from tax import tax_cents

def test_ten_percent_on_1999_cents():
    # 1999 * 0.10 = 199.9 cents; integer vs half-up disagree
    assert tax_cents(1999, 1000) == 200

git stash push -u -m candidate
python -m pytest -q test_tax.py
# record exit code on parent
git stash pop
python -m pytest -q test_tax.py
# record exit code on candidate

If you cannot get a red parent run, you do not have an oracle. Stay on Leaf B. Rewrite the assertion until the parent fails for the reason you care about.

Leaf C — local red/green pair

The assertion above fails on parent (199 vs 200) and passes on the candidate. The command finishes in seconds. Fixtures are in-process. That is a local gate.

Session report:

{
  "behavioral_diff": true,
  "reproducer": "python -m pytest -q test_tax.py",
  "red_on_parent": true,
  "green_on_candidate": true,
  "local_gate": true,
  "log_path": "artifacts/tax-local.txt"
}

python done_gate.py session_c.json
mkdir -p artifacts
python -m pytest -q test_tax.py | tee artifacts/tax-local.txt

Merge bit is true only after the log exists. Paste the command, both exit codes, and the log path into the PR. The chat transcript is optional. The log is not.

Leaf D — off-laptop red/green pair

Same rounding change, but the real risk is not tax_cents in isolation. Invoices are scored in a batch job that reads a fixture of 50k rows and writes a CSV the finance script diffs. The laptop thermal-throttles. The job is still deterministic. It is not a local gate.

Do not lower the standard to “the unit test passed, ship the batch.” Move the gate, not the claim. Author the job on the laptop. Run it where it can finish. A free server option is the relevant tool when the machine that authored the patch is the wrong machine to witness the batch.

# author locally — labeled as a job spec, not as a completed gate
cat > job_tax_batch.sh <<'EOF'
#!/usr/bin/env bash
set -euo pipefail
python -m pytest -q test_tax.py
python batch_invoices.py --in fixtures/invoices_50k.json --out /tmp/out.csv
diff -u fixtures/invoices_50k.golden.csv /tmp/out.csv
EOF
chmod +x job_tax_batch.sh

Run the same script on the free server, capture stdout/stderr, and store artifacts/tax-remote.txt. The session report flips local_gate to false and still requires both red and green. done_gate.py returns Leaf D. Merge is allowed only with that remote log. If the server run cannot be reproduced from job_tax_batch.sh, you are back on Leaf B with extra latency.

How to walk a live session (numbered)

1. Freeze the parent revision with git rev-parse HEAD.
2. Accept patches only as files, never as “applied in the air.”
3. Fill session_report.json before you type “looks good.”
4. Run python done_gate.py session_report.json.
5. If the leaf is B, spend the next prompt on the reproducer, not on a second implementation.
6. If the leaf is C, tee the local log into artifacts/.
7. If the leaf is D, submit the job spec to the free server and refuse merge until the remote log matches the spec.
8. Attach leaf, command, and log path to the PR. Stop talking about vibes.

What this does not decide

The tree does not choose a rounding rule. Half-up versus integer truncation is a product decision. The tree only refuses to call that decision “done” without a command.

It also does not replace CI. A free server job is a witness for one candidate. It is not a protected branch, a required check, or a security boundary. Secrets, production data, and non-deterministic flaky suites do not belong on a shared free host. If the change is safety-critical, regulated, or irreversible, this method is too small. Use the real pipeline.

Free model access does not make Leaf B safer. It only makes it cheaper to draft test_tax.py. If the model writes a test that never fails on parent, you have a green that means nothing. That failure mode is silent. The red/green pair is the countermeasure.

Limitations, and who should skip this

Skip the tree if your repository already blocks merge on required checks and those checks already encode the red/green pair. You do not need a second ritual.

Skip the free server path if the job needs credentials, customer fixtures, or a GPU story this article does not have. Nothing here states model identity, token ceilings, uptime, or hardware. Those numbers go stale. Commands do not.

Skip first-person “it worked in prod” stories that you cannot back with a log. This article does not claim one. The invoice function is a fixture. Your service will differ. The leaf names stay the same.

The assistant will keep saying done. That is its job. Your job is to name the leaf. If you want a drafting surface for Leaf B tests or a place to run a Leaf D job spec, MonkeyCode’s free model access and free server option are one way to host those two steps — after the glossary, not instead of it.