DEV Community: Sam Chen

Your Model Graded Its Own Homework

Sam Chen — Sun, 11 Oct 2026 21:23:57 +0000

A green log is not a finished patch. I start from that rule every single time. If the generator also wrote the test, you graded nothing.

I keep a short catalog for free model drafts. Each entry lists a symptom, a cause, and a replacement. I do not need a new slogan to spot the miss.

What I refuse to trust

Free model access can draft a unified diff. A free server can run that diff one time. Neither option is allowed to define the expected behavior.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Do you want a draft, or do you want a verdict? I want the draft, and I keep the verdict. I use those two options as a rehearsal bench only.

I do not treat a chat reply as a specification. I do not treat a remote exit code as proof. A friendly log can still hide a fake oracle.

Anti-pattern 1: the model writes the oracle

Symptom

The new tests mention the same names as the patch. They pass on the first remote run you try. You feel finished, and that feeling is the trap.

Root cause

One generator filled both sides of the check. The test encodes the implementation you just accepted. It cannot catch a wrong idea with a pretty shape.

Replacement

I write two fixture files before I ask for any code. One file holds the input object I care about. The other file holds the output I will accept.

The model may not edit either fixture file. Could a stranger fail this test without your patch? If the answer is no, your oracle is fake.

Write a stricter fixture, then ask the model again. Then rerun the compare before you trust the color. These commands are an unexecuted example, not a lab report.

mkdir -p rehearsal/contract
printf '%s\n' '{"n": 4}' > rehearsal/contract/input.json
printf '%s\n' '{"ok": true, "double": 8}' > rehearsal/contract/expect.json
test -s rehearsal/contract/expect.json && echo "fixture ready"

Change the expected numbers to match your real rule. Do not let the model pick them after it sees the diff. Pair the files with a short note you wrote yourself.

# Behavior note

Input n is an integer from 0 to 20.
Return ok true only when n is even.
Return double as n times 2 when ok, else 0.

That note is the spec the model is allowed to see. If the note and the fixture disagree, fix the note first. Do not ask the model to reconcile your own contradiction.

Anti-pattern 2: fixtures arrive after the diff

Symptom

The patch lands in the branch before any example. The sample numbers show up later in the same chat. Those numbers match the code you already wanted.

Root cause

You let the result choose the requirement after the fact. Time order is the actual bug, not the syntax. A contract should be older than the candidate patch.

Replacement

I record file times before I send any prompt. The gate fails when the fixture is newer than the patch. That rule feels strict, and it should feel strict.

stat -c '%Y %n' rehearsal/contract/expect.json rehearsal/patch.diff

On macOS I switch to stat -f '%m %N' for the same check. I label this snippet as a local proposal, not a benchmark. Which file did you create first, the fixture or the diff?

If you cannot tell, you do not have a contract yet. Stop the prompt and write the fixture in git first. A chat bubble is not a timestamp you can audit.

Anti-pattern 3: the free box becomes the spec

Symptom

The script runs on the free server and looks fine. The same script fails on your laptop five minutes later. The missing import was never written into your lockfile.

Root cause

An unnamed runtime quietly became your real design. Locale, timezone, and preinstalled packages hid the gap. You then copied host notes into the repo as requirements.

Replacement

I pin the command, the workdir, and the expected files. The remote host may execute the candidate one time. It may not invent requirements I did not write down.

Check	Free server may do	You must own
Run the candidate once	Yes	The command text
Print stdout and stderr	Yes	The fixture files
Add a missing package	No	Your lockfile
Decide pass or fail	No	Your local compare
Store a secret or token	No	Your redaction step

If a package exists only on that host, I stop. I add it locally, or I drop the idea entirely. Does your lockfile explain every import the log used?

Anti-pattern 4: you skip the deleted lines

Symptom

The viewer paints the new lines in a friendly green. A helper function vanished near the bottom of the file. The new test never called the path you just lost.

Root cause

Your eyes follow additions because additions look like work. Generated patches delete quietly while you read the headline. Addition bias is how bounds checks disappear unnoticed.

Replacement

I count removed lines before I read any new line. A deletion without a contract note is an automatic reject. I want a human sentence that explains each removal.

git diff --numstat -- candidate.py
git diff --unified=0 -- candidate.py | grep '^-' | grep -v '^---'

Why did that line leave the file at all? If you cannot answer in one plain sentence, stop. Leave the branch dirty until the note exists.

Anti-pattern 5: comments pretend to be a fix

Symptom

The diff is long, but behavior barely moved at all. New comments restate the old function in softer words. The free server still prints the same wrong value.

Root cause

Activity got mistaken for progress during a fast review. A verbose patch feels safer than a small sharp edit. The generator optimized for looking busy, not the fixture.

Replacement

I reject patches whose comment lines dominate the behavior lines. Docs belong in the notes file when I asked for docs. Otherwise I want a smaller diff that moves the output.

Did the output change, or did the narration change? Grade the fixture, not the paragraph under the function. A longer file is not a stronger fix.

Run this gate after the remote log comes home

This script is a proposal you can read today. I am not publishing timing numbers or success rates. It checks shape, notes, and a plain JSON compare.

It does not deploy, install, or call a network. Treat every command below as unexecuted sample code. Adapt paths if your repo layout differs from this example.

#!/usr/bin/env python3
"""Local contract gate. Proposal, not a measured benchmark."""
import json
import sys
from pathlib import Path

ROOT = Path("rehearsal")
NEED = ("input.json", "expect.json", "notes.md")

def fail(msg: str) -> None:
    print(f"REJECT {msg}")
    sys.exit(2)

def main() -> None:
    box = ROOT / "contract"
    if not box.is_dir():
        fail("missing contract dir")
    for name in NEED:
        if not (box / name).is_file():
            fail(f"missing {name}")
    expect = json.loads((box / "expect.json").read_text())
    got_path = ROOT / "server_out.json"
    if not got_path.is_file():
        fail("missing server output")
    got = json.loads(got_path.read_text())
    if got != expect:
        fail("output != fixture")
    notes = (box / "notes.md").read_text().strip()
    if len(notes) < 40:
        fail("notes too thin")
    print("PASS local contract compare")

if __name__ == "__main__":
    main()

Copy the remote stdout into rehearsal/server_out.json before the run. Then invoke the gate and read the exit code yourself. A non-zero exit means reject, not a warmer prompt.

python3 rehearsal/gate_patch.py
echo "exit=$?"

A pass means your output matched the fixture you wrote. A pass does not mean the patch is wise or safe. A pass does not mean the remote host was clean.

Preflight before the prompt

I run four checks before I ask for any draft.

Confirm both fixture files exist and are non-empty.
Confirm notes.md states the behavior in plain words.
Scan the prompt text for keys before you paste.
Send the contract, not your whole repository tree.

grep -nE '(AKIA|BEGIN PRIVATE KEY|password\s*=)' prompt.txt && echo "stop"

That sniff is a seatbelt, not a full secret scanner. If it prints stop, I rewrite the prompt without the secret. I never paste a live environment file to save time.

Where a free model and a free server fit

I ask for a patch against notes.md, not against a vibe. I send no secrets and no environment file with the ask. I run the candidate once on the free server option.

I bring stdout home and compare it on my machine. MonkeyCode is the bench this rehearsal is written around. Free model access drafts the patch from the notes I wrote.

The free server option runs that draft away from my laptop. I still own the contract, the fixture, and the reject. Remove the product name and the gate still stands.

I am not stating a token quota in this article. I am not naming a model identifier or a hardware size. I am not promising how long any free offer stays up.

Check the live offer before you plan a week around it. Terms move, and stale numbers become confident lies. A missing quota line is safer than a fabricated one.

Is a fake quota more helpful than an honest gap? No, because a fake number schedules work you cannot finish. I would rather send you to the current page.

Who should not use this approach

Skip this if you cannot read a diff line by line. Skip this if the change touches auth, payments, or private data. Skip this if you need a certified or reproducible release build.

A free remote host is a rehearsal room, not a release gate. Skip this if your fixture says only "make it better." That sentence is a mood, not a contract you can test.

Write the input, write the output, then ask for code. Teams with no human reviewer should not adopt this flow. The script can reject a mismatch, not a bad product idea.

Judgment stays with a person who can own the merge. If that person is absent, do not automate the yes. Wait until a reviewer can read the deletions.

What I keep on the next pass

I keep four questions on a card beside the keyboard.

Who wrote the oracle that this run claims to satisfy?
Which file is older, the fixture or the candidate patch?
What did the host add that my lockfile does not list?
Which lines disappeared, and where is the note?

If any answer is mush, I stop the merge right there. The draft can stay on the branch until the note exists. The verdict stays with me, not with the remote log.

If you already keep fixtures in git, rehearse this once. Use a free model pass only to draft against your notes. Use one free server run only to collect the remote log.

Then let the local script be the only grade you trust. That is the whole offer I am willing to make today. No countdown, no ranking claim, and no borrowed story.

The Free Host Said Yes. Your Fence Card Is Still Blank.

Sam Chen — Sat, 10 Oct 2026 17:48:38 +0000

A free coding host is not a production fence. I treat a trial login as a risk, not as a vault. That shortcut turns into debt before the first merge.

Do you want the short version before the long catalog? Write the fence card on your laptop before any send. If that card fails, the free box never sees the prompt.

This piece is an anti-pattern catalog, not a product tour. Each item names a symptom, a root cause, and a replacement. I offer the checker as a proposal, not as a measured benchmark.

What the card is for

You have a coding task that a free model can draft. A free server can hold the scratch step off your laptop. That pair is useful, but it is not a vault or an SLA.

I keep the exploratory step behind a fence card. The card does not trust a happy status page or a long chat. Would you merge a patch you cannot explain tomorrow?

Anti-pattern 1: One shared trial key

The symptom is one free key shared across every chat tool. The root cause is a trial login used as a team identity. The replacement is a task id bound to one prompt file.

Shared keys hide who sent the prompt in the first place. They also hide which client the pasted text belonged to. Can you name the owner of yesterday's session right now?

Store the key in a local secret store, never in the repo. Rotate it when a teammate leaves the scratch workflow. A key in a dotfile will leak the moment you share a zip.

Anti-pattern 2: The prompt becomes the spec

The symptom is a requirement that lives only in a chat scroll. The root cause is speed, because the answer made the thread into law. The replacement is a short task file checked in before any call.

A chat scroll is not a reviewable spec for the next person. A task file is small, diffable, and deliberately boring to read. I want the next reviewer to open a file, not a vibe.

Put the goal, the non-goals, and the stop rule in that file. If you cannot write the stop rule, you are not ready. Why would a remote host know your stop rule better than you?

Anti-pattern 3: Two clients, one scratch disk

The symptom is two client notes sharing one server workspace. The root cause is reuse of a disk that stayed dirty. The replacement is one task, one empty workspace, and a wipe note.

Leftover files are not harmless clutter on a scratch disk. They become the next prompt's context by accident. Would you leave two customers' notes in one unlocked drawer?

Name the workspace after the task id before you copy files in. Refuse the run when a foreign task id is still on disk. Wipe is part of the task, not a chore you might remember later.

Anti-pattern 4: Trial host as a CI runner

The symptom is a pull request job starting on the trial host. The root cause is spare-capacity thinking, and free looked like owned CI. The replacement keeps CI on a runner you administer yourself.

Untrusted pull request code plus a shared host is a bad mix. I cannot cite an isolation promise for a trial box in this draft. If you need a hardened runner, this path is the wrong path.

Do not forward webhooks at the free host to save a bill. A saved bill is a poor trade for an unreviewed remote shell. Who owns the blast radius when that shell runs a stranger's script?

Anti-pattern 5: Pins with no local resolver

The symptom is a generated lockfile committed without a local resolve. The root cause is fluency, so a finished-looking file skipped the resolver. The replacement regenerates pins with your own package tool.

A fluent lockfile can still name a package you never chose. Your resolver is the check, and the model output is only a draft. Did you run the install on a machine you actually control?

Diff the regenerated pins against the model draft before you commit. If the diff is huge, stop and read the package names aloud. A huge pin diff is a question, not a gift.

Anti-pattern 6: A pool limit dressed up as a bug

The symptom is a mid-file death that you treat as a code bug. The root cause is a missing local budget file. A pool limit then looks like a logic crash.

The replacement writes a cap beside the task and stops on it. I will not invent a token count, a duration, or a machine size. Those numbers move, and a stale blog figure becomes a trap.

Check the vendor page on the same day you plan the run. Write the cap as a plain integer you are willing to spend. When the run hits the cap, save state and stop the session.

Guessing that the pool is infinite is how scratch work eats the day. A false crash can burn an afternoon that should have been a stop. Do you want a budget line, or another hour of fake debugging?

The replacement pattern

The replacement pattern is a fence card, not a longer prompt. You fill seven lines in a local card file. A script refuses the send when any line is blank.

The free model and the free server stay behind that refusal. Here is the card I want sitting next to the task file. Treat it as a proposal you can adopt, not as a lab result.

The card protects the next reviewer from a chat-only spec.
The card protects a second client from a dirty scratch disk.
The card protects your CI from a trial host you do not run.

task_id=2026-10-10-fence-01
workspace=empty
clients=one
secrets_in_prompt=no
ci_target=owned-runner
budget_cap=budget.txt
wipe_after=yes

Seven lines keep the rule obvious to a reviewer. A blank line is a failed review, not a minor nit. If a line is blank, you are not ready for a remote draft.

Run the checker before the send

Fail closed on your laptop

The script below is unexecuted example code for the local check. It reads the card file and the task file only. It does not open a network connection of any kind.

This parser reads KEY=value lines and does not source the card. It is not a full secret scanner, so do not pretend otherwise. I am not shipping this as a tested release.

#!/usr/bin/env bash
# Proposal only. Fail closed before any free-host prompt.
set -euo pipefail

card="${1:-fence-card.env}"
prompt="${2:-task.md}"

test -f "$card" || { echo "missing card"; exit 2; }
test -f "$prompt" || { echo "missing task file"; exit 2; }

get() {
  key="$1"
  line=$(grep -E "^${key}=" "$card" | head -n 1 || true)
  test -n "$line" || { echo "missing $key"; exit 2; }
  printf '%s' "${line#*=}"
}

task_id=$(get task_id)
workspace=$(get workspace)
clients=$(get clients)
secrets_in_prompt=$(get secrets_in_prompt)
ci_target=$(get ci_target)
budget_cap=$(get budget_cap)
wipe_after=$(get wipe_after)

test "$workspace" = "empty" || { echo "workspace must start empty"; exit 3; }
test "$clients" = "one" || { echo "one client per scratch task"; exit 3; }
test "$secrets_in_prompt" = "no" || { echo "prompt must stay free of secrets"; exit 3; }
test "$ci_target" = "owned-runner" || { echo "do not aim CI at the trial host"; exit 3; }
test -f "$budget_cap" || { echo "budget cap file missing"; exit 3; }
test "$wipe_after" = "yes" || { echo "plan the wipe before the run"; exit 3; }

if grep -E -q 'AKIA[0-9A-Z]{16}|-----BEGIN |api[_-]?key[[:space:]]*[:=]' "$prompt"; then
  echo "prompt looks like it carries a secret marker"
  exit 4
fi

mkdir -p .fence
printf '%s\n' "task_id=$task_id" "card=$card" "prompt=$prompt" > ".fence/${task_id}.decision"
echo "fence card passed for $task_id"

Run the checker like this before you open a remote session. Create a one-line budget file before you expect a pass. A missing budget file is a failed card, not a warning.

printf '%s\n' "set-your-cap" > budget.txt
chmod +x fence-card.sh
./fence-card.sh fence-card.env task.md
echo "exit=$?"

A non-zero exit means you stay on the laptop and fix the card. A zero exit means the card is complete, not that the code is correct. You still review the diff on a machine you trust.

Where the free step belongs

I send the free step only after the checker prints a pass. The model may draft, and the server may run one scratch command. The laptop still owns the spec, the pins, and the final diff.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode, for this draft, is relevant only after the card passes. The operator-supplied note is free model access plus a free server option. The operator also describes the project as open source.

I am not linking a repository, because no primary URL was supplied here. I am not stating a quota, a chip type, a time box, or permanence. Read the current project docs before you plan around either option.

If those docs disagree with this paragraph, trust the docs. A blog post is a workflow note, not a contract or a status page. Would you put a client export on a host you cannot describe?

That pair can hold an exploratory draft and a scratch command. It should not hold customer data, production deploys, or shared CI. Silence in the docs is a stop sign, not a blank check.

Six questions before the send

Use these six checks when you decide whether the send is allowed. One failed check blocks the send, even if the other five pass. I would rather lose a draft than mix two clients on one disk.

Stay local when the task file is not already in git.
Wipe first when the workspace does not start empty.
Stop now when the prompt shows a secret marker.
Do not retarget CI onto a host you do not administer.
Do not start when the budget file is missing from disk.
Do not start when you cannot wipe the workspace afterward.

These checks are a habit, not a certification of the remote host. A printed list still needs a human who will honor a failed row.

What the card does not prove

Leave the numbers on today's docs

The grep line is a tripwire, not a certified secret scanner. A clever paste can still sneak past a short pattern list. Use a real scanner when your org already runs one.

The card does not prove isolation on a remote host. It does not prove uptime for the remote host. It does not prove that a zero price will last.

Free access can change, so keep a local fallback ready. I did not run a benchmark for this script. I did not time a model, and I did not measure a server.

The checker only tests that the budget file exists. It does not parse the integer, so you still have to honor it. A file full of jokes is a pass, and that is a real limit.

If you need numbers, measure your own task and date the note. Paste the date next to the figure so the next reader sees the age. An undated metric is how old limits sneak into new posts.

Who should skip this path

Skip this path when the data is regulated or under a client contract. Skip it when you need a stated service level or a fixed machine shape. Skip it when the team will not keep the card inside review.

Also skip it if you cannot wipe the scratch workspace after the task. A free server you cannot reset is a shared disk with a login page. Are you comfortable reading that disk out loud in a retro?

A five-step local test

Drop a fake key marker into the task file and confirm exit code four.
Set the workspace value to dirty and confirm exit code three.
Remove the budget file and confirm the script stops before a pass.
Pass a clean card and confirm the decision file appears.
Keep that file next to the task so review can see the refusal trail.

Do not point this test at production, because the pass is only local. That plan is enough to trust the gate you actually have. It is not enough to trust a remote host you have not read.

Read the host policy on the same day you decide to use it. If the policy is silent on isolation, do not invent a promise. A quiet policy page is not the same thing as a guarantee.

Close the loop on your laptop

Free model access can speed a draft you already scoped. A free server can hold scratch work after the card passes. Neither one replaces the card, the task file, or the wipe plan.

I start with the card, and the remote step stays optional after that. If you already use that free model access, run this checker first. Then read today's limits in the project docs before you plan load.

Keep the final merge on a repository you control and can revert. A remote draft is only a guest in your workflow. Your repo is the host, and the host sets the rules.

The Transcript Looked Done. You Still Have No Receipt.

Sam Chen — Fri, 09 Oct 2026 14:16:09 +0000

A clean transcript is not a review record. I want a local receipt before any free session touches my branch. If that receipt is empty, I do not share the work.

Does a confident last reply count as proof? No, I keep seeing summaries outrun the diff. The chat can sound finished while the tree is still a guess.

The hole I am naming

I am not repeating my older stop-loop note. I am not repeating the allowlist note either. This catalog is about the transcript you keep after the model goes quiet.

A transcript is only a story of tokens. A review record is a set of files you can rerun. I mix those two views when I am tired.

Do you mix those two views as well? Yes, I still catch it in late reviews. A receipt is how I catch that mix.

Free model access makes another turn feel cheap. A free server makes a scratch run feel disposable. Cheap and disposable are not the same as checked.

I still need the receipt on my laptop. No news page was attached to this draft. I will not invent a headline, a score, or a quota.

If a number is missing here, that is on purpose. Stale product figures can waste a whole sprint. I would rather show the check than a stale number.

What sits in the receipt

I keep four files next to the diff. They are plain text, and they are boring. Boring is what I want on a Friday merge.

prompt.txt holds the exact text I am willing to send.
claim-ledger.txt maps each claim to a file and a command.
replay.txt stores the command, the exit line, and a log hash.
receipt.sha256 names those files so a later edit is obvious.

None of these files makes the model correct. They make my shortcut visible to a reviewer. That is the whole point of this catalog.

Anti-pattern 1: The summary replaces the diff

Symptom

The pull request text is the model's closing paragraph. Reviewers reply to tone, and nobody names a line. The bug can hide under a smooth ending.

Root cause

The chat is ordered by time, and the diff is not. I let the easier view win the review. Have you merged a paragraph you never opened?

Replacement

I write the ledger before I open the request. Each claim points at one path and one check. If I cannot point, I delete the claim.

claim: empty names are rejected
file: src/parser.py
check: python -m unittest tests.test_parser

Ask yourself this before you paste a summary. Can a teammate run the check without opening the chat? If the answer is no, the summary is still a story.

Anti-pattern 2: Price becomes a privacy control

Symptom

A key, a customer id, or a raw trace sits in the prompt. The word free made that paste feel small. A small paste can still become a large leak.

Root cause

I treated a billing label as a safety boundary. A billing label does not redact my files. Who told me that price was a control?

Nobody told me that, and I just hoped. Hope does not redact a prompt file either. I still have to run the search myself.

Replacement

I run a loud search before I send anything. This is not a full secret scanner at all. It catches the obvious leaks I keep repeating.

# proposal - unexecuted in this article
rg -n -i 'api[_-]?key|secret|password|BEGIN ' prompt.txt

A hit means I stop and rewrite the sample. I use fake values instead of live ones. I never send a live token into a free model box.

Someone with a mandated scanner should keep that scanner. Do not swap a policy tool for one search line. This pattern is a tripwire, not a program of record.

Anti-pattern 3: The model invents a missing field

Symptom

A tool blob lacks a key the patch still uses. The chat then fills that gap in prose. The code branches on a field nobody returned.

Root cause

I trusted the shape I hoped to see. I did not pin the shape I actually got. Hope is not a schema I can ship.

Replacement

I record observed keys, and I leave gaps empty. I do not ask the model to invent fields. A missing key means stop, then a fresh command.

# proposal, not a captured production trace
tool: repo.status
observed_keys: [branch, dirty]
missing_keys: [ahead]
action: stop

If the action line says stop, I do not merge. I fetch status with a command I can read. Then I edit that status note by hand.

Anti-pattern 4: Comments stand in for a test

Symptom

The diff is mostly comments and renamed variables. The claim says bugfix, but the test hash did not move. The chat still calls that change a fix.

Root cause

Comments feel like progress inside a long chat. Tests feel like friction, so the feeling wins. I refuse that trade on any branch I share.

Replacement

A bugfix claim must name a test path. Same hash as yesterday means the claim is void. I either write the test or I drop the label.

# proposal - unexecuted here
sha256sum tests/test_parser.py
git diff --stat -- tests/test_parser.py

Which of those two did I actually finish? The saved hash will answer that question cleanly. A feeling in the chat will not answer it.

Anti-pattern 5: Yesterday's log becomes the prompt

Symptom

I resend the full chat so the next turn has context. The prompt then holds dead ends and reversed decisions. Old secrets can ride back in with that log.

Root cause

I used the provider as my working notebook. The provider is not my system of record. A free model will not remember my correction for me.

Replacement

I keep a twenty-line digest instead of the novel. The digest lists the decision and the rejected option. I hash it, and I send the digest only.

decision: reject empty names in parser
rejected: coerce empty names to guest
ledger: claim-ledger.txt
stop: do not add a network call

Would I want that digest in the repo later? If I would not, I should not send the longer log. The longer log is how old mistakes return.

The order I follow

I follow the same order on every branch. I do not reorder it because a session feels friendly. A friendly session is still not a receipt.

I write prompt.txt with fake sample values only.
I write the claim ledger before I ask for a patch.
I run the redaction search and stop on any hit.
I ask for a patch only after those files exist.
I save the command and the exit line in the replay file.
I run the receipt script and read every bad exit.
I read the diff myself before I share the branch.

That order is the replacement pattern for this catalog. Each step leaves a file or an exit code. If a step has neither, I am still in the anti-pattern.

The check I run before a share

Here is the bundle I keep beside the branch. It is a proposal, not a timed benchmark. Do not quote this script as a measured result.

#!/usr/bin/env bash
# receipt-check.sh - local check before share
set -euo pipefail

need() {
  test -s "$1" || exit 2
}

need prompt.txt
need claim-ledger.txt
need replay.txt

if rg -n -i 'api[_-]?key|secret|password|BEGIN ' prompt.txt; then
  printf '%s\n' 'redaction hit'
  exit 3
fi

if rg -n '^claim:.*bugfix' claim-ledger.txt >/dev/null; then
  if ! rg -n '^test:' claim-ledger.txt >/dev/null; then
    printf '%s\n' 'bugfix claim without test line'
    exit 4
  fi
fi

sha256sum prompt.txt claim-ledger.txt replay.txt > receipt.sha256
printf '%s\n' 'receipt ok'

I invoke it from the worktree, not from memory. A bad exit means I stay local and fix the file. I do not negotiate with that exit code in chat.

bash receipt-check.sh

Exit 2 means a required file is missing or empty. Exit 3 means the prompt search hit a secret-like string. Exit 4 means a bugfix claim has no test line.

I fix the file that failed, then I run the script again. I do not negotiate with the exit code in chat. A model apology is not an exit code.

How I read the exits

Exit	Meaning	Next move
0	Crude checks passed on the three files	Read the diff once more, then share
2	A required file is missing or empty	Write the file before any session
3	Prompt search hit a secret-like string	Strip it and use a fake sample
4	Bugfix claim without a test line	Add a test path or drop the claim

This table is a rule for my own branches. It is not a service level from any vendor. If your team needs signatures, this table is too small.

Where the free options fit

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

I mention MonkeyCode once, as one optional workbench. It offers free model access and a free server option. I do not name a model in this draft.

I do not state a token quota or a duration. An outreach note mentioned a free token pool. I will not repeat an unverified figure here.

No primary page was attached for me to check. Stale quotas are how teams plan the wrong week. Confirm the current docs yourself before you depend on them.

The workflow stays on my laptop until the receipt check passes. I write the prompt file, then I run the script. Only then do I ask a free model for a patch.

A free server can run the command stored in the replay file. I still commit the receipt next to the diff. The session log does not become the record.

If your team already hashes prompts, try this check on one branch. Tell me which table row feels too strict. That is the only ask I will make here.

Who should not use this

Do not use this script as a compliance archive. It has no signature, no clock sync, and no reviewer identity. A regulated repo needs a heavier trail than this.

Do not use the check to hide a diff you cannot read. The script checks that files exist and look non-empty. It does not understand your parser or your threat model.

Do not point the search at live production secrets. Use a fake sample when you test the pattern. I mean that as a hard stop, not a tip.

Skip the free server when the job needs private paths. Customer data and long-lived credentials stay off that box. Keep those runs on a machine you already trust.

The free option is for scratch work you can lose. If losing the run would hurt, do not start it there. A missing receipt is cheaper than a leaked trace.

Limits I am willing to say out loud

The search pattern misses secrets that look like normal words. The ledger can still be filled with confident lies. I can hash a bad prompt and still be wrong.

The check raises the cost of skipping a review step. It does not make me honest by itself. I still have to read the diff with my own eyes.

I did not execute these commands against a live product. Treat every block as a proposal until you run it. Verify current product terms before you plan a sprint.

Terms move, and my files should still make sense. That is the bar I am willing to keep. If the bar feels low, add a signature and keep the files.

The Free Loop Kept Going. I Never Set a Stop.

Sam Chen — Thu, 08 Oct 2026 09:53:58 +0000

A free model reply is not a release decision. A free server log is not an audit trail either. I still want a local fence before any agent runs.

That fence sits on the branch I plan to merge. I do not trust a scratch host to invent it later. If the stop is missing, the green line is theater.

This is a composite review rule, not a measured outage report. I am not timing a cluster for this post. I am writing the stop I want in git.

Does a green free run mean the patch is safe to merge? It does not, and I want that said first. It only means some host answered a prompt I sent.

Why free access keeps lying to me

Free model access feels like permission to wander. I add another file because the call looks cheap. I retry a vague prompt and call the result research.

Vibe-coded patches are filling review queues this season. The model often sounds sure of the fix. The run record is still empty of stops.

Would I ban a free server for scratch work? No. I would ban unbounded use of that server. A scratch host can help, then it must stop.

The debt is not the model voice. The debt is the missing fence around retries, context, secrets, and model identity. Those four failures stack quietly.

Anti-pattern 1: Retry until something looks green

Symptoms

The same file shows three overlapping edits.
Logs repeat one tool name without a new goal.
A flaky check passes only on the final try.

Root cause

I treated free calls as an endless budget. The price felt like zero, so I never set a max. The loop then chased a green line, not a reviewable diff.

Replacement

Cap attempts in the runner before the first call. Exit when the cap hits. Do not ask the model whether it feels done.

# Proposal only. This cap is local policy, not a benchmark.
MAX_ATTEMPTS = 2

def allow_attempt(n: int) -> None:
    if n < 0:
        raise SystemExit("attempt fence: bad counter")
    if n >= MAX_ATTEMPTS:
        raise SystemExit("attempt fence: stop and read the diff")

Two is my personal policy, not a law of nature. You can pick three if your suite is slow. Write the number down before you prompt.

What should the runner do after that exit? Leave the worktree dirty and visible to me. Do not auto-commit, and do not open a pull request.

If I raise the cap mid-run, I have restarted the anti-pattern. The new number belongs in a new commit. Reviewers should see the change of mind.

Anti-pattern 2: Upload the world because the call is free

Symptoms

Answers cite paths I deleted last week.
Latency jumps without a real code change.
Two files in the bundle disagree on one type.

Root cause

I skipped a context budget and called it thorough. Free access is not a design for relevance. More files often add noise, not truth.

Replacement

Build a short manifest, then count bytes on disk. Refuse the send when the local cap breaks. Send a map of files, not the whole world.

# Unexecuted teaching example. Tune the byte cap locally.
find src tests -type f \( -name '*.py' -o -name '*.md' \) \
  | sort > /tmp/fence-manifest.txt
wc -c $(cat /tmp/fence-manifest.txt) \
  | awk '$1 > 80000 { print "bundle fence"; exit 2 }'

Eighty thousand bytes is a classroom number, not a vendor quota. I want that refusal on my laptop first. The free server should never see a rejected bundle.

Should tests enter the bundle by default? Sometimes yes, when they explain intent. Fixtures with live customer rows should not. I keep a deny file beside the manifest.

# deny.txt — paths that never leave the machine
.env
.env.*
secrets/
fixtures/live/

The deny file is plain text on purpose. I can diff it like any other review. A hidden client toggle would fail that simple test.

Anti-pattern 3: Treat the free session as a vault

Symptoms

A key shape shows up in a shared transcript.
A teammate can replay the prompt from logs.
Rotation starts in chat instead of the secret store.

Root cause

I confused a free server option with a private enclave. Convenience is not custody of my keys. The host may log prompts for its own reasons.

Replacement

Scan the outbound bundle for obvious secret shapes. Block the upload on the first match. Then read the bundle anyway, because scanners miss novel formats.

import re
import sys

PATTERNS = (
    r"AKIA[0-9A-Z]{16}",
    r"-----BEGIN ",
    r"(?i)api[_-]?key\s*=",
    r"(?i)secret[_-]?key\s*=",
)

def main() -> None:
    blob = open(sys.argv[1], encoding="utf-8", errors="replace").read()
    for pattern in PATTERNS:
        if re.search(pattern, blob):
            raise SystemExit("redaction fence: secret shape in bundle")

if __name__ == "__main__":
    main()

Is this a full data-loss program? It is not, and I will not pretend. It catches the paste mistakes I worry about most. New token formats will still slip through.

I also refuse to send shell history to any scratch host. History is full of one-off exports. Those lines look harmless until a key is sitting there.

Who owns the deny file on a team? A named reviewer, not the agent. If nobody owns redaction, do not use this flow at all.

Anti-pattern 4: Swap the model when the free path blips

Symptoms

Monday notes name one model id.
Wednesday's patch reads like another writer.
Style checks fail while behavior checks still pass.

Root cause

I treated free model access as a stable contract. The client was allowed to pick whatever was up. I never stored a pin in the run record.

Replacement

Require a model id in a local policy file. If the server cannot echo that id, stop the job. Do not improvise a cousin model to save the demo.

# run-fence.yml — local policy, not a published quota
model_id: "set-by-operator"
max_attempts: 2
max_bundle_bytes: 80000
fail_closed: true

I will not invent a model name for you in this note. Pin the id your current docs actually list. If that id disappears, fail closed and reread the docs.

How do I check the echo without a stale URL in print? I keep the HTTP path out of this article. A copied endpoint rots faster than the rule. Fill the path from the page you trust today.

def echo_matches(pin: str, echoed: str) -> bool:
    if not pin or pin == "set-by-operator":
        raise SystemExit("model fence: pin is still a placeholder")
    return pin.strip() == echoed.strip()

A placeholder pin must never count as a match. That guard looks petty until review day. A surprise substitute is a different experiment, so stop.

The local fence, step by step

I run this as a preflight, before any scratch host is called. Each step is local. Each failure exits nonzero. None of these steps ask the model for permission.

Read run-fence.yml and reject a placeholder model id.
Build the manifest, then subtract every path in deny.txt.
Fail if bundle bytes exceed max_bundle_bytes.
Run the secret scan and stop on the first hit.
Increment the attempt counter and stop at max_attempts.
Call the scratch host only after those checks pass.
Compare the echoed model id with the pin, then stop on drift.
Write a tiny report file and leave the merge decision to me.

# Proposal. I have not executed this as a published bench.
def ready_to_review(report: dict) -> bool:
    return (
        report["attempts"] <= report["max_attempts"]
        and report["secret_scan"] == "pass"
        and report["bundle_bytes"] <= report["max_bundle_bytes"]
        and report["model_echo"] == report["model_pin"]
    )

If ready_to_review is false, I do not open the pull request. I fix the fence or the bundle first. I do not ask the model to talk me out of a red check.

Where does the report live? Next to the diff, in the branch, as a small text file. Chat scrollback is not the record. A future reviewer cannot see my closed tab.

Where a free server is allowed to help

MonkeyCode is one host I can point at this fence. It offers free model access and a free server option for a scratch run.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

I am not stating a token quota, a chip type, or a duration. I am not stating that the free option is permanent. I have not published latency numbers here. If the current docs disagree with this paragraph, believe the docs.

What do I send that host? Only a bundle that passed the deny list and the secret scan. What do I require back? An echoed model id that matches my pin. What do I do on a mismatch? I stop, then I pick a pin the docs still list.

# Sketch only. Set FENCE_MODEL from run-fence.yml yourself.
test -n "$FENCE_MODEL" || { echo "missing pin"; exit 2; }
test "$FENCE_MODEL" != "set-by-operator" || exit 2
echo "local fence ready pin=$FENCE_MODEL"

Would I call this path a production deploy? I would not. It is a scratch path with brakes on it. Production still needs your normal CI, review, and secret store.

Question	I require locally	A free server may do
Who sets the stop?	The fence file in git	Nothing, until I upload
Where do secrets live?	The secret store only	Nowhere in the prompt
Which model ran?	The pinned id	Echo that id or fail
What does green mean?	Tests I trust already ran	A sample, not a merge vote
What if the host blips?	Fail closed	No silent model swap

Can the free server be my only CI? No, and I do not want that shortcut. Can it be a first pass before CI? Yes, if the fence file is in the same commit. Reviewers should see the policy with the patch.

Who should skip this approach

Skip this if you need a signed availability contract today. Free access is an option, not an SLA I can hand an auditor. I will not dress it up as one.

Skip this if production keys live in the repo and nobody owns redaction. A sample regex is not your compliance boundary. Fix ownership before you automate uploads.

Skip this if you need a fair public benchmark. I gave no hardware notes and no latency table. Anyone inventing those numbers is guessing, including me.

Use it if you maintain a small service and want a cheaper scratch loop. Use it if reviewers already read diffs and you only need a hard stop. The fence is the product of this workflow, not the host.

What I refuse to count as proof

A calm paragraph from the model is not proof. A timestamp on a free server is not proof. A green badge from an unpinned model is not proof either.

Proof, for this rule, is boring and local. Attempts stayed under the cap I wrote down. The secret scan exited clean on the bundle. The echoed model id matched the pin I set.

Then I still read the diff myself. The fence only earns a review, not a merge. If I skip that read, I have rebuilt the original anti-pattern with extra files.

If you want a scratch host for that first pass, start from the current MonkeyCode docs for free model access and the free server option. Pin an id your docs name today. I would rather hear what the fence blocked than hear that the chat felt smart.

Bring a Kill Card or Leave the Scratch Host Alone

Sam Chen — Thu, 08 Oct 2026 07:41:19 +0000

A free server is a scratch lane, not a release gate. Free model access is a budgeted probe, not a blank check. I keep both ideas in one fence file.

Do you keep a fence, or only a chat log? What dies first when the free box disappears tonight?

The point I will not soften

Speed is useless if the session cannot be killed. A green page on a free box proves almost nothing. It does not prove secrets stayed off the box.

It does not prove your laptop can replay the change. Are you debugging the app, or a host that vanished?

What I am willing to claim

I want a scratch lane when the question is still small. Disclosure: This article was prepared as part of MonkeyCode's product outreach. That brief names free model access and a free server option.

The same brief calls the project open source. I still verify the repo license before I depend on it.

I do not have a live quota sheet in front of me. I will not invent model names, caps, or uptime. Check the project docs before you plan a demo.

Docs move, and a blog number goes stale fast. A stale cap in a post is worse than no cap. Would you bet a demo on a number I refused to invent?

How this catalog is built

Each item below has a symptom, a cause, and a fix. I want you to steal the fix, not the slogan. Would you merge a patch you cannot replay tomorrow?

Anti-pattern: no step meter

Symptom: the agent edits files long after the task ends. You look up and the branch is a junk drawer.

Root cause: the prompt never named a maximum step count. Free model access makes that omission feel cheap. Cheap is not the same thing as safe.

Replacement: put max_steps in a card before you prompt. I stop the run when the card hits that number. Do you stop, or do you hope the model gets bored?

Anti-pattern: secrets hitch a ride

Symptom: a dotenv file sits in the folder you upload. The free server boots, and the key boots with it.

Root cause: you treated a scratch box like a private vault. A free option does not change secret handling rules.

Replacement: deny env files and key paths in the card. I refuse the session if those paths exist in the tree. Can you name every secret that left your laptop today?

Anti-pattern: the box never dies

Symptom: an old probe URL still answers a week later. You forgot which task created that public endpoint.

Root cause: the session card never required a teardown step. Free servers still leave traces if you skip cleanup.

Replacement: teardown equals required, or the checker fails. I write the kill command next to the start command. If I cannot kill it, I do not start it.

Anti-pattern: lockfile drift

Symptom: tests pass on the hosted box and fail at home. The import error names a version you do not run.

Root cause: nobody diffed the box image against the lockfile. A free server image is not your source of truth.

Replacement: record the lockfile hash in the session card. I compare that hash before I trust a green log. Which hash did your last green run actually use?

Anti-pattern: live rows in the prompt

Symptom: the prompt includes a real customer email or token. The model reply quotes that row back at you.

Root cause: you used live data because fake data felt slow. Free model access is still a data exit.

Replacement: synthetic fixtures only, checked into a tiny folder. I paste a fixture id, never a production record. Would you read that prompt aloud in a team channel?

Anti-pattern: the scratch host hardens

Symptom: a frontend constant points at the free server host. A teammate clones the repo and hits your old probe.

Root cause: the demo URL never had an expiry note. Replacement: keep the host in an untracked local file.

I add that filename to gitignore before the first commit. Did your last commit contain a host you do not own?

The four moves I make instead

I want one card, one checker, and one local receipt. The card names the task, the cap, and the deny list. The checker fails closed when a required key is absent.

The receipt is a local file, not a chat scrollback. This checker is a proposal I have not executed live. Label it as a sample, then run it on a toy tree.

I write the fence card before I touch the prompt box.
I run the checker and I keep the red output.
I probe on the free server only after a local accept.
I tear the box down and I keep the local receipt.

Sample card and checker

The format below is mine, not a vendor schema. Change the keys if you must, but keep the fail-closed rule.

# session.fence — proposal format, not a product schema
task_id=probe-184
max_steps=12
max_minutes=25
teardown=required
network=off
lock_hash=unset
deny_glob=.env
deny_glob=*.pem
allowed_path=src/parser

#!/usr/bin/env python3
# Proposal checker. Not executed against a live host.
import argparse
import pathlib
import sys

REQUIRED = ('task_id', 'max_steps', 'max_minutes', 'teardown', 'network')

def parse_card(text):
    card = {'deny_glob': [], 'allowed_path': []}
    for raw in text.splitlines():
        line = raw.split('#', 1)[0].strip()
        if not line or '=' not in line:
            continue
        key, val = [part.strip() for part in line.split('=', 1)]
        if key in ('deny_glob', 'allowed_path'):
            card[key].append(val)
        else:
            card[key] = val
    return card

def main():
    parser = argparse.ArgumentParser()
    parser.add_argument('--card', required=True)
    parser.add_argument('--root', default='.')
    args = parser.parse_args()
    root = pathlib.Path(args.root)
    card = parse_card(pathlib.Path(args.card).read_text(encoding='utf-8'))
    missing = [key for key in REQUIRED if not card.get(key)]
    if missing:
        print('fence reject: missing ' + ','.join(missing))
        return 2
    if card.get('teardown') != 'required':
        print('fence reject: teardown must be required')
        return 2
    if card.get('network') != 'off':
        print('fence reject: network must be off for this probe')
        return 2
    try:
        steps = int(card['max_steps'])
        minutes = int(card['max_minutes'])
    except ValueError:
        print('fence reject: caps must be integers')
        return 2
    if steps < 1 or steps > 20 or minutes < 1 or minutes > 40:
        print('fence reject: caps outside the probe window')
        return 2
    hits = []
    for pattern in card['deny_glob']:
        hits.extend(path for path in root.glob(pattern) if path.is_file())
    if hits:
        print('fence reject: denied files present')
        for hit in hits:
            print(hit)
        return 3
    receipt = root / 'tmp' / 'fence-receipt.txt'
    receipt.parent.mkdir(parents=True, exist_ok=True)
    body = 'task_id={}\nmax_steps={}\nlock_hash={}\n'.format(
        card['task_id'], steps, card.get('lock_hash', 'unset')
    )
    receipt.write_text(body, encoding='utf-8')
    print('fence accept: ' + card['task_id'])
    return 0

if __name__ == '__main__':
    sys.exit(main())

Local policy, not a product promise

The numeric caps in code are my policy, not a vendor limit. I picked a small window so a probe cannot wander all day. You should pick your own window, then keep it fail closed.

The receipt stores the task id, the step cap, and the lock hash. It does not store prompts, secrets, or customer rows.

I create the card before I open any chat window. I run the checker from the repo root, not from memory.

mkdir -p scripts tmp
printf '%s\n' 'tmp/' 'session.local.host' >> .gitignore
python3 scripts/fence_check.py --card session.fence --root .
sha256sum package-lock.json | awk '{print $1}'

I store the receipt under tmp, and I commit nothing there. I delete the scratch host before I close the laptop.

A reject on missing teardown is a success, not a bug. A reject on a pem file is the whole point. Do not weaken the card to get a green line.

A table I use before I upload

Situation	Free server probe	Laptop only
Toy bug, synthetic input, teardown written	Allow	Also fine
Dotenv, pem, or a customer row nearby	Deny	Required
You need the run tomorrow, or an audit trail	Deny	Required
Lockfile hash on the host is unknown	Deny	Required
Step cap and kill command already written	Allow	Also fine

I allow a free server only in the probe column. I deny it for secrets, production data, and long jobs. If a row says deny, I stay on my laptop.

Why would you rent risk for a task you can run locally?

Toy test plan

Seed a toy tree with a fake dotenv file. Expect exit code 3, and expect the path printed.

Delete the teardown line and expect exit code 2. Set network to on and expect a hard reject.

Use a clean tree and expect a receipt under tmp. Confirm that receipt path is listed in gitignore.

I treat a failed check as a blocked session, not a warning. I do not continue in chat while the checker is red. Have you ever talked a red gate into silence?

Limits I will not hide

A glob deny list is not a full secret scanner. The checker cannot see steps inside a remote agent. It cannot prove the free server stayed up.

It cannot prove a vendor quota will remain tomorrow. Hash notes do not replace a real lockfile install. Dotfile matches depend on the Python you actually run.

This card does not replace diff review or test ownership. I assume a free server can vanish without notice. Keep a laptop copy, or you do not have a copy.

Who should skip this lane

Skip it if you need an audit letter for a customer. Skip it if the data is regulated or personal. Skip it if the only copy of the work lives remotely.

Skip it if you cannot delete the box the same day. A laptop-only repro is slower, and that is fine. I would rather be slow than stuck with a mystery host.

One next step

Read current MonkeyCode docs before you trust a free lane. Then run a fence card before the first prompt. I would rather you bounce off the checker than leak a key.

The Scratch Server Went Green. You Still Need a Local Gate.

Sam Chen — Wed, 23 Sep 2026 16:45:32 +0000

A free agent box is not your merge queue.
You still need a local, boring, repeatable gate.
Would you ship code from a laptop you do not own?

That is the whole article, stated up front.
The rest is a catalog of ways teams skip it.
I wrote the checks as commands you can run.

Why this keeps breaking

Cheap inference changed the cost of a first draft.
It did not change who owns the blast radius.
Did your policy file follow the model, or stay in chat?

I keep six failures on a short review list.
Each failure looks like speed on a busy afternoon.
Each failure still ships as untracked engineering debt.

Anti-pattern 1: Remote green, local skip

Symptom

The scratch server printed a passing test line.
The pull request skips CI because the agent ran tests.
Main now trusts a machine you do not control.

Root cause

People confuse a demo host with a trusted runner.
A free server is a sandbox, not an attesting builder.
Green output is not a signed provenance record.

Replacement

Replay every agent patch on your own runner.
Fail the merge when local tests never executed.
Keep the remote box for drafts, not releases.

#!/usr/bin/env bash
# local_gate.sh — labeled recipe, not a CI platform.
set -euo pipefail

if [[ ! -f .agent/origin.json ]]; then
  echo "missing .agent/origin.json" >&2
  exit 1
fi

replayed="$(jq -r '.replayed_locally' .agent/origin.json)"
if [[ "${replayed}" != "true" ]]; then
  echo "patch was not replayed locally" >&2
  exit 1
fi

if [[ "${CI:-}" != "true" ]]; then
  echo "refusing to bless a merge outside CI" >&2
  exit 1
fi

git diff --check
pytest -q

Did those agent tests even use your lockfile?
If you cannot answer, you cannot merge yet.

Anti-pattern 2: Secrets on a scratch host

Symptom

Someone pasted a dotenv file into the agent thread.
The free server now holds a production token.
Rotation starts only after the screenshot already leaked.

Root cause

Convenience still beats threat modeling under time pressure.
A shared scratch box is not your secret store.
Free compute does not include any free confidentiality.

Replacement

Redact first, then inject secrets only in local CI.
I keep a dumb scanner in the pre-push path.
It looks ugly, and it is enough for drafts.

# secret_scan.py — labeled example, not a security product.
from __future__ import annotations

import re
import sys
from pathlib import Path

PATTERNS = [
    re.compile(r"AKIA[0-9A-Z]{16}"),
    re.compile(r"-----BEGIN (RSA |OPENSSH )?PRIVATE KEY-----"),
    re.compile(r"(?i)(api[_-]?key|secret)\s*=\s*\S+"),
]


def main(paths: list[str]) -> int:
    failed = False
    for raw in paths:
        path = Path(raw)
        if not path.is_file():
            continue
        text = path.read_text(errors="ignore")
        for pat in PATTERNS:
            if pat.search(text):
                print(f"secret-like match in {path}")
                failed = True
    return 1 if failed else 0


if __name__ == "__main__":
    sys.exit(main(sys.argv[1:]))

# recipe: scan only staged files
git diff --name-only --cached -z | xargs -0 -r python secret_scan.py

Would you paste that token into a hallway laptop?
Then do not paste it into a free agent server.

Anti-pattern 3: Prompt-as-policy

Symptom

The system prompt says never touch production files.
The agent still edits the deploy workflow anyway.
Nobody encoded the rule as a failing check.

Root cause

Natural language feels like a real control plane.
Models drift over sessions, while files stay put.
A policy that cannot fail CI is a wish.

Replacement

Put deny paths in an executable policy file.
I like a tiny allow and deny list in-repo.
The agent may read it, but CI must enforce it.

# .agent/policy.yml — proposal format
deny_globs:
  - ".github/workflows/**"
  - "infra/prod/**"
  - "**/*.env"
  - "**/credentials.json"
allow_globs:
  - "src/**"
  - "tests/**"
  - "docs/**"

# policy_check.py — unexecuted example
from fnmatch import fnmatch
from pathlib import Path
import subprocess
import sys
import yaml

policy = yaml.safe_load(Path(".agent/policy.yml").read_text())
diff = subprocess.check_output(
    ["git", "diff", "--name-only", "origin/main...HEAD"],
    text=True,
).splitlines()

blocked = False
for path in diff:
    if any(fnmatch(path, glob) for glob in policy["deny_globs"]):
        print(f"denied path: {path}")
        blocked = True

sys.exit(1 if blocked else 0)

If the check is optional, the anti-pattern remains.
Make it blocking, then argue exceptions in the PR.

Anti-pattern 4: Session soup

Symptom

One long agent thread spans three private repositories.
Context from repo A leaks into a patch for B.
The summary sounds confident, but the imports are wrong.

Root cause

Sessions are cheap, and isolation is not the default.
A free server often outlives your actual attention.
Yesterday's stack trace becomes today's invented client API.

Replacement

Use one repo, one worktree, and one short session.
Destroy the thread when the branch finally has a name.
Record the origin file before you close the tab.

{
  "generated_at": "2026-09-23T12:00:00Z",
  "repo": "payments-api",
  "base_sha": "REPLACE_WITH_LOCAL_SHA",
  "session": "scratch-only",
  "replayed_locally": false
}

Set replayed_locally true only after local CI passes.
Anything else is still a draft, not a candidate.

Anti-pattern 5: Prod-shaped data on a demo box

Symptom

Someone says the model needs a realistic payload.
A customer export then lands on the scratch server.
The model is free, but that dataset is not.

Root cause

Realism gets confused with actual processing permission here.
Free inference still does not grant data-processing rights.
A redacted fixture is slower, and it is legal.

Replacement

Build a tiny fixture set inside the repository.
I would rather ship boring JSON than a real dump.
Name the file fake, and then keep it fake.

{
  "order_id": "ord_test_001",
  "amount_cents": 1999,
  "region": "lab",
  "email": "user@example.test"
}

# recipe: refuse prod-shaped filenames in the diff
if git diff --name-only | grep -E '(customers|pii|prod-dump)'; then
  echo "prod-shaped filename in the diff" >&2
  exit 1
fi

Do you have a deletion ticket for that upload?
If not, you already lost the data conversation.

Anti-pattern 6: No local replay of the patch

Symptom

The patch arrived as a blob from the chat.
Nobody can regenerate it from a recorded prompt.
Reviewers argue with a screenshot, not a command.

Root cause

Generation starts to feel like real authorship too quickly.
Authorship without a replay path is just folklore.
Folklore does not bisect when production later breaks.

Replacement

Export a patch file, then apply it locally.
Run the same tests your CI will run tomorrow.
If apply fails, the remote box lied about the tree.

#!/usr/bin/env bash
# replay.sh — labeled recipe
set -euo pipefail

base="$(jq -r '.base_sha' .agent/origin.json)"
git checkout -B "replay/${USER}" "${base}"
git apply --check /tmp/agent.patch
git apply /tmp/agent.patch

tmp="$(mktemp)"
jq '.replayed_locally = true' .agent/origin.json > "${tmp}"
mv "${tmp}" .agent/origin.json

pytest -q

Can a stranger reproduce this without the original chat?
If they cannot, you do not have a change.
You only have a vibe from a closed tab.

A decision table I actually use

I print this table near the merge button.
I still miss a row when the review is rushed.
The table is the review, not the model output.

Signal	Merge	Why
Remote tests only	No	Host is not your runner
Secrets in the thread	No	Rotate first, then rewrite
Policy only in the prompt	No	Encode a failing check
Mixed-repo session	No	Split and regenerate
Real customer payload	No	Replace with fixtures
Local apply and CI green	Yes	Now it is your patch

Where a free model and free server fit

I still want a scratch pad for ugly first drafts.
Throwaway refactors do not deserve a paid cluster.
They also do not deserve a silent path to main.

MonkeyCode offers free model access and a free server option.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
I treat that option as a disposable editor, nothing more.

The workflow stays boring on purpose for a reason.
Drafts can be remote without becoming merge artifacts.
Merges cannot be remote without a local replay.

Draft on the scratch server with fake fixtures only.
Export a patch file and a small origin file.
Scan for secrets, then enforce the deny globs.
Apply the patch on a clean local worktree.
Run CI, and only then open the pull request.

Skip step one if your code cannot leave the building.
Skip none of the later steps if it can.

Limitations, loudly

This catalog will not make a model honest.
It only makes a dishonest merge much harder.
That is the actual job of these gates.

Do not use a free scratch server in these cases.

You handle regulated data, even as supposed context.
Long-lived credentials still sit in the working tree.
No local runner matches the production environment closely.
Your license forbids sending source to another host.
You cannot explain the change without the chat.

I also will not claim tokens, uptime, or model names.
Those numbers go stale by the next product page.
Trust the commands you can rerun after lunch.

What I do tomorrow morning

Clone the repo, apply the patch, scan, then test.
If a step fails, the agent did not finish.
The chat window is not a merge witness.

Want a disposable box for that first ugly draft?
Use a free server, then run the local gate at home.

The Tool Returned 200. You Still Have No Allowlist.

Sam Chen — Tue, 22 Sep 2026 15:02:25 +0000

A successful tool call is not a reviewed change.
Cheap inference only makes those side effects cheaper.
Do you gate the tool, or trust the chat?

I keep watching agents celebrate a raw HTTP 200.
The demo parses JSON and calls the job done.
Nobody asks which tools were even legal.

That missing allowlist is the whole incident.
The model proposed a call. The runner obeyed it.
Git never saw a policy, only a mutated tree.

The problem is not the model

Tool calling is loud in developer feeds this month.
People wire HTTP, shell, and files into one loop.
Then a green status gets treated like a contract.

A green status code is not a contract.
It is an unscoped side effect with a nickname.
Who signed the allowlist for that curl?

I do not care that inference was complimentary.
I do not care that the box had no invoice.
I care that the tool could mutate production.

How I read a tool-calling failure

I start from symptoms, not from the prompt text.
The chat window is a story about intent.
The transcript is the only evidence I trust.

If there is no transcript, I already lost.
I cannot replay the call, only the narrative.
Would you debug production from a Slack summary?

Here is the catalog I actually use on reviews.
Each entry has a symptom, a cause, a replacement.
Steal the replacement. Leave the branding on the floor.

Anti-pattern 1: The open tool belt

Symptom. The agent can hit HTTP, shell, SQL, files.
One schema dump lists twenty tools with vague copy.
A "search" tool can POST, and nobody noticed.

Root cause. More tools feel more autonomous in demos.
Teams drop a kitchen-sink pack into every task.
Default allow becomes the real architecture, quietly.

Replacement. Default deny. One task, very few tools.
Name every tool in a committed allowlist file.
If the task changes, the allowlist changes in git.

Would you hand that belt to a weekend contractor?
If the answer is no, stop handing it to a model.

Anti-pattern 2: Secrets ride in the prompt

Symptom. Tool args contain tokens, cookies, or PEM blobs.
Those args went out with the remote model context.
Your so-called local agent was never actually local.

Root cause. Pasting the key into the prompt is easy.
A complimentary endpoint feels disposable, so people slacken.
Disposable compute is still somebody else's machine.

Replacement. The runner injects secrets at call time.
The model sees a handle like secret:billing_read.
The gate strips anything that looks like a credential.

If the model can print the key, design already failed.
Rotate that key. Then fix the proposal path.

Anti-pattern 3: Stdout is treated as truth

Symptom. The agent reads stdout and patches source.
A truncated JSON payload still "looks right enough."
Hallucinated fields survive because nobody schema-checks.

Root cause. Models trust the last tokens they saw.
Tool output is just more tokens in the window.
There is no type boundary after the 200.

Replacement. Every tool returns a typed envelope.
Validate with JSON Schema before the model sees it.
Reject extra fields. Reject missing required keys.

A 200 with a bad body is still a failed call.
Do not let the agent launder that into a diff.

Anti-pattern 4: No first-class transcript

Symptom. Slack has a summary. Disk has a mutation.
You cannot replay which tool ran, with which args.
The chat closed. The side effect stayed on disk.

Root cause. People log model prose, not tool IO.
Demos optimize for narrative, not for later forensics.
Forensics needs bytes, hashes, and honest timestamps.

Replacement. Write JSONL for every gated tool call.
Store tool name, arg hash, result hash, and decision.
CI fails if a call happened outside the gate.

Can you rebuild the incident without the chat UI?
If you cannot, you shipped a ghost.

Anti-pattern 5: The free box is "localhost"

Symptom. The agent workspace is a complimentary server.
Your checkout, your .env, and your SSH agent followed.
Egress is wide open "so the tools can work."

Root cause. Free compute feels like an extra laptop.
People confuse the invoice with the trust boundary.
A free host is still a different machine.

Replacement. Use an ephemeral workspace. No prod credentials.
Pin egress to the allowlisted hosts, nothing else.
Destroy the box after the transcript is copied off.

Would you scp your wallet onto a shared runner?
Then stop treating the free box as home.

The replacement pattern, in one pipeline

I want three files before any agent tool may run.
An allowlist. A schema. A transcript path.
No file, no call. That is the whole policy.

tools/
  allowlist.yml
  schemas/http_get.json
  schemas/repo_read.json
  transcripts/.gitkeep
scripts/
  tool_gate.py

The gate sits between the model and the real tool.
The model proposes. The gate decides. The OS executes.
If that order flips, you are already in production.

Artifact: a local tool gate you can run

This is a proposed runner, not a production IAM system.
I am not claiming latency numbers or pass rates.
Copy it, break it, then tighten the schemas.

`tools/allowlist.yml`

# Proposed policy. Commit this beside the task, not in chat.
version: 1
task: "read-only docs probe"
max_calls: 8
max_arg_bytes: 4096
default: deny
tools:
  - name: repo_read
    schema: schemas/repo_read.json
    network: false
    secrets: []
  - name: http_get
    schema: schemas/http_get.json
    network: true
    hosts:
      - api.github.com
    secrets:
      - github_readonly

`tools/schemas/http_get.json`

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "type": "object",
  "additionalProperties": false,
  "required": ["url"],
  "properties": {
    "url": {
      "type": "string",
      "pattern": "^https://api\\.github\\.com/"
    },
    "timeout_ms": {
      "type": "integer",
      "minimum": 100,
      "maximum": 5000
    }
  }
}

`scripts/tool_gate.py`

#!/usr/bin/env python3
"""Proposed local tool gate. Not IAM. Run it yourself."""
from __future__ import annotations

import argparse
import hashlib
import json
import re
import sys
from datetime import datetime, timezone
from pathlib import Path
from urllib.parse import urlparse

import yaml
from jsonschema import Draft202012Validator

SECRET_RE = re.compile(
    r"(?i)(api[_-]?key|secret|token|bearer\s+[A-Za-z0-9\-._]+|-----BEGIN )"
)


def sha256_bytes(data: bytes) -> str:
    return hashlib.sha256(data).hexdigest()


def looks_like_secret(payload: str) -> bool:
    return bool(SECRET_RE.search(payload))


def decide(allowlist: dict, proposal: dict, schema_root: Path) -> dict:
    tool = proposal.get("tool")
    args = proposal.get("args") or {}
    raw_args = json.dumps(args, sort_keys=True).encode("utf-8")
    record = {
        "ts": datetime.now(timezone.utc).isoformat(),
        "tool": tool,
        "arg_sha256": sha256_bytes(raw_args),
        "decision": "deny",
        "reason": "",
    }
    if not tool:
        record["reason"] = "missing tool name"
        return record
    if len(raw_args) > int(allowlist.get("max_arg_bytes", 4096)):
        record["reason"] = "args exceed max_arg_bytes"
        return record
    if looks_like_secret(raw_args.decode("utf-8")):
        record["reason"] = "secret-shaped value in args"
        return record
    spec = next(
        (t for t in allowlist.get("tools", []) if t.get("name") == tool),
        None,
    )
    if spec is None:
        record["reason"] = "tool not in allowlist"
        return record
    schema_path = schema_root / spec["schema"]
    schema = json.loads(schema_path.read_text(encoding="utf-8"))
    errors = sorted(
        Draft202012Validator(schema).iter_errors(args),
        key=lambda e: list(e.path),
    )
    if errors:
        record["reason"] = "schema: " + errors[0].message
        return record
    if spec.get("network"):
        url = args.get("url", "")
        host = urlparse(url).hostname or ""
        if host not in (spec.get("hosts") or []):
            record["reason"] = "host not pinned"
            return record
    record["decision"] = "allow"
    record["reason"] = "ok"
    return record


def append_transcript(path: Path, record: dict) -> None:
    path.parent.mkdir(parents=True, exist_ok=True)
    with path.open("a", encoding="utf-8") as fh:
        fh.write(json.dumps(record, sort_keys=True) + "\n")


def main() -> int:
    parser = argparse.ArgumentParser(description="Gate one proposed tool call.")
    parser.add_argument("--allowlist", required=True)
    parser.add_argument("--transcript", required=True)
    parser.add_argument("--proposal", required=True)
    ns = parser.parse_args()
    allowlist_path = Path(ns.allowlist)
    transcript = Path(ns.transcript)
    allowlist = yaml.safe_load(allowlist_path.read_text(encoding="utf-8"))
    existing = transcript.read_text(encoding="utf-8").count("\n") if transcript.exists() else 0
    if existing >= int(allowlist.get("max_calls", 8)):
        record = {
            "ts": datetime.now(timezone.utc).isoformat(),
            "tool": None,
            "arg_sha256": None,
            "decision": "deny",
            "reason": "max_calls exceeded",
        }
        append_transcript(transcript, record)
        print(json.dumps(record, indent=2))
        return 2
    proposal = json.loads(ns.proposal)
    record = decide(allowlist, proposal, allowlist_path.parent)
    append_transcript(transcript, record)
    print(json.dumps(record, indent=2))
    return 0 if record["decision"] == "allow" else 2


if __name__ == "__main__":
    sys.exit(main())

Install the two libraries the gate actually imports.

python3 -m pip install pyyaml jsonschema

Now throw a bad host at it on purpose.

python3 scripts/tool_gate.py \
  --allowlist tools/allowlist.yml \
  --transcript tools/transcripts/run.jsonl \
  --proposal '{"tool":"http_get","args":{"url":"https://evil.example/steal"}}'

You should see "decision": "deny" and host not pinned.
That deny is the feature. Keep the JSONL line.

Now throw a pinned host at the same gate.

python3 scripts/tool_gate.py \
  --allowlist tools/allowlist.yml \
  --transcript tools/transcripts/run.jsonl \
  --proposal '{"tool":"http_get","args":{"url":"https://api.github.com/repos/octocat/Hello-World"}}'

You should see "decision": "allow" and a new arg hash.
Still do not execute the real HTTP call from this script.
The gate only judges the proposal. Execution stays elsewhere.

Secret-shaped args should die before schema even runs.

python3 scripts/tool_gate.py \
  --allowlist tools/allowlist.yml \
  --transcript tools/transcripts/run.jsonl \
  --proposal '{"tool":"http_get","args":{"url":"https://api.github.com/?token=ghp_example"}}'

Expect deny. Expect secret-shaped value in args.
If that call had reached a remote model, you already leaked.

Decision table I keep next to the gate

Signal	Meaning	Action
Tool missing from allowlist	Open belt	Deny, fail the job
Args fail JSON Schema	Model improvised	Deny, keep transcript
Secret-shaped string in args	Prompt leak	Deny, redact the log
Host not in the pin list	Egress drift	Deny
Call count exceeds `max_calls`	Runaway loop	Deny, kill the runner
HTTP 200 with invalid body	Fake success	Treat as an error
Transcript file missing	Blind run	Do not merge

Read the table before you read the model recap.
The recap will sound confident. The table will not lie.
Which column are you actually merging?

A debugging workflow I follow

Reproduce the proposal as JSON, never as chat paste.
Run that JSON through tool_gate.py with the task allowlist.
Check deny or allow against the table above.
Execute the real tool only in a throwaway workspace.
Diff the transcript against git, not against memory.

If step one is hard, the agent UI is hiding the call.
Fix the UI. Do not "just ask it again."
A hidden call is an ungated call.

I also grep the transcript before I open the diff.

python3 - <<'PY'
import json, pathlib, sys
p = pathlib.Path("tools/transcripts/run.jsonl")
if not p.exists():
    print("no transcript"); sys.exit(2)
rows = [json.loads(line) for line in p.read_text().splitlines() if line]
denies = [r for r in rows if r.get("decision") == "deny"]
print(f"calls={len(rows)} denies={len(denies)}")
for r in denies:
    print(r["reason"])
sys.exit(1 if denies else 0)
PY

Zero denies does not mean the change is good.
It only means the gate stayed on during the run.
You still review the code like a human wrote it.

Where a cheap model loop actually helps

You still need a disposable place to practice dry-runs.
Not to touch production. To generate illegal proposals.
The gate gets stronger when you feed it bad calls.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode has free model access and a free server option.
I treat that pair as a sandbox for proposal generation only.
The allowlist, schemas, and gate stay in the application repo.
If you try the sandbox, keep customer data and prod keys off it.

Limitations, stated plainly

This gate is not IAM, and it is not a TEE.
It will not save you from a malicious runner image.
It will not sign an audit log your regulator accepts.

Do not use this approach when any of these apply:

You handle regulated personal data in tool args
The tool can move money or mutate cloud IAM
You cannot destroy the workspace after the run
You need a vendor-backed, signed audit trail
The provider cannot see even redacted proposals

Complimentary model access can change without notice.
A free server is not your compliance program.
Price zero is not the same as risk zero.

Who should skip this

Skip it if your agents never call tools at all.
Skip it if a human already types every request.
Skip it if security already brokers every egress path.

This catalog is for teams wiring agents to real APIs.
If that is not you, keep the tools in a drawer.
You do not need a gate for a function you never invoke.

What I will not merge

I will not merge a diff that needed a mystery tool call.
I will not merge "the agent said the API looked fine."
I will not merge a workspace that still holds prod secrets.

Show me the allowlist.
Show me the transcript.
Show me the deny that proved the gate was on.

If those three files are missing, the 200 never happened.
Not in a way I can defend later.
And later is when this stuff actually matters.

The Agent Diff Has No Parent SHA. Stop Merging It.

Sam Chen — Mon, 21 Sep 2026 13:14:56 +0000

A parentless agent patch is not a change. It is a rumor about some other tree. I will not merge that rumor today.

Did the model even start from your current HEAD?

Parent SHA first

I review AI patches by parent SHA first. I read the patched code only after that. Most failures hide between two commits, not inside one file.

The agent branched from an unnamed commit. Your teammates pushed new commits onto main. And nobody recorded the original base SHA.

Now the diff applies with hunks of fiction. Does this sequence already sound familiar to you?

How to read this catalog

Each anti-pattern below has three named parts. I list symptoms, root cause, and a runnable replacement. I keep every replacement small on purpose.

You should keep those replacements small too.

Anti-pattern 1: The orphan patch

The pull request shows a diff without a parent. git apply works Monday and then fails Tuesday. The chat says done and dumps files, not commits.

Symptoms

The pull request never names a recorded base_sha.
The same patch fails after an unrelated main push.
The agent returns file blobs instead of commits.

Root cause

The agent never serialized its real starting tree. You treated the chat blob as history. Git still disagrees with that entire chat story.

Replacement

Record the parent SHA before the first prompt. Refuse any patch that omits that SHA.

# example: persist the only parent that matters
git rev-parse HEAD > .agent-base-sha
git status --porcelain > .agent-dirty-index

Would you accept a human PR with no merge base?

Anti-pattern 2: The dirty overlay

Local edits sit unstaged while the agent writes files. The same path exists in your tree and the patch. Tests pass because your leftovers filled the holes.

Symptoms

Unstaged local files share paths with the agent patch.
Tests pass only with your leftover working tree.
git status was never clean at agent start.

Root cause

Two authors shared one working tree by accident. Neither person truly owned that shared tree. The agent overlayed a dirty index without checking.

Replacement

Start the agent on a clean worktree. Use a second clone if the tree is busy.

# example: refuse to start on a dirty tree
if [ -n "$(git status --porcelain)" ]; then
  echo "dirty tree; refuse agent start"
  exit 1
fi

Is that extra clone disposable on purpose then? Keep that clone disposable after every agent run.

Anti-pattern 3: The silent rebase

The patch applies with fuzz and with offset. Reviewers read a diff that HEAD will never see. CI on main fails after a "successful" agent run.

Symptoms

git apply reports offsets you did not review.
The reviewed hunks never match git show HEAD.
Main CI fails after the agent session looked green.

Root cause

HEAD moved while the agent was writing. Nobody replayed the patch onto current main. You reviewed a ghost of yesterday's tree.

Replacement

Rebase the agent commit onto current HEAD. Review that replayed diff with fresh eyes. Ship only that replayed commit after review.

# example: replay onto today's origin/main
BASE=$(cat .agent-base-sha)
git fetch origin
git rebase --onto origin/main "$BASE"

If the rebase conflicts, the original review is void. Run the review again on the result.

Anti-pattern 4: Generated-file amnesia

The source changed and the lockfile stayed put. Generated protobuf stubs lag behind the .proto edit. CI generates files the agent never opened.

Symptoms

Source imports a package the lockfile does not pin.
Build artifacts differ between agent host and CI.
Reviewers never opened the generated file diff.

Root cause

The agent patched only the file it was shown. The build graph stayed invisible to the session. You merged half of a real change.

Replacement

List generated outputs inside the receipt file. Diff those outputs after the local replay.

# example receipt fragment, not a live config
must_regenerate:
  - package-lock.json
  - src/generated/
commands:
  - npm install --package-lock-only
  - make proto

Did you read the generated diff at all? If not, you skipped the real change.

Anti-pattern 5: Remote-only green

The agent host printed a cheerful all-tests-passed line. Your laptop cannot even find that command. No artifact ever left the remote session.

Symptoms

Success exists only as a sentence in chat.
The command line is not checked into the receipt.
Local replay is skipped because the host looked green.

Root cause

That success lived on another machine entirely. You imported the story, not the bytes. A free remote host makes that swap easy.

I sometimes point the first pass at MonkeyCode. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode offers free model access and a free server option. That host is fine for a first pass. It is not my merge oracle though.

Copy the command list off the host. Run every command on your local checkout. Store the exit codes beside both SHAs.

A green remote log without a local receipt is theater. Do not import that theater into main.

The artifact: a two-SHA receipt

I want two SHAs sitting on disk. I want the command transcript beside them.

Here is an example script for the receipt. Treat it as a proposal, not a benchmark.

#!/usr/bin/env python3
"""Example only: write a two-SHA agent receipt."""
from __future__ import annotations

import json
import subprocess
import sys
from datetime import datetime, timezone
from pathlib import Path


def sh(*args: str) -> str:
    proc = subprocess.run(args, check=True, capture_output=True, text=True)
    return proc.stdout.strip()


def main() -> int:
    if len(sys.argv) != 2:
        print("usage: receipt.py write|verify", file=sys.stderr)
        return 2

    root = Path(".agent-receipt.json")
    mode = sys.argv[1]
    head = sh("git", "rev-parse", "HEAD")
    branch = sh("git", "rev-parse", "--abbrev-ref", "HEAD")
    dirty = sh("git", "status", "--porcelain")

    if mode == "write":
        payload = {
            "recorded_at": datetime.now(timezone.utc).isoformat(),
            "base_sha": head,
            "branch": branch,
            "dirty": bool(dirty),
            "dirty_paths": dirty.splitlines(),
            "result_sha": None,
            "commands": [],
        }
        root.write_text(json.dumps(payload, indent=2) + "\n")
        print(f"wrote base_sha={head}")
        return 0 if not dirty else 1

    if mode == "verify":
        data = json.loads(root.read_text())
        base = data["base_sha"]
        merge_base = sh("git", "merge-base", base, head)
        if merge_base != base:
            print(f"base {base} is not an ancestor of {head}")
            return 1
        if data.get("dirty"):
            print("receipt started dirty; refuse verify")
            return 1
        data["result_sha"] = head
        data["verified_at"] = datetime.now(timezone.utc).isoformat()
        root.write_text(json.dumps(data, indent=2) + "\n")
        print(f"ok base={base} result={head}")
        return 0

    print("unknown mode", file=sys.stderr)
    return 2


if __name__ == "__main__":
    raise SystemExit(main())

How I run the example

This run sequence is a labeled example. Change the names to match your repo.

# 1. freeze the parent on a clean tree
python3 receipt.py write || exit 1

# 2. let the agent work, then commit
# git add -A && git commit -m "agent: named change"

# 3. prove the parent still ancestors HEAD
python3 receipt.py verify || exit 1

# 4. rebase if origin/main moved under you
git fetch origin
git rebase origin/main
python3 receipt.py verify

No receipt means no review from me. That is the whole review rule here.

Test plan for the receipt

Do not trust this example script blindly. Break it on purpose before any agent run.

Run write on a dirty tree and expect exit 1.
Commit a dummy file, then run verify for exit 0.
Reset --hard to HEAD~1 and expect verify to fail.
Rebase onto an unrelated commit and expect ancestor failure.
Delete .agent-receipt.json and confirm the review stops.

If step 3 still prints ok, the script is lying. Fix it before any real agent run.

Decision table

Use this table during review, not during prompting. When a row says no, stop merging.

Signal	Trust the patch?	Next move
No `base_sha` on disk	No	Restart the agent on a named SHA
Dirty tree at `write`	No	Clean clone, then run `write` again
`merge-base` is not `base_sha`	No	Rebase, then re-review the new diff
Remote tests green only	No	Replay the same commands locally
Generated files missing from the diff	No	Run generators, then diff outputs
Two SHAs plus a clean local replay	Candidate only	Human review of the replayed diff

Notice the last row still says only candidate. People still need to read the replayed diff.

What this does not catch

A parent SHA is not a behavior specification. It does not prove runtime behavior by itself. The example script ignores several real failure classes.

Flaky tests that passed once on the remote host.
Secrets that existed only in the remote environment.
Prompt text that never entered the git history.
Semantic bugs that still compile and link cleanly.

I still read the replayed diff slowly. Do you actually read that replayed diff?

Who should not use this

Skip this workflow if you have no local clone. Skip it if hermetic CI is already your only trusted runner.

Skip it for throwaway spikes you will delete before lunch. Also skip it if you cannot run the project's tests locally.

A receipt without a replay is another pretty story. Remote-only teams need a named CI SHA.

Closing

The model will keep returning patches anyway today. HEAD will keep moving under those patches. Those two facts do not care about your deadline.

I want a parent SHA, a result SHA, and a local replay. Everything else is narration from a model.

If you draft on a free remote host, steal this receipt script. Make the host prove its parent SHA. Then decide whether the replayed diff is mergeable.

You Closed the Chat. Five Anti-Patterns Still Merged.

Sam Chen — Sun, 20 Sep 2026 11:21:13 +0000

The chat went quiet, so the work felt finished. That feeling is the bug, not the model.

I keep a five-row anti-pattern catalog for this moment. Each row lists symptoms, root cause, and a local replacement.

A free session can draft code without any shame. Your repo still owns the merge gate, always.

Why this catalog exists

AI coding sessions now end in minutes, not days. Speed is not the same thing as a merge contract.

Have you ever pasted a green snippet as review? I have done that, and the next outage was local.

Trending posts ask if models outgrew our tests. Wrong question for this catalog on your machine. Those tests never ran against your actual checkout.

How to read each row

Use the catalog during the last ten minutes. Do not use it as a vibe check.

For every row I want three things only:

Symptoms you can spot in the diff or logs
Root cause stated in one blunt sentence
A replacement pattern with a real command

No score. No leaderboard. Just fail-closed checks you can rerun.

Anti-pattern 1: Clipboard as CI

You copy the patch into the tree. You never run a command that can fail.

Symptoms

The PR quotes the chat, not a failing command
Generator output never touches a scripts/ gate
Review says "looks good" with zero pasted output

Root cause

The session produced text. You treated that text as an execution receipt.

Replacement: a local fail-closed gate

Write one script the model cannot mark green by talking.

#!/usr/bin/env bash
# scripts/local-gate.sh
set -euo pipefail
cd "$(dirname "$0")/.."

test -f reproduce.sh
test -f decisions.md

if command -v rg >/dev/null; then
  if rg -n "AKIA|BEGIN PRIVATE|api_key\s*=" --glob '!decisions.md' .; then
    echo "secret-shaped text in the tree" >&2
    exit 1
  fi
fi

./reproduce.sh
python3 scripts/check_defaults.py
pytest -q tests/gate

Did the chat approve the patch in prose? Fine. Did local-gate.sh exit zero on your laptop?

Anti-pattern 2: Anonymous runtime

You ran the demo on a borrowed box. Nobody on the team can replay it.

Symptoms

The README says it worked on "the server"
No commit SHA, seed, or interpreter version is recorded
A teammate cannot hit the same path on a laptop

Root cause

A shared runtime became the source of truth. Your laptop never signed the result.

Replacement: a reproduce file that pins the command

#!/usr/bin/env bash
# reproduce.sh
set -euo pipefail
echo "git=$(git rev-parse HEAD)"
echo "python=$(python3 --version)"
python3 -m venv .venv
# shellcheck disable=SC1091
. .venv/bin/activate
pip install -r requirements-gate.txt
python3 -m app.probe --fixture tests/fixtures/min.json

The box can be disposable and even free. The script cannot vanish with the session.

Anti-pattern 3: Model-chosen defaults

The generator picked timeouts, pool sizes, and log levels. You shipped those numbers unchanged.

Symptoms

timeout=30, retries=3, workers=4 with no comment
Config keys appear only inside generated files
No test asserts a deadline or a fail-closed limit

Root cause

Defaults look like expertise in a hurry. They are the model's prior, frozen into your repo.

Replacement: an explicit defaults review

# scripts/check_defaults.py
from pathlib import Path
import re
import sys

ALLOWED = {
    "timeout_s": {5, 10, 15},
    "retries": {0, 1},
    "workers": {1, 2},
}

text = Path("app/config.py").read_text(encoding="utf-8")
fail = False
for key, allowed in ALLOWED.items():
    match = re.search(rf"{key}\s*=\s*(\d+)", text)
    if not match or int(match.group(1)) not in allowed:
        print(f"unreviewed default: {key}")
        fail = True
if fail:
    sys.exit(1)
print("defaults reviewed")

Would you defend retries=3 in an incident channel? If not, do not merge that line.

Anti-pattern 4: Secrets in the prompt

You pasted .env so the model could "see the real shape." That thread now outlives the key rotation.

Symptoms

Chat history contains hostnames, tokens, or customer ids
Generated fixtures still point at production URLs
.gitignore looks clean, but the session log is not

Root cause

Context dumping felt faster than making a fake fixture. The leak is now immortal in the thread.

Replacement: a fixture that cannot authenticate

{
  "base_url": "http://127.0.0.1:9",
  "token": "test-token-not-secret",
  "user_id": "fixture-user"
}

If the model needs shape, give it a dead socket. Never give it a live key.

Scan the tree before the gate returns green:

# last line of local-gate.sh before pytest
python3 - <<'PY'
from pathlib import Path
bad = ("AKIA", "BEGIN PRIVATE", "sk-live", "xoxb-")
root = Path(".")
for path in root.rglob("*"):
    if path.suffix in {".png", ".pyc"} or ".venv" in path.parts:
        continue
    try:
        text = path.read_text(encoding="utf-8", errors="ignore")
    except OSError:
        continue
    if any(token in text for token in bad):
        raise SystemExit(f"secret-shaped text: {path}")
print("no secret-shaped text")
PY

Did the fixture still need a routable host? Then it is not a fixture.

Anti-pattern 5: Forty-turn design doc

The architecture lives in a scrolling chat. The repo only has a shrug.

Symptoms

The PR links a transcript instead of a table
Two engineers recount different "final" decisions
Re-asking the model quietly changes the story

Root cause

Conversation is cheap and feels like design. A twelve-line table is the actual design.

Replacement: a decision table in the repo

Keep decisions.md tiny. Blank cells mean the chat is not done.

Decision	Options	Chosen	Why	Revisit when
Auth	session cookie / static token / none	static token in tests	local gate must not hit IdP	we add a real user
Runtime	laptop / borrowed server / CI	laptop owns merge	borrowed boxes lie about paths	CI is green twice
Timeouts	5s / 30s / none	5s	fail closed on hang	p99 is measured
Retries	0 / 3 / infinite	0	retries hide probe bugs	idempotency is proven
Fixtures	prod dump / synthetic	synthetic	secrets stay out of prompts	schema changes

Print this table in the PR body. If a cell is blank, stop generating more code.

The artifact: one gate, one table

Here is the workflow I want on the last commit. Treat it as a proposal, not a published study.

Fill decisions.md before asking for more code.
Generate against fixtures, never against live secrets.
Run ./scripts/local-gate.sh on your laptop.
Treat any borrowed server as a scratch lane only.
Merge only when the gate exits 0 and the table is complete.

I am not attaching latency charts on purpose. Charts without a pinned reproduce.sh are theater.

A tiny pytest file keeps the happy-path lie from shipping:

# tests/gate/test_fail_closed.py
from app.probe import probe
from app import config

def test_dead_socket_fails_closed():
    result = probe("tests/fixtures/min.json")
    assert result.ok is False
    assert result.error in {"timeout", "connection_refused"}

def test_retries_are_zero():
    assert config.retries == 0

def test_timeout_is_reviewed():
    assert config.timeout_s in {5, 10, 15}

Does a dead socket fail closed on the first try? If that probe "succeeds," your gate is theater.

Wire a stub module so the example is runnable:

# app/config.py
timeout_s = 5
retries = 0
workers = 1

# app/probe.py
from dataclasses import dataclass
from pathlib import Path
import json
import socket

@dataclass
class Result:
    ok: bool
    error: str | None

def probe(fixture_path: str) -> Result:
    data = json.loads(Path(fixture_path).read_text(encoding="utf-8"))
    host, port = "127.0.0.1", 9
    if ":" in data["base_url"].split("//", 1)[-1]:
        hostport = data["base_url"].split("//", 1)[-1]
        host, port_s = hostport.split(":")[0], hostport.split(":")[1].split("/")[0]
        port = int(port_s)
    try:
        socket.create_connection((host, port), timeout=0.2).close()
        return Result(ok=True, error=None)
    except OSError as exc:
        name = "timeout" if isinstance(exc, socket.timeout) else "connection_refused"
        return Result(ok=False, error=name)

Run it. Watch the dead port refuse you. That refusal is the point.

Where a free model and free server fit

I still use a throwaway lane for drafts. Scratch generation is fine. Merge authority is not.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode offers free model access and a free server option. I treat that pair as the scratch lane: draft the probe, try the fixture, throw the box away. The local gate above still decides the merge. I am not claiming model names, quotas, hardware, or benchmarks. Those claims go stale. These five anti-patterns do not.

If you want a disposable place to practice the gate, that scratch lane is enough. Do not promote it into CI.

Limitations

This catalog will annoy you on tiny scripts. Good. Friction on a one-line typo is a smell, not a virtue.

It will not catch semantic bugs the tests never named. It will not replace a threat model. It will not make a weak fixture honest.

Skip this approach when:

You are pairing live and already running the suite
The change is a comment, rename, or typo
You cannot run anything locally at all
You need a signed production deploy, not a laptop gate

Cannot run locally? Then you have a bigger problem than chat quality. Borrowed servers drift. Free sessions forget. Your reproduce.sh should not.

What I will not do

I will not call the chat a review. I will not paste live keys into a prompt. I will not ship retries=3 because it looked standard.

Would you bet an incident channel on a transcript? I would not.

Close the tab after the gate is green. Not before.

The Chat Approved It. Git Still Has No Receipt.

Sam Chen — Fri, 18 Sep 2026 07:48:44 +0000

Cheap inference does not replace a human review gate.
It makes five skip patterns cheaper to repeat.
I still catch them in my own branches.

What this piece is not

This is not another vibe-coding culture rant.
I will not grade your editor or your chat model.
I will show symptoms, root causes, and replacements.

The public debate this week is still noisy.
People argue about whether AI writes real software.
The useful question is smaller and much meaner.

Did this diff actually get a review receipt?
Or did a cheap run just feel like shipping?

The catalog at a glance

I use the same three columns every time.

Symptom: what shows up in git or chat.
Root cause: why cheap inference encourages the skip.
Replacement: the pattern I want instead.

The proposed scripts and tables live right below.
Treat the scripts as templates, not production gospel.

Anti-pattern 1: Chat as the source of truth

Symptoms

The pull request says "details in the thread."
Commits read like "update stuff" and nothing else.
Reviewers paste screenshots instead of file paths.

Root cause

The sidebar answered faster than the commit template.
The answer felt documented because it was long.
Git never stored the actual product decision.

Replacement: put the decision in git

I put the tradeoff in the commit body.
I link the chat only as appendix, never as proof.
If the chat dies, the repo must still explain the change.

git commit -m "$(cat <<'EOF'
reject unsigned webhook payloads

Why: the agent added a debug bypass on staging.
Decision: verify signatures in the wrapper, not the prompt.
Chat transcript is appendix only, not source.
EOF
)"

Ask yourself one rude and practical question.
Could a new teammate review this without the chat?
If the answer is no, the chat still owns your history.

Anti-pattern 2: Secrets riding the free prompt

Symptoms

Env fragments appear inside a pasted stack trace.
The prompt file contains hostnames from production.
The agent reprints a token in the session log.

Root cause

A free box still feels disposable and unofficial.
Disposable does not mean the prompt is private.
Cheap inference can still persist that text somewhere.

Replacement: redact, then hash

I redact the raw prompt first, without exceptions.
I hash the cleaned prompt file second.
I never paste live credentials into a sandbox prompt.

# Proposed local redaction before any remote prompt.
sed -E \
  -e 's/[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+/[redacted-email]/g' \
  -e 's/(api[_-]?key|token|secret)[=:][^[:space:]]+/\1=[redacted]/Ig' \
  -e 's/[0-9]{1,3}(\.[0-9]{1,3}){3}/[redacted-ip]/g' \
  prompt.raw.txt > prompt.clean.txt

sha256sum prompt.clean.txt > prompt.clean.sha256

Run that before the model sees the file.
If redaction fails, the session does not start.

Would you paste that prompt into a public gist?
If not, do not send it to a free endpoint either.

Anti-pattern 3: The agent owns the lockfile

Symptoms

The lockfile moves with a one-line behavior fix.
The model tidies dependencies during a simple rename.
Nobody can explain the version jump in review.

Root cause

Cheap retries invite extra cleanup turns from the agent.
Those extra cleanup turns always love the lockfile.
Lockfiles are easy to generate and hard to audit.

Replacement: lockfile needs a human ACK

I split lockfile diffs from the behavior diffs.
I require an explicit ACK file in the pull request.
No ACK means the CI job fails closed.

# proposed_lockfile_ack.py
# Example only. Adapt paths before you trust it.
from pathlib import Path
import subprocess
import sys

LOCK_NAMES = {"package-lock.json", "poetry.lock", "Cargo.lock", "go.sum"}
ACK = Path("LOCKFILE_ACK.md")


def changed_files() -> set[str]:
    out = subprocess.check_output(
        ["git", "diff", "--name-only", "origin/main...HEAD"],
        text=True,
    )
    return {Path(p).name for p in out.splitlines() if p}


changed = changed_files()
touched_lock = sorted(changed & LOCK_NAMES)
if not touched_lock:
    print("no lockfile change")
    sys.exit(0)
if not ACK.exists():
    print("lockfile changed without LOCKFILE_ACK.md")
    sys.exit(1)
text = ACK.read_text(encoding="utf-8").lower()
if "i reviewed the lockfile diff" not in text:
    print("ACK is missing the review sentence")
    sys.exit(1)
print("lockfile ACK present for:", ", ".join(touched_lock))

Put that ACK in the same pull request.
Name every package version that actually moved.
Name the risk in one blunt sentence.

Did the agent bump a compiler to make tests pass?
That compiler bump is not just a tidy.
That bump is a silent product change too.

Anti-pattern 4: Policy living in the system prompt

Symptoms

"Do not call production" exists only in a prompt.
The tool wrapper still accepts the prod hostname.
Safety is a paragraph, not a check.

Root cause

Prompts are easy to type on a free model.
Guards in code take an extra hour.
Cheap inference rewards the hour you skip.

Replacement: policy in code, prompt as hint

I keep the system prompt short and polite.
I keep the policy in a denylist.
The model cannot override a failed hostname check.

# proposed_tool_guard.py
from urllib.parse import urlparse

BLOCKED_HOST_SUFFIXES = (".prod.internal", ".corp.example")
BLOCKED_HOSTS = {"127.0.0.1"}


def assert_safe_url(url: str) -> None:
    host = (urlparse(url).hostname or "").lower()
    if not host:
        raise ValueError("missing host")
    if host in BLOCKED_HOSTS:
        raise ValueError(f"blocked host: {host}")
    if any(host.endswith(sfx) for sfx in BLOCKED_HOST_SUFFIXES):
        raise ValueError(f"blocked suffix: {host}")

The prompt may still say "stay in staging."
The wrapper must still refuse the URL.
Which layer do you trust when the model is tired?

Anti-pattern 5: Idle free capacity as a release signal

Symptoms

The merge happened because the server was free tonight.
The reviewer is whoever is still online in chat.
Monday on-call inherits an unexplained diff.

Root cause

Cheap idle time feels like a wasted gift.
That waste anxiety beats review patience every time.
The review calendar never becomes a real gate.

Replacement: capacity is not a criterion

I keep capacity out of the ship list.
A free server can wait until review exists.
Idle machines do not approve the architecture change.

Would you merge this if the box were busy?
If the answer flips, capacity was the reviewer.
Ship criteria belong in the receipt, not the queue.

The artifact: a review receipt

I want one file per pull request.
I call that file REVIEW_RECEIPT.md on purpose.
This file is not a model freeze file.

A freeze file only pins the model inputs.
This receipt pins the review skips I just named.

# REVIEW_RECEIPT.md

- Decision lives in commit: yes/no
- Prompt redacted and hashed: yes/no (sha256: ...)
- Lockfile ACK present: yes/no/n/a
- Tool policy enforced in code: yes/no
- Capacity was not a ship criterion: yes/no
- Reviewer: name
- Date: ISO-8601

Use this table when the sandbox looks too green.

Claim you want	Cheap sandbox can show	Cheap sandbox cannot show
Script parses on a clean clone	Yes	Your dirty production working tree
Unit tests pass with fixtures	Yes	Real traffic shape
Tool guard blocks fixture URLs	Yes	A hostname someone adds later
Prompt hash matches redacted text	Yes	Extra secrets pasted only in chat
Feature is safe to ship	No	Data, auth, clocks, or load

If the table cell says no, the merge still needs a human.

Proposed checker:

# proposed_receipt_check.py
from pathlib import Path
import re
import sys

receipt = Path("REVIEW_RECEIPT.md")
if not receipt.exists():
    print("missing REVIEW_RECEIPT.md")
    sys.exit(1)

text = receipt.read_text(encoding="utf-8")
required = [
    "Decision lives in commit:",
    "Prompt redacted and hashed:",
    "Lockfile ACK present:",
    "Tool policy enforced in code:",
    "Capacity was not a ship criterion:",
    "Reviewer:",
]
missing = [line for line in required if line.lower() not in text.lower()]
if missing:
    print("receipt missing fields:", ", ".join(missing))
    sys.exit(1)

if re.search(r"sha256:\s*$", text, re.I | re.M):
    print("hash field is empty")
    sys.exit(1)

print("review receipt fields present")

Then you wire the checker into CI.

# proposed .github/workflows/review-receipt.yml
name: review-receipt
on: [pull_request]
jobs:
  receipt:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: python proposed_receipt_check.py

I have not executed this workflow on your org.
Copy it only after you change the names.

A cheap rehearsal, not a release

I still want a cheap place to practice the gate.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode offers free model access and a free server option.
I treat those as a rehearsal room for the receipt checker.
I do not treat them as production capacity, an SLA, or a benchmark.

If you rehearse there, run the redaction step first.
Then run the receipt checker on a throwaway branch.
Skip the demo prompt that never touches git.

Who should not use this approach

Do not use a free server for regulated customer data.
Do not use prompt text as your only access control.
Do not skip this catalog because the tokens were cheap.

This receipt is only a minimum bar.
It will not catch a clever exfil path.
It will not replace a real threat model.

Teams with no git history should not start here.
You should fix basic source control first here.
Then you add the review receipt file.

Limitations

The checker only proves that a file exists.
A bored reviewer can type "yes" everywhere.
That is still a process failure, not a model failure.

Cheap inference will keep getting even cheaper.
Your gates will not appear by accident.
Build them while the diffs are still small.

Would I merge a green sandbox without the receipt?
I would not merge that green branch yet.
Would I trust the receipt without reading the diff?
I would also refuse that lazy shortcut.

The Agent Wrote the Tests. They Still Echoed the Patch.

Sam Chen — Thu, 17 Sep 2026 05:44:43 +0000

Passing tests do not prove the agent understood the contract.
They often prove the agent quoted its own patch.
That gap is the merge risk, not the model.

I keep reviewing pull requests that look fully engineered.
The prose stays calm while the checklist stays green.
So why does production still surprise the on-call engineer?

We launder chat-driven patches through review theater.
Fluency becomes evidence and echoes become fake oracles.
This catalog names four anti-patterns I keep seeing.

Each entry has symptoms, a root cause, and a replacement.
I also include a copyable oracle check and a command pin.
Treat the snippets as labeled examples, not production history.

What this is not

This is not a ranking of coding models today.
This is not a claim that agents cannot write code.
This is a review workflow you can run tomorrow.

I am not publishing timings, winners, or customer stories.
Those claims go stale and I will not invent them.
If you need signed capacity data, use your own lab.

Anti-pattern 1: Confidence as a merge gate

Symptoms

The summary reads like a senior engineer wrote every line.
Reviewers praise tone, structure, and seemingly clear reasoning.
Nobody can name the invariant that actually changed here.

Root cause

Fluency is cheap while true invariants stay expensive.
A model can narrate a patch without owning its failure.
Your process then confuses narration with actual technical review.

Replacement

Require one written invariant before anyone opens the diff.
If the author cannot state it, the PR remains a spike.
Spikes get a ticket, a date, and no merge button.

INVARIANT: refunds never exceed the captured amount
PROOF: test_refund_cap in tests/billing_oracle.py
OWNER: @oncall-owner

Ask one rude question during every model-authored review.
What still fails if we delete the generated comments now?
If the answer is nothing, you reviewed a blog post.

Anti-pattern 2: Echo tests

Symptoms

Tests arrived in the same chat as the implementation.
Assertions repeat literals copied from the generated body.
Renaming a constant makes the test fail before production.

Root cause

The agent tested its story, not the external contract.
The test file is a mirror sitting beside the patch.
Green then only means I agreed with myself again.

Did the test exist before the implementation existed?
If not, you probably bought an echo, not an oracle.
Oracles come from specs. Echoes come from stdout.

Replacement

Write the oracle before the implementation is allowed.
Pin expected values from the spec, never from chat output.
If you lack a spec, you still lack a real test.

Here is a labeled example, not a measured suite.

# example_oracle.py — proposal, not a production run
from decimal import Decimal

SPEC_CAPTURED = Decimal("40.00")
SPEC_REQUESTED = Decimal("99.99")
SPEC_REFUND = Decimal("40.00")  # from the billing spec, not stdout


def refund_cap(captured: Decimal, requested: Decimal) -> Decimal:
    if requested < 0:
        raise ValueError("requested must be >= 0")
    return min(captured, requested)


def test_refund_never_exceeds_capture():
    assert refund_cap(SPEC_CAPTURED, SPEC_REQUESTED) == SPEC_REFUND


def test_oracle_does_not_quote_the_function():
    source = open(__file__, encoding="utf-8").read()
    banned = "assert refund_cap(SPEC_CAPTURED, SPEC_REQUESTED) == refund_cap"
    assert banned not in source

Run the example with a boring, repeatable command.

python -m pytest example_oracle.py -q

If your only assertion is output equals output, stop.
That check is a quote. It is not a contract.

Echo versus oracle, on one page

Signal	Echo test	Oracle test
Source of expected value	Generated code or chat	Spec, RFC, or ticket
Same session as the patch	Usually yes	Must be no
Rename a literal	Test dies first	Behavior still pinned
Delete the implementation comments	Still green	Still meaningful
Who can explain a failure	The chat log	A named owner

Print that table in the PR template if you must.
I would rather see one filled row than ten adjectives.
Which column did your last AI patch actually occupy?

Anti-pattern 3: Prompt until green

Symptoms

The chat log is longer than the resulting diff.
Failures vanish after another polite try again.
Nobody saved the first command that actually failed.

Root cause

You optimized the prompt instead of pinning the system.
The failure never became a hashed, repeatable command.
The next model, or the next morning, revives it.

Is the fix a new behavior, or a new pep talk?
If you cannot tell, you do not have a fix.
You have a mood swing with a green badge.

Replacement

Freeze the failing command, not the motivational thread.
Re-run that exact command after every prompt change.
If the command drifts, the supposed fix is theater.

# pin the failure; do not paraphrase it in chat
cat > /tmp/failing_cmd.sh <<'EOF'
#!/usr/bin/env bash
set -euo pipefail
python -m pytest tests/billing_oracle.py -q
EOF
chmod +x /tmp/failing_cmd.sh
sha256sum /tmp/failing_cmd.sh
/tmp/failing_cmd.sh; echo EXIT:$?

Paste the hash into the pull request body.
Did this command change, or did the behavior change?
Only the second answer counts as an actual fix.

Keep the script in the repo when the check matters.
A gist in chat will rot before the next on-call shift.
CI should run the same bytes your laptop ran.

install -d scripts
cp /tmp/failing_cmd.sh scripts/oracle_refund.sh
git add scripts/oracle_refund.sh tests/billing_oracle.py
git commit -m "pin refund oracle and the failing command"

Anti-pattern 4: Sandbox as production proof

Symptoms

It ran on a free box, so we called it done.
No owner, no data class, no rollback, no page path.
The demo path is the only path anyone executed.

Root cause

A disposable environment proves one thing: reproducibility.
It does not prove capacity, identity, or blast radius.
People collapse those proofs because the demo felt good.

Would you page this sandbox at three in the morning?
If the answer is no, it never signed production.
Stop letting a clean demo launder an unsigned change.

Replacement

Split the checks. Keep them boring and explicitly named.

Check	Sandbox may answer	Production still needs
Does the oracle fail closed?	Yes	Same oracle in CI
Can a stranger reproduce it?	Yes	README command plus hash
Will it hold real load?	No	Your own capacity test
Who gets paged at 03:00?	No	A named owner
Is this data class allowed?	No	Policy, not a chat

I use a free coding environment only as a reproduction box.
MonkeyCode is an open source option with free model access and a free server. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

That box can rerun scripts/oracle_refund.sh from a clean tree.
It cannot sign your incident, your identity, or customer data.
If you need those signatures, you already left the sandbox.

A workflow you can copy tomorrow

Do this on the next model-authored pull request.
Do not skip the writing. The writing is the review.
Empty fields mean you are still in chat-driven mode.

State the invariant in one plain sentence.
Add one oracle that never imports patch literals.
Pin the failing command and store its sha256.
Mark sandbox results as reproduced, never shipped.
Name an owner who can revert without the chat log.

PR template
- Invariant:
- Oracle test path:
- Failing command hash:
- Reproduced on (sandbox/CI):
- Ship check (yes/no + owner):

Fill it in public comments, not in a private sidebar.
If any field is empty, the merge is still a spike.
Call it a spike. Do not call it engineering.

Want a tiny gate before humans even look?
This hook only checks that the template fields exist.
It will not prove the oracle is wise. Nothing local can.

# scripts/check_pr_template.sh — proposal
set -euo pipefail
file="${1:-pr_body.md}"
for key in Invariant "Oracle test path" "Failing command hash" "Ship check"; do
  grep -q "^- ${key}:" "$file"
  grep -vq "^- ${key}: *$" "$file"
done

Who should not use this

Do not use this catalog as a reason to ban models.
Do not use a free server as a load or soak test.
Do not paste secrets, prod dumps, or customer records there.

Do not treat my examples as measured benchmarks either.
I did not name models, quotas, hardware, or durations.
Those numbers rot. These anti-patterns do not rot.

Skip this sermon when the change is already specified.
A owned RFC with golden tests does not need this ritual.
You already have an oracle. Keep feeding it in CI.

Also skip it for throwaway spikes you will delete today.
Just do not merge the spike because the prose sounded sure.
Sounding sure is the cheapest part of the whole stack.

Limitations

Short review questions will not catch every failure mode.
Oracles are wrong when the spec itself is wrong.
Pinned commands rot if nobody runs them inside CI.

This workflow also slows a true exploratory spike.
That cost is the point, not an accident of process.
If the slowdown hurts, you were shipping narration.

I cannot prove your billing rules from a blog example.
Your domain invariants have to come from your own spec.
Copy the method. Do not copy my refund numbers blindly.

What I actually want in review

I want a sentence a stranger can falsify later.
I want a test that would fail if the spec changed.
I want a command hash that does not depend on charm.

Did you review behavior, or did you only review tone?
Did the tests quote the patch, or pin a contract?
If you cannot tell, the agent did not fail. Review did.

Pin the command. Name the owner. Then consider merge.
That is the whole method, and it still fits in a template.
The model can draft. It still cannot sign the incident.

The Model Replied. You Still Have No Freeze File.

Sam Chen — Wed, 16 Sep 2026 04:03:43 +0000

A green chat is not a green build.
The missing freeze file is the real outage.
Free remote inference does not pin a compiler.

The public feed keeps arguing about model skill.
Did AI already beat most working developers?
That question skips the only check I run.

Did you pin the run at all?
Or did you screenshot a lucky chat?
I keep a blunt catalog for this failure.

Each entry lists symptoms and a root cause.
Each one also names a replacement pattern.
I am not ranking vendors in this post.

I am not posting latency or quality charts.
I am talking about unpinned AI coding runs.
Folklore patches still look brilliant in a thread.

What I mean by a freeze file

A freeze file is a local contract.
It records what the model actually saw.
It also records the patch you accepted.

The proposed schema sits below.
Treat it as an example until you execute it.
Do not paste it into production CI untested.

{
  "prompt_sha256": "replace-me",
  "patch_sha256": "replace-me",
  "test_command": "pytest -q tests/test_billing.py",
  "created_at": "2026-09-16T12:00:00Z",
  "runtime_kind": "scratch-pad",
  "reproduced_locally": false,
  "secrets_present": false
}

I only trust prompt, patch, and test hashes.
Drop one hash and the run becomes folklore.
Git then stores a story, not a replay.

I gate merges on that file.
No freeze file, no merge.
The validator is boring on purpose.

#!/usr/bin/env python3
"""Refuse an AI coding run with no freeze file.

Local gate only. It never calls a model.
"""
from __future__ import annotations

import json
import sys
from pathlib import Path

REQUIRED = (
    "prompt_sha256",
    "patch_sha256",
    "test_command",
    "created_at",
    "runtime_kind",
)
ALLOWED_RUNTIME = {"local", "scratch-pad", "ci-pinned"}


def load(path: Path) -> dict:
    data = json.loads(path.read_text(encoding="utf-8"))
    if not isinstance(data, dict):
        raise ValueError("freeze file must be an object")
    return data


def validate(data: dict) -> list[str]:
    errors: list[str] = []
    for key in REQUIRED:
        if not data.get(key):
            errors.append(f"missing {key}")
    runtime = data.get("runtime_kind")
    if runtime not in ALLOWED_RUNTIME:
        errors.append(f"runtime_kind {runtime!r} is not pinable")
    if runtime == "scratch-pad" and not data.get("reproduced_locally"):
        errors.append("scratch-pad run was never reproduced locally")
    if data.get("secrets_present") is True:
        errors.append("freeze file claims secrets in the prompt")
    return errors


def main() -> int:
    if len(sys.argv) != 2:
        print("usage: validate_freeze.py path/to/freeze.json", file=sys.stderr)
        return 2
    path = Path(sys.argv[1])
    if not path.is_file():
        print(f"no freeze file at {path}", file=sys.stderr)
        return 1
    errors = validate(load(path))
    if errors:
        print("unpinned run:")
        for item in errors:
            print(f"- {item}")
        return 1
    print("freeze file ok")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Run the local gate like this.

sha256sum prompt.txt patch.diff
python3 validate_freeze.py freeze.json
pytest -q tests/test_billing.py

Want a shorter rule of thumb?
If validate_freeze.py cannot run, you vibe-coded.
The chat UI will not argue with that.

Anti-pattern 1: Unpinned inference

You asked the box. It answered.
You shipped the vibe anyway.
You never named the runtime kind.

Symptoms

Tomorrow's answer flips on the same prompt.
Nobody can name the runtime kind.
"Works on my machine" now means "works in chat."

Root cause

You treated a remote completion like gcc -v.
Compilers publish versions you can pin.
Many chat boxes do not.

Replacement pattern

Record runtime_kind before you accept a patch.
Reproduce the patch with local tests.
Refuse to merge folklore from a thread.

jq -r .runtime_kind freeze.json
# expected: local | scratch-pad | ci-pinned

Did the model get smarter overnight?
Or did the unnamed runtime drift?
I assume drift until the freeze says otherwise.

Anti-pattern 2: Chat log as the audit trail

The thread looks complete and friendly.
It even has jokes in the margins.
Auditors cannot replay jokes later.

Symptoms

The only spec lives in a vendor UI.
Patch files have no prompt hash.
New hires cannot reconstruct the change.

Root cause

You stored conversation, not evidence.
A chat log is not an artifact.
It is a diary with extra tokens.

Replacement pattern

Export three files, always, in one commit.
Keep prompt.txt, patch.diff, and freeze.json.
Then hash them before anyone reviews the patch.

git add prompt.txt patch.diff freeze.json
git commit -m "pin billing patch to freeze file"

Did the assistant summarize the diff for you?
I do not care about that summary.
I care whether patch_sha256 matches git diff.

python3 - <<'PY'
import hashlib, pathlib, json, sys
patch = pathlib.Path("patch.diff").read_bytes()
digest = hashlib.sha256(patch).hexdigest()
frozen = json.loads(pathlib.Path("freeze.json").read_text())
sys.exit(0 if frozen.get("patch_sha256") == digest else 1)
PY

Anti-pattern 3: Free server as CI

Exploration is cheap, and that is fine.
CI is a different machine with different trust.
Do not blur those two on purpose.

I still want a scratch pad for messy prompts.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
MonkeyCode offers free model access and a free server option.

I map that pair to runtime_kind: scratch-pad only.
Draft there if you need a pad.
Then copy the patch home and run pytest locally.

Symptoms

Shared scratch boxes absorb flaky tests.
Failures get blamed on "the model."
Secrets leak into a communal prompt.

Root cause

You confused availability with isolation.
A free server is a shared whiteboard.
Whiteboards do not sign releases.

Replacement pattern

Keep the scratch pad in the draft lane.
Never point test_command at a shared box.
Set reproduced_locally only after local pytest.

{
  "runtime_kind": "scratch-pad",
  "reproduced_locally": true,
  "test_command": "pytest -q tests/test_billing.py"
}

If reproduced_locally is still false, stop.
You have a sketch, not a candidate patch.
The validator will say the same thing.

Anti-pattern 4: Tests written after the patch

The model "solved" the ticket first.
Then you asked it for tests.
The tests learned the first answer.

Symptoms

Tests assert the first returned string.
Edge cases never appear in the file.
Refactors break tests that never encoded rules.

Root cause

You used tests as applause.
Applause is not a specification.
The model taught the oracle.

Replacement pattern

Write the failing test first.
Commit it on main before any prompt.
Only then paste the prompt that cites it.

# tests/test_billing.py — proposed example, not executed here
def test_prorate_ignores_trailing_partial_day():
    assert prorate("2026-09-01", "2026-09-16", 3100) == 1500

Run this loop without skipping step one.

pytest must fail on main.
Freeze the prompt that cites that test.
Accept a patch only if the same test passes.
Re-run python3 validate_freeze.py freeze.json.

If step one was already green, you cheated.
The model did not invent your spec.
You let it grade its own homework.

Decision table I actually use

What I saw	Story I tell myself	Anti-pattern	Replacement
Same prompt, new answer	The model got worse	Unpinned inference	Freeze runtime, replay locally
Long chat history	We have an audit	Chat log as audit	Commit prompt, patch, freeze
Tests on a shared box	CI is covered	Free server as CI	Local pytest gate
Tests after the patch	We are thorough	Post-hoc tests	Failing test first

Print that table next to the PR.
If a row matches, name the anti-pattern.
Do not argue about model IQ instead.

A concrete debugging workflow

I use this when a "smart" patch rots.
It is a proposed workflow, not a benchmark.
I have not executed it against your repo.

Print the freeze file. Missing file means stop.
Check secrets_present. True means rotate keys.
Confirm reproduced_locally is actually true.
Replay test_command on a dirty checkout.
If replay fails, blame the freeze, not the model.

test -f freeze.json || { echo "unpinned"; exit 1; }
python3 validate_freeze.py freeze.json
git stash push -u -m "temp"
bash -lc "$(jq -r .test_command freeze.json)"

Why stash first?
Because dirty trees hide lucky passes.
I want the freeze to survive a clean tree.

Limitations

A freeze file is not magic.
Hashing a prompt does not freeze a remote model.
The validator cannot see server-side drift.

scratch-pad is an honesty label.
It does not claim isolation or uptime.
It does not claim a service level.

Do not use this approach when you need these.

Production traffic on the same box
Secret-bearing prompts on a shared server
A contractual SLA for inference
A merge that skips local replay

Regulated workflows need a runtime you control.
This catalog will not give you that.
It will only stop you lying to git.

Solo explorers can skip the freeze for toys.
Throwaway katas do not need freeze.json.
Anything you might merge tomorrow does.

What I ship

I ship a patch, a test, and a freeze file.
I do not ship a chat screenshot.
A green chat is still an unpinned run.

If the freeze is missing, I failed.
The model did not fail me first.
Pin the run or admit you vibe-coded.