Cheap inference does not replace a human review gate.
It makes five skip patterns cheaper to repeat.
I still catch them in my own branches.
What this piece is not
This is not another vibe-coding culture rant.
I will not grade your editor or your chat model.
I will show symptoms, root causes, and replacements.
The public debate this week is still noisy.
People argue about whether AI writes real software.
The useful question is smaller and much meaner.
Did this diff actually get a review receipt?
Or did a cheap run just feel like shipping?
The catalog at a glance
I use the same three columns every time.
- Symptom: what shows up in git or chat.
- Root cause: why cheap inference encourages the skip.
- Replacement: the pattern I want instead.
The proposed scripts and tables live right below.
Treat the scripts as templates, not production gospel.
Anti-pattern 1: Chat as the source of truth
Symptoms
- The pull request says "details in the thread."
- Commits read like "update stuff" and nothing else.
- Reviewers paste screenshots instead of file paths.
Root cause
The sidebar answered faster than the commit template.
The answer felt documented because it was long.
Git never stored the actual product decision.
Replacement: put the decision in git
I put the tradeoff in the commit body.
I link the chat only as appendix, never as proof.
If the chat dies, the repo must still explain the change.
git commit -m "$(cat <<'EOF'
reject unsigned webhook payloads
Why: the agent added a debug bypass on staging.
Decision: verify signatures in the wrapper, not the prompt.
Chat transcript is appendix only, not source.
EOF
)"
Ask yourself one rude and practical question.
Could a new teammate review this without the chat?
If the answer is no, the chat still owns your history.
Anti-pattern 2: Secrets riding the free prompt
Symptoms
- Env fragments appear inside a pasted stack trace.
- The prompt file contains hostnames from production.
- The agent reprints a token in the session log.
Root cause
A free box still feels disposable and unofficial.
Disposable does not mean the prompt is private.
Cheap inference can still persist that text somewhere.
Replacement: redact, then hash
I redact the raw prompt first, without exceptions.
I hash the cleaned prompt file second.
I never paste live credentials into a sandbox prompt.
# Proposed local redaction before any remote prompt.
sed -E \
-e 's/[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+/[redacted-email]/g' \
-e 's/(api[_-]?key|token|secret)[=:][^[:space:]]+/\1=[redacted]/Ig' \
-e 's/[0-9]{1,3}(\.[0-9]{1,3}){3}/[redacted-ip]/g' \
prompt.raw.txt > prompt.clean.txt
sha256sum prompt.clean.txt > prompt.clean.sha256
Run that before the model sees the file.
If redaction fails, the session does not start.
Would you paste that prompt into a public gist?
If not, do not send it to a free endpoint either.
Anti-pattern 3: The agent owns the lockfile
Symptoms
- The lockfile moves with a one-line behavior fix.
- The model tidies dependencies during a simple rename.
- Nobody can explain the version jump in review.
Root cause
Cheap retries invite extra cleanup turns from the agent.
Those extra cleanup turns always love the lockfile.
Lockfiles are easy to generate and hard to audit.
Replacement: lockfile needs a human ACK
I split lockfile diffs from the behavior diffs.
I require an explicit ACK file in the pull request.
No ACK means the CI job fails closed.
# proposed_lockfile_ack.py
# Example only. Adapt paths before you trust it.
from pathlib import Path
import subprocess
import sys
LOCK_NAMES = {"package-lock.json", "poetry.lock", "Cargo.lock", "go.sum"}
ACK = Path("LOCKFILE_ACK.md")
def changed_files() -> set[str]:
out = subprocess.check_output(
["git", "diff", "--name-only", "origin/main...HEAD"],
text=True,
)
return {Path(p).name for p in out.splitlines() if p}
changed = changed_files()
touched_lock = sorted(changed & LOCK_NAMES)
if not touched_lock:
print("no lockfile change")
sys.exit(0)
if not ACK.exists():
print("lockfile changed without LOCKFILE_ACK.md")
sys.exit(1)
text = ACK.read_text(encoding="utf-8").lower()
if "i reviewed the lockfile diff" not in text:
print("ACK is missing the review sentence")
sys.exit(1)
print("lockfile ACK present for:", ", ".join(touched_lock))
Put that ACK in the same pull request.
Name every package version that actually moved.
Name the risk in one blunt sentence.
Did the agent bump a compiler to make tests pass?
That compiler bump is not just a tidy.
That bump is a silent product change too.
Anti-pattern 4: Policy living in the system prompt
Symptoms
- "Do not call production" exists only in a prompt.
- The tool wrapper still accepts the prod hostname.
- Safety is a paragraph, not a check.
Root cause
Prompts are easy to type on a free model.
Guards in code take an extra hour.
Cheap inference rewards the hour you skip.
Replacement: policy in code, prompt as hint
I keep the system prompt short and polite.
I keep the policy in a denylist.
The model cannot override a failed hostname check.
# proposed_tool_guard.py
from urllib.parse import urlparse
BLOCKED_HOST_SUFFIXES = (".prod.internal", ".corp.example")
BLOCKED_HOSTS = {"127.0.0.1"}
def assert_safe_url(url: str) -> None:
host = (urlparse(url).hostname or "").lower()
if not host:
raise ValueError("missing host")
if host in BLOCKED_HOSTS:
raise ValueError(f"blocked host: {host}")
if any(host.endswith(sfx) for sfx in BLOCKED_HOST_SUFFIXES):
raise ValueError(f"blocked suffix: {host}")
The prompt may still say "stay in staging."
The wrapper must still refuse the URL.
Which layer do you trust when the model is tired?
Anti-pattern 5: Idle free capacity as a release signal
Symptoms
- The merge happened because the server was free tonight.
- The reviewer is whoever is still online in chat.
- Monday on-call inherits an unexplained diff.
Root cause
Cheap idle time feels like a wasted gift.
That waste anxiety beats review patience every time.
The review calendar never becomes a real gate.
Replacement: capacity is not a criterion
I keep capacity out of the ship list.
A free server can wait until review exists.
Idle machines do not approve the architecture change.
Would you merge this if the box were busy?
If the answer flips, capacity was the reviewer.
Ship criteria belong in the receipt, not the queue.
The artifact: a review receipt
I want one file per pull request.
I call that file REVIEW_RECEIPT.md on purpose.
This file is not a model freeze file.
A freeze file only pins the model inputs.
This receipt pins the review skips I just named.
# REVIEW_RECEIPT.md
- Decision lives in commit: yes/no
- Prompt redacted and hashed: yes/no (sha256: ...)
- Lockfile ACK present: yes/no/n/a
- Tool policy enforced in code: yes/no
- Capacity was not a ship criterion: yes/no
- Reviewer: name
- Date: ISO-8601
Use this table when the sandbox looks too green.
| Claim you want | Cheap sandbox can show | Cheap sandbox cannot show |
|---|---|---|
| Script parses on a clean clone | Yes | Your dirty production working tree |
| Unit tests pass with fixtures | Yes | Real traffic shape |
| Tool guard blocks fixture URLs | Yes | A hostname someone adds later |
| Prompt hash matches redacted text | Yes | Extra secrets pasted only in chat |
| Feature is safe to ship | No | Data, auth, clocks, or load |
If the table cell says no, the merge still needs a human.
Proposed checker:
# proposed_receipt_check.py
from pathlib import Path
import re
import sys
receipt = Path("REVIEW_RECEIPT.md")
if not receipt.exists():
print("missing REVIEW_RECEIPT.md")
sys.exit(1)
text = receipt.read_text(encoding="utf-8")
required = [
"Decision lives in commit:",
"Prompt redacted and hashed:",
"Lockfile ACK present:",
"Tool policy enforced in code:",
"Capacity was not a ship criterion:",
"Reviewer:",
]
missing = [line for line in required if line.lower() not in text.lower()]
if missing:
print("receipt missing fields:", ", ".join(missing))
sys.exit(1)
if re.search(r"sha256:\s*$", text, re.I | re.M):
print("hash field is empty")
sys.exit(1)
print("review receipt fields present")
Then you wire the checker into CI.
# proposed .github/workflows/review-receipt.yml
name: review-receipt
on: [pull_request]
jobs:
receipt:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: python proposed_receipt_check.py
I have not executed this workflow on your org.
Copy it only after you change the names.
A cheap rehearsal, not a release
I still want a cheap place to practice the gate.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
MonkeyCode offers free model access and a free server option.
I treat those as a rehearsal room for the receipt checker.
I do not treat them as production capacity, an SLA, or a benchmark.
If you rehearse there, run the redaction step first.
Then run the receipt checker on a throwaway branch.
Skip the demo prompt that never touches git.
Who should not use this approach
Do not use a free server for regulated customer data.
Do not use prompt text as your only access control.
Do not skip this catalog because the tokens were cheap.
This receipt is only a minimum bar.
It will not catch a clever exfil path.
It will not replace a real threat model.
Teams with no git history should not start here.
You should fix basic source control first here.
Then you add the review receipt file.
Limitations
The checker only proves that a file exists.
A bored reviewer can type "yes" everywhere.
That is still a process failure, not a model failure.
Cheap inference will keep getting even cheaper.
Your gates will not appear by accident.
Build them while the diffs are still small.
Would I merge a green sandbox without the receipt?
I would not merge that green branch yet.
Would I trust the receipt without reading the diff?
I would also refuse that lazy shortcut.
Top comments (0)