Start From a Red Test. Keep the Model Off Your Shell.

The model does not get a shell. It does not get your working tree. It gets a failing test node, a handful of AST signatures, and a path allowlist. You keep merge.

Want the agent to “just fix the repo”? That is how a helper rewrite deletes a fixture and “simplifies” your config. Engineering is the gate, not the prompt. This walkthrough is a from-zero loop you can run on a laptop. Each stage has a command and a pass/fail check. No fake latency numbers. No invented model names.

What you will have when it works

A tiny broken Python package. A red pytest node that you wrote by hand. A job file that never includes production bodies. A checker that rejects any unified diff walking off an allowlist. A disposable worktree that applies the patch and runs one test.

If a stage fails, you stop. You do not “let it try again in the same tree.”

Stage 1 — Make a red repo you actually own

Do this in an empty directory. Keep it boring on purpose.

mkdir -p taxlib tests
cat > taxlib/__init__.py << 'EOF'
from .compute import add_tax
EOF

cat > taxlib/compute.py << 'EOF'
def add_tax(cents: int, rate_bps: int) -> int:
    # Bug: integer truncation, no rounding.
    return cents + (cents * rate_bps) // 10_000
EOF

cat > tests/test_compute.py << 'EOF'
from taxlib.compute import add_tax

def test_add_tax_rounds_half_up():
    # 199 cents at 7.25% should round to 213, not 212.
    assert add_tax(199, 725) == 213
EOF

python -m pytest tests/test_compute.py::test_add_tax_rounds_half_up -q

Verify: pytest exits non-zero. If it is green, you have nothing to ask a model. Why call a remote box to celebrate a passing test?

Stage 2 — Extract signatures, not source

I refuse to paste compute.py into a prompt. The implementation is the thing I do not trust a model to “improve” wholesale. I send names, arguments, and return annotations. Bodies stay on disk.

# extract_sigs.py
from __future__ import annotations

import ast
import json
import sys
from pathlib import Path

def signatures(path: Path) -> list[dict]:
    tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
    out: list[dict] = []
    for node in tree.body:
        if isinstance(node, ast.FunctionDef):
            args = [a.arg for a in node.args.args]
            ret = ast.unparse(node.returns) if node.returns is not None else None
            out.append({"name": node.name, "args": args, "returns": ret})
    return out

if __name__ == "__main__":
    target = Path(sys.argv[1])
    print(json.dumps({"file": str(target), "sigs": signatures(target)}, indent=2))

python extract_sigs.py taxlib/compute.py > /tmp/sigs.json
python -c "import json; d=json.load(open('/tmp/sigs.json')); assert d['sigs'][0]['name']=='add_tax'"

Verify: the JSON has add_tax and no substring of integer truncation. rg -n "truncation" /tmp/sigs.json must print nothing. If the body leaked, your extractor is the bug, not the model.

Stage 3 — Write the allowlist before the prompt

An allowlist is not a hint. It is a reject rule. I put it in a file so I cannot “just this once” expand it in chat.

printf '%s\n' 'taxlib/compute.py' > allowlist.txt
test "$(wc -l < allowlist.txt)" -eq 1

Verify: exactly one path. If you feel the urge to add tests/ or pyproject.toml, stop. The test is already green or red on your machine. The model does not get to edit evidence.

Stage 4 — Build a job the model cannot wander out of

This is not “here is my repo, go.” It is a contract. Proposed shape — label it as such until you run it:

# make_job.py
from __future__ import annotations

import json
from pathlib import Path

sigs = json.loads(Path("/tmp/sigs.json").read_text(encoding="utf-8"))
test_src = Path("tests/test_compute.py").read_text(encoding="utf-8")
allow = Path("allowlist.txt").read_text(encoding="utf-8").splitlines()

job = {
    "goal": "Make tests/test_compute.py::test_add_tax_rounds_half_up pass.",
    "constraints": [
        "Reply with a unified diff only. No prose. No shell.",
        "Touch only paths in allowlist.",
        "Do not modify tests.",
    ],
    "allowlist": allow,
    "signatures": sigs,
    "failing_test": {
        "nodeid": "tests/test_compute.py::test_add_tax_rounds_half_up",
        "source": test_src,
    },
}
Path("job.json").write_text(json.dumps(job, indent=2) + "\n", encoding="utf-8")

python make_job.py
python -c "import json; j=json.load(open('job.json')); assert 'compute.py' in ''.join(j['allowlist'])"

Verify: job.json exists, contains the node id, and still has no implementation body from compute.py. Open it. Read it. Would you paste this into a vendor chat by accident? If yes, shrink it again.

Stage 5 — Ask for a unified diff, nothing else

I do not give the model bash. I do not give it apply_patch with a live tree. I POST the job and I expect a diff. If you do not have a paid API, this is the only place a hosted coding endpoint belongs.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. I use MonkeyCode here as a free-model endpoint plus an optional free server so the HTTP call is not tied to my laptop. I am not claiming model names, quotas, hardware, or how long that free tier lasts. Treat those as operator-stated availability, then verify them yourself before you depend on them.

Proposed client — unexecuted until you point BASE_URL at a real host:

# request_diff.py
from __future__ import annotations

import json
import os
import sys
import urllib.request
from pathlib import Path

job = Path("job.json").read_text(encoding="utf-8")
url = os.environ.get("BASE_URL", "").rstrip("/") + "/v1/diff"
if not os.environ.get("BASE_URL"):
    sys.stderr.write("Set BASE_URL to your model endpoint. No silent localhost fallback.\n")
    sys.exit(2)

req = urllib.request.Request(
    url,
    data=job.encode("utf-8"),
    headers={"Content-Type": "application/json"},
    method="POST",
)
with urllib.request.urlopen(req, timeout=60) as resp:
    body = resp.read().decode("utf-8")

Path("patch.diff").write_text(body if body.endswith("\n") else body + "\n", encoding="utf-8")
print("wrote patch.diff", len(body), "bytes")

Until BASE_URL is set, keep a fixture so the rest of the tutorial is runnable:

cat > patch.diff << 'EOF'
--- a/taxlib/compute.py
+++ b/taxlib/compute.py
@@ -1,3 +1,6 @@
 def add_tax(cents: int, rate_bps: int) -> int:
-    # Bug: integer truncation, no rounding.
-    return cents + (cents * rate_bps) // 10_000
+    n = cents * rate_bps
+    q, r = divmod(n, 10_000)
+    if r * 2 >= 10_000:
+        q += 1
+    return cents + q
EOF

Verify: head -n 1 patch.diff starts with --- or diff. If you got a markdown apology, you do not have a patch. Do not “clean it up by hand” and pretend the loop worked.

Stage 6 — Reject walks off the allowlist

git apply --check is useful. It is not enough. A patch can be syntactically fine and still edit README.md. I parse the diff myself.

# check_diff_allowlist.py
from __future__ import annotations

import re
import sys
from pathlib import Path

PLUS = re.compile(r"^\+\+\+ b/(.+)$")
MINUS = re.compile(r"^--- a/(.+)$")

def paths_in(diff: str) -> set[str]:
    found: set[str] = set()
    for line in diff.splitlines():
        for rx in (PLUS, MINUS):
            m = rx.match(line)
            if m and m.group(1) != "/dev/null":
                found.add(m.group(1))
    return found

def main() -> int:
    diff = Path("patch.diff").read_text(encoding="utf-8")
    allow = {p.strip() for p in Path("allowlist.txt").read_text(encoding="utf-8").splitlines() if p.strip()}
    got = paths_in(diff)
    extra = sorted(got - allow)
    missing_touch = sorted(allow - got)
    if extra:
        print("REJECT extra paths:", extra)
        return 1
    if not got:
        print("REJECT empty diff")
        return 1
    print("OK paths:", sorted(got), "allowlist unused:", missing_touch)
    return 0

if __name__ == "__main__":
    raise SystemExit(main())

python check_diff_allowlist.py

Verify: exit code 0 and the only path printed is taxlib/compute.py. Now sabotage the fixture on purpose:

printf '\n--- a/README.md\n+++ b/README.md\n@@ -0,0 +1 @@\n+owned\n' >> patch.diff
python check_diff_allowlist.py; echo exit:$?

Verify: exit code 1. If that still passes, your checker is theater. Restore patch.diff from Stage 5 before you continue. Did you skip the sabotage? Then you never tested the reject path.

Stage 7 — Apply in a worktree, run one node

The live tree stays red until the oracle says otherwise. A second clone is cheap. A silent in-place apply is how you lose an afternoon.

git init -q
git add taxlib tests extract_sigs.py make_job.py check_diff_allowlist.py allowlist.txt job.json
git commit -qm 'red tax helper'

git worktree add /tmp/tax-oracle HEAD
cp patch.diff /tmp/tax-oracle/
(
  cd /tmp/tax-oracle
  git apply patch.diff
  python -m pytest tests/test_compute.py::test_add_tax_rounds_half_up -q
)
echo oracle:$?

Verify: the oracle directory exits 0. Your original tree still fails the same node. That split is the whole point. Promote only with a copy you can read:

git -C /tmp/tax-oracle diff HEAD -- taxlib/compute.py
# if you like the diff:
cp /tmp/tax-oracle/taxlib/compute.py taxlib/compute.py
python -m pytest tests/test_compute.py::test_add_tax_rounds_half_up -q

Verify: now the original tree is green. If you skipped the worktree and applied in place, you do not have a loop. You have hope.

Optional: run the oracle on a free server, still without a shell for the model

Generation and merge are different jobs. The model still returns bytes. If you want the pytest oracle off your laptop, ship only allowlist.txt, patch.diff, the allowlisted sources, and the single test file to an isolated runner. Not .env. Not .git. Not the rest of the tree.

MonkeyCode’s free server option is one place you can park that runner. Same gates. Same reject script. If the remote box cannot run check_diff_allowlist.py before git apply, do not use it. A free GPU that executes arbitrary tool calls is not a gate. It is a bigger blast radius.

Limitations

This does not prove rounding is correct for negative cents, for zero, or for huge rate_bps. One node is an oracle for that node. AST extraction misses decorators you care about, nested functions, and comments that encode law. Unified diff parsing misses rename-heavy patches unless you extend the checker. A model can still write a “fix” that hard-codes return 213. Your test would go green. That is a test design hole, not a hosting hole.

I did not benchmark tokens. I did not time the HTTP call. If someone quotes a million-token figure at you, ask them to show the current product page. Do not copy a number from a blog, including this one.

Who should not use this

Skip it if you need the model to explore an unfamiliar codebase by reading everything. Skip it if your “test” is a screenshot. Skip it if policy forbids sending even a failing test to a hosted endpoint. Skip it if you will ignore a reject and paste the diff anyway. The loop only works if a non-zero exit means stop.

The free-model path is for people who want a stateless diff compiler. It is not for people who want an unsupervised intern with rm.

Steal the scripts either way. If you need a hosted endpoint so Stage 5 is not a stub, point BASE_URL at MonkeyCode’s free model access and keep the allowlist checker on your side of the cable.