Scratch-Box the Agent Loop. Promote the Diff Only When the Hash Matches.

Stop applying agent diffs on the laptop that holds your secrets. Freeze a fixture, run the loop on a scratch box, and promote the tree only when the output hash matches the lock you already checked in. Everything else is theater.

You already know the failure. A free remote model “fixes” a test. It also rewrites a Makefile target you still needed, and git status looks like a yard sale. Why would you let an untrusted compiler write into the same tree you deploy from?

This is a from-zero walkthrough. Each stage has a command and a verification step. The model never touches your working copy. It emits a tree on a scratch box. You accept that tree only after sha256 agrees with a lockfile you wrote before the call.

What you will have when this works

A job folder you can rsync. A fixture that pins inputs. A receipt that records what the runner actually did. A promote step that copies files only on hash match. If the hash misses, the laptop stays clean. That is the whole contract.

I am not asking the model to be honest. I am asking the filesystem to be boring.

Stage 1 — Make a job that cannot see your real repo

Start in an empty directory. Do not run this inside an app you care about. The scratch box should look like a parcel, not like $HOME.

mkdir -p agent-scratch/{job,fixture,scratch,out}
cd agent-scratch
printf 'def add(a, b):\n    return a - b\n' > job/calc.py
printf 'from calc import add\n\ndef test_add():\n    assert add(2, 3) == 5\n' > job/test_calc.py
printf 'add is broken on purpose\n' > job/README.md

Verification:

find job -type f | sort
python3 -c "from pathlib import Path; print(Path('job/calc.py').read_text())"

You should see three files and a function that subtracts. If add already works, you have nothing to gate. Broken on purpose is the point.

Stage 2 — Freeze the fixture before any model call

The fixture is not a prompt dump. It is the bytes you allow the runner to read, plus the bytes you will hash after the run. Write that down first. If you freeze after the agent “helps,” you are locking a crime scene.

cat > freeze_fixture.py << 'PY'
#!/usr/bin/env python3
from pathlib import Path
import hashlib, json, sys

ROOT = Path(sys.argv[1] if len(sys.argv) > 1 else "job")

def sha256(p: Path) -> str:
    h = hashlib.sha256()
    h.update(p.read_bytes())
    return h.hexdigest()

files = sorted(
    p for p in ROOT.rglob("*")
    if p.is_file() and p.name != ".DS_Store"
)
fixture = {
    "root": str(ROOT),
    "inputs": {str(p.relative_to(ROOT)): sha256(p) for p in files},
    "expect_paths": ["calc.py", "test_calc.py"],
    "forbid_paths": ["README.md"],
    "must_contain": {"calc.py": "return a + b"},
    "tests": ["python3", "-m", "unittest", "test_calc.py"],
}
Path("fixture/lock.json").write_text(json.dumps(fixture, indent=2) + "\n")
print(f"locked {len(files)} files")
PY
python3 freeze_fixture.py job

Verification:

python3 -m json.tool fixture/lock.json | head
grep forbid_paths fixture/lock.json

README.md is in the job, but it is forbidden as an output. The agent may read it. It may not rewrite it. If your lock does not name a forbid list, the model will “improve” docs until they lie.

Stage 3 — Give the runner a receipt schema, not a vibe

I want one JSON object back. Not a chat. Not a patch with a smile. A receipt. If the runner cannot fill the schema, the run is a miss. No retry storm. No “it almost compiled.”

cat > fixture/receipt.schema.json << 'JSON'
{
  "type": "object",
  "required": ["ok", "changed", "sha256", "skipped", "reason"],
  "additionalProperties": false,
  "properties": {
    "ok": {"type": "boolean"},
    "changed": {"type": "array", "items": {"type": "string"}},
    "sha256": {"type": "object", "additionalProperties": {"type": "string"}},
    "skipped": {"type": "array", "items": {"type": "string"}},
    "reason": {"type": "string"}
  }
}
JSON

Verification:

python3 -c "import json; json.load(open('fixture/receipt.schema.json')); print('schema ok')"

Why a schema here? Because a free remote model will happily narrate. Narration is not a tree. If you cannot hash it, you cannot promote it. Simple.

Stage 4 — Write a runner that treats the model as a byte source

This runner copies job/ into scratch/, asks a function for a proposed tree, writes only allowed paths, then hashes. The function is a stand-in. Swap it for an HTTP call later. Do not swap it until the gate works against a deterministic fake. If you cannot pass a fake, a live model will just hide the bugs.

cat > runner.py << 'PY'
#!/usr/bin/env python3
"""Proposed local/remote runner. No network. Deterministic fake model."""
from __future__ import annotations
from pathlib import Path
import hashlib, json, shutil, subprocess, sys

JOB = Path("job")
SCRATCH = Path("scratch/tree")
LOCK = json.loads(Path("fixture/lock.json").read_text())
OUT = Path("out/receipt.json")

def sha256(p: Path) -> str:
    h = hashlib.sha256()
    h.update(p.read_bytes())
    return h.hexdigest()

def fake_model(tree: dict[str, str]) -> dict[str, str]:
    # Proposed replacement for a remote completion. Returns a full tree.
    proposed = dict(tree)
    proposed["calc.py"] = "def add(a, b):\n    return a + b\n"
    proposed["README.md"] = "# rewritten by the model, ignore me\n"
    proposed["extra.py"] = "print('side quest')\n"
    return proposed

def main() -> int:
    if SCRATCH.exists():
        shutil.rmtree(SCRATCH)
    shutil.copytree(JOB, SCRATCH)

    tree = {p.name: p.read_text() for p in SCRATCH.iterdir() if p.is_file()}
    proposed = fake_model(tree)

    allowed = set(LOCK["expect_paths"])
    forbidden = set(LOCK["forbid_paths"])
    changed, skipped = [], []

    for name, body in proposed.items():
        if name in forbidden or name not in allowed:
            skipped.append(name)
            continue
        target = SCRATCH / name
        if target.read_text() != body:
            target.write_text(body)
            changed.append(name)

    for name, needle in LOCK["must_contain"].items():
        text = (SCRATCH / name).read_text()
        if needle not in text:
            receipt = {
                "ok": False,
                "changed": changed,
                "sha256": {},
                "skipped": skipped,
                "reason": f"{name} missing {needle!r}",
            }
            OUT.write_text(json.dumps(receipt, indent=2) + "\n")
            print(receipt["reason"])
            return 2

    proc = subprocess.run(
        LOCK["tests"], cwd=SCRATCH, capture_output=True, text=True
    )
    hashes = {
        p.name: sha256(p)
        for p in SCRATCH.iterdir()
        if p.is_file() and p.name in allowed
    }
    ok = proc.returncode == 0
    receipt = {
        "ok": ok,
        "changed": sorted(changed),
        "sha256": hashes,
        "skipped": sorted(set(skipped)),
        "reason": "tests passed" if ok else proc.stderr[-400:],
    }
    OUT.write_text(json.dumps(receipt, indent=2) + "\n")
    print(json.dumps(receipt, indent=2))
    return 0 if ok else 3

if __name__ == "__main__":
    sys.exit(main())
PY
python3 runner.py

Verification:

python3 -m json.tool out/receipt.json
grep extra.py out/receipt.json
grep README.md out/receipt.json
python3 -c "from pathlib import Path; print(Path('scratch/tree/calc.py').read_text())"

ok should be true. changed should list calc.py only. skipped should list README.md and extra.py. The scratch calc.py should add. The job copy should still subtract. If job/calc.py changed, you ran the runner against the wrong root. Stop.

See the fake model? It tried to rewrite the README and drop a side-quest file. The runner threw those writes away. That is the lesson. A free endpoint will do the same thing with more adjectives.

Stage 5 — Lock the output hash, not the prompt

A passing receipt is not a promote. Passing today and passing tomorrow must be the same bytes. Write the output lock from a receipt you actually inspected.

cat > lock_output.py << 'PY'
#!/usr/bin/env python3
from pathlib import Path
import json, sys

receipt = json.loads(Path("out/receipt.json").read_text())
if not receipt["ok"]:
    sys.exit("refusing to lock a failed receipt")
lock = {
    "changed": receipt["changed"],
    "sha256": receipt["sha256"],
    "skipped": receipt["skipped"],
}
Path("fixture/output.lock.json").write_text(json.dumps(lock, indent=2) + "\n")
print("output lock written")
PY
python3 lock_output.py

Verification:

cat fixture/output.lock.json
test -f fixture/output.lock.json && echo locked

Commit fixture/lock.json and fixture/output.lock.json together. If you only commit the input fixture, the next run can “pass” with a different calc.py that still contains return a + b. Passing is not identical. Identical is identical.

Stage 6 — Promote only on match

Promotion is a copy. It is not a merge. It is not “looks good in the chat.” If the receipt hash drifts, you do nothing to job/.

cat > promote.py << 'PY'
#!/usr/bin/env python3
from pathlib import Path
import json, shutil, sys

receipt = json.loads(Path("out/receipt.json").read_text())
lock = json.loads(Path("fixture/output.lock.json").read_text())
scratch = Path("scratch/tree")
job = Path("job")

if not receipt["ok"]:
    sys.exit("receipt not ok")
if receipt["sha256"] != lock["sha256"]:
    sys.exit(f"hash miss: {receipt['sha256']} != {lock['sha256']}")
if receipt["changed"] != lock["changed"]:
    sys.exit("changed set drifted")

for name in receipt["changed"]:
    shutil.copyfile(scratch / name, job / name)
    print(f"promoted {name}")
PY
python3 promote.py

Verification:

python3 -c "from pathlib import Path; print(Path('job/calc.py').read_text())"
grep 'return a + b' job/calc.py
python3 -m unittest job/test_calc.py

job/calc.py should now add. job/README.md should still say it is broken on purpose. If the README moved, promotion leaked. That is a runner bug, not a model bug.

Stage 7 — Prove the fail-closed path

A gate you never fail is a comment. Break the fake model and watch promote refuse.

python3 - << 'PY'
from pathlib import Path
p = Path("runner.py")
text = p.read_text().replace("return a + b", "return a * b", 1)
p.write_text(text)
print("mutated fake model")
PY
python3 runner.py; echo "runner exit $?"
python3 promote.py; echo "promote exit $?"

Verification: runner should write ok: false or a hash that is not in output.lock.json. Promote should exit non-zero. job/calc.py should still contain return a + b from the previous good promote, not a multiply. Restore the runner when you are done poking it.

git checkout -- runner.py 2>/dev/null || true

No git? Put the + back by hand. Do not leave a multiplied fake in the tree. You will confuse yourself in an hour.

Stage 8 — Move the runner, not the secrets

The laptop copy is a rehearsal. The real dry-run belongs on a box that does not mount your SSH keys, your .env, or your Docker socket. Same files. Same commands. Different machine.

Proposed remote hop — labeled, because your host names will differ:

# Proposed: sync the parcel, not your home directory.
rsync -a --delete \
  --exclude '.git' --exclude '.env' --exclude '*.pem' \
  ./job ./fixture ./runner.py ./promote.py ./freeze_fixture.py \
  scratch-host:agent-scratch/

ssh scratch-host 'cd agent-scratch && python3 runner.py'
rsync -a scratch-host:agent-scratch/out/receipt.json ./out/receipt.json
python3 promote.py

Verification on the far side:

ssh scratch-host 'test ! -e ~/.ssh/id_rsa && test ! -e .env && echo "scratch looks empty of secrets"'
ssh scratch-host 'python3 -m json.tool agent-scratch/out/receipt.json | head'

If the scratch host can see production credentials, it is not a scratch host. It is a second laptop. Call it what it is.

Where do you get a box and a model without standing up a lab? Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode is an open-source project with free model access and a free server option. That pair maps onto this layout: park runner.py on the free server, keep promote.py on the laptop, move receipts not secrets. The fixture files do not care which host hashed them. If you already have a throwaway VM, use that. The workflow is the product here.

A small decision table, because vibes do not promote

Receipt field	Laptop action
ok is false	Do not copy. Do not “just take calc.py.”
sha256 ≠ output lock	Do not copy. Re-inspect. Maybe re-freeze.
extra paths in changed	Do not copy. The runner leaked.
skipped lost a forbid path	Do not copy. The forbid list regressed.
exact match	Copy only changed. Leave everything else.

Read that last row again. Promotion is not rsync scratch/ job/. Promotion is a whitelist copy of names you already locked.

Limitations — read these before you automate it

This does not bound a model that needs live network side effects. If the “fix” is “charge the card” or “migrate prod,” a hash match is not a safety story. Do not use a scratch box as a fig leaf for irreversible actions.

The fake model is deterministic. A live free model is not. If you skip Stage 4 and jump to a real endpoint, you will chase flakes and blame the gate. Freeze the runner against a fake first. Then wrap the HTTP call behind the same proposed dict. One seam.

Hashes do not capture meaning. Two different calc.py files can both contain return a + b and still be wrong for your API. The must_contain check is a tripwire, not a proof. Add real tests to LOCK["tests"] or you are hashing fan fiction.

I also would not run this on data you cannot put on a third-party box. Free servers are still someone else’s disk. Fixture jobs should be synthetic. If the bug only reproduces with a customer dump, redact it or keep the runner local and accept that you lost the isolation.

Who should skip this? Anyone who needs a one-shot refactor across 400 files with no tests. Anyone whose “agent” already has write mounts on prod. Anyone who will ignore a hash miss because the chat sounded confident. This workflow is for small, test-backed edits you can freeze. It is not a personality.

What I want you to run tomorrow

One parcel. One lock. One receipt. One promote. If you cannot show me the hash that failed, you did not run the gate. You ran a demo.

Questions I ask myself before I even open the model pane: Can this job die if the network dies? Can I delete the scratch box and still rebuild the fixture? Does job/ still subtract until promote says otherwise? If any answer is no, I am back to letting a remote completion write my tree. I have watched that movie. The Makefile never survives.