The Patch Was in Git. The Agent Run Had No Receipt.

Have you ever stared at a merged agent patch and failed to reconstruct the prompt that produced it? I did that for forty-eight hours after a supposedly simple test-fix session, and the chat log was not enough. The model had rewritten two helpers, the transcript looked confident, and my local tree had extra files I never asked it to touch. Why did a successful-looking run become unreproducible the moment I tried to explain it to myself?

What I actually needed

I did not need another dashboard, and I did not need a longer system prompt dressed up as process. I needed a receipt that bound the git SHA, the prompt files, and the interpreter together before any tool call ran. Without that bundle, claiming the agent fixed the tests is a story you cannot replay on a clean machine. Is a chat transcript really evidence of a run, or is it just theater with timestamps attached?

A receipt, in this field note, is a small JSON file plus a hashed prompt directory. It is boring on purpose, because boring artifacts survive after the conversation UI forgets a turn. I now treat a missing receipt as a failed run, even when the diff looks tasteful in git show. Would you ship a binary without the compiler flags that produced it, just because the bytes look small?

Hours 0–12: what I tried

I started the way most of us start, which is to trust the UI and skim the diff like a code review. That failed for three separate reasons that only became obvious after I printed the hashes myself. Have you ever been completely sure the prompt file on disk was the one the agent loaded?

1. The prompt file on disk was not the prompt the agent had loaded earlier.
2. The test command in the transcript used a different virtualenv than which python in my shell.
3. The agent had created a scratch file that never entered git, so the next session could not see it.

I copied the transcript into a gist and tried to re-run the same English instructions by hand. The second run edited a different helper because my notes file had gained a paragraph overnight. Have you noticed how a phrase like use the project conventions quietly changes meaning when README.md moves?

git rev-parse HEAD
python -V
which python
sha256sum prompts/*.md
pytest -q tests/test_receipt_demo.py

Those five commands already contradicted the chat log I had trusted during the first session. The SHA was clean, the interpreter was 3.12, and the prompt hash did not match the filename I remembered from the first session. I had been reviewing the wrong inputs, so every later judgment about the model was unearned.

Hours 12–36: what broke

The breakage was not a cinematic model hallucination, and it was not a mysterious cloud outage either. The breakage was missing inputs, because I had never frozen the prompt files before the first tool call. I kept asking the agent to continue, which meant the hidden context window carried stale file dumps from the first hour. Here is the ugly part that I should have caught during the first code review pass.

I grepped the transcript for def test_ and found an assertion that never existed in the repo. The agent had described a test it wanted to write, and I had read that description as a test that already passed. Do you review proposed tests with the same suspicion that you already apply to production patches?

I also learned the hard way that local convenience is a quiet trap for agent work. My laptop had extra env files, a dirty pip cache, and an editor plugin that auto-formatted on save. Replaying on a throwaway host without those helpers made the green session turn red immediately.

local session:  pytest exit 0, two files changed, prompt hash abc
clean host:    pytest exit 1, zero files matched, prompt hash def

That mismatch is the whole article, and it is the reason I stopped trusting local green badges. If you cannot move the run to a clean host, you do not actually have a run. Would you accept a CI log that can only be produced on one person's laptop with plugins enabled?

The receipt workflow I would repeat

This is a labeled field-notes script, not a claim that I ran a published benchmark. Copy it, change the paths, and refuse to start an agent until receipt.json exists. I keep it at the repo root so the agent cannot helpfully hide it under a nested build directory.

# receipt.py — snapshot the inputs before any agent tool call
from __future__ import annotations

import hashlib
import json
import os
import subprocess
import sys
from datetime import datetime, timezone
from pathlib import Path

ROOT = Path(__file__).resolve().parent
PROMPT_DIR = ROOT / "prompts"
RECEIPT = ROOT / "receipt.json"


def git_sha() -> str:
    return subprocess.check_output(
        ["git", "rev-parse", "HEAD"], cwd=ROOT, text=True
    ).strip()


def hash_tree(path: Path) -> dict[str, str]:
    out: dict[str, str] = {}
    for p in sorted(path.rglob("*")):
        if p.is_file():
            digest = hashlib.sha256(p.read_bytes()).hexdigest()
            out[str(p.relative_to(ROOT))] = digest
    return out


def interpreter() -> dict[str, str]:
    return {
        "executable": sys.executable,
        "version": sys.version.split()[0],
        "prefix": sys.prefix,
    }


def main() -> None:
    if not PROMPT_DIR.is_dir():
        raise SystemExit("prompts/ is missing; refuse to start the agent")
    payload = {
        "created_at": datetime.now(timezone.utc).isoformat(),
        "cwd": os.getcwd(),
        "git_sha": git_sha(),
        "interpreter": interpreter(),
        "prompt_hashes": hash_tree(PROMPT_DIR),
        "command_template": os.environ.get(
            "AGENT_CMD", "echo 'set AGENT_CMD before launching'"
        ),
    }
    RECEIPT.write_text(json.dumps(payload, indent=2) + "\n")
    print(f"wrote {RECEIPT} for {payload['git_sha'][:12]}")


if __name__ == "__main__":
    main()

Run it like this from the repo root, and keep the JSON next to the eventual patch. If prompts/ is missing, the script exits on purpose, because I am done launching agents against vibes. Commit the receipt before the model is allowed to touch application code in that same session.

export AGENT_CMD='pytest -q'
python receipt.py
git add prompts receipt.json
git status --short

Failing-first gate

Then I added a failing-first gate, because I no longer accept an agent that fixes tests that never failed on the receipt host. The tests below are a labeled example, and they exist to block a session, not to flatter a model. If they are already green before the agent starts, I stop the session immediately and refuse to continue.

# tests/test_receipt_demo.py — failing-first fixture, labeled example
from pathlib import Path
import json

RECEIPT = Path(__file__).resolve().parents[1] / "receipt.json"


def test_receipt_exists_before_agent_runs():
    assert RECEIPT.is_file(), "write receipt.json before any agent patch"


def test_prompt_hashes_are_non_empty():
    data = json.loads(RECEIPT.read_text())
    assert data["prompt_hashes"], "prompts/ hashed empty; check the bundle"

A green suite with no receipt is exactly how I wasted the first twelve hours. After the receipt exists, I still want one real product test failing for the bug I asked the agent to fix. Otherwise the model can satisfy the harness by inventing a complementary assertion that never exercised the bug.

Numbered loop I actually follow now

1. Freeze prompts/ and print receipt.json.
2. Prove one product test fails on the same interpreter the receipt recorded.
3. Launch the agent against that frozen tree only.
4. Reject any extra file that is not in the planned path list.
5. Re-run the receipt hashes; if they drifted, the session is invalid.

Would I skip step two again after watching a model invent a passing test in prose? I would not, because a transcript that says the suite is green is not a test runner. The loop is slow on purpose, and that slowness is cheaper than another two days of archaeology.

Local laptop versus a throwaway server

I needed isolation more than I needed a bigger prompt or a friendlier chat theme. The laptop lied by being helpful, and a clean host told the truth without extra plugins. Here is the decision table I now keep in the same repo as receipt.py.

Situation	Stay local	Move the run
You are editing the receipt script itself	Yes	No
The agent needs network credentials from your shell	No	Still no; do not copy secrets
You must prove a stranger can replay the patch	No	Yes
Pip cache or editor formatters keep "helping"	No	Yes
You only have a dirty working tree	No	Yes, after a clean clone

I used MonkeyCode here only as the isolated place to replay the same receipt, not as a magic reviewer.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The operator-supplied piece that mattered for this workflow was free model access plus a free server option. That let me replay the frozen tree without turning my laptop into the agent host. I am not going to invent model names, quotas, or hardware claims, because those would rot faster than a missing receipt. If you already have access, point receipt.py at a clean clone on that server and compare hashes first.

That is the only product-shaped sentence I need, and the method still works on a VM you already own. Reader value has to survive if you delete the product name and keep the receipt. Would you rather have a branded chat or a JSON file you can email to future you?

Limitations, and who should not do this

This receipt does not prove the model was honest, and it does not prove the tests were meaningful. It only proves which prompt files and which interpreter you claimed to use for the run. If your agent can shell out, it can still rewrite receipt.json unless you copy the file off-box first.

Do not use this approach when you need a signed supply chain, a guaranteed model identity, or a production SLA. Do not paste secrets into prompt files that you are about to hash and copy onto a shared server. Do not treat free model access as a reason to skip code review, because a replayable bad patch is still a bad patch.

Also, skip it if your repo cannot even run pytest without a dozen private services. A receipt of an unreproducible stack is just a prettier form of folklore with timestamps. People who need formal evaluations should log traces with their own vendor contracts, not a JSON file I sketched in a field note.

What I would repeat

I would repeat the boring snapshot, the failing-first test, and the move to a clean host. I would not repeat trusting a transcript, and I would not repeat launching from a dirty tree. Forty-eight hours of confusion compressed into one rule: if you cannot hash the prompt, you cannot claim the agent did a specific thing.

Would I still read the chat after receipt.json and git diff finally agree with each other? The agent can keep its personality in the transcript, but I just want the inputs on disk. If the hashes moved, I throw the session away and start from the last committed receipt, no negotiation.

If you try the receipt script, paste the JSON shape you end up with, not another screenshot of a green badge. A green badge without hashes is exactly how this whole forty-eight hour mess got started for me. Future me only believes files that have digests.