A Receipt for Every Agent Patch: Seeds, Digests, and a Freeze File

A green suite after an agent patch is not a result. It is a claim. If the same change can rewrite assertions, refresh golden files, or skip flickering tests, the claim is circular.

This article proposes a receipt-based gate. Three inputs stay immutable while the patch is applied. One JSON receipt is the only artifact a merge bot should trust.

The protocol is small on purpose. It fits a single function or a single bounded module. It does not replace design review.

The circular-green problem

Agent patches fail in a short list of mechanical ways. They loosen assertions until the new code matches. They rewrite fixtures so a bug becomes the baseline. They delete or skip tests that flicker.

None of those paths require malice. A model that is scored on "tests passed" takes the cheapest route. Cheap often means mutating the oracle.

A larger suite does not fix that. A split does. The agent may edit production code under test. It may not edit the evidence that decides pass or fail.

Three immutable inputs

Keep these paths out of the agent's write set. Enforce the rule in CI. Prompt text is not a control.

1. properties/ — seedable checks that encode invariants, not copied examples.
2. fixtures/ — canonical inputs plus expected digests. Avoid full output blobs when a blob is easy to regenerate wrongly.
3. freeze.json — the only list of tests that may be skipped, each with an expiry and a closed reason code.

If a patch touches those paths, fail before tests run. Ordering matters. A rewritten freeze file can make a red suite look green.

Proposed layout

Label: this is a proposed workflow, not a report from a private production suite.

repo/
  src/allocate.py
  properties/test_allocate_properties.py
  fixtures/weights_v3.json
  freeze.json
  tools/patch_receipt.py

The example domain is integer allocation. Given a total in cents and a list of weights, parts must sum to the total, stay non-negative, and stay stable for a recorded seed when weights tie. Remainder handling is a frequent agent footgun. It is also cheap to check.

Step 1 — Lock the write set

A path check is enough to start. Run it on the merge ref, not on a workspace the agent still owns.

# proposed: fail if the patch edits evidence files
git diff --name-only origin/main...HEAD \
  | grep -E '^(properties/|fixtures/|freeze\.json$|tools/patch_receipt\.py$)' \
  && { echo "evidence paths are read-only for agent patches"; exit 1; }

Put tools/patch_receipt.py on the same list. Otherwise the agent can weaken the hasher. Lock workflow files the same way if the runner definition lives in-repo.

Step 2 — Property checks with a recorded seed

Example checks below are proposed. They are not claimed as executed results.

# properties/test_allocate_properties.py
import os
import random
from src.allocate import allocate

SEED = int(os.environ.get("PROPERTY_SEED", "20260906"))
TRIALS = int(os.environ.get("PROPERTY_TRIALS", "200"))

def test_allocation_invariants():
    rng = random.Random(SEED)
    for i in range(TRIALS):
        n = rng.randint(1, 8)
        weights = [rng.randint(0, 50) for _ in range(n)]
        if sum(weights) == 0:
            weights[0] = 1
        total = rng.randint(0, 10_000)
        parts = allocate(total, weights, seed=SEED + i)
        assert len(parts) == n, (i, parts)
        assert sum(parts) == total, (i, total, parts)
        assert all(p >= 0 for p in parts), (i, parts)

Record PROPERTY_SEED and TRIALS in the receipt. A later rerun with a different seed is a new experiment. It is not confirmation of the same run.

Two hundred trials is a default for a tiny function, not a coverage proof. Raise trials when the input has more axes. Drop them when each trial is expensive, then compensate with fixtures.

A second, cheaper property is useful when ties exist: equal weights must not swap when the seed is held constant. Stability is a merge requirement. Novelty is not.

def test_tie_order_is_seed_stable():
    weights = [1, 1, 1, 1]
    a = allocate(100, weights, seed=SEED)
    b = allocate(100, weights, seed=SEED)
    assert a == b
    assert sum(a) == 100

Step 3 — Fixture digests, not copy-paste goldens

Full golden files invite "update the snapshot." Store a digest of the canonical output instead. Compute it in the gate. Do not let the patch compute it.

# excerpt from tools/patch_receipt.py (proposed)
import hashlib
import json
import pathlib

def digest_bytes(data: bytes) -> str:
    return hashlib.sha256(data).hexdigest()

def digest_file(path: pathlib.Path) -> str:
    return digest_bytes(path.read_bytes())

def check_fixture(path: pathlib.Path, produced: bytes) -> dict:
    expected = json.loads(path.read_text())["sha256"]
    actual = digest_bytes(produced)
    return {
        "path": str(path),
        "expected": expected,
        "actual": actual,
        "ok": expected == actual,
    }

A fixture file on disk holds the digest and a pointer to the input. It does not hold the output blob. Moving a digest is an oracle change. It needs a human review, not an agent commit message that says "fix tests."

Example fixture record:

{
  "id": "weights_v3",
  "input": {"total": 1000, "weights": [50, 30, 20]},
  "sha256": "replace-with-real-digest-after-human-approval"
}

Step 4 — Freeze flakes without handing the agent an eraser

Flakes happen. Letting the agent skip them is how flakes become holes.

{
  "version": 1,
  "frozen": [
    {
      "id": "tests/test_allocate.py::test_concurrent_realloc",
      "reason": "timing",
      "expires": "2026-09-20",
      "owner": "human",
      "added": "2026-09-06"
    }
  ]
}

Rules for the proposed freeze file:

1. Only a human, or a separate bot with a different credential, may edit freeze.json.
2. Every entry has an expiry. CI fails if expires is in the past and the test is still skipped.
3. Reason codes are a closed set: timing, network, order, infra. Not unknown. Not agent_said_so.
4. Frozen tests still run on a scheduled job. The freeze applies only to the agent-patch pipeline.

If a patch happens to pass a frozen test, record that fact on the receipt. Do not auto-delete the freeze. Unfreezing remains a human action. A noisy pass is not evidence that the flake is gone.

A tiny expiry check keeps the list honest:

import datetime
import json

def expired_freezes(path="freeze.json", today=None):
    today = today or datetime.date.utcnow()
    data = json.loads(open(path).read())
    bad = []
    for row in data["frozen"]:
        expires = datetime.date.fromisoformat(row["expires"])
        if expires < today:
            bad.append(row["id"])
    return bad

Fail closed when bad is non-empty. An expired skip is a process failure, not a soft warning buried in logs.

Decision table

The table is the policy. Prompts are not.

Symptom	Agent-patch pipeline	Human follow-up
Property check fails on the recorded seed	Reject patch	Inspect allocate
Property check fails only on a new seed	Reject, or open a new receipt	Decide if the invariant is too tight
Fixture digest mismatch	Reject patch	Review whether the oracle should move
Unfrozen test flakes once	Reject patch	Freeze with expiry, or fix infra
Frozen test still failing after expiry	Reject any merge that relies on the skip	Fix the test or extend freeze with review
Patch edits freeze.json or properties/	Reject before tests	Treat as a process failure

Step 5 — Emit a receipt the merge bot can parse

# tools/patch_receipt.py (proposed)
import datetime
import json
import os
import pathlib
import subprocess
import sys

def git_sha() -> str:
    return subprocess.check_output(
        ["git", "rev-parse", "HEAD"], text=True
    ).strip()

def main() -> int:
    freeze = pathlib.Path("freeze.json")
    receipt = {
        "schema": "agent-patch-receipt/v1",
        "git_sha": git_sha(),
        "generated_at": datetime.datetime.now(
            datetime.timezone.utc
        ).isoformat(),
        "property_seed": int(os.environ["PROPERTY_SEED"]),
        "property_trials": int(os.environ["PROPERTY_TRIALS"]),
        "properties_ok": True,  # set by the test runner
        "fixtures": [],         # filled by digest checks
        "freeze_file_sha256": digest_file(freeze),
        "expired_freezes": expired_freezes(str(freeze)),
        "evidence_paths_clean": True,
    }
    ok = (
        receipt["properties_ok"]
        and receipt["evidence_paths_clean"]
        and not receipt["expired_freezes"]
        and all(row.get("ok", False) for row in receipt["fixtures"])
    )
    pathlib.Path("receipt.json").write_text(
        json.dumps(receipt, indent=2) + "\n"
    )
    print("wrote receipt.json")
    return 0 if ok else 1

if __name__ == "__main__":
    sys.exit(main())

Store receipt.json as a CI artifact. Merge requires four facts at once: properties passed on the recorded seed, fixture rows match, the evidence diff is empty, and the freeze-file digest matches main.

Proposed local run:

export PROPERTY_SEED=20260906
export PROPERTY_TRIALS=200
python -m pytest properties/ -q
python tools/patch_receipt.py

A second command compares freeze digests across refs:

git show origin/main:freeze.json | sha256sum
sha256sum freeze.json

If those hashes diverge, stop. Do not interpret test output until the freeze file is explained.

Where a free model and a free server belong

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode's free model access can propose the change in src/. MonkeyCode's free server option can run patch_receipt.py and the property file on a machine the model does not write to. That split is the point. The generator and the grader should not share a writable workspace.

This article does not claim model names, quotas, hardware, runtimes, or benchmark scores. Treat the free model as a patch source. Treat the free server as an isolated runner for the receipt. If either role collapses into the same writable tree, the receipt is theater.

Limitations

Seeded property checks are not proofs. Two hundred random trials miss structured counterexamples. Fixture digests detect mismatch; they do not explain it. A freeze file with generous expiry dates becomes a second skip list.

The path grep is bypassable if the agent can change CI config. Lock the workflow directory the same way you lock freeze.json. Clock skew can expire freezes early or late. Use UTC dates and fail closed.

The workflow assumes one tame unit of work. Multi-service patches need one receipt per bounded context. A single digest list across five services becomes noise, and nobody reads noise.

Random trials also assume you can inject clocks, networks, and other seams. If allocate secretly reads time.time(), the seed is decoration.

Who should not use this

Do not use a freeze file if nobody owns expiry review. An unmaintained freeze list is skipped tests with extra JSON.

Do not run the grader on a disk the agent can wipe. Do not apply this to spikes where the oracle is still being invented. Freeze an unstable spec and you will reject every honest patch.

If the function under test needs the network, the clock, or shared mutable hardware, fix those seams first. A receipt that cannot replay is a log line, not a gate.

Closing

A patch receipt turns "the agent said tests passed" into three checkable facts: the seed, the fixture digests, and a freeze file the agent could not edit. That is a small protocol. It is also the difference between a merge and a tautology.

If patches come from MonkeyCode's free model access, keep the receipt job on the free server and keep evidence paths off the agent's write list.