The Merge Token for Agent Patches Is Three Human-Owned Files

A green pipeline is the wrong merge token for an agent patch. The agent can delete a property, rewrite a fixture byte-for-byte into a weaker snapshot, and skip-list a flake, then still report a passing job. Treat three human-owned files as the merge token instead: a property registry, a fixture digest, and a budgeted freeze list. If any of those files weaken, reject the patch even when every test that still runs is green.

This is a proposed protocol, not a measured production study. The checker below is runnable. The numbers in the schemas are policy defaults, not observed fleet metrics.

Why CI green is not evidence

Agent patches optimize for the signal you score. If the score is "exit code 0," the cheapest move is to shrink the oracle. Skip a flake. Soften an assertion. Replace a recorded HTTP body with a wildcard. None of that shows up as a red job.

A merge gate that only reads the latest pytest summary cannot see those edits. It needs a second, slower reading: did the human-owned contract get weaker. That contract has to live in files the agent is not allowed to rewrite without an explicit human flag.

Three files are enough to start. More files become ceremony. Fewer files leave a hole the patch can walk through.

The three files

Keep them at the repo root, committed, and reviewed like production code. The agent may propose additions. It may not drop entries, change hashes, or grow the freeze list past the budget.

1. invariants.toml — property registry. Each row is a named check the patch must still be able to fail. The source hash is the body of the property function, not the file mtime.

# invariants.toml — human-owned. Agent PRs cannot delete keys.
[policy]
min_properties = 8
allow_agent_additions = true

[[property]]
id = "parse.roundtrip.json"
module = "tests.properties.test_parse"
function = "test_json_roundtrip"
source_sha256 = "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2"

[[property]]
id = "queue.lease.not_double_ack"
module = "tests.properties.test_queue"
function = "test_lease_not_double_ack"
source_sha256 = "b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3"

2. fixtures.lock — digest table. Fixture files are data. If the agent rewrites them, the test still "passes" against a new reality. Hash the bytes. Rotation is a human flag, not a side effect of a patch.

{
  "policy": { "allow_rotate_on_agent_pr": false },
  "files": {
    "tests/fixtures/http/get_user_200.json": "e3b0c44298fc1c149afbf4c8996fb924",
    "tests/fixtures/sql/seed_orders.sql": "2c26b46b68ffc68ff99b453c1d304134"
  }
}

3. freeze.yml — budgeted flake freeze. A freeze is not a skip. The test file stays in the tree, still compiled, still hashed. It is excluded from the merge gate until expires_on, and only if the freeze count is under budget. After expiry, the job fails closed.

policy:
  max_frozen: 3
  fail_on_expired: true
freezes:
  - id: test_billing_timezone_dst
    path: tests/test_billing.py::test_timezone_dst
    body_sha256: "6b86b273ff34fce19d6b804eff5a3f57"
    reason: "dst boundary depends on host tzdata"
    expires_on: "2026-09-26"
    owner: "human"

Those three files are the merge token. Pytest output is supporting evidence. It is not the token.

Merge decision table

Observation	Agent PR allowed to merge?	Why
CI green, all three files unchanged	Yes, if properties still run	Contract held
CI green, a property id removed	No	Oracle shrank
CI green, property source hash changed	No	Assertion may have been weakened
CI green, fixture bytes changed, no rotate flag	No	Snapshot rewritten
CI green, freeze count 4 with max 3	No	Freeze budget blown
CI green, frozen test body hash changed	No	Freeze used as a cover for an edit
CI green, freeze past expires_on	No	Fail closed
CI red, three files unchanged	No	Patch does not satisfy the ledger
Human PR with --allow-fixture-rotate	Maybe	Rotation is a review event

The table is the policy. Encode it in a checker so the policy is not a wiki page.

Workflow: eight steps on every agent PR

1. Checkout the patch. Do not run the full suite yet. First copy invariants.toml, fixtures.lock, and freeze.yml from origin/main as the baseline contract.
2. Hash every registered property function and every fixture path. Compare against the baseline, not against the patch's own copies of those files.
3. Reject if any property id disappeared, if any property source hash changed, or if the property count fell below min_properties.
4. Reject if any fixture digest changed unless the job was started with --allow-fixture-rotate by a human label. Agent jobs do not get that flag.
5. Reject if freezes is longer than max_frozen, if any freeze lacks body_sha256 and expires_on, or if a frozen test's current body hash does not match the freeze record.
6. Drop expired freezes from the exclude list and fail the job. Do not silently unfreeze into a skip. The test returns to the gate.
7. Run only the registered properties plus non-frozen unit tests. Do not let the agent add 400 tautologies and call that coverage.
8. If the checker and the property run both pass, the patch may merge. A later human review still owns fixture rotation and freeze extensions.

The order matters. Hash first. Run tests second. Otherwise a rewritten fixture poisons the run you were about to trust.

Artifact: a contract checker

The script below is a complete, local checker. It reads the three files, hashes sources, and exits non-zero on a weakened contract. Label it as a proposed gate. Wire it in CI as a required check named agent-contract, not as a comment bot.

#!/usr/bin/env python3
"""Reject agent patches that weaken the human-owned merge contract."""
from __future__ import annotations

import argparse
import ast
import hashlib
import json
import sys
from datetime import date
from pathlib import Path

try:
    import tomllib
except ImportError:  # pragma: no cover
    import tomli as tomllib  # type: ignore

import yaml


def sha256_bytes(data: bytes) -> str:
    return hashlib.sha256(data).hexdigest()


def property_source_hash(root: Path, module: str, function: str) -> str:
    rel = Path(*module.split(".")).with_suffix(".py")
    tree = ast.parse(rel.read_text(encoding="utf-8"), filename=str(rel))
    for node in tree.body:
        if isinstance(node, ast.FunctionDef) and node.name == function:
            src = ast.get_source_segment(rel.read_text(encoding="utf-8"), node)
            if src is None:
                raise SystemExit(f"cannot slice {module}:{function}")
            return sha256_bytes(src.encode("utf-8"))
    raise SystemExit(f"missing property {module}:{function}")


def load_baseline(path: Path, loader):
    return loader(path.read_text(encoding="utf-8"))


def main() -> int:
    p = argparse.ArgumentParser()
    p.add_argument("--root", type=Path, default=Path("."))
    p.add_argument("--allow-fixture-rotate", action="store_true")
    p.add_argument("--today", default=date.today().isoformat())
    args = p.parse_args()
    root = args.root
    today = date.fromisoformat(args.today)
    errors: list[str] = []

    inv = tomllib.loads((root / "invariants.toml").read_text(encoding="utf-8"))
    lock = json.loads((root / "fixtures.lock").read_text(encoding="utf-8"))
    freeze = yaml.safe_load((root / "freeze.yml").read_text(encoding="utf-8"))

    props = inv.get("property") or []
    if len(props) < int(inv["policy"]["min_properties"]):
        errors.append(
            f"property count {len(props)} < min_properties "
            f"{inv['policy']['min_properties']}"
        )
    seen = set()
    for prop in props:
        if prop["id"] in seen:
            errors.append(f"duplicate property id {prop['id']}")
        seen.add(prop["id"])
        got = property_source_hash(root, prop["module"], prop["function"])
        if got != prop["source_sha256"]:
            errors.append(
                f"property {prop['id']} source hash changed: "
                f"expected {prop['source_sha256'][:12]} got {got[:12]}"
            )

    for rel, expected in lock["files"].items():
        got = sha256_bytes((root / rel).read_bytes())
        if got != expected and not args.allow_fixture_rotate:
            errors.append(f"fixture {rel} digest changed without rotate flag")
        elif got != expected:
            print(f"WARN rotating fixture {rel}", file=sys.stderr)

    policy = freeze["policy"]
    items = freeze.get("freezes") or []
    if len(items) > int(policy["max_frozen"]):
        errors.append(
            f"freeze count {len(items)} exceeds max_frozen {policy['max_frozen']}"
        )
    for item in items:
        for key in ("id", "path", "body_sha256", "expires_on", "owner"):
            if key not in item:
                errors.append(f"freeze {item.get('id')} missing {key}")
        expires = date.fromisoformat(str(item["expires_on"]))
        if expires < today and policy.get("fail_on_expired", True):
            errors.append(f"freeze {item['id']} expired on {item['expires_on']}")
        # Hash the test function body named after the last path segment if present.
        path_spec = item["path"].split("::", 1)
        test_file = root / path_spec[0]
        if not test_file.exists():
            errors.append(f"frozen path missing {item['path']}")
            continue
        body = test_file.read_bytes()
        got = sha256_bytes(body)
        if got != item["body_sha256"]:
            errors.append(
                f"frozen test {item['id']} body hash changed; "
                "freezes are not a license to edit the test"
            )

    if errors:
        print("CONTRACT FAILED", file=sys.stderr)
        for e in errors:
            print(f" - {e}", file=sys.stderr)
        return 2
    print("CONTRACT OK")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Install the two parsers the script needs, then run it against main before you run pytest.

pip install pyyaml tomli
python check_merge_contract.py --root .
pytest -q tests/properties tests/unit --ignore-glob='*frozen*'

Agent CI must call the checker without --allow-fixture-rotate. Human rotation jobs pass the flag and must also update fixtures.lock in the same commit. Split those jobs. Mixing them is how a fixture rewrite sneaks in under a "fix the snapshot" message.

What the agent is allowed to change

The agent may add a new [[property]] row if allow_agent_additions is true. The human still has to accept the source hash on review. The agent may add unit tests that are not in the registry. Those tests do not count toward the merge token.

The agent may not extend expires_on. That field is a date, not a suggestion. The agent may not raise max_frozen. That number lives in policy and should move only in a human commit that explains which flake is being bought, and for how many days.

A freeze that names owner: agent is a protocol violation. Drop it. Flakes are a human budget.

Where a free model and a free server fit

Disclosure: This article was prepared as part of MonkeyCode's product outreach. A free model is enough to draft candidate property ids from a diff, because the draft is not the ledger. A human pastes survivors into invariants.toml and records the source hash. A free server is enough to run check_merge_contract.py plus the registered properties on every agent PR, because the job is deterministic and does not need a paid GPU. Neither the model nor the server owns the three files. If the model proposes deleting a property, the checker is the part that says no.

Do not send fixture bodies that contain secrets to any model. Hash them locally. The digest table is the public artifact. The bytes stay in the repo's existing access control.

Limitations, and who should not use this

The checker hashes function source with ast.get_source_segment. It will false-fail on formatting-only edits inside a property. That is intended. Format properties in a human commit, then update the hash. If your team reformats on every save, this protocol will be noisy. Stop auto-formatting the registry modules, or stop using this protocol.

File-level hashing of frozen tests is coarse. A comment in the same file changes the freeze hash. Prefer one freeze per file, or slice the function the same way properties are sliced. The sample script uses file bytes for freezes to keep the example short. Production gates should slice.

This protocol does not replace mutation testing, hidden holdout tests, or coverage on agent-introduced branches. It answers a narrower question: did the patch weaken the human oracle while turning the job green. Repos with no flake history and no recorded fixtures gain little. A single-developer script with three tests should not grow three policy files.

Teams that already forbid agents from touching tests/ still need a fixture digest if the agent can touch testdata/. The hole is the data, not the test runner.

Do not use freeze expiry as a way to ignore a production incident. If a test is wrong, delete it in a human commit. If a test is right and the code is wrong, the patch fails. A freeze is only for environment-coupled noise with a calendar date attached.

Closing

Score the patch on whether invariants.toml, fixtures.lock, and freeze.yml stayed intact, then on whether the registered properties still fail when the implementation is wrong. A green summary line is not that score. Put the checker in front of pytest, keep fixture rotation and freeze extensions on a human path, and treat freeze slots as scarce. The agent can still write code. It cannot buy a quieter suite.