Block the Agent Patch Until Parent Drift Has a Signed Ledger

An agent patch is not ready to merge when its new tests were recorded after the edit. Capture a characterization baseline on a pinned fixture pack at the parent commit, rerun that same pack on the patch, and reject unexplained drift. A snapshot refreshed after the edit is a copy of the patch, not an oracle.

That order is the whole gate. Stability policy belongs in other reviews. This gate asks only whether observed behavior moved, and whether a named reviewer accepted the move.

Why a post-edit golden fails too late

Characterization tests store outputs and compare a later run to that store. They help when the written spec is thin. They stop being evidence when the same change both edits the code and rewrites the store.

An agent can do both in one turn. The suite goes green because the golden file now matches the candidate, not because the candidate matches the parent contract. The diff is large, the note is short, and the reviewer has no independent witness.

A parent baseline breaks that loop. Drift becomes a row. Acceptance becomes a ledger with an allow-listed reason code. The patch does not get to mint the behavior and the proof in one unreviewed write.

This is ordinary characterization testing, sometimes called a golden master, plus one constraint: the master is taken before the agent edit. The constraint is the part that matters for generated patches.

What has to be true first

Three conditions make the ceremony worth running.

1. The module already has callers and a fixture pack that runs on the parent.
2. You can execute that pack in two clean worktrees, parent and candidate.
3. The review stores both result files and the ledger, not only a buffer on one laptop.

Skip the gate for greenfield code with no callers, for a patch whose job is a broad error-text rewrite, and for a suite you cannot run twice on the same inputs. If a field is nondeterministic, exclude it in the pack definition. Do not hide that choice inside a refreshed snapshot.

Figures below are policy knobs for your repo. They are not measured limits of any tool. The scripts are unexecuted proposals until you run them on your tree.

Step 1. Pin a pack the patch cannot resize

Start with 20 to 50 fixtures. That range is a review budget, not an optimum found by a study. Each fixture is a JSON object with an id, a callable name, an input, and a one-line note on why the row exists.

Put the file on a path the agent change must not edit, such as review_fixtures/pack.json. If the diff touches that path, fail closed. The pack is the ruler. A patch that resizes the ruler and the cloth in one commit has not been measured.

Drop rows that assert a literal true, or that compare a value to itself. They cannot show drift. Prefer inputs taken from past bugs, plus empty, single-element, and rejected-type cases. The pack still under-approximates the domain. Say so in the file header so a match is not treated as a proof.

Step 2. Capture the parent, not the dirty tree

Add a worktree at the parent revision and run the pack there. Write one row per fixture: id, callable, normalized output, and a hash of the input. Sort object keys. If a field is a timestamp, exclude it in the pack or render a fixed token, and record that choice beside the fixture.

Proposed commands, not a log from a run:

git worktree add ../baseline "$BASE_SHA"
python scripts/capture_baseline.py \
  --pack review_fixtures/pack.json \
  --out artifacts/baseline.json

Import the library from that worktree. A capture that imports the candidate branch will confirm the patch against the code it already replaced. Record the parent commit and the blob hash of baseline.json in the review note. The scorer below does not check that binding yet. Add the check before the gate is mandatory.

Step 3. Recapture on the candidate

Apply the agent diff in a second worktree. Recapture with the same pack and the same normalizer. Diff by fixture id, never by file order.

git worktree add ../candidate "$PATCH_SHA"
python scripts/capture_baseline.py \
  --pack review_fixtures/pack.json \
  --out artifacts/candidate.json
python scripts/score_drift.py \
  --baseline artifacts/baseline.json \
  --candidate artifacts/candidate.json \
  --ledger review_fixtures/drift_ledger.json \
  --new-symbols "$(python scripts/list_new_symbols.py artifacts/patch.diff)"

list_new_symbols.py can be a thin scan of the diff: public function names added under the package, ignoring names that start with an underscore. Keep the scan boring. A clever scanner becomes another oracle the agent can game.

Step 4. Class every row before merge talk

The scorer emits four classes. Read the table as policy, not as a benchmark.

Class	Rule	Result
match	Normalized output unchanged	Pass this fixture
explained	Output changed; ledger has this id, an allow-listed reason, and a reviewer id	Pass this fixture
unexplained	Output changed; ledger row missing or incomplete	Fail the run
uncovered_new	Diff adds a public symbol with no property name in the ledger	Fail the run

Allow-list the reason codes. A short set is enough: bugfix, spec-change, error-text, performance-neutral. Free text is a comment, not a code. Require a reviewer id so an unsigned file in the tree cannot pose as approval.

If the patch also adds tests, those tests must point at a fixture id that drifted, or at a property name listed for a new symbol. The checker will not prove the property is strong. It rejects one obvious cheat: an expected value that exists only as the candidate output, with no parent row. A separate review can still judge weak assertions. Do not fold that review into this script.

Step 5. Run the proposed scorer

The program below is a proposal. It uses the standard library only. It has not been executed for this article, and it reports no pass rate.

#!/usr/bin/env python3
"""Proposal: score a candidate capture against a parent baseline."""
from __future__ import annotations

import argparse
import json
import sys
from pathlib import Path

ALLOWED = {"bugfix", "spec-change", "error-text", "performance-neutral"}

def load_rows(path: str) -> dict:
    data = json.loads(Path(path).read_text(encoding="utf-8"))
    rows = data["rows"] if isinstance(data, dict) else data
    return {row["id"]: row for row in rows}

def main() -> int:
    parser = argparse.ArgumentParser()
    parser.add_argument("--baseline", required=True)
    parser.add_argument("--candidate", required=True)
    parser.add_argument("--ledger", required=True)
    parser.add_argument("--new-symbols", default="")
    args = parser.parse_args()

    base = load_rows(args.baseline)
    cand = load_rows(args.candidate)
    ledger_doc = json.loads(Path(args.ledger).read_text(encoding="utf-8"))
    ledger = {row["id"]: row for row in ledger_doc.get("drifts", [])}
    problems = []

    for fid, brow in sorted(base.items()):
        crow = cand.get(fid)
        if crow is None:
            problems.append(f"dropped: {fid}")
            continue
        if brow.get("norm") == crow.get("norm"):
            continue
        row = ledger.get(fid)
        reason = None if row is None else row.get("reason")
        reviewer = None if row is None else row.get("reviewer")
        if reason not in ALLOWED or not reviewer:
            problems.append(f"unexplained: {fid}")
            continue
        if row.get("parent_norm") != brow.get("norm"):
            problems.append(f"ledger-mismatch: {fid}")
        if row.get("candidate_norm") != crow.get("norm"):
            problems.append(f"ledger-mismatch: {fid}")

    covered = set(ledger_doc.get("new_symbol_properties", []))
    for name in [s for s in args.new_symbols.split(",") if s]:
        if name not in covered:
            problems.append(f"uncovered_new: {name}")

    if problems:
        print("\n".join(problems))
        return 1
    print(f"scored {len(base)} fixtures, 0 unexplained")
    return 0

if __name__ == "__main__":
    sys.exit(main())

Expected shape of a failure, illustrative only, not a captured log:

unexplained: fx-014
uncovered_new: parse_header

Exit code 1 blocks the merge. Exit code 0 means this gate is satisfied, not that the patch is correct.

A ledger row that clears fx-014 has this shape:

{
  "drifts": [
    {
      "id": "fx-014",
      "reason": "bugfix",
      "reviewer": "reviewer-on-call",
      "parent_norm": "null",
      "candidate_norm": "ok"
    }
  ],
  "new_symbol_properties": ["parse_header_rejects_empty"]
}

The norm fields are copies for the human reader. The checker compares them with the artifact files and rejects a ledger that describes a drift the artifacts do not show.

How to read a week of ledgers

Do not invent a single quality score. Count classes. If most changed rows are explained with reason bugfix, reviewers are signing a break with the baseline on habit. Re-read those rows on a fixed cadence, such as once a week, and ask whether the parent output was actually wrong.

If uncovered_new is rare because every new symbol lists a property file that only echoes the candidate output, the file-exists check has been gamed. Open the property. When the expected value equals the candidate norm and the parent has no row, send it back. That is a post-edit snapshot under another name.

If match dominates on a patch that claims a behavior change, the pack missed the branch. Add a fixture for the claimed branch and recapture both trees. Do not update only the candidate file.

Drafting ledger text, and where the run can sit

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The gate still works if a person writes every reason code and a laptop runs both captures. Two operator-supplied options only remove friction. Free model access can draft a reason-code sentence and a property name for each unexplained id. A free server option can run the two captures and score_drift.py off the laptop that produced the patch. This article does not name models, quotas, hardware, durations, or how long either option stays available. Read the current terms before you depend on them. Nothing here is a benchmark.

Keep the draft prompt narrow. Pass the parent norm, the candidate norm, and the allow-list. Ask for one code and one property that would have failed on the parent. Discard a code that is not in ALLOWED. Discard a property whose expected value is only the candidate output. The draft is a comment. The reviewer id in the ledger is the decision.

If those free options are already on the account, point the draft at unexplained ids only, and point the remote run at the two worktrees. Do not ask either one to approve the merge.

Limitations

Baselines lock in current behavior, bugs included. bugfix is the escape hatch. Unreviewed use of that code deletes the gate, because every drift then looks signed.

The pack is small on purpose. A match is not a proof. New branches need a property, or a metamorphic check you judge by reading, not by this script's exit code. Normalization can hide real changes: replace every timestamp and a broken clock will match. Keep the replace list short and in the pack header.

The checker trusts the files you pass it. Bind baseline.json to the parent blob hash before the gate is mandatory. The listing leaves that binding as a follow-up so the gap stays visible. Matching snapshots also says nothing about injection flaws, auth checks, or data races. Do not cite this gate in a security review.

Who should skip it

Skip it when the agent can edit the fixture pack in the same change, and skip it on a one-file prototype with no callers. Skip it when the task is a wholesale copy rewrite and almost every row will be error-text. Use a dedicated copy review instead of a ledger full of identical codes.

Teams that already fail the build on unexplained golden diffs can keep that system. If anything here is worth copying, copy the order: parent capture, then the patch, then a ledger a person signs. Do not copy a claim that an automated draft reviewed the patch. It drafted a sentence.