If a Hunk Survives Revert, It Does Not Belong in the Merge

An agent patch that is green on HEAD is not a correctness result. It is a snapshot of the patched tree. Merge it only when every production hunk has at least one reliable test that goes red when that hunk is reverted, when property checks read versioned fixtures instead of live I/O, and when flaky or xfailed tests are excluded from the kill map.

That is the strategy. Green on the candidate SHA is a necessary condition. It is not a sufficient one.

The failure mode is empty kill sets, not missing coverage

Agent patches often land tests in the same turn as production edits. Those tests describe the new code. They do not have to interrogate it. A rewritten matcher, a deleted branch, or an assertion on a freshly invented helper can raise coverage while leaving large hunks unkilled.

File-level coverage does not answer the merge question. The merge question is narrower. If this hunk disappeared, which stable test would fail?

If the answer is none, the hunk is commentary. It should not ship.

Layer 1: hunk-level revert-kill

Treat the production diff as a list of independently reversible units. Keep the incoming tests. Revert one unit at a time. Record which tests fail. Demand a non-empty kill set per unit.

This is a proposal for a local gate, not a measured campaign. Label every command below as a template until you run it on your repository.

1. Isolate production paths from test paths

# .ci/revert-kill.paths
production:
  - "src/"
  - "pkg/"
  - "internal/"
tests:
  - "tests/"
  - "src/**/test_*.py"
exclude:
  - "**/*.md"
  - "**/testdata/generated/"

If a path can affect runtime behavior, it is production. If a path only asserts, it stays on disk during revert. Generated golden files belong in fixtures (Layer 2), not in this glob, or the revert will delete the oracle.

2. Build a file list from the merge base

#!/usr/bin/env bash
# tools/list_prod_files.sh — proposal
set -euo pipefail
BASE="${1:-$(git merge-base HEAD origin/main)}"
git diff --name-only "$BASE"...HEAD -- src/ pkg/ internal/ \
  | grep -v -E '(^tests/|/test_|_test\.py$)"' \
  | sort -u

Start at file granularity. Hunks inside a file come next, and only if the file-level kill set is mixed: some functions killed, others not.

3. Revert one production file, keep tests, run pytest

#!/usr/bin/env bash
# tools/revert_kill_file.sh — proposal
set -euo pipefail
BASE="${1:?merge-base}"
FILE="${2:?production file}"
SHA="$(git rev-parse HEAD)"
STAMP=".ci/revert-kill/$(echo "$FILE" | tr '/.' '__').json"

mkdir -p .ci/revert-kill
git show "${BASE}:${FILE}" > "${FILE}.pre" 2>/dev/null || {
  # file added by the patch: revert means delete
  mv "$FILE" "${FILE}.new"
  python -m pytest -q --maxfail=20 --json-report --json-report-file="$STAMP" || true
  mv "${FILE}.new" "$FILE"
  exit 0
}

cp "$FILE" "${FILE}.new"
cp "${FILE}.pre" "$FILE"
python -m pytest -q --maxfail=20 --json-report --json-report-file="$STAMP" || true
mv "${FILE}.new" "$FILE"
rm -f "${FILE}.pre"

The patched tests stay in the tree. Only the production file rolls back. Pytest must see failures. A clean pass on the reverted file is a blocker for that file.

4. Reduce to hunks when a file is only partly tested

# proposal: split the file diff, revert one hunk via git apply -R
git diff -U3 "$BASE" -- "$FILE" > /tmp/full.patch
# split /tmp/full.patch on hunk headers, write /tmp/hunk-N.patch
git apply -R --recount /tmp/hunk-N.patch
python -m pytest -q tests/ --maxfail=20 || true
git checkout -- "$FILE"

Do not stop at "the file has a killer." A 400-line agent rewrite can hide an untested refactor next to a one-line bugfix that the suite already covered. Hunk-level revert is how you find the silent half.

5. Write a kill map, not a narrative

{
  "base": "a1b2c3d",
  "head": "e4f5a6b",
  "units": [
    {
      "path": "src/billing/ledger.py",
      "hunk": "@@ -80,12 +80,31 @@",
      "killers": ["tests/billing/test_ledger.py::test_unpaid_balance_cannot_go_negative"],
      "flake_hits": [],
      "status": "killed"
    },
    {
      "path": "src/billing/ledger.py",
      "hunk": "@@ -210,4 +229,40 @@",
      "killers": [],
      "flake_hits": ["tests/billing/test_retry.py::test_window"],
      "status": "block"
    }
  ]
}

killed means at least one test not on the flake freeze failed. block means the kill set is empty after that exclusion. flake_hits never promote a unit to killed.

Layer 2: property checks that only read versioned fixtures

Revert-kill catches hunks that no example test watches. It does not catch a patch that preserves those examples while breaking an invariant the examples never stated.

Put invariants in property tests. Feed them fixtures that are versioned directories, not whatever the development server returns today.

# tests/properties/test_ledger_invariants.py
# Illustrative example — not an executed campaign.
from decimal import Decimal
from pathlib import Path
import json

import pytest

from billing.ledger import Ledger

FIXTURE_ROOT = Path(__file__).parent / "frozen" / "ledger_v4"
MANIFEST = json.loads((FIXTURE_ROOT / "manifest.json").read_text())


def load_cases():
    for name in MANIFEST["cases"]:
        yield json.loads((FIXTURE_ROOT / name).read_text())


@pytest.mark.parametrize("case", list(load_cases()), ids=lambda c: c["id"])
def test_balance_equals_posted_minus_voided(case):
    ledger = Ledger.from_events(case["events"])
    posted = sum(Decimal(e["amount"]) for e in case["events"] if e["type"] == "post")
    voided = sum(Decimal(e["amount"]) for e in case["events"] if e["type"] == "void")
    assert ledger.balance() == posted - voided


def test_fixture_manifest_hash_is_pinned():
    import hashlib, os
    h = hashlib.sha256()
    for root, _, files in os.walk(FIXTURE_ROOT):
        for name in sorted(files):
            path = Path(root) / name
            h.update(path.relative_to(FIXTURE_ROOT).as_posix().encode())
            h.update(path.read_bytes())
    assert h.hexdigest() == MANIFEST["sha256"]

The property does not call the network. It does not read a developer database. It does not regenerate fixtures during the test. Regeneration is a separate, reviewed change to ledger_v4/ and to the pinned hash.

When an agent patch needs a new invariant, add a new fixture version (ledger_v5/) instead of editing v4 in place. Old versions stay as regression oracles for the revert-kill loop.

Candidate properties can be drafted off the critical path. They still do not merge themselves.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. A free server and free model access are enough to propose extra invariants for hunks that the kill map marked block, and to run the revert loop without burning the main CI queue. They are not an oracle. A generated property enters tests/properties/ only after a human accepts the fixture pack and the pinned hash.

Layer 3: flake freeze is exclusion, not evidence

Timing tests, order-dependent tests, and xfailed tests will fail on some reverts by accident. That noise looks like a kill set if you let it.

Freeze them. Then ignore them when you score revert-kill.

# .ci/flake-freeze.yml
fail_closed: true
# statuses that may appear for a frozen node
allowed_statuses: [skipped, xfailed, error]
entries:
  - nodeid: tests/billing/test_retry.py::test_window
    reason: wall-clock backoff
    until: "2026-10-15"
  - nodeid: tests/http/test_search.py::test_eventual_consistency
    reason: concurrent index
    until: "2026-10-01"

CI rule, in order:

1. Parse the pytest report.
2. If a frozen nodeid is passed, fail the job. A freeze that goes green is a change in the suite, not a reason to merge the agent patch.
3. Subtract frozen nodeids from every unit's killers list.
4. If any production unit then has an empty killers list, fail the job.
5. Reject edits to .ci/flake-freeze.yml in the same commit as production code unless the commit message references a tracking ticket.

The freeze is not a license to skip Layer 1. It is a filter on which reds count.

Compose the three layers in CI

Number the job so the cheap checks run first.

1. Path filter: production vs tests vs freeze file vs fixture manifest.
2. Example suite on HEAD. If this is red, stop. The patch is not a candidate yet.
3. Property suite against frozen/vN. No network.
4. Revert-kill matrix over production files, then hunks for mixed files.
5. Kill-map policy: every unit killed, zero freeze promotions, fixture hash unchanged unless the change set is fixtures-only.

# tools/gate.sh — proposal
set -euo pipefail
BASE="$(git merge-base HEAD origin/main)"
python -m pytest -q tests --ignore=tests/properties
python -m pytest -q tests/properties
python tools/revert_kill.py --base "$BASE" --out .ci/revert-kill/map.json
python tools/policy.py --map .ci/revert-kill/map.json --freeze .ci/flake-freeze.yml

Parallelize step 4 per file. That is the expensive part. It is also the part that does not belong on a shared, quota-sensitive runner if a spare machine can hold a git checkout and pytest.

Decision table

Observation	Merge?	Next action
HEAD examples red	No	Fix the patch or the new tests first
HEAD green, a production hunk reverts clean	No	Add an example or a property that reads frozen fixtures
Hunk only killed by a frozen flake	No	Write a deterministic killer; do not unfreeze to pass the gate
Frozen nodeid turns passed	No	Treat it as a suite change; require a ticket
Fixture hash changes with production code	No	Split the fixture bump into its own review
All hunks killed, properties pass, freeze untouched	Yes, pending human review of the diff	Review still required for security and product intent

The last row is the only passing row. Coverage percentage is not a row.

Limitations

Revert-kill assumes you can name production paths. A monorepo that mixes generated protobufs, vendored snapshots, and runtime code in one directory will misclassify oracles as production and delete them during revert. Fix the path file first.

It also assumes tests are deterministic enough that red on revert is causal. If the suite is globally racy, the kill map is noise. Quarantine first. Do not use this gate to launder a flaky integration farm.

Property checks on frozen fixtures will not see production configuration drift, clock-dependent tax rules, or third-party API shape changes. Those need contract tests against a recorded cassette versioned the same way as ledger_v4/. Live-endpoint properties do not belong in the merge gate.

Hunk split quality depends on the diff. A one-hunk rewrite of a whole module is a single unit. If the agent replaced 800 lines, revert-kill can only tell you the module is or is not tested as a blob. In that case require a smaller patch or an explicit characterization fixture for the module's public functions.

Who should not use this

Do not use hunk-level revert-kill as a substitute for review on authentication, crypto, or privacy-sensitive diffs. An empty kill set is a blocker. A full kill set is not a shipping license.

Do not use it on teams that cannot keep a linear merge-base. Revert against the wrong parent produces a fake kill map.

Do not feed free-model output straight into tests/properties/. Generated invariants are drafts. Unreviewed drafts recreate the original problem: the author of the patch writing the oracle that blesses the patch.

Do not apply the freeze file to tests that are simply failing. xfail is for known nondeterminism with an expiry. It is not a parking lot for broken assertions.

What to implement first

Start with file-level revert-kill on src/ and a one-page freeze file. Add fixture-hashed properties only for the module the agent touches most. Expand to hunks when a file-level killed status stops being credible.

The artifact to keep is the kill map plus the fixture manifest hash. Those two files explain a merge in a way a coverage total never will. If you want a spare checkout to run the matrix while the main pipeline stays thin, run the same scripts there and keep merge authority in policy.py, not in the process that proposed the extra properties.