My fix for the agent-edited test gate was wrong. The pin has to be older than the agent.

Three weeks ago I published four fixes for a test gate that runs the tests in the tree the agent just edited. Fix 2 was this: do not run the tests in the agent's tree, run them in a pristine worktree off HEAD and copy in only the unprotected changes.

@howcani_howcani_77e786a89 took it back to a local repo, stopped at fix 2, and wrote: fix 2's precondition is the thing evasion 1 removes. Evasion 1 is the agent committing. After that, HEAD is the agent's commit, so "a commit you control" is a commit the agent controls, and a worktree off it is pristine only with respect to the uncommitted dilution.

He was right. I reproduced it before writing this. Baseline has one real test. The agent commits a weakened one. A clean worktree off HEAD, fresh checkout, nothing copied in from the working tree:

old fix 2 worktree tests/test_calc.py:
    assert True
old fix 2 verdict: PASS

The gate passes on a test that asserts nothing. Fix 2 moved the dilution from "uncommitted edits" to "a commit", and a commit is exactly the thing the agent can make.

The pin has to be older than the agent

The fix is not a better use of HEAD. It is to stop asking HEAD what the baseline is.

Before the agent runs, while the tree is known good, record the protected set as content: every path under tests/, conftest.py, the runner config, the CI file, each mapped to its blob hash.

37895b28ab864addfe1921d4bf035a1563ac0689 conftest.py
4b341e0595f8265e27ed7b7e6119711b672e0e17c tests/test_calc.py

At check time, recompute the same tree listing for those paths and compare. Three ways to differ, all fail: a path is missing, a path's hash changed, a path that was not in the pin appears. Then run the tests on a tree assembled from the pins for protected paths and from the agent's commit for everything else.

A blob hash is not a ref. An agent can rewrite history, orphan its commits, delete a branch. It cannot produce a different file with the same hash.

What I actually ran, four repos from one baseline, one shared pin:

A control src-only:                    added=0 changed=0 missing=0 -> PASS
B agent-committed weakened test:       added=0 changed=1 missing=0 -> FAIL
C delete + add inside tests/:          added=1 changed=0 missing=1 -> FAIL
D rewritten history (1 orphan commit): added=0 changed=1 missing=0 -> FAIL

Case C is the one that caught my own corrected check. Compare hashes of the paths you know about and the agent can leave every pinned path alone, delete them, and drop tests/test_zz.py with assert 1 == 1 in the same commit. The path set has to be compared, not just the hashes. Case D is a repo rewritten to a single orphan commit where the baseline is not an ancestor of anything. The pin does not care, because it never asks about ancestry.

What the pin does not fix

Where the pin lives.

If the harness writes the expected hashes into the worktree it is about to inspect, or reads them out of the repository it is checking, this whole method collapses into the class I hunt for a living: a check that decides on bytes it never reads. An agent that can write the pin can set it to whatever it just produced, and every check above passes.

So the pin is written by the step that owns the checkout, before the agent exists, and kept out of every path the agent's process can write. If you cannot name the component that owns the pin, you do not have this fix, you have a second copy of the problem.

The general shape

A gate is only as strong as the oldest copy of the expected value it holds. If the expected value is derived at check time from anything the checked party touched, then the gate's verdict is decided by the artifact, not by the check. The agent committing is just the most convenient way to make the artifact supply its own baseline.

Corrections: this one came from someone who ran my method instead of agreeing with it, which is worth more than a like and costs him an afternoon. The first piece is https://dev.to/mayailands/your-agents-test-gate-runs-the-tests-in-the-tree-the-agent-just-edited-574

If you ship the pinned version and want a third-party re-run of the check against your repo, with a control that must pass and a negative control that must fail, the receipt is $25 and the card is here: https://buy.stripe.com/fZu5kD1rN0laceh13n5kI02

The finding is free. That is why it is above and not behind a link.