Originally published at tddbuddy.com.
Related reading: Agent PRs Do Not Converge Under Design Review argued the current-PR review turn is the wrong artifact for an agent author. The Bounded Context Is the Scope the Agent Actually Needs names the scoping primitive at the service tier; this post adds the per-PR scope assertion at the change-set tier. Tamper-Resistant Test Design is the structural companion.
The agent fixed the loyalty discount. It also rewrote the rounding helper. The reporting pipeline broke a week later on a customer the ticket never named.
The MSR 2026 reverted-changes study analyzed 33,580 agentic pull requests across five coding agents. 2.66% of those PRs contain at least one reverting commit. The top cause categories are unintended side effects and overengineering (22.33%) and functional incorrectness (22.13%). The split is the whole story. Agents are not primarily failing at writing the fix the ticket asked for; they are failing at not writing more than the fix at roughly the same rate they fail at correctness. The industry conversation keeps underselling the first category.
A sampler notices an adjacent helper looks wrong and changes it. The ticket shipped. The helper changed. The audit log rounds differently in a reporting path the ticket never mentioned. The PR template said "keep changes focused." The reviewer said "LGTM." Neither artifact asked the question the test suite was supposed to answer: what should this PR specifically NOT touch.
The MSR 2026 Numbers Land on a Scope Diagnosis
The 2.66% headline undersells. The structural claim lives in the split. A quarter of reverted agentic PRs reverted because the change extended beyond its ticket. Another quarter reverted because the change was wrong. The two failure modes are different, and the industry's attention has been on the second one. The first one has an answer the second one does not, because scope is a thing a test can verify directly, and correctness often is not.
Humans typed slowly enough that scope drift was a slow-bleed problem the review loop could mostly manage. A junior developer touching a rounding helper on their way through a loyalty ticket got caught at review nine times out of ten, because the diff was legible to the reviewer who read it. The sampler removed the typing-speed cap and the review loop did not scale with it. The 22.33% is the receipt.
The question that moves is not "how do we train the agent not to touch adjacent code." The question is "what does the suite enforce when the agent touches adjacent code anyway."
Agents Fail at Not Writing More Than the Fix
The failure is structural, not incidental. The sampler's incentive gradient points at consistency: a helper that reads awkwardly, a comment that looks stale, a parameter named oddly all look like candidates for improvement given the context the agent has in that moment. The agent is doing the thing a careful human reviewer would compliment. The agent is also doing it without the human reviewer's context about who else depends on the helper, what else the comment is tracking, or why the parameter was named that way.
An agent-generated PR to adjust a loyalty discount rule changes a rounding helper two files away because the sampler noticed the helper looked wrong. The helper is used by both the loyalty path and the audit log path. The loyalty path's tests pass because the agent updated them. The audit log path's tests are elsewhere in the suite, in a file the agent's working context never loaded, so the agent cannot update them. CI runs the whole suite, and if the audit log has an invariant test that pins the current rounding, the invariant fails. If the audit log has no such test, the invariant breaks silently and the reporting consumer breaks the following week.
The structural failure is not that the agent was careless. The structural failure is that the agent had no artifact in its context that said "the rounding helper is load-bearing for the audit log, and this PR is not supposed to change it." The template said scope matters. The template was not in the test runner.
PR Templates Catch at Reading Speed
A reviewer who reads a 400-line diff and asks "was all of this in scope?" is doing the work at reading speed, PR by PR. The sampler is writing 400-line diffs faster than the reviewer can read them. The arithmetic does not close.
Diff-size caps are the same shape of intervention. A 400-line cap tells the agent to write smaller PRs. The agent complies by writing two 200-line PRs that between them contain the same scope drift as the original 400. The cap optimizes an incidental property (line count) rather than the one that matters (invariants held constant). Reviewer checklists that mark "out-of-scope changes" as a review item are honest about the problem and still run at the review loop's cadence, which is the loop the sampler already outran.
The direction worked for human authors because the typing speed capped the damage. It does not work for agent authors because the sampling throughput removes the cap. The template approach assumes the review loop can keep up with the author. In the agent era, that assumption is quantitative, not qualitative, and the quantities moved.
A Scope Assertion Is a Test on What Must Not Change
The architectural move is one level up, in the suite. A scope assertion is a test the PR is not supposed to touch: an invariant on the billing calculation that is NOT the subject of this change, a contract on the audit log schema this PR is NOT supposed to migrate, a scenario on the order-total rounding this PR is NOT supposed to adjust.
[Fact]
public void Order_rounding_matches_fiscal_2025_treatment_for_all_wholesale_customers()
{
foreach (var order in theWholesaleOrderFixtureSet)
{
order.roundedTotal.Should().Be(fiscal2025ReferenceTotal(order));
}
}
If the agent changes the rounding helper as a side effect of a loyalty ticket, CI fails on an assertion the agent never saw. The agent cannot "fix" the test; the test is named for a behavior the PR was never supposed to touch. The failure is immediate, the signal is specific, and the enforcement is agent-blind. The scope the template was supposed to guard becomes a property the suite owns, enforceable at commit speed.
The organization layout matters. tests/scope/billing-invariants.cs, tests/scope/audit-log-schema.cs, tests/scope/order-rounding.cs. Each file holds the invariants the active work is not supposed to touch. A domain tag on the PR selects which scope files run alongside the regular suite. A ticket in the loyalty area runs the loyalty-adjacent scope assertions; a ticket in billing runs the billing-adjacent ones. The test runner is doing the sorting the reviewer was doing by eye.
Refactor PRs Carry Their Own Scope Assertion
The honest counterexample is the refactor. A refactor PR is a change whose scope is "no observable behavior change." A rule that any PR tripping an unrelated invariant test fails CI collapses legitimate refactor work into a thousand tiny tickets, which is a worse workflow than the one the post is trying to fix.
The concession is small. A refactor's scope assertion is a different shape: a property test or a characterization test asserting that the behavior under refactor is byte-identical before and after.
[Property]
public void DiscountChain_refactor_preserves_total_for_every_order_in_the_fixture_set(
Order order)
{
discountedTotalBefore(order).Should().Be(discountedTotalAfter(order));
}
The refactor is accepted on the strength of the property, not on the reviewer's eye for subtlety. The scope question has an answer; the answer belongs in the suite. The rule is not "no PR may touch adjacent code." The rule is "every PR must be honest about its own scope, and the honest scope is a thing the suite verifies." A ticket PR tripping an unrelated billing invariant and a refactor PR failing to preserve observable behavior are different structural failures caught by different scope assertions of the same discipline.
Scope Is a Design Property of the Suite, Not a Policy of the Template
An instruction that says "keep PRs focused" is a request. A test that fails when the PR drifts out of scope is a receipt.
The discipline the series has argued for all year applies here too. Structure wins over directive. The suite is where the structure lives. A rules file that tells the agent to stay in its lane is a prompt the agent reads and does its best to follow; a scope test file that fails CI on the invariant the agent crossed is a fence the agent cannot talk its way through. The two artifacts carry different kinds of evidence, and the fence is the one that survives contact with a sampler that writes faster than its reviewer reads.
The house position on governed judgment extends here directly. A scope assertion can be a deterministic invariant (schema diff, rounding behavior, fixed property) or a governed model judgment (a second agent with independent context reads the diff and judges whether the touched files match the ticket's scope, with a human-authored rubric and a deterministic gate on the judge's verdict). The deterministic form is the default because it is cheaper and sharper. The governed form is the extension for scope questions where the invariant is harder to pin.
The agent fixed the loyalty discount. It also rewrote the rounding helper. The reporting pipeline broke a week later. The template said "keep changes focused." The reviewer said "LGTM." The scope assertion the suite did not carry was the only artifact that could have caught the drift before merge.
Scope was always a test-design question. Humans typed slowly enough that the review loop could mostly cover for the suite's silence on the question. The agent era removed the cover and left the question exposed. The MSR 2026 numbers are the measurement; the scope assertion is the response. The suite that answers "what does this PR promise not to change" catches the 22.33% at commit speed. The suite that leaves the question to the template catches the ones the reviewer notices.
The question has an answer. The answer belongs in the suite, in a test file named for the invariant, loaded by the domain the ticket touches, agent-blind by construction. The template was always a request. The suite is a receipt. The agent era is the one that made the difference obvious.
Top comments (0)