Over-Mocking Is a Governance Problem, Not a Review Smell

Originally published at tddbuddy.com.

Related reading: Agents Pattern-Match Your Test Smells named over-mocking as one of five smells; this post is the dedicated argument backed by the empirical study. The Contract Test Is the Only Witness the Agent Cannot Author owns the seams the allowlist opens. The Instruction File Is Not the Discipline is the same failure mode at a smaller scale.

The study counted the mocks. The answer was a policy question.

A 2026 paper by Hora and Robbes (arxiv 2602.00409, MSR 2026) measured 1.2 million commits across public repositories, isolated the 48,563 authored by coding agents, and compared the rate at which those commits introduce test doubles against the rate at which humans introduce them. Agent test commits add mocks 36% of the time. Non-agent test commits add them 26% of the time. The agents concentrate on a single test-double style where humans spread across stubs, fakes, spies, and mocks. The suites that result are more brittle than the human baseline on the measures the paper reports.

The gap is directional, not deterministic. Ten percentage points on a measured dataset is a clear signal; it is not evidence that agents mock everything in sight. What the signal describes is a tilt. A codebase that already leans on mocks gets leaned on harder. A codebase that leans harder teaches the next session to lean harder still. The industry's instinct is to catch this in review. The industry's instinct will not scale.

Over-mocking stopped being a review smell on the day the sampler got faster than the reviewer. What has to replace the review is governance, and the convention layer is where the fix lives.

The Numbers Land on a Governance Question

The MSR study is not a thought piece. It is 1.2 million commits, 48,563 of them agent-authored, measured on a comparison the review-pass-at-a-time workflow has no way to see. The 36% versus 26% delta is the headline. The subsidiary findings matter as much. Agent-generated test commits cluster narrowly on one test-double style, where human commits distribute across the toolbox in a way that tracks the shape of the problem. Brittleness on the agent-generated portion reads higher than the human baseline on the paper's measures.

That shape is a governance shape, not a taste shape. A review-taste finding would show scattered individual lapses the reviewer could flag one at a time. A governance finding shows a measurable, directional, dataset-scale tilt that recurs regardless of which reviewer is on the PR. The reviewer did not cause the tilt. The reviewer cannot fix the tilt. The tilt is a property of how the sampler reads the codebase and extends it.

The intervention that moves the number lives at the layer that shapes what the sampler sees.

Governance beats hunting.

Review-Smell Framing Catches One at Reading Speed

The earlier post in this arc listed over-mocking as one of five smells agents reproduce from the local suite. That framing is honest and it is necessary. It is also insufficient on its own. The review-smell lens looks at a single PR, surfaces a single mock, asks a question, and moves on. The workflow fits the production rate of a human writing one test at a time. It does not fit the production rate of a sampler writing twenty.

The math is against the reviewer structurally. A reviewer who catches eight mocks in a diff of twenty tests has caught eight mocks and shipped twelve. The twelve ship with precedent weight: each becomes an example the next session samples. Next week's PR arrives with thirty tests built on top of the twelve. The reviewer who was running an uphill fight starts running one on a steeper hill.

What the review-smell framing earns is the diagnosis. What it does not earn is the fix. The fix is one level up, at the convention the reviewer is defending. A reviewer defending a strong convention catches the exceptions. A reviewer defending nothing in particular is reading diffs at reading speed against a sampler running at generation speed, and the arithmetic has one answer.

The smell outruns the reviewer. The reviewer's job is to hold the convention that decided whether the mock was allowed.

The Precedent Is Load-Bearing

Agents mock what the codebase mocks. Every merged PR adds a vote for the pattern it carried, and the next session reads the vote. A suite with a mock-heavy convention will produce mock-heavy tests at generation throughput for as long as the convention stands. The precedent is not a historical artifact. It is the active specification for what the next hundred tests will look like.

This is the mechanism the earlier posts in this arc have been circling. Agents are faithful pattern-matchers. The faithfulness is a feature when the pattern is good and a liability when the pattern is a smell. A codebase that teaches "every collaborator is a mock" by example gets that teaching reproduced with structural fidelity, in every file, at generation speed. The sampler does not know the convention is a smell. The sampler knows the convention is what the suite says to do.

The suite is self-instructing. The question is which instruction it carries.

No Mocks Without an Allowlist Entry

Flipping the default is the move. Mocks are not banned. Mocks are not allowed without a named seam. A static analyzer rejects Mock<T> everywhere by default and permits it only in paths the team has listed: the specific boundary files that touch external systems the suite cannot legitimately stand up in process. The rule runs in CI. A test that adds a mock outside the allowlist fails the gate. The gate does not negotiate at the PR layer.

The composition primitive the suite reaches for by default is not a mock. It is a builder plus a real domain object:

[Fact]
public void Loyalty_members_get_a_ten_percent_discount_on_orders_over_fifty_dollars()
{
    var cart = aCartReadyForCheckout()
        .forCustomer(aLoyaltyMember())
        .containing(aBookCosting(60.dollars()));

    var receipt = checkout.process(cart);

    receipt.discount.Should().Be(6.dollars());
    receipt.total.Should().Be(54.dollars());
}

The mock-heavy version of the same test would set up three mocks, stub the pricing service, stub the membership service, stub the repository, call the method under test, and verify an invocation on one of the mocks. The mock version tests the mock. The builder version tests the behavior. The second one is shorter, reads in the domain's vocabulary, and does not grant the next session a precedent that says "wire these three mocks together."

The honest concession has one shape. Some seams need mocks. A payment gateway cannot run inside a unit suite. A third-party API with rate limits cannot be exercised on every PR. A filesystem call to a device that is not there has to be faked. The allowlist names those seams by path. The rule is not "no mocks." The rule is "no mocks without an allowlist entry," and the entry is the convention the team signs.

An allowlist with ten names is a convention. A codebase with no allowlist is also a convention, and it is the one the study measured.

Contracts Own the Seams the Allowlist Opens

At the seam where mocks are legitimate, the mock is not the whole story. The mock verifies the local shape of the call. The contract test verifies the truth at the seam. The companion post on contract tests pinned the mechanism: a test the agent wrote against code the same agent wrote is a mirror, and the integration seam is where a second author lives.

Pair the allowlist with the pact. Every allowlisted mock has an attached contract test, authored on the counterparty's cadence, stored outside the writable tree of the sampler that owns the client. The mock keeps the unit suite fast. The contract keeps the mock honest. Remove one and the other is defeatable. Keep both and the seam is covered by two independent witnesses, which is what the seam has needed since before the word "agent" meant what it means now.

The mock verifies the shape the client sent. The contract verifies the shape the counterparty will accept. The two are not substitutes.

Governance Is a Test-Design Property, Not a Policy Document

A markdown file in the root of the repository that says "avoid mocks" is theatre. The writer-agent can read it, agree with it, and ignore it on the next sample. An unstructured directive that lives in prose and depends on the sampler to self-police is a request, not a rule. The request is honored exactly as reliably as the sampler finds it convenient to honor, which is unreliably. The study measured the unreliability directly, in points of a tilt, across a dataset larger than any team's workflow.

The nuance matters. The argument is not that an instruction file can never work. The argument is that an unstructured directive the writer-agent can see and reinterpret is theatre; a structural rule the writer-agent cannot bypass is design. A lint rule is design. A codeowners entry on the test directory is design. A failing CI gate is design. An AGENTS.md paragraph that no gate enforces is prose. Prose captures intent. Structure enforces it. The enforcement stack has been the house frame throughout this series: prose, tests, contracts, policies, governed judgment, production as verdict. The policy file sits at the prose tier. The convention lives at the policy tier. Confusing the two is how teams install discipline that does not install.

The cost of installing the convention is small. Ten lines of analyzer configuration, one week to migrate the current offenders, one honest conversation per named seam. The payoff compounds across every future test the sampler ever writes against the codebase. The asymmetry is the argument.

The study counted the mocks. The answer was never going to come from the PR queue. The answer lives at the convention layer, where a rule the sampler cannot talk its way around decides whether the next thousand tests look like the ones the paper measured or like the ones the team meant to ship.

Over-mocking is a governance problem. Governance is where the governance work goes.