Agents Pattern-Match Your Test Smells

Originally published at tddbuddy.com.

Test smells were a slow-bleed problem. With agents, they are a fast-bleed problem.

For most of software's history, a bad test habit entered the suite at the speed a developer could type it. One overworked teammate reached for a mock-of-mock on a Friday afternoon, a junior saw it, copied it on Tuesday, and by the end of the quarter the pattern had drifted into maybe a dozen tests. Slow enough to catch in review, slow enough to refactor in a focused cleanup sprint, slow enough that the damage stayed local.

That speed limit is gone. An agent reading a suite full of eager-mock tests will write more eager-mock tests. It is not imitating anyone's bad habit by accident. It is doing exactly what it is supposed to do: learning the local convention and extending it. The convention happens to be wrong. The agent does not know that. The agent knows only what the test suite already says.

The smells in your suite are now the smells in your roadmap. Every sprint that ships agent-written tests without a smell-focused review is a sprint that ships the existing bad patterns at generation speed. The damage curve changed shape. The team that hasn't noticed is running a different risk model than it thinks it is.

Smells Were Always Sampled

Developers have always learned test style by reading existing tests.

This is not a critique. It is how skills transfer in a codebase. A new team member opens the repo, finds the nearest test file, and writes their first test in the style they find there. If the team writes scenario-named tests with fluent builders, the new member writes that. If the team writes test_1 with twenty lines of primitive setup, the new member writes that. The first test they encounter sets the mold.

The same process applies to experienced developers when they move into unfamiliar areas of a large codebase. Nobody reads the full suite before adding a test for the payment module. They open the existing payment tests, absorb the pattern, apply it, and move on. The style propagates by example because that is the most efficient path.

Style by example means smells propagate the same way virtues do. A codebase with well-factored builders and scenario-named tests tends to produce more of the same, because the example sitting next to the new test is already correct. A codebase with mystery setup and assertion-on-call tends to produce more of the same, because the example sitting next to the new test is already wrong.

The containing factor was typing speed. A developer producing ten to fifteen tests per day caps the blast radius. A smell entering the suite on Monday might infect thirty tests by Friday. Painful, but bounded. Review cycles, pairing sessions, and periodic cleanup sprints could stay ahead of it.

The taxonomy of these smells is not new. Meszaros catalogued many of them in xUnit Test Patterns in 2007. Van Deursen and colleagues described a related set in the early 2000s. The community has known what the smells look like, what they cost, and how to remove them for the better part of two decades. The reason the taxonomy did not solve the problem is that the propagation rate was slow enough to tolerate. Nobody treated smell elimination as urgent because the alternative (living with the smells) was manageable. Generation speed removes the "manageable" option.

The Cap Just Came Off

An agent writing tests does not work at human typing speed.

A reasonably capable agent, given a PR with twenty new production behaviors, will produce twenty tests in roughly the same time it takes a human to write three. For large features or for agents tasked with expanding coverage across an existing module, the output is an order of magnitude faster. This is not a projection. It is the current state of teams using agentic workflows seriously.

The smell-propagation rate scales with the generation rate. A codebase carrying the eager-mock pattern at ten percent density will, after one sprint of agent-generated tests, carry it at thirty or forty percent. Not because the agent was careless. Because the agent was correct. The agent found the local convention, weighted it appropriately, and applied it uniformly. The uniformity is the problem. What a human would have copied inconsistently, the agent copies with fidelity.

This is the uncomfortable inversion: the agent's accuracy is what makes the smell dangerous. A human who copies a bad pattern often mutates it enough in transcription that the copy is slightly less harmful than the original. The agent copies it cleanly. Ten bad tests in the sample produce ten bad tests in the output, in the same structural family, with the same shape, legible as a pattern to anyone who looks at the suite level.

The smell that was a nuisance when humans wrote tests is a structural liability when agents do. The blast radius is generation-speed-shaped now, not typing-speed-shaped.

The Smells That Matter Most

Not all smells replicate equally. Five categories deserve attention by name because they are the ones agent-written tests carry most often, and they are the ones that do the most downstream damage.

Eager mocking is the habit of substituting every dependency a class touches, then testing only the wiring. The test creates five mocks, sets up three of them, calls the method under test, and verifies that specific methods were called on specific mocks. The production behavior under test is the collaboration graph, not the domain outcome. Eager-mock tests survive refactors badly: rename a private method, extract a collaborator, or shuffle responsibility between classes and the tests break, even when the observable behavior is unchanged.

Assertion-on-call is eager mocking's closest cousin. The test passes if the mock recorded the right invocation. Whether the system produced a correct result for the end user is not verified. warehouse.Received().Reserve(...) is not wrong in isolation; it becomes a smell when it replaces, rather than accompanies, an assertion about what the customer actually experienced.

Mystery setup is twenty lines of arranged test data with no named intent. The reader can see that an order is being created with a customerId of "abc-123", a shippingZone of 3, and a memberSince date of 2019-01-15. The reader cannot tell whether any of those values are load-bearing, whether the combination is meaningful, or what business scenario this test is actually covering. Mystery setup is not the absence of builders; it is builders used as low-level scaffolding without wrapping the result in a named scenario.

Magic-number assertions are the mirror of mystery setup. result.Should().Be(42) tells the reader the expected output but not why that output is correct. Was forty-two computed? Fixed? A boundary? The assertion verifies the system under a specific condition but provides no documentation of why the condition matters. Agents reproducing magic-number assertions reproduce the value without the reasoning, and the reasoning is what makes the assertion legible to the next developer who changes the underlying logic.

Brittle fixtures are shared world objects that multiple tests mutate. The fixture works for the first test written against it, and quietly corrupts the second, the fifth, and the twelfth. Tests that pass in isolation fail in the suite. Tests that pass in one order fail in another. Agents writing against a codebase with brittle fixture conventions write more brittle fixtures, because fixture-first setup is the pattern they find.

Reviewers Will Miss Them

The review culture built around human-generated tests is poorly calibrated for agent-generated ones.

When a developer writes a PR with three new tests and four smell-carrying violations, the violations arrive one at a time. A reviewer reading the diff sees the first mystery setup, flags it, asks a question, and the author fixes it. The review tooling, the mental model, and the feedback loop all fit the production rate.

An agent-generated PR arrives with twenty tests, twelve of which carry one or more of the five smells. The reviewer is reading the same way, line by line, looking for logic errors and naming issues. The smell pattern is not visible at the line level. It is visible at the suite level. The reviewer who would have caught a single mystery setup in a human-written test will miss the structural pattern when twelve tests share the same unintentional vocabulary.

There is a second failure mode specific to agent-written batches: the smells are consistent. Human-written smells are varied because different developers reach for different bad patterns. An agent trained on a suite carrying eager-mock conventions writes eager-mock tests with high structural fidelity, and a reviewer scanning for smells is less likely to notice a consistent pattern than an inconsistent one. The pattern looks, superficially, like deliberate style.

There is a third failure mode that compounds the first two: reviewers are reading the test diff at the same time as the production diff, and the production diff gets the cognitive budget. Test review is typically the last thing on the checklist, done quickly, with diminished attention. The smell that slips past an alert reviewer at the start of a review session will almost certainly slip past a tired one at the end. Agent-generated tests arrive in volume, which means the test diff is often the largest part of the PR, which means the most review fatigue lands exactly where the most smell-detection work needs to happen.

The review that catches individual bad tests is not the review that catches suite-level structural smells. These are different cognitive tasks. Most teams are running the first kind and thinking they have covered the second.

The Review Pass That Is Missing

There is a review step that almost no team runs on agent-generated PRs: explicit smell-detection at the test-suite level, as a separate pass from the production-code review.

The production diff gets scrutinized for logic, naming, architecture, and security. The test diff gets skimmed for coverage and whether the assertions look plausible. The smell-detection pass does not happen because it was never formalized, and it was never formalized because the human-speed propagation rate made it optional. It is not optional anymore.

A smell-detection pass is not complicated. It is a short checklist run against the test diff before merge:

• Are any new tests mocking their way through multiple layers, then asserting only on mock invocations?
• Are any assertions verifying that a method was called without also verifying a domain outcome?
• Does the test setup name a scenario, or does it just assemble properties?
• Are any expected values numeric or string literals with no named explanation?
• Do any new tests share mutable state with other tests?

Five questions. Run them at the test-suite level, not the line level. Most agent-written PRs will fail at least one item in the first sprint after introducing agentic test generation. That is not a failure of the tooling. It is a signal about what the existing suite looks like, amplified.

The checklist is not a permanent overhead. It is a forcing function that surfaces the existing smells before they replicate further. Run it seriously for two sprints and the team will have a clear map of which areas of the suite need restructuring. The agent will then pattern-match against the restructured areas and the smell rate will drop. The checklist does not eliminate smells; it stops them from compounding while the underlying structure improves.

Builders Are the Defense

The good news is that the smell-free versions of these tests are shorter, not longer.

This is the counterintuitive part that teams resist. Mystery setup feels thorough. Twenty lines of arranged data feels like documentation. The builder-and-scenario rewrite feels too brief. The brief version is actually the better artifact, because the names are doing the documentation work and the values are invisible unless they deviate from the scenario's default.

Walk through each smell and its repair:

Eager mocking dies when the test composes a real domain object for the main subject and reserves mocks for the explicit collaboration boundary. If the behavior under test is a discount calculation, mock the external pricing feed and wire in a real DiscountCalculator with a real Order. The test then verifies a domain outcome, not a call graph.

Assertion-on-call becomes honest when a domain assertion is added alongside the collaboration assertion. The warehouse mock that received a Reserve call is fine to verify; the test should also verify that the customer got a confirmation with the right items. Both are true at once. The call assertion documents the collaboration; the domain assertion documents the behavior.

Mystery setup dissolves into a named scenario factory. The twenty lines of field assignments become aFirstYearLoyaltyMemberWithExpiredPaymentCard(). The name is the documentation. The fields inside the factory are an implementation detail the test does not need to see.

Magic-number assertions become legible when domain types are in play and the expected value is composed, not typed. Compare these two:

// Before: magic-number assertion, mystery setup
[Fact]
public void ProcessOrder_WithLoyaltyMember_ReturnsCorrectDiscount()
{
    var mockRepo = new Mock<IOrderRepository>();
    var mockPricing = new Mock<IPricingService>();
    var mockMember = new Mock<IMembershipService>();

    mockRepo.Setup(r => r.GetOrder(It.IsAny<string>()))
            .Returns(new Order { Id = "x1", CustomerId = "c1", Subtotal = 60.00m });

    mockMember.Setup(m => m.GetTier("c1")).Returns("loyalty");
    mockPricing.Setup(p => p.GetRate("loyalty")).Returns(0.10m);

    var svc = new OrderService(mockRepo.Object, mockPricing.Object, mockMember.Object);
    var result = svc.ProcessOrder("x1");

    mockPricing.Verify(p => p.GetRate("loyalty"), Times.Once);
    Assert.Equal(54.00m, result.Total);
}

// After: named scenario, domain types, composed assertion
[Fact]
public void Loyalty_members_receive_ten_percent_discount_on_qualifying_orders()
{
    var order = anOrder()
        .forCustomer(aLoyaltyMember())
        .withSubtotal(60.dollars());

    var receipt = checkout.process(order);

    receipt.discount.Should().Be(6.dollars());
    receipt.total.Should().Be(54.dollars());
}

The second test is twelve lines shorter. It names the scenario in domain language. It asserts the domain outcome (not the mock invocation). The expected values are composed from 6.dollars() and 54.dollars(), which is not cryptic because the test name already told the reader the discount rate. The tenPercent calculation is implicit in the named relationship between sixty, six, and fifty-four.

Brittle fixtures disappear when each test constructs its own world through the builder API. The upfront cost is one builder method per scenario shape. The ongoing benefit is tests that never interfere with each other regardless of execution order.

The builder-and-scenario rewrite is almost always shorter than the smelly original. The argument that smell-free tests require more effort is false. The smelly tests appear to require less effort because the friction is deferred: the developer who writes the magic-number assertion does not pay the cost; the next developer who changes the pricing logic and can't tell which tests represent intentional specifications versus accidental regressions pays it.

There is an additional leverage point here that matters specifically in the agent context. The builder API is the ontology the agent samples. Every named factory method, every fluent builder verb, every domain type in the test layer is a word the agent can use when composing its next test. The more richly that vocabulary is built out, the less often the agent has to invent primitive-level setup from scratch, and the more its output stays inside the domain language the team has established. Investing in builders is not just a defense against existing smells. It is the infrastructure that prevents the next category of smells from forming at all.

The Smells in Your Suite Are Your Roadmap

The agent is a faithful pattern-matcher.

This is a feature during healthy operation: the agent finds the local vocabulary, uses it, extends it, and keeps the suite coherent. Teams that invested in builder APIs and scenario-named tests get that investment amplified. Every agent-generated test follows the convention, which means the convention becomes the dominant idiom in the suite faster than any human team could achieve.

The same property is a liability when the convention is a smell. The agent does not distinguish between a pattern that represents intentional design and a pattern that represents accumulated technical debt. It sees frequency and structural consistency. A smell that appears in forty percent of the tests reads as the local norm. The agent writes the norm.

This means the smells in the current suite are not a historical record of past mistakes. They are the active specification for what the next hundred agent-generated tests will look like. Every eager-mock test already present is a vote for more eager-mock tests. Every magic-number assertion already present is a vote for more magic-number assertions. The suite is self-instructing, and it is instructing the agent right now.

The team's choice is binary: fix the smells deliberately, or ship them at scale.

There is no middle path where the agent generates clean tests against a smelly suite. Pattern-matching is the mechanism. The mechanism does not care about the quality of the pattern. Fix the suite and the agent generates clean tests. Leave the suite and the agent generates smelly ones, faster than the team can review them, compounding with every sprint.

Vocabulary is the product. Every concept in the test suite is a word in the language the agent learns. Smells are vocabulary in the negative direction: they teach the agent the wrong words for the right ideas. Mystery setup teaches the agent that scenarios are assembled from primitive fields, not named intent. Assertion-on-call teaches the agent that the collaboration graph is more important than the domain outcome. These are lessons the suite teaches silently, by example, every time the agent reads it.

The earlier post in this arc, The Bar for TDD Just Moved, named the new floor: scenario-named, builder-driven, domain-typed tests treated as a product surface. That floor matters for exactly this reason. A suite at the new floor gives the agent the right vocabulary. A suite below it gives the agent whatever vocabulary drifted in during the years when the propagation rate was too slow to notice.

The smells were always telling the team something. The team learned to live with what they were saying. The agent is not going to live with them. It is going to say the same thing, loudly, in every PR, until the team decides to fix the source.

The smells in your suite are your roadmap.

Fix them deliberately, before the agent does it for you at scale.

Test smells were a slow-bleed problem. With agents, they are a fast-bleed problem. The difference is not the smells. The difference is the speed at which the existing ones compound. The suite that could be cleaned up over a quarter now has a quarter to compound before the damage becomes structural. The window is not closed. It is closing.