Originally published at tddbuddy.com.
Related reading: Your Test Suite Is Your API for Agents named the suite as the persistent interface for agents. The Bar for TDD Just Moved named the floor. The Instruction File Is Not the Discipline named another directive-shaped artifact sold as the discipline. This post names the wrapper.
The industry named a new discipline in six months.
Mitchell Hashimoto shipped a February 2026 post coining "harness engineering" as the practice of engineering the environment an agent runs inside. The naming caught. Arxiv papers followed (2602.14690, 2604.17025, 2606.22678). The major model vendors published engineering guides. Benchmark tables added a "harness" heading. Curated awesome-lists appeared. An industry reliability synthesis landed in May. A widely-read researcher essay on harness engineering for self-improvement ran in July. The formula given to the field was Agent = Model + Harness, and inside of one season the harness was the wrapper carrying the discipline.
Read the load-bearing parts of every taxonomy that shipped and the same shape repeats. CI gates. Test-suite feedback. Deterministic verification loops. Sandbox permissions that fail closed. The layers that decide whether shipped code is correct are layers the disciplined test suite has always owned. The genuinely new pieces (persistent memory, planning artifacts, retry logic, scratchpads, sub-agent isolation) change what the agent remembers between turns. They do not decide what code gets shipped.
Calling both by one name blurs which piece is doing the correctness work. The honest split is two disciplines.
The Naming Arrived Before the Substance
Trace the arc.
February 2026: a single practitioner post gives the practice a name. The observation underneath it is sharp. The environment around a coding agent (its context window, its tool set, its retry rules, its permission graph) does shape what the agent produces, and the industry had been treating those knobs as incidental. Naming the discipline made the knobs visible. Hashimoto deserves the credit for the observation.
The generalization happened faster than the observation earned. By April, arxiv had a harness paper wave. By June, curated awesome-lists indexed dozens of harness patterns. By July, benchmark tables added a harness column and the framing had drifted from "engineer the environment" to "harness engineering is the new practice that replaces the old one." What started as a useful lens on the run-time environment became a superset claim about where correctness lives, and the substance never had a chance to catch up with the branding.
That gap is what the rest of this post interrogates. The observation was real. The superset claim is not, and pretending otherwise costs teams a clear view of what actually decides whether their agent ships correct code.
Every Harness Taxonomy Rediscovers the Same Load-Bearing Layer
Pull any published harness taxonomy and count the top-tier layers.
A representative one from the arxiv wave lists eight: CI gates, test suites, sandbox permissions, deterministic verification loops, sub-agent isolation, memory eviction, planning artifacts, retry logic. Strip the rebranding off the first four and the answer to "did this PR ship correct code" lives entirely in them. The remaining four decide what the agent held in context at each step, which is a real question about run-time behavior. It is not the question of whether the code that landed is right.
Every taxonomy that shipped has this ratio. Half the layers are the maintained verification surface under a new label. Half are new plumbing about the agent's own state. The verification half is where the correctness signal actually gets produced. The plumbing half decides how efficiently the agent reaches for that signal.
The verification half is not exclusively the unit test file. It is also contract tests at the seam, mutation-testing gates, and, where a team runs one, a governed model reviewer with independent inputs, a human-authored rubric, and a deterministic gate around its output. Those all live in the same row. They are surfaces the shipped code has to satisfy, authored separately from the code, and they collectively decide the "was this correct" question. The harness wraps them. It does not replace them.
Rename the wrapper all you like. The bottom row is where the receipt gets printed.
Memory, Planning, and Scratchpads Change What the Agent Remembers, Not What Ships
The genuinely new pieces of harness engineering deserve honest credit.
Persistent memory lets an agent carry context across turns without regenerating it. Planning artifacts let it commit to a route before it executes. Scratchpads let it externalize intermediate reasoning without polluting the visible answer. Sub-agent isolation lets it delegate a bounded sub-task and merge the result. Retry logic lets it recover from a specific class of transient failure without human intervention. Tool-permission graphs let it operate on production-adjacent systems without an unbounded blast radius.
Every one of those changes what the agent knows, remembers, or is allowed to touch during a run. That is a real category. It has real consequences for cost, for latency, for how many turns an agent needs to complete a task, and for what the agent is prevented from doing when it is wrong.
None of them, on its own, decides whether the code that finally lands is correct. Memory eviction picks the wrong context on a bad day and the agent proposes a plausible change against a stale understanding of the codebase. The change still has to pass the tests. If the tests catch the mistake, the wrong context was expensive but not fatal. If the tests do not catch it, the memory was not the load-bearing failure. The suite was.
The run-time hygiene layer moves failure modes about what the agent did during the run. The verification surface moves failure modes about what shipped. They are two different disciplines. They belong to different investment plans. Treating them as one hides which lever moves which failure.
A Rich Harness Around a Weak Suite Is a Scaffold Around a Rotten Frame
Consider two teams working the same task.
The first team has invested heavily in harness engineering. Sub-agent isolation. Persistent memory. A permission graph tuned to their infrastructure. Deterministic replay of every turn. Their test suite is thin: primitive assertions, magic-number expectations, no domain vocabulary. The agent runs the loop cleanly. It never crashes. It never loses context. It ships the wrong discount calculation, and the suite goes green.
// Rich harness, weak suite: the agent's confident output goes green.
result.Should().NotBeNull();
// Maintained suite: the agent has to reach the receipt, not just avoid null.
var receipt = checkout.process(anOrder()
.forCustomer(aLoyaltyMember())
.containing(aBookCosting(60.dollars())));
receipt.discount.Should().Be(6.dollars());
receipt.Should().Satisfy(sumOfLineTotalsEqualsSubtotal());
receipt.total.Should().BeGreaterThanOrEqualTo(Money.Zero);
The second team has a minimal harness and a maintained suite. Their tests read like sentences in the domain: aLoyaltyMember().withExpiredCard() composes into scenarios named after behaviors, and the assertions pin the actual receipt shape. The agent's context is less carefully curated. It sometimes wastes a turn reading a file it did not need. When it proposes the wrong discount calculation, three tests go red: the example, the property that says the discount never exceeds the subtotal, and the invariant that says the total never goes negative. The agent cannot ship until it fixes the code.
The first team shipped confidently. The second team shipped correctly.
That is the split. A rich harness amplifies whatever signal the suite produces. When the signal is strong, the amplification is worth the investment. When the signal is weak, the amplification is a faster loop into the same bug, dressed up in run-time telemetry that suggests something rigorous is happening. The ceiling on an agent-generated PR is bounded by the discrimination power of the tests it must satisfy. The harness reaches the ceiling faster. It does not raise it.
Sub-agent isolation on a suite that certifies the writer's assumptions is a scaffold around a rotten frame. The frame decides how much weight the whole structure can carry. The scaffold decides how quickly a construction crew can move around it. Naming the scaffold a new discipline does not turn it into the frame.
Two Disciplines, Not One
Rewrite the field's opening formula honestly.
Agent = Model + Harness became the shorthand. The reality is Agent = Model + (Runtime Hygiene + Verified Test Surface). The parenthetical is the split the industry blurred, and it is the split that lets a team see which investment moves which failure.
Runtime hygiene is its own discipline, and it is genuinely new. Nothing in the TDD literature covers permission graphs, sub-agent orchestration, memory eviction strategies, or planning-artifact schemas. Those are practices that emerged because agents run differently from human developers, and they deserve their own vocabulary, their own experts, their own arxiv track. Credit the observation. Fund the research. Ship the tooling.
The verified test surface is the older discipline, doing new work. It is what tells the run-time-hygiene layer whether the run produced correct code. Without it, the harness has no signal to hygiene against. With it, the harness gets to do its job. That is the load-bearing role the suite carried into this era from the last one, and no amount of rebranding moved the load.
A team with a rich harness and a weak suite is a team investing in the wrong layer. A team with a maintained suite and no harness is a team leaving efficiency on the floor. A team with both can tell which failure came from which discipline, which is the precondition for fixing either one on purpose.
Harness engineering as a superset was a marketing move. Harness engineering as run-time hygiene sitting on top of a maintained verification surface is the honest split. The suite did not get renamed. The wrapper around it did, and the wrapper matters. It is not the load-bearing wall.
Every harness taxonomy that gets published rediscovers this quietly by naming test coverage as the first layer to build. The naming was six months old. The layer it sits on has been load-bearing for twenty years, and it is still what decides whether the code the agent shipped is code the team wanted shipped.
Top comments (0)