Two teams, same Claude Sonnet model, same benchmark tasks. Team A scores 52.4, Team B scores 76.2. The only difference: the harness wrapping the model.
The harness is the binding constraint, not the model. Architecture decisions about loop structure, state durability, tool surface, and verification gates produce performance gaps larger than any model upgrade can close. Most teams optimize prompts when they should be engineering loops.
23.8 Points Separate Harnesses, Not Models
Harness-Bench (arXiv:2605.27922, May 2025) ran identical models on identical benchmark tasks inside two different harnesses: NanoBot scored 76.2, OpenClaw scored 52.4. A 23.8-point gap with no model difference. The same study found that 24.6% of harness failures were tool errors without any recovery mechanism, and 9.3% were interrupted workflows that left no durable progress behind. Cursor's internal benchmarks show the same pattern at different magnitudes: identical models score 46% with one harness and 80% with another.
The implication is uncomfortable. A team that spends six months waiting for a better model will still underperform a team that spent two weeks fixing their harness. 65% of enterprise AI failures in 2025 traced to harness defects -- context drift, schema misalignment, state degradation -- not model capability limits. The model ceiling is not what most teams are hitting.
The specific failure categories matter. Tool errors without recovery (24.6%) happen when the harness lacks retry logic differentiated by error type: a schema validation error and a rate limit error require different recovery paths, and a harness that treats both as "failed" will retry unrecoverable errors indefinitely. Interrupted workflow failures (9.3%) happen when the loop stores progress in context rather than in durable storage -- one process death erases everything.
Five Composable Loop Primitives Every Harness Combines
Inside the Scaffold (arXiv:2604.03515, April 2025) analyzed 13 open-source coding agents and found that all of them combine five universal primitives: ReAct (thought-action-observation sequential), generate-test-repair (generate patch, run tests, repair on failure), plan-execute (DAG upfront then execute nodes), multi-attempt retry, and tree search (MCTS with scored branches). 11 of 13 agents compose multiple primitives simultaneously. SWE-agent and the majority of production systems use ReAct as the inner primitive; Aider's --auto-test and TDFlow implement generate-test-repair; LLMCompiler and HyperAgent use plan-execute for long-horizon tasks; Moatless Tools implements tree search for multi-path exploration.
The harness engineering decision is which combination to select for which task class. A single code repair task calls for generate-test-repair inside a multi-attempt retry shell. A large refactor with multiple independent modules calls for an outer plan-execute layer wrapping per-module generate-test-repair cycles. The most common production pattern -- outer plan-execute with inner generate-test-repair -- is exactly what a file-based queue feeding per-cycle fresh context windows implements. The harness encodes this structure explicitly; leaving it implicit in the agent's prompt is what separates research prototypes from production systems.
State Without Durability Means Recovery Without Memory
OpenHands (arXiv:2511.03690, November 2025) took a clear architectural position: all interactions are immutable events appended to a log, ConversationState is the single source of truth, and agents are stateless. They replay the event log to resume interrupted sessions. The result was 61% fewer system-attributed errors on interrupted session recovery compared to stateful context approaches. The agents do not "remember" anything; the log remembers everything, and the agent reconstructs from it.
The failure mode of stateful context is silent. When a process dies mid-cycle, stateful agents restart from zero. The work done -- files read, decisions made, tests run -- evaporates. Event-sourced agents restart from last verified state. Each queue entry must carry enough context for a cold agent to orient: spec path, affected files, current test status, last action taken, and measurable acceptance criteria. Without that, every interrupted cycle is a complete restart, and long-horizon tasks become expensive retries.
Infinite Loops Are Not Edge Cases -- They Are the Default Failure Mode
IAL-Scan (arXiv:2607.01641, July 2026) analyzed 6,549 agent repositories and found 68 confirmed infinite loop failures in 47 projects. These are not theoretical risks. SyncMind (arXiv:2502.06994, February 2025) found that agents show consistently low collaboration willingness at 4.86% -- agents almost never signal being stuck and almost never request help. The harness cannot wait for the agent to report a problem; the harness must detect it externally.
Five safeguards are non-negotiable. A hard iteration cap per cycle (30 steps is a reasonable ceiling). A no-progress detector: halt if git diff HEAD returns empty after five consecutive steps. A token budget ceiling to kill runaway cost. Action deduplication via hash(tool + args): the same tool call with identical arguments twice in a row is a loop signal, not normal retry behavior. Named terminal states with measurable done criteria: the cycle entry must specify what "done" means in observable terms, not prose.
Beyond pass@1 (arXiv:2603.29231, March 2026) found that higher-capability models have higher catastrophic failure rates on long-horizon tasks due to more ambitious strategies. More capable models create longer loops, not safer ones. The hard caps exist precisely because model quality does not substitute for harness discipline.
Maker-Checker Gates Prevent Yes-Spirals
When an agent verifies its own work, it converges toward declaring success. The pressure to terminate the loop -- cost, step count approaching cap, token budget -- creates a structural incentive to find the work good enough. TDAD (arXiv:2603.17973, March 2026) quantified this precisely: bare TDD instructions given to an implementing agent increased regressions from 6.08% to 9.94%, worse than no TDD at all. The agent evaluated its own tests against its own implementation, a tight loop with no external check. Introducing dependency-aware test impact maps -- structurally separate verification -- cut regressions to 1.82%.
The structural enforcement is simple: cycle status cannot advance to Done without a REVIEW:APPROVED event written by a verifier role, not the implementer. This is not a convention or a prompt reminder. The harness checks the event log before allowing a terminal state transition. Without this gate, yes-spirals are not a risk -- they are the path of least resistance. Pipeline accuracy data makes the consequence concrete: relay-based handoffs degrade from 90.7% accuracy at one stage to 41.2% at two stages and 22.5% at five stages. Structured artifact handoffs are the only pattern that resists this degradation.
Context Compression Kills Constraints Unless You Anchor Them First
Governance Decay (arXiv:2606.22528, June 2026) documented what happens to safety constraints and invariants during context compaction: they get dropped. Compaction pipelines optimize for task continuity, which means they preserve recent tool outputs while discarding the earlier system-level constraints that bound the loop's behavior. The result is a loop that continues executing but is no longer governed by the rules it started with.
JetBrains' observation masking work (arXiv:2508.21433, NeurIPS 2025) tested deterministic masking against LLM-based summarization over 250-turn trajectories. Deterministic masking produced a 52.7% cost reduction with equal or better solve rates. LLM summarization produced trajectories 13-15% longer by obscuring stop signals -- the model read a summarized test output and missed that all tests were already passing. The correct architecture: spec content, invariants, and active constraints belong in the system prompt (the pre-compaction zone), not in conversation history. Tool outputs are deterministically masked before context injection, never summarized. Reducing tool surface from 37 to 5 tools improved local model performance from 2/10 to 10/10 on SWE-agent benchmarks -- fewer hallucinated tool calls, fewer recovery events, less context consumed per step.
The model is commodity. The harness is the product. Measure your loop by three signals: does state survive process death, does a structurally separate verifier gate every terminal transition, and does your no-progress detector fire before the token bill does.
Top comments (0)