What is the harness?
Claude Opus scored 93% inside Cursor on Terminal-Bench 2.0. The same model scored 77% inside Claude Code — a 16-percentage-point swing on the same benchmark, with the same model weights, the same training data. The only variable was the harness: the layer between the model API and you.
The harness is everything between the model and you — the layer that decides what the model sees, what it can do, and how it recovers when things go wrong. Most developers think the model is the product. The model is the engine. The harness is the car.
The honest version of that number makes the point better: with a community-tuned configuration — the "Mythos" CLAUDE.md — Claude Code scores 92.1% on the same benchmark, within a percentage point of Cursor. The gap isn't that one tool is better. It's that the harness configuration moves the score. And the configuration is your declaration.
Five things the harness controls:
Tool definitions. What the model knows it can do — file editing, terminal access, search, web fetch. Descriptions sit in the system prompt; vague ones cause wrong tool calls.
Rules and system prompt. Your AGENTS.md, CLAUDE.md, whatever rules file the harness reads. Behavioral steering — how to approach tasks, when to stop, what to prioritize.
Context management. What the model sees at each turn — old outputs compressed, summarized, or cut. That policy decides whether the model remembers what it did five steps ago or starts over.
Permission model. Ask before acting, or auto-approve. Speed against risk. A harness decision, not a model decision.
Recovery logic. What happens when the agent gets stuck — retries the same command, detects the stale loop, or escalates to you.
The benchmarks below show what happens when you change the harness and keep the model the same.
The evidence: same model, different results
I didn't run these benchmarks. Others did. Eight papers and three real-world benchmarks — and they all point the same way: the harness moves the score more than the model.
LogRocket (May 2026) tested Claude Opus 4.6 in both Claude Code and OpenCode on the same task — migrating a Next.js dashboard from prop drilling to Zustand. Claude Code: 14 minutes, 2 code mistakes. OpenCode: 7 minutes, zero mistakes. Same Opus weights, same task, same codebase. This was a single-task, informal benchmark — not a controlled study. But the direction is consistent with everything that follows.
Builder.io (January 2026) ran four tasks on a TypeScript project with Sonnet 4.5 in both tools. Claude Code finished in 9 minutes 9 seconds with 73 tests. OpenCode finished in 16 minutes 20 seconds with 94 tests — 29% more tests, but nearly twice the time. Disclosure: Builder.io sells its own coding agent, so read these results with vendor context in mind. MorphLLM later reported the same figures.
SWE-bench Verified (April 2026) is the gold standard — 500 human-validated software engineering tasks. Augment Code's harness scored 72.0% with Opus 4.6. OpenHands scored 68.4% with the same model. A 3.6 percentage point gap from the harness alone.
On a separate leaderboard run with Qwen 3.6-flash, five different harnesses produced a spread from 38.6% to 66.0% — a 27.4 percentage point gap. Same model weights, five wrappers.
CLAW-SWE-Bench (June 2026) tested the same GLM 5.1 backbone across five harnesses. Pass rates ranged from 60.9% to 73.4% — a 12.5 percentage point spread from the harness alone. The paper concluded: "Changing only the harness-specific agent loop, tool interface, workspace management, and stopping policy can produce performance differences comparable to, or larger than, neighboring model tiers."
Worth noting: both wide spreads came from open-weight models — GLM 5.1 and Qwen. The harness effect shows up across model tiers, but it's loudest where the scaffold does more of the heavy lifting. On a frontier model like Opus 4.6, the same leaderboard shows a 3.6-point swing. Expect the harness to matter either way; expect it to matter more on smaller models.
A harness change can outperform a model upgrade.
Lewis (August 2026) tested the same harness with two configurations — a control and a treatment that compressed old tool outputs and detected stuck turns. The metric: fail-to-pass fraction (F2PF), the share of previously-failing tests that a patch fixes. On Devstral, F2PF jumped from 17% to 37%. On Qwen3.6-35B, it jumped from 28% to 49%. One context management change. No model change.
Vats and Golev (June 2026) found a striking number: the same model consumed up to 41 times more tokens in OpenCode than in Goose. Same model, same tasks, same sandbox. But the pass-rate difference was only 0-8 percentage points — the harness burned more tokens to reach similar outcomes. And the failure modes were harness-specific: Goose loops on reasoning, OpenHands gets stuck verifying, OpenCode spins idle. Same model, different pathologies.
The evidence cuts both ways. Builder.io shows Claude Code winning on speed. Vats and Golev show that 41× more tokens don't always produce 41× better results. Princeton tested only Codex on small pull requests. Each study has limits. But the pattern across all of them is consistent: the harness is a variable, and most developers aren't measuring it.
One honest limitation: this synthesis can only cite studies that set out to measure the harness variable. The null results that never got published aren't in the record — that blind spot is built into any literature-based synthesis.
If you've run the same model in two harnesses and seen the opposite, I want to hear it — counter-evidence is exactly what makes this pattern useful. I'll compile the replies into a follow-up.
What the harness actually controls
The benchmarks show the gap in coding agent performance. Here's the mechanism.
Tool descriptions shape selection. When the model receives a list of available tools with descriptions, it decides which to use based on those descriptions. Vague descriptions produce wrong tool calls. Precise descriptions produce correct ones. The CLAW-SWE-Bench paper showed that tool interface changes alone produce 12+ percentage point swings. The model doesn't choose its tools — the harness presents them.
System prompt structure steers behavior. In April 2026, Anthropic's own postmortem revealed that a single line in Claude Code's system prompt — keeping text between tool calls to 25 words — caused a 3% coding quality drop. The API was unaffected, and the change was reverted in mid-April. A single line. The harness is the system prompt, and the system prompt is load-bearing. Lulla et al. at Princeton found that AGENTS.md presence alone — the rules file — produced 28.6% faster task completion and 16.6% fewer output tokens. ETH Zurich found that human-written AGENTS.md improved performance by 4%, while LLM-generated files degraded it by 0.5% to 20%. The quality of your harness declaration matters more than its presence.
Context management is the biggest lever. Lewis's paper showed that compressing old tool outputs and detecting stuck turns produced up to a 21 percentage point improvement in fail-to-pass fraction — without touching the model. At tight context windows (20K tokens), the effect is dramatic. At wide windows (262K), it still saves 7% of tokens. The harness decides what the model remembers at each step. Most developers never touch this setting.
Permission model affects speed and risk. Ask-first is safer but slower. Auto-approve is faster but riskier. The choice changes how the agent handles multi-step tasks — and it's entirely a harness decision. Neither is wrong, but the tradeoff matters.
Recovery logic determines failure modes. Vats and Golev's harness-specific fingerprints: Goose fails by reasoning in circles (REASON-dominated). OpenHands fails by over-verifying (VERIFY + MAX_TURNS). OpenCode fails by idling (TIME + no-action turns). The same model produces different pathologies because the harness determines what happens when the first approach doesn't work. ProcCtrlBench documented "Dead Step" — a step that never completes — and "Duplicate Step" — the same action repeated — as harness-level failures, not model failures, with a fragile success rate of 10.8-20.2% across systems.
Two supporting papers round out the count. Sghaier et al. — "Don't Blame the LLM" — traced performance regressions to harness evolution, not model updates. And the Agentic Harness Engineering team found automated harness optimization reached 77% resolution, against 47-72% for human-tuned configurations.
The gap between research and your AGENTS.md
Daniel Vaughan coined "the harness effect" in April 2026, focused on Codex CLI. Eight academic papers have since explored it. But that conversation lives in research papers and tool docs — not the file your agent reads every session. The config principle — "the agent is only as good as what's declared to it" — gets quoted a lot and demonstrated rarely. Here's the demonstration.
Your AGENTS.md is harness engineering
Your AGENTS.md isn't documentation. It's a harness declaration. It tells the system prompt what to include, which shapes how the model behaves, which determines which tools it selects and how it recovers from errors. Every section of your AGENTS.md is a harness variable.
Be precise about the two layers this evidence separates. The biggest swings — Lewis's 21-percentage-point context gain, CLAW-SWE-Bench's 12-point tool-interface spread, Anthropic's 3% from one system-prompt line — come from how the tool is architected: context management, tool interface, stopping policy. You mostly choose those once, when you pick the harness. AGENTS.md is the layer you can change every session — the declaration that steers the architecture you chose. It's the smaller lever, and it's still worth real gains: 28.6% faster task completion in Princeton's study, 4% better performance in ETH Zurich's.
Here's what that looks like in practice. If your AGENTS.md says "always run the full test suite before committing," the harness includes that instruction in the system prompt. The model sees it as a constraint and follows it. If you remove that line — or copy the file to a new tool that doesn't read it — the model has no instruction to run tests. Same model. Different harness configuration. Different behavior.
Your skills are harness declarations too. They define available tools and how the model is supposed to use them. When you port a skill from one harness to another and it silently degrades — that's the harness effect in action. Your MCP configuration is a harness declaration. The tools available to the model are part of the harness. Changing MCPs changes the harness.
The migration diary lesson — re-declare, don't copy — is a harness effect lesson. When you copy your CLAUDE.md to a new tool and expect identical results, you're ignoring the harness variable. The file content matters. The harness that reads it matters more.
I saw this firsthand during my migration from Claude Code to OpenCode. The rules were the same. The model was the same. The behavior was different — because the harness changed.
Port the knowledge. Rebuild the harness.
The harness audit
If you're about to spend $200/month on a different model, spend 30 minutes on this first.
0. Pick the harness with the right defaults first. Context management, tool interface, and recovery logic are architecture — most of the evidence above says that's where the points hide. If the tool you chose scores the same model far behind in every benchmark, no rules file closes that gap. Choose for the defaults, then tune what you can.
1. What tools does the harness present to the model? Check the tool definitions. Are they specific or generic? Do they match what you actually need? The CLAW-SWE-Bench paper showed that tool interface changes alone produce 12+ percentage point swings. Quick test: ask your agent "what can you do?" and compare the answer against the tools you know exist. A generic answer means the descriptions are underspecified.
2. How are your rules loaded? Open your coding agent's AGENTS.md or equivalent. Is it specific? Does it tell the model what to do, or just what not to do? ETH Zurich found that the quality of rules matters more than their presence. A three-line instruction that says "run tests before committing, check for type errors, never modify .env files" outperforms a 200-line rules file that says "be careful." Quick test: cut the file to three concrete do-rules and rerun a task you do often. If behavior improves, the old file was noise, not guidance.
3. What's the context management strategy? Does the harness compress old tool outputs? Does it detect stuck turns? Lewis's paper showed this single variable can produce a 21 percentage point improvement. If your harness doesn't compress context, you're leaving performance on the table. Quick test: run a five-step task, then scroll the transcript. If early tool outputs still sit there in full, there's no compression policy.
4. What's the permission model? Auto-approve is faster but riskier. Ask-first is safer but slower. Neither is wrong — but the choice affects how your agent handles multi-step tasks. Quick test: run the same multi-step task once with ask-first and once with auto-approve. If the agent spends more time waiting than working, the tradeoff just became visible.
5. How does it recover from errors? Run a task that requires multiple steps. When the first approach fails, what does the agent do? If it retries the same command three times, your harness has a stuck-turn detection gap.
The decision framework: (each row tested against your own runs, not a benchmark)
- Agent reasons incorrectly → model issue — the output is plausibly wrong even with the right files and clear rules
- Agent selects wrong tools → harness issue — it re-greps a file it already saw because the tool descriptions are vague
- Agent loses context mid-task → harness issue — five steps in, it re-explores files it already mapped
- Agent can't solve the problem at all → model issue — with perfect context and correct tools, it still produces broken code
Most problems developers blame on the model are harness problems. Vats and Golev's data makes the economics concrete: if a Goose harness costs $100/month in tokens for a given workload, the same model in an OpenCode harness costs $4,100/month for the same tasks. That's not a model problem. That's a harness problem with a dollar sign. The math is illustrative — raw list token prices from a 50-task study, no caching. Real bills would be lower with prompt caching; the point is the multiplier, not the price tag.
Conclusion
The evidence says one thing: the agent is only as good as what's declared to it.
Eight papers. Three real-world benchmarks. A 16-percentage-point swing from Cursor to Claude Code. A 21-percentage-point gain from one context management change. A 41× token cost difference from harness choice alone.
Before you switch models, audit your harness. The full playbook for harness declarations — AGENTS.md, rules, skills, and memory — lives here: Why Your Coding Agent Keeps Making the Same Mistakes — AGENTS.md Fixes It.
Stop optimizing the engine. Start optimizing the wrapper.
FAQ
What is the harness effect in AI coding agents?
The phenomenon where the same model produces measurably different results depending on the wrapper it runs in. On Terminal-Bench 2.0, Claude Opus scored 93% inside Cursor, 92.1% inside Claude Code with a community-tuned "Mythos" configuration, and 77% inside default Claude Code.
Does the coding agent matter more than the model?
Both matter — and the harness is the variable most developers ignore. Anthropic's own postmortem showed a single system-prompt line dropping Claude Code's quality by 3%.
How do I audit my coding agent harness?
Run the five checks in the audit section above. Fastest signal: ask your agent "what can you do?" — if the answer is vague, your tool descriptions are underspecified.
What is AGENTS.md and how does it affect agent performance?
A rules file your agent reads as part of its system prompt. Good files help — Princeton measured 28.6% faster task completion, ETH Zurich +4% performance. Bad files hurt: LLM-generated rules degraded performance by 0.5-20%.
Why does the same model perform differently in different tools?
Because each tool implements the five harness variables differently — tool definitions, system prompt, context management, permissions, recovery logic. Same weights, different wiring.
This is the sixth post in my agent-workflow migration series. Previous: The Portability Trap: When 'It Loads' Doesn't Mean 'It Works'. Next: re-declaring MCP connection config when you switch agents.
I write about AI engineering stacks, autonomous developer tools, and structural agent design. If you're building in this space, follow @buildloops for weekly breakdowns!
Top comments (1)
Have you tested the same model in different harnesses? What changed? I'm especially interested in counter-evidence — if your results contradict this, I want to hear it. I'll compile the replies into a follow-up.