LangChain Says Agent = Model + Harness. Here's What Broke in My Agent When I Skipped the Harness.

The model swap that didn't help

I had a coding agent that was fine on small tasks and mediocre on long ones. My first instinct was the one everyone has: swap the model. Cheaper model to a stronger one. A few days later the agent felt about the same. Maybe a hair better. Not the leap I was expecting for the extra tokens I was now paying for.

What I hadn't touched was everything around the model. The loop that called it. How tool results got fed back in. What happened when a tool returned 40KB of garbage. What happened after 30 turns when the context started to slide. All of that was the same code I'd written in a weekend.

Then LangChain's Anatomy of an Agent Harness post landed with a claim that sat uncomfortably close to what I was ignoring.

The line that reframed it

LangChain's framing is short:

Agent = Model + Harness. The model contains the intelligence. The harness makes that intelligence useful.

The number attached to that line is what made me stop scrolling. On Terminal-Bench 2.0, they rebuilt only the harness — same model, no weights touched — and moved from 52.8% to 66.5%. That's a jump from outside the top 30 to rank 5. Not from a bigger model. From a better harness around the same model.

Written the way a bench-press chart is written, that gap is embarrassing. I'd spent a week on model selection. I hadn't spent a day on the harness.

What LangChain means by "the harness"

The word can feel fuzzy the first time you hear it. LangChain splits it into concrete pieces. Roughly:

• The loop. How the model, tools, and observations chain together turn after turn. Who decides when to stop.
• Tools. What the agent can actually reach. Their schemas. How results come back.
• Context management. What survives from turn to turn, what gets summarized, what gets dropped.
• State and memory. Anything that persists across turns or sessions.
• Recovery and guardrails. What happens when a tool times out. When the model produces malformed JSON. When it wanders off task.

If you're not the model, you're the harness. That's the reframe. The model is one component out of six. And the model is the one you probably can't change; the other five are yours.

Why the model swap didn't move the needle

Rereading the list, the answer to my own agent got obvious. The model was never the bottleneck in the runs where the agent lost. What broke was somewhere else on that list.

The three things that had actually been failing, once I looked:

1. The loop had no verification step. The agent produced output, and my loop trusted it. If a tool returned an error, I fed the error text back in and hoped the next turn would fix it. There was no "did this actually work" check. On short tasks this is fine because you'd catch it by eye. On long tasks the errors compound. A bad output at turn 4 poisons the plan at turn 12.

2. Tool results overwhelmed the context. One of my tools returned a full HTTP response. Fifty kilobytes of it. Every turn. The model spent a growing chunk of its context re-reading response headers it already knew about. By turn 20 there was barely room for the actual task. I hadn't set any tool-output budget — no truncation, no summary, no separate scratchpad. The context was doing itself in.

3. There was no recovery. When a tool timed out, the whole run died. There was no retry with backoff, no fallback tool, no "log this and continue." A single flaky endpoint could end a 30-minute run at minute 4. The model wasn't the fragile part. My harness was.

Same model. Same task shape. The reason the runs failed had nothing to do with intelligence and everything to do with the six-component list I'd been treating as one component.

What "just add a harness" looks like next to two peers

The reason the harness framing is useful is that it maps cleanly across the big three frameworks people actually reach for. Different names, similar boxes, different defaults.

Component	LangChain / LangGraph	CrewAI	AutoGen
Loop	Graph with explicit nodes and edges. You draw the flow.	Sequential or hierarchical crews. Less state, less code.	Conversational. Agents talk to each other in turns.
Tools	Typed tool schemas. Structured results.	Tools attached to agents. Simple.	Tools available inside the conversation.
Context management	Checkpointing per node. Explicit state carried between nodes.	Task-level, mostly implicit.	Message history is the state.
Recovery	Retries, human-in-the-loop, durable execution.	Manual mostly. Retry a task.	Manual mostly. Restart the conversation.
Observability	LangSmith. Traces every node.	Basic logging.	Basic logging. Microsoft has moved development attention to the newer Agent Framework.

There is no single right answer here. The point isn't "LangGraph wins." The point is that if you skip the harness entirely and write your own weekend loop, you inherit the defaults of neither. You get whatever accidentally happened when you were prototyping. That was me. That's the anti-pattern.

Independent 2026 comparisons put LangGraph around 62% on complex-task completion, AutoGen around 58%, CrewAI around 54%, and note that the gap on simple tasks is almost nothing — all four sit in the 79–88% range. Which lines up with the model-vs-harness point neatly. When the task is short, the harness barely matters. As the task gets longer, the harness gets louder and louder.

The measurement problem

LangChain didn't just say "the harness matters." They put a number on it: 52.8 → 66.5, same model. That's the part of their post I keep coming back to. If you can't measure the harness, you can't defend improving it. The default failure mode of harness work is that it looks like plumbing to whoever pays your salary.

Some things worth counting, cheapest first:

• Task success rate. Runs that ended in the shape you wanted, divided by total runs. Boring, hard to game.
• Rework rate. Runs where a human had to step in and fix the output before shipping. If this is high and success rate is high, your metric is lying to you.
• Tokens per completed task. Not tokens per turn. Per completed task. This punishes runs that succeed only because they ran 40 turns to get there.
• First-pass gate rate. How often the output cleared the tests, the lint, the schema check on the first attempt without a retry loop.

These are RAG metrics with the noun swapped. Retrieval evaluation figured out this shape years ago — precision, recall, cost per query — and harness evaluation is the same idea over a bigger surface. If you already trust RAG dashboards to steer a search pipeline, you can trust the same shape of dashboard to steer an agent loop.

What I actually changed

Not a rewrite. Three small changes in order of leverage:

1. Added a verification step to the loop. Before the agent declares done, a separate check runs — the tests, the schema, the "did the file actually get written" question. If it fails, the loop feeds the failure back in with the specific reason. This alone killed the "agent said it worked, it didn't" class of bug.
2. Put every tool result through a summarizer with a token budget. Full response goes to a scratchpad file the agent can grep. Only a short summary goes back into the model's context. The context stopped drowning around turn 15.
3. Wrapped every external tool in a small retry + fallback. Timeout, one retry with jitter, then a graceful "skipped, here's why" record the agent can see and route around.

None of that touched the model. I didn't swap providers. I didn't tune a prompt. The success rate on long runs moved more than the model swap had. Not 52.8 → 66.5 numbers, because I don't have Terminal-Bench set up, but the shape was the same: same model, better harness, better outputs.

The one-line version

If your agent feels weak, the model is the easiest thing to blame and the hardest thing to actually be the cause. Before you swap it, check the five things around it that you probably wrote in a weekend. That's where the 13-point jumps live.

If you want the full mental model — how the five components fit together, how to instrument them, and how to make the case for harness work to a boss who wants to hear about model upgrades — I wrote the long version in Harness Engineering: The Real Layer Where Your Agent Wins or Loses.