DEV Community

Redreamality
Redreamality

Posted on

What an agent harness actually owns: 8 reads on loops, cost routing, exactly-once, durability and spec authority

Ask five engineers what an "agent harness" is and you'll get five answers. For some it's the while-loop around the model call. For others it's the retry wrapper, the spec checker, the durable store, or the whole repository the agent works in. I spent the last few weeks reading longer write-ups on each of those layers, and the eight below are the ones I keep coming back to. Each gets a short summary in my own words and a link to the full piece.

1. Where the other 1,695 lines go

A textbook ReAct loop fits in five lines of pseudocode. Claude Code's query.ts is roughly 1,700. This walkthrough reads that file and pulls out a five-layer pattern: a compaction pipeline that runs before every API call, tools that start executing while the model is still streaming, recovery paths for overflow and rate limits, and so on. It's a good map if you're writing your own loop and wondering what you'll end up adding anyway.

Read: the agent harness pattern behind Claude Code's core loop

2. The price sheet isn't the bill

A preprint from Accenture's Responsible AI group argues that enterprise coding-agent spend is decided less by per-token rates than by the harness: which model answers, what it reads, and whether switching models throws away the prompt cache. In their emulated 10,000-seat company, cache-safe routing recovers about 14–21% of model spend. The post explains why a single turn with 18 tool steps re-sends the same context 19 times, and what procurement should be asking vendors.

Read: routing and cost governance when you control the harness

3. Who makes sure the charge only happens once?

A write times out. Did it go through? Retrying blindly means a second charge or a second deploy; giving up skips work. The Limbo sandbox paper ran 25,930 graded episodes across nine models, three production harnesses and two contract variants to ask where exactly-once should live. The short answer: it depends on whether an immediate read-back can show what happened. When it can't, no model or wrapper saves you, and the fix belongs in the tool's API (idempotency keys, status queries).

Read: exactly-once between model, harness and tool contract

4. The harness is moving into the environment

This one connects three very different stories: Anthropic replacing git with an in-house system after daily merges jumped from about 2,500 to over 30,000, a game developer turning his game into something an agent can drive, and OpenAI's Codex-only repository. The common thread is that once code is cheap, the bottleneck is whether the environment is queryable, reproducible, measurable and enforceable. My favorite line of advice in it: almost nobody needs custom version control, but almost everybody benefits from headless tests that print text and numbers.

Read: why environment legibility is the new harness work

5. When nobody is watching the terminal

Most coding agents assume a human will notice a crash and hit "continue". Pi Durable, shipped with Pi 1.0, targets Slack on-call bots and shared incident workspaces where sessions must survive deploys and dropped clients. The article lines it up against Temporal, Durable Objects and LangGraph checkpointers and asks a sharp question of each: when the process dies, do you lose progress, or can a write fire twice?

Read: what durable agent harnesses actually guarantee

6. Let the spec sign off, not the agent

In most setups the SKILL.md, the schema and the task instructions just become more context for the same model that later declares "done". SpecHarness compiles those specs into source-linked obligations and a versioned ledger, so the agent proposes and the runtime commits. The write-up separates two gaps worth naming in your own reviews: understanding a requirement is not executing it, and claiming completion is not proving the required state exists.

Read: SpecHarness and who holds the pen

7. Training the planner and the harness together

Mobile planning agents are expensive to develop because real-device runs are slow and hard to parallelize. Alibaba's Qwen-Planner-Agent paper describes a closed loop: a human-gated data flywheel, online RL with a competence-aware reward scheme, and runtime memory, skills and tools that get revised from failure traces. The 27B model tops MobilePA-Bench at 77.05 overall. The post is careful to note that the authors frame co-evolution as a pathway, not something already automatic.

Read: planner–harness co-evolution in Qwen-Planner-Agent

8. A smarter harness, or a better cheater?

If you let an optimizer keep rewriting prompts, memory and control code until the score rises, it may learn the benchmark's habits rather than the task: filename patterns, the order of retrieval channels, where unit notes usually sit. The CHASE paper adds a Challenger that searches for meaning-preserving protocol changes that wipe out the gain. Task holdout stops item memorization; it doesn't stop a harness from eating the protocol.

Read: Bad Genius and counterfactual checks on harness evolution


If you only have time for two, I'd start with #3 and #8. They're the ones most likely to change what you test before shipping an agent. Earlier lists in this series covered agent memory and context and agent skills.

Top comments (0)