Claude Code 2.1.318 added a new knob: subagent effort (CLAUDE_CODE_SUBAGENT_EFFORT). The pitch is easy to believe: hand the searching to an Explore subagent, give it more effort, get a better answer.
So I measured it. 288 runs later: the answers were the same, the bill was not.
- Found: 97% to 100% of the right answers in every condition. No condition moved it by as much as 2 points.
- Cost per run: Haiku 5.5 +74% to +116%. Opus 5.5 +143% to +221%.
- Time per run: roughly 2x to 3x.
- Subagent model: always the same as the main agent. Opus calls an Opus subagent, so you pay Opus prices twice.
The setup
12 "find every place" questions in 6 open-source repos (undici, fastify, cachetools, Python-Markdown, anyio, zod), at fixed commits. Things like "find every place in lib/ where a RequestAbortedError is created, with the enclosing function."
The answer key came from a regex over the source plus a parser for the enclosing function or class: 151 items. Claude Code 2.1.318, headless, read-only tools, Haiku 5.5 and Opus 5.5, 3 runs each. 12 × 2 × 4 × 3 = 288.
| Condition | What changed |
|---|---|
| No instruction | The question only |
| Subagents, effort low / medium / high | + "Use subagents (the Explore agent) for the searching, in parallel where it helps, and write the final answer yourself." and CLAUDE_CODE_SUBAGENT_EFFORT set to low, medium or high |
I pre-registered the questions, the answer key (sha256) and a 15-point threshold on GitHub before the first run.
Results
| Condition | Haiku found | Haiku cost/run | Opus found | Opus cost/run |
|---|---|---|---|---|
| No instruction | 98% | $0.0056 | 100% | $0.16 |
| Subagents, low | 97% | $0.0098 (+74%) | 99% | $0.39 (+143%) |
| Subagents, medium | 98% | $0.011 (+87%) | 99% | $0.51 (+221%) |
| Subagents, high | 99% | $0.012 (+116%) | 100% | $0.48 (+204%) |
Higher subagent effort didn't buy better searches either. For Opus, medium cost more than high.
And with no instruction, Claude Code never reached for a subagent on its own: 0 of 72 runs here, 0 of 504 in my previous dataset.
The surprise: headless runs answer late
In 2.1.318 the Explore agent runs in the background. In headless mode (claude -p) that means one run can emit several result events:
- "The search agent is running in the background. I'll write up the list once its results come back."
- The actual answer.
- Sometimes: "Nothing to act on: the lookup is finished [...] so I've stopped the autonomous loop."
In 210 of 216 subagent runs, the first result held no answer. A script that reads only the first one gets 0% to 9% of the answers. A script that reads only the last one can also miss, because the last one is sometimes the "stopped the loop" note. 4 Haiku runs gave the answer and then never exited until my 20-minute timeout.
My own scorer read the last result at first, which is how I found this. I changed it to join all result events and logged the change in the pre-registration. Both readings are in the data.
If you call Claude Code from CI or a script and ask it to use subagents, read every result event, not just one.
What to do
- Don't add "use subagents" for ordinary code search. Here it only added cost and time.
- If you do use them on Opus, know the subagent is Opus too.
- Leave subagent effort at the default unless you've measured a gain on your own tasks.
Limits
Both models were near the ceiling with no instruction, so there was little room to gain. Mid-sized repos; a huge monorepo might be different. One prompt wording, one Claude Code version, 3 runs per cell.
Data
All 288 runs (recall under three readings, cost, time, subagent calls and models, and every answer text) are public under CC BY 4.0:
- Data page: https://tokensave.app/data/claude-subagent-effort
- GitHub (pre-registration, answer key, scorer): https://github.com/johnisking/token-cost-by-language/tree/main/claude-subagent-effort
Have you seen subagents actually help on a search? Tell me the task and I'll test it next round.
Top comments (0)