We benchmarked Claude Code effort levels on 100 runs. Above medium, nothing improved.

If you run Claude Code all day, you have probably done what we did: Opus, effort on high or above, everywhere. It feels safe. My colleague Ayoub Hammoudi decided to measure what that safety actually costs, and the answer changed how we work.

This post sums up his benchmark and the open-source skill that came out of it, rightsize. Disclosure: Ayoub wrote it, I co-maintain it.

The short answer

On tasks that a test or an oracle can grade, raising the effort level above medium bought nothing measurable, while the bill kept climbing. Routing each step to the right model cut costs by 73% to 98% with the same pass rate.

Benchmark 1: effort levels (100 runs)

• 100 valid headless runs, October 2026, Claude Code 2.1.308
• Haiku, Sonnet and Opus 5.5, 3 runs per cell
• 4 tasks graded by hidden tests or an oracle. The hardest: write a regex engine and match Python's re on 3,000 random patterns

Results:

• Every valid run passed from low up, on all three models.
• Haiku at low built the regex engine, matching re on all 3,000 cases, 3 times out of 3, for $0.02.
• Above medium, cost went up and quality did not:

Cost vs high, same model	low	medium	xhigh	max
Sonnet 5.5	0.4–0.6×	0.5–0.8×	1.4–1.9×	5.0–8.1×
Opus 5.5	0.6–1.1×	0.9–1.0×	1.5–2.2×	3.7–9.3× (4–7× slower)

Benchmark 2: routing (19 runs)

Baseline: Opus at high for everything.

Task	Baseline	rightsize	Cost	Pass
Find a value in a repo, with decoys	$0.118	haiku/low $0.002	−98%	2/2 vs 2/2
Rename across files, add type hints	$0.104	haiku/low $0.004	−96%	2/2 vs 2/2
Fix a parser against hidden tests	$0.130	sonnet/medium $0.035	−73%	2/2 vs 2/2

Scoped testing also matters: 11.4 s for the full suite against 0.7 s for the tests a one-module change actually touches.

What the numbers do not say

These tasks are well specified and verifiable, which is exactly where low effort is safest. Nothing here measures open-ended work (architecture, design, ambiguous bugs), and the samples are small. We read these results as "let a check carry the quality", not "always use the cheapest model".

What rightsize does with that

The rule is quality first, then savings on everything around the deliverable:

1. Score each task on scope, risk, ambiguity and depth.
2. Pick model and effort from that score: a deterministic tool when one exists (formatter, compiler), Haiku for lookups and mechanical edits, Sonnet for standard logic, Opus with high effort for unknown root causes, migrations, security.
3. Let a check carry the quality: a verified step starts one effort level lower and climbs only when the check fails. Work no check can grade (design, review, open-ended tasks) keeps a high effort.
4. Test what the change touches, and run the full suite at the end of the work.
5. Escalate on evidence, never by habit: same check failing twice, a contradiction, a low-confidence report.

It also ships expert playbooks per work type (bug, feature, refactor, research, review, migration, performance, docs), so the hard thinking happens once, up front, on the strong model.

Install

In Claude Code:

/plugin marketplace add spar-x-fr/spar-x-skills
/plugin install spar-x-skills@spar-x-skills

Or copy skills/rightsize/ into ~/.claude/skills/.

MIT licensed. Method, raw numbers and limits: github.com/spar-x-fr/spar-x-skills.

If it routes badly on your workload, open an issue: the failure cases are the most useful data we can get.

---

Zakaria Khchiche is a freelance Tech Lead Data & AI (200+ AI agents in production) and a contributor to Microsoft Agent Framework and Mistral AI. LinkedIn · Website. rightsize was created by Ayoub Hammoudi.