DEV Community

Cover image for Claude Fable 5.1 Benchmark Results: The Gains Are Mostly in Execution
Ryan Cole
Ryan Cole

Posted on Originally published at cometapi.com

Claude Fable 5.1 Benchmark Results: The Gains Are Mostly in Execution

Claude Fable 5.1 looks less like a general reasoning leap and more like a substantial upgrade for agents that have to keep working.

Its biggest reported improvements are in scientific research, business automation, terminal coding, and other tasks that require planning, tool calls, verification, and recovery over multiple steps. The gains are much smaller on closed-ended reasoning and shorter IDE coding tasks.

That distinction matters more than any single leaderboard number.

The short version

  • Fable 5.1 scored 52.6% on Terminal-Bench-Science 0.1, compared with 24.7% for Fable 5.
  • AutomationBench improved from 17.1% to 31.4%, a large increase in end-to-end business workflow completion.
  • Terminal-Bench 4.0 rose from 42.0% to 55.8%.
  • CursorBench increased only from 70.5% to 73.4%, and tool-assisted Humanity’s Last Exam from 63.8% to 65.0%.
  • Standard input and output pricing remains unchanged from Fable 5. Cache reads are cheaper at $0.25 per million tokens, which can reduce the cost of context-heavy agents.
  • GPT-6 Astra is the newer OpenAI comparison, but it was not part of Anthropic’s September 1 benchmark table. A direct ranking requires a controlled evaluation using the same harness.

Anthropic released Claude Fable 5.1 on September 1, 2026, positioning it for demanding reasoning, long-horizon agents, software engineering, research, and professional knowledge work.

My read is straightforward: this is primarily an agentic-execution release. Fable 5.1 appears most useful when the model must preserve a goal, operate an environment, detect mistakes, and eventually verify a completed state.

Model specification and pricing

Anthropic’s documentation lists a 1-million-token context window, 128K maximum output, text and image input, always-on adaptive thinking, and a June 2026 knowledge cutoff. The Claude API model ID is claude-fable-5-1.

Specification Claude Fable 5.1
Provider Anthropic
Release date September 1, 2026
Model ID claude-fable-5-1
Context window 1,000,000 tokens
Maximum output 128,000 tokens
Input / output Text and images → text
Thinking Adaptive, always on
Default effort High
Knowledge cutoff June 2026
Input price $10 / MTok
Output price $50 / MTok
Cache read $0.25 / MTok
Comparative latency Slower
Primary positioning Demanding reasoning and long-horizon agentic work

The regular input and output rates are the same as Fable 5. The meaningful pricing change is cache reads, which fell by 75% from $1 to $0.25 per million tokens.

Anthropic estimates approximately 25% lower cost for typical workloads and up to approximately 45% lower cost for highly agentic workloads. That estimate makes sense for systems repeatedly sending repository context, tool definitions, instructions, and accumulated state.

The relevant metric for these systems is not headline token price. It is cost per completed or accepted task.

Anthropic also reports 60.9% for Claude Mythos 5.1 on Terminal-Bench 4.0. Mythos 5.1 is separately deployed, has different safeguards, and should not be treated as the generally available Fable 5.1 score.

Anthropic’s reported benchmark table

These figures come from Anthropic’s September 2026 evaluation. They describe a model plus harness configuration, rather than an immutable capability score.

Benchmark What it measures Fable 5.1 Fable 5 Opus 5 GPT-5.6 Sol
Terminal-Bench-Science 0.1 Agentic scientific research 52.6% 24.7% 29.0% 22.4%
Terminal-Bench 4.0 Agentic terminal coding 55.8% 42.0% 52.3% 37.3%
GDPval-AA v2 Professional knowledge work, Elo 1853 1723 1824 1711
OSWorld 2.0, partial Long-horizon computer use 77.9% 72.9% 75.4%
OSWorld 2.0, strict Fully completed computer tasks 41.7% 36.1% 39.6%
Humanity’s Last Exam, no tools Expert multidisciplinary reasoning 60.9% 57.8% 56.6%
Humanity’s Last Exam, with tools Expert reasoning with tools 65.0% 63.8% 63.6%
AutomationBench End-to-end business workflows 31.4% 17.1% 26.9% 19.6%
CursorBench 3.2.0 Agentic IDE coding 73.4% 70.5% 70.0% 67.2%

Fable 5.1 leads Fable 5 and Opus 5 in all nine reported rows. But the distribution of those gains is more informative than the fact that it leads every row.

Where the release actually moves the needle

Scientific research: the clearest generational jump

Terminal-Bench-Science 0.1 is the standout result:

  • Fable 5.1: 52.6%
  • Fable 5: 24.7%
  • Opus 5: 29.0%
  • GPT-5.6 Sol: 22.4%

The 27.9-point improvement over Fable 5 is substantially larger than the reported per-model standard error of approximately ±3.5–4.5 points.

This benchmark evaluates complete scientific and engineering workflows. The agent may need to write code, perform analysis, produce proofs, run simulations, or generate data products. It is not simply answering a sequence of isolated questions.

That makes the result more relevant to research agents than a conventional academic reasoning score.

Terminal coding: better autonomous execution

On Terminal-Bench 4.0, Fable 5.1 scores 55.8%, compared with 42.0% for Fable 5, 52.3% for Opus 5, and 37.3% for GPT-5.6 Sol.

The gain over Fable 5 is 13.8 points. The advantage over Opus 5 is only 3.5 points, so the latter comparison should be treated more cautiously given the reported error range.

Terminal-Bench measures execution in an environment: navigating files, changing code, running commands, and validating outcomes. Because Terminal-Bench 4.0 uses a different task set from earlier releases, scores should only be compared within the same benchmark version.

Business automation: much better, still far from solved

AutomationBench improved from 17.1% to 31.4%, an absolute increase of 14.3 points, or approximately an 84% relative gain.

The tasks span sales, marketing, operations, support, finance, and HR. Scoring checks the final environment state, which is more useful than counting whether the agent proposed individually correct actions.

The limitation is equally clear: approximately two thirds of tasks still did not complete successfully under Anthropic’s reported configuration.

Computer use: partial progress, strict completion remains low

Fable 5.1’s OSWorld 2.0 scores are:

  • 77.9% with partial credit
  • 41.7% under strict completion

For comparison, Fable 5 scores 72.9% and 36.1%, while Opus 5 scores 75.4% and 39.6%.

The strict result is the one I would use for production planning. Fewer than half of tasks were fully completed. That leaves plenty of room for checkpoints, permissions, monitoring, and recovery logic around the model.

IDE coding and closed-ended reasoning: smaller changes

CursorBench 3.2.0 shows a more modest improvement:

  • Fable 5.1: 73.4%
  • Fable 5: 70.5%
  • Opus 5: 70.0%
  • GPT-5.6 Sol: 67.2%

That is just a 2.9-point gain over Fable 5.

Humanity’s Last Exam shows the same pattern. Fable 5.1 improves by 3.1 points without tools and only 1.2 points with tools:

  • No tools: 60.9% versus 57.8%
  • With tools: 65.0% versus 63.8%

The evidence points toward sustained execution as the main improvement, not a universal improvement across every kind of reasoning.

Fable 5.1 compared with the alternatives

Fable 5.1 is the strongest option in Anthropic’s published long-horizon table. Opus 5 remains a reasonable starting point when its smaller performance gap is acceptable. Fable 5 is the previous baseline. GPT-6 Astra needs a same-harness test.

Dimension Claude Fable 5.1 Claude Fable 5 GPT-6 Astra
Context window 1M tokens 1M tokens 1.05M tokens
Maximum output 128K 128K 128K
Standard input / output $10 / $50 per MTok $10 / $50 per MTok $10 / $50 per MTok
Cache read $0.25 / MTok $1 / MTok Verify current provider terms
Terminal-Bench-Science 0.1 52.6% 24.7% Not included in Anthropic’s launch table
Terminal-Bench 4.0 55.8% 42.0% Not included in Anthropic’s launch table
AutomationBench 31.4% 17.1% Not included in Anthropic’s launch table
Primary positioning Demanding reasoning and long-horizon agents Previous Fable-class baseline Difficult end-to-end professional work

GPT-6 Astra was not included in Anthropic’s September 1 evaluation, so there is no valid direct comparison from that table. The safe approach is to run both models against identical prompts, tools, permissions, effort settings, and scoring.

My selection rule would be:

  • Start with Opus 5 when cost is the primary constraint.
  • Use Fable 5.1 when long-horizon completion and recovery justify the premium.
  • Keep Fable 5 for compatibility-sensitive deployments.
  • Test GPT-6 Astra separately rather than inferring its position from unrelated evaluations.

Improvement by workload

Capability Fable 5.1 Change vs Fable 5 Reading
Agentic scientific research 52.6% +27.9 pts Very large gain
Business workflow automation 31.4% +14.3 pts Very large gain
Terminal coding 55.8% +13.8 pts Large gain
Computer use, strict 41.7% +5.6 pts Moderate gain
Computer use, partial 77.9% +5.0 pts Moderate gain
Expert reasoning, no tools 60.9% +3.1 pts Small gain
IDE agentic coding 73.4% +2.9 pts Small gain
Expert reasoning, tools 65.0% +1.2 pts Small gain
Professional knowledge work 1853 Elo +130 Elo Strong, configuration-sensitive

The pattern is consistent across the table: the larger gains appear where the agent has to persist, interact with an environment, use tools, verify state, and recover from intermediate failures.

Limits of the benchmark results

The benchmark numbers are useful for narrowing a model shortlist, but they are not production guarantees.

  • The main comparison table is vendor-run.
  • Effort settings affect quality, latency, and cost.
  • Agent benchmarks measure the model, harness, tools, and permissions together.
  • Production safeguards were enabled for Fable 5.1 and affected some outcomes.
  • Benchmark versions change, so results from different task releases may not be comparable.
  • AutomationBench remains at 31.4%.
  • OSWorld strict completion remains at 41.7%.
  • Several workloads therefore retain substantial failure rates.

A practical process is to use the public scores for initial selection, reproduce a smaller comparison with identical settings, and then measure the real production workflow.

Cost and model choice

Anthropic prices Fable 5.1 at $10 per million input tokens and $50 per million output tokens, compared with $5 and $25 for Opus 5.

That premium is rational only if the stronger model delivers a measurable operational benefit: higher task acceptance, fewer retries, fewer tokens per successful result, or less human review.

The lower cache-read price can narrow the effective gap for long-running agents, but I would still measure cost per accepted task rather than relying on token pricing.

For teams that need to compare several models through one endpoint, CometAPI can be useful when the evaluation harness needs to remain constant while the underlying model changes.

How I would evaluate it

The test set should look like the workload being deployed.

  • Coding: Run complete issues and record patch acceptance, tests passed, retries, tool calls, elapsed time, and human correction time.
  • Research: Score source quality, factual accuracy, evidence coverage, subtask completion, and goal retention over long runs.
  • Business agents: Evaluate the final environment state rather than counting correct individual actions.
  • Computer use: Include pop-ups, slow pages, interrupted sessions, and inconsistent application state.
  • Model comparisons: Keep prompts, tools, harnesses, permissions, effort settings, and scoring rules constant.
  • Economics: Track cost per accepted task, not only cost per token.

Bottom line

Fable 5.1 looks most valuable for work that requires sustained execution: scientific research, autonomous terminal coding, complex business automation, and workflows with repeated verification and recovery.

It is not automatically the best choice for every task. CursorBench and Humanity’s Last Exam show smaller gains, OSWorld strict completion remains below half, and GPT-6 Astra was absent from Anthropic’s launch evaluation.

The right deployment strategy is to test Fable 5.1 where improved long-horizon success can justify premium inference, then compare it with Opus 5, GPT-5.6 Sol, and GPT-6 Astra on representative tasks under the same conditions.

FAQ

What is Fable 5.1’s highest reported benchmark score?

Among Anthropic’s percentage-based results, it reaches 77.9% with partial credit on OSWorld 2.0 and 73.4% on CursorBench 3.2.0. Its largest improvement over Fable 5 is on Terminal-Bench-Science: 52.6% versus 24.7%.

How much better is it than Fable 5?

The improvement depends heavily on the workload:

  • Terminal-Bench-Science: +27.9 points
  • AutomationBench: +14.3 points
  • Terminal-Bench 4.0: +13.8 points
  • CursorBench: +2.9 points
  • Tool-assisted Humanity’s Last Exam: +1.2 points

Is Fable 5.1 better than Opus 5?

It scores higher across Anthropic’s reported table, but several differences are small. Opus 5 remains the more economical starting point when the additional long-horizon capability is not required.

Does it beat GPT-5.6 Sol?

Anthropic reports higher Fable 5.1 scores on five shared metrics. Since the evaluations are vendor-run and configurations differ, the defensible conclusion is that Fable 5.1 is highly competitive, not universally superior.

Are the results independently verified?

Only partially. Some benchmarks have public leaderboards, but effort, harness behavior, fallback behavior, and task versions must align before scores can be compared directly with Anthropic’s launch run.

What is Fable 5.1 best at?

Its benchmark profile is strongest for long-running scientific research, autonomous coding, complex agent workflows, business automation, and tasks requiring planning, tools, verification, and recovery.

How do I access it?

The API model ID is claude-fable-5-1. Availability and provider terms should be checked before deploying, especially when comparing cache pricing and latency across routes.


Originally published at cometapi.com

Top comments (0)