Gemini 4 Carbon is the new #1 on the Artificial Analysis Intelligence Index v5 with 59 points. Claude Opus 5.5 has 58. On coding, which is what most of us actually pay these models for, Opus still wins: 62 to 59 on the Coding Agent Index and 56% to 51% on FrontierCode. And you can't call Carbon on the public API yet.
The headline is true. It also tells you almost nothing if you're picking a model for your editor or your agent loop. I went through the published numbers for Gemini 4 Carbon vs Claude Opus 5.5 vs GPT-6.1 (per-benchmark scores, price per million tokens, cost per task, speed, context window) to see where each one wins. I didn't run anything myself. Every number below comes from Artificial Analysis or the vendors' pricing pages.
Gemini 4 Carbon vs Opus 5.5 vs GPT-6.1 Sol: the full scoreboard
First, "GPT-6.1" isn't one model. OpenAI has GPT-6.1 Sol (released September 29) and the cheaper GPT-6.1 Terra. Sol scores higher on the Index (55), so that's what I'm comparing. OpenAI's highest scorer is still the older GPT-6 Astra at 57.
| Gemini 4 Carbon (high) | Claude Opus 5.5 (max) | GPT-6.1 Sol (max) | |
|---|---|---|---|
| Intelligence Index v5 | 59 | 58 | 55 |
| Coding Agent Index | 59 (Antigravity CLI) | 62 (Claude Code) | n/a |
| FrontierCode v1.1 | 51% | 56% | n/a |
| AA-Omniscience v1.1 | 54 | 48 | n/a |
| Hallucination rate | 11% | 50% | n/a |
| Price in/out per 1M tokens | $4 / $20 | $4 / $20 | $2 / $10 |
| Cost per Index task | $7.76 | $10.04 | $1.27 |
| Output speed | 82.5 t/s | 96.6 t/s | 81.1 t/s |
| Context window | 1M | 1M | 1M |
| Can you use it today? | selected users only | yes | yes |
Sources: the Carbon launch write-up, the Index v5 announcement, and the AA model pages for Carbon, Opus 5.5 and GPT-6.1 Sol. The n/a cells are Sol numbers I couldn't find in any of them.
Look at the list prices first. Carbon and Opus 5.5 are both $4 input and $20 output per million tokens, and both get a 95% cache discount down to $0.20. Calling Carbon "cheaper" only works for two reasons. One is a 50% launch discount that Google hasn't put an end date on. The other is token count.
Where Gemini 4 Carbon wins: fewer hallucinations
On AA-Omniscience v1.1 Carbon scores 54 to Opus 5.5's 48. Its hallucination rate is 11%. Opus 5.5 (max) and GPT-6 Astra (max) both sit at 50%. If you're building something that answers questions from its own knowledge (support bots, research assistants, anything where a confident wrong answer costs you), that gap is far bigger than the one-point gap on the Index.
One caveat: that 11% is up from 4% on Gemini 4 Argon. Google traded some caution for accuracy. And the Hacker News thread has people saying Argon made things up anyway: invented paragraphs in project summaries, a fake espresso machine. A benchmark hallucination rate and your production hallucination rate aren't the same number.
Carbon also uses fewer tokens. AA measured 123k output tokens per Index task, against 173k for Opus 5.5 at max effort, which is 29% fewer. That's where the ~23% cost-per-task advantage ($7.76 vs $10.04) comes from. The per-token price is identical, so the saving is all in how much each model writes.
It's #1 on 4 of the 9 Index evaluations: HLE-Diamond, Terminal-Bench-Science, AA-Omniscience and AutomationBench-AA. Three of those sit in the Science and General categories; AutomationBench-AA is the only Knowledge Work one.
Coding: Opus 5.5 is still ahead
One of the key results in AA's own write-up is literally titled "Behind only Claude Opus 5.5 on coding agents." The details:
- Coding Agent Index: Carbon in Antigravity CLI scores 59. Claude Code with Opus 5.5 (max) scores 62.
- FrontierCode v1.1: Carbon 51%, Opus 5.5 56%.
- Terminal-Bench 4.0: Carbon 64%, which AA says is second place.
Three points on a composite isn't a blowout. But the Coding Agent Index scores the model inside its agent harness, which is closer to how you'll actually use it, and Opus is ahead there. One HN commenter put it bluntly: "Carbon scoring below Opus 5.5 on coding highlights how magnificent Opus 5.5 is as a coding agent."
So Carbon took the overall #1 by winning science and knowledge, while Opus is ahead on the Coding Agent Index and FrontierCode. In Index v5 the categories are weighted equally: Knowledge Work 25%, Coding 25%, Science 25%, General 25%. Terminal-Bench-Science alone is 15% of the total. If you never ask a model to do lab-style science in a terminal, 15% of that "59" isn't measuring anything you'll use.
GPT-6.1 Sol is the one that should worry Google
Everyone's arguing about first and second place, and meanwhile GPT-6.1 Sol scores 55 for $1.27 per Index task. That's about 6x cheaper than Carbon and about 8x cheaper than Opus 5.5 at max effort, for a model 3-4 points behind.
On CritPt v2 (hard subset), Sol scores 69%. Opus 5.5 scores 61%. On Terminal-Bench-Science, Sol gets 58% and Opus gets 59%, which is close to a tie.
There's a catch if you work with big contexts. OpenAI's pricing page charges more above 272K input tokens. Sol goes from $2/$10 to $4/$15, and Terra goes from $1/$5 to $2/$7.50. Anthropic bills Opus 5.5 at standard rates for the whole 1M window. A 900k-token request costs the same per token as a 9k one.
Long context: the Index stopped measuring it
All three advertise roughly 1M tokens. What bothers me is that Index v5 removed AA-LCR, the long-context reasoning benchmark, along with GDPval-AA and SciCode. So the #1 ranking contains zero direct evidence about how any of these models reasons over 500k tokens of your codebase.
What you can compare is cost. Opus 5.5 has flat pricing across the full window. Sol charges double for input above 272K. For Carbon, AA lists only the $4/$20 headline, the cache discount and the launch discount, and you can't call it publicly yet anyway.
Speed matters for long-context work too. AA lists Opus 5.5 at 96.6 tokens/s, Carbon at 82.5 and Sol at 81.1. Time to first token is ugly across the board at these reasoning settings: 103.54s for Carbon, 277.18s for Sol, and 644.22s for Opus 5.5 at max. Nobody is running max effort interactively.
Which model I'd pick for coding, pipelines and Q&A
- Coding agents, refactors, repo-wide changes: Claude Opus 5.5 in Claude Code. It leads the Coding Agent Index (62 vs 59) and FrontierCode, and you can use it today.
- High-volume pipelines, CI bots, eval loops: GPT-6.1 Sol. 55 points for $1.27 a task. Keep prompts under 272K or the price jumps.
- Factual Q&A, research, anything where hallucinations hurt: Gemini 4 Carbon, once you can actually get access. An 11% vs 50% hallucination rate is the biggest real gap in this whole comparison.
- Long context on a budget: Opus 5.5 for predictable billing up to 1M, but test it on your own data, because the index no longer measures this.
Before you switch anything, check two settings.
- Effort. Opus 5.5 scores 35 at low effort and 58 at max. At low effort it costs $0.98 per task. GPT-6.1 Sol's low-to-max spread is 20 points. The "same model" can be a different product depending on one parameter, so compare at the effort level you actually run.
- Whether you're caching. Opus 5.5 and Carbon both discount cached input by 95%, down to $0.20 per 1M tokens. If your agent re-sends the same repo context every turn and you aren't caching, you're paying for it again on every turn.
About that #1
The thread's other big argument is about trust. AA now says 60% of the Index comes from private test sets or held-out answers. One camp calls that unverifiable. The other says it's the only defense against benchmaxxing. I'm in the second camp, but private tests also mean you can't check how much the weighting decides the ranking. A one-point lead under a reweighting that happened in the same week as the launch deserves some skepticism.
My take: Carbon earned the headline by being the most careful model on knowledge questions. It hasn't earned your coding workflow, and you can't even buy it on the API yet.
Would a #1 ranking get you to switch away from your current model, or does your coding workflow keep you where you are? If you've had early Carbon access, tell me whether that 11% hallucination number matches what you're seeing.
Top comments (3)
In multi-turn coding loops with repo context, the real cost driver is the cache invalidation rate. When an agent edits files across sequential turns, breaking the prefix cache on a 400k context turns a ninety-five percent discount back into full retail rates for every subsequent tool step. If cache hit rates slip below eighty percent from frequent system prompt churn or shifting tool schemas, the effective price per completed feature converges right back toward headline token tariffs.
The illusion of competence in agent-assisted coding usually stems from relying on recognition rather than retrieval. Reading a clean diff feels identical to understanding the design until an unhandled edge case appears in production. Forcing an active recall checkpoint before marking a milestone complete re-inserts the friction where failure modes and boundary conditions actually get internalized. Separating the architectural decision log from the repository itself also keeps the record from turning into post-hoc justification for whatever the model generated.
The trickiest ASR failure mode on newly coined jargon is acoustic substitution into older dictionary priors. When a term has zero unigram weight in the language model, speech decoders often force it into common homophones like 'new cloud' or 'near cloud' before the text token reaches your regex. Slicing by speaker role is the cleanest part of this dataset. The migration of terminology from vendor presentations into debt covenants and restructuring discussions on bank calls usually tracks the turning point of the capex cycle.