Senthil Kumar MS

Posted on Nov 2 Originally published at phpscientist.com

Gemini 4 Carbon: Benchmarks, Key Differences and a Decision Guide for Leaders

#enterpriseai #generativeai #cybersecurity #technologystrategy

The short answer

Most organisations cannot adopt Gemini 4 Carbon yet. Google announced it on 28 October 2026, but access is limited to a subset of cyber defenders, with more trusted partners to follow. On Google’s published numbers it leads 15 of 17 benchmarks, so it belongs on your evaluation list. The practical decision today is how to prepare: build on a model you can buy, keep the model swappable, and budget at the $4 and $20 list price.

59 - Artificial Analysis Intelligence Index v5.0, ranked #1
15 - of 17 benchmarks led in Google’s comparison table
96.2% - long-context score (GraphWalks); next non-Gemini model 71.8%
$4 / $20 - per million input and output tokens after the introductory period

Every new frontier model arrives with a table in which it wins. The useful question for a leadership team is narrower: where is the lead big enough to change a decision, how was it measured, and can you buy the thing yet? This guide answers those questions for Gemini 4 Carbon.

**A note on sources. **All model facts and scores here come from Google: its launch announcement, its Gemini model page and its evaluation methodology document. Google published these figures and they should be read as vendor claims, including the scores it lists for competitors. Differences between scores and the recommendations are my own reading of those figures.

What Google says Gemini 4 Carbon is

Google describes Carbon as its strongest model, with state-of-the-art cybersecurity and long-horizon capabilities. It follows Gemini 4 Argon, which is the Gemini model you can use through the API today. Three claims define its positioning:

**Long-running work. **Google says its internal teams have seen Carbon run for days on complex engineering tasks, such as root-causing and patching subtle bugs across large codebases.
**Cyber defence. **Google reports leading scores for finding, exploiting and patching vulnerabilities, and more than 2x uplift over Argon in generating secure code on internal benchmarks.
**More dependable behaviour. **Google reports fewer shortcuts, fewer unsupported claims and stronger resistance to prompt injection than Argon.

Gemini 4 Carbon vs GPT-6 Astra, Claude and Argon: the full table

These are the scores on Google’s model page, as of October 2026. A dash means Google lists no score. The highest score in each row is marked in bold on Google’s page; here, the last column names the leader.

BenchmarkCarbonArgonGPT-6 AstraClaude Opus 5.5Claude Fable 5.1LeaderAA Index v5.05950575849CarbonVals Index v2.270.4%69.2%65.0%70.2%—CarbonSWE-sweep9.6%4.3%9.5%3.6%2.9%CarbonFrontierSWE 261.2%55.0%65.5%62.3%56.3%GPT-6 AstraTerminal-Bench 4.0 (AA)65.2%57.1%59.1%59.6%52.0%CarbonAgents’ Last Exam v1.177.4%69.0%73.1%71.6%—CarbonOSWorld-2.1 (offline subset)81.5%74.7%80.9%87.2%76.8%Opus 5.5AutomationBench (Zapier)53.9%51.3%41.4%42.5%31.4%CarbonVending-Bench 2 (bank balance)$16,484$13,718$15,515$9,235$5,422CarbonChartography72.5%71.6%71.0%66.3%46.2%CarbonBlueprint-Bench 258.6%54.4%49.7%51.2%41.9%CarbonGraphWalks (256k to 1M, BFS F1)96.2%84.2%71.8%66.8%65.0%CarbonRiemannBench80.0%76.0%72.0%69.6%65.6%CarbonTerminal-Bench-Science 0.1 (AA)64.8%52.9%63.3%59.0%43.3%CarbonCWE-bench71.0%68.0%68.0%67.0%58.0%CarbonExploitGym54.8%42.0%42.4%34.5%—CarbonSEC-bench Pro88.2%83.4%85.4%——Carbon

Key differences: where the lead is large, narrow or missing

Counting wins hides the most useful signal. Sorting by the gap to the best non-Gemini model on the same row gives a clearer picture.

Size of leadBenchmarkCarbon vs best non-Gemini modelWhat it means for a buyerLargeGraphWalks+24.4 points over GPT-6 AstraThe clearest gap. Relevant if you work across very large contexts.LargeExploitGym+12.4 points over GPT-6 AstraStrong offensive capability, which explains the restricted rollout.LargeAutomationBench+11.4 points over Opus 5.5Relevant to multi-step business automation.ModerateRiemannBench+8.0 points over GPT-6 AstraRelevant to mathematical research work.ModerateBlueprint-Bench 2+7.4 points over Opus 5.5Relevant to reading plans and diagrams.ModerateTerminal-Bench 4.0+5.6 points over Opus 5.5Relevant to agentic work in the terminal.NarrowAA Index, Vals Index, SWE-sweep+1 point, +0.2 and +0.1Treat as level. Do not switch vendors on these alone.BehindFrontierSWE 2−4.3 points to GPT-6 Astra; also behind Opus 5.5Carbon places third of the five listed models.BehindOSWorld-2.1−5.7 points to Opus 5.5Computer use is not a clean win: Carbon leads Agents’ Last Exam but trails here.

The pattern matters more than any single row. Carbon’s strongest case is long context, cybersecurity and sustained automation. For general capability, the top three are close enough that your own workload should decide. For agentic coding, the results are genuinely split across three benchmarks.

Carbon vs Argon: is the upgrade worth waiting for?

For most teams this is the real comparison, since Argon is the model available now and Google says the two launch at the same introductory price.

MeasureGemini 4 ArgonGemini 4 CarbonAvailabilityStable in the Gemini APISubset of cyber defenders; trusted partners nextIntroductory price per 1M tokens$2 input, $10 output$2 input, $10 outputPrice after the introductory periodSee Google’s pricing page$4 input, $20 outputAA Index v5.05059SWE-sweep4.3%9.6%Terminal-Bench 4.057.1%65.2%GraphWalks, 256k to 1M84.2%96.2%ExploitGym42.0%54.8%Gray Swan indirect prompt injection, attack success0.7%0.0%Generating secure code (internal)BaselineMore than 2x uplift in success rateUnsupported claims (internal)BaselineCut by about 2xContext window1,048,576 tokensNot stated in the announcement

Carbon scores higher than Argon on all 17 benchmarks, though some gaps are small: 0.9 points on Chartography and 1.2 on the Vals Index. If Argon already clears your quality bar, there is little reason to hold a project for Carbon. Argon’s published specifications give you a firm base to design against today.

Safety and reliability claims

Google says Carbon builds on Argon’s safety stack and reports these results:

**Cheating: **successful cheating on Terminal-Bench 4.0 fell to 0.0%, and attempts fell to 0.4%.
**Misalignment: **high-severity alerts dropped by almost 80% in live coding environments with internal teams.
**Staying in scope: **overreach fell by 26% and under-service by 29%.
**Prompt injection: **Carbon resisted every evaluated attempt on the Gray Swan indirect prompt injection benchmark, with a 0.0% attack success rate.

A 0.0% score means no evaluated attack succeeded on that benchmark. It does not mean an agent is safe to run unsupervised on your systems, so keep your own permissions, sandboxing and approval steps. Our AI governance framework covers the controls to keep in place whichever model you choose.

What the cost looks like

Google states the introductory price as $2 per million input tokens and $10 per million output tokens, with cached input at 95% off the input price. After the introductory period, $4 and $20 apply. Google has not said when that period ends for Carbon. Here is an illustration for a workload of 100 million input tokens and 20 million output tokens a month:

ScenarioIntroductoryList priceNo caching$400 a month$800 a month60% of input tokens cached$286 a month$572 a month*

*Assumes the 95% cache discount continues at list price. Google states the discount alongside the introductory price and does not confirm it beyond that. Token prices are also only part of the cost: a model that runs for days can consume far more tokens than one that answers a single prompt, so measure cost per completed task. Our guide to evaluating AI ROI shows how to build that business case.

How to read Google’s numbers

Google’s methodology document is worth reading before anyone quotes this table in a board paper. Five details affect how far you can lean on it:

**Sources differ by row. **Some scores come from third parties such as Artificial Analysis, Vals AI, Zapier, Andon Labs and Surge. Others are self-computed by Google, including SWE-sweep, Agents’ Last Exam, OSWorld, ExploitGym and SEC-bench Pro for Carbon.
**Competitor scores are often self-reported. **Unless noted, non-Gemini results come from the providers’ own numbers, using maximum reasoning settings or the best available results.
**Setups are not always identical. **Carbon’s SWE-sweep score uses a mini-swe-agent harness, while rivals’ scores come from the public leaderboard. That makes a 0.1-point lead especially weak evidence.
**Some scores are best-case. **Carbon and Argon ran on the highest thinking settings unless noted, and OSWorld scores are the maximum over three runs. Expect higher cost and latency from those settings.
**Cyber scores were run without guardrails. **Google says the external cyber benchmarks were evaluated without guardrails, and ExploitGym with mitigations off. Since Google is still iterating on guardrails, the released product may not behave identically.

Two headline claims also cannot be checked from what is published. The Chrome team’s “more than 10X better” patch result is against “another lab’s large frontier model”, which Google does not name, and the 2x secure code uplift is from internal benchmarks.

What Google has not published yet

The announcement and evaluation document leave open several items that a procurement or architecture review would normally require:

A general availability date. Google says only “as soon as possible”.
Carbon’s context window and output limits.
When the introductory price ends.
Latency, throughput and rate limits.
The final guardrails, including what security work they may block for ordinary customers.

Should you adopt Gemini 4 Carbon? A decision guide

Your situationRecommended decision nowYou run security or vulnerability workAsk your Google contact about the Fairwind Program. This is the group Google is prioritising.You have a live project on ArgonStay on Argon. Add an evaluation set so you can test Carbon on the day it opens.You are starting a new AI projectBuild on a model available today, and keep the model behind a thin interface so you can swap it.You work across very large documents or codebasesPut Carbon first on your test list. GraphWalks is its largest published lead.You rely on agentic coding or computer useRun a head-to-head trial. Google’s own table shows other models leading FrontierSWE 2 and OSWorld-2.1.You are in a regulated sectorWait for final guardrails and contractual terms before committing.

Whichever row fits, moving from a promising demo to dependable operation is usually harder than picking the model. Our guide on moving AI pilots into production covers that step.

Key takeaways

What would a benchmark win have to change in our cost, quality or risk to justify switching models?
Do we have an evaluation set built from our own tasks, ready to run when Carbon opens?
Can we swap the model behind our applications without rewriting them?
Does the business case still work at $4 and $20, measured per completed task?
If an agent runs for days, who sets its permissions and budget, and who reviews its output?
Which missing facts, such as availability date, limits and guardrails, would block our approval?

AI consulting

Choosing a model for a real workload?

We can build an evaluation set from your own tasks, compare the models you can buy today, and design your application so that you can change models later.

Book a free 30-minute call

Sources

Originally published at phpscientist.com.

DEV Community