A security agent can produce a convincing explanation and a plausible patch while leaving the underlying vulnerability intact. For developers building automated remediation workflows, the useful deliverable is a change that blocks the exploit, preserves expected behavior, and gives a reviewer enough evidence to approve it.
Google’s Gemini 4 Carbon announcement makes that distinction particularly relevant. Announced on October 28, 2026, Carbon targets cybersecurity and sustained engineering work. Google says initial access will go to selected cyber defenders, followed by additional trusted partners through its Fairwind Program, with broader availability planned. Google’s announcement therefore gives developers a reason to investigate the model while keeping integration plans tied to actual access.
My view: Carbon’s most actionable signal is the possibility of carrying a security investigation through to a verifiable repair. That puts the quality of the surrounding test environment at the center of any adoption decision.
Read the security scores as separate capabilities
Google’s performance table reports 88.2% on SEC-bench Pro, 54.8% on ExploitGym, and 71.0% on CWE-bench. These represent different parts of security work: finding vulnerabilities, validating exploitability, and repairing code. Combining them into a single claim about autonomous security would obscure where a workflow can fail.
A model might identify a suspicious code path without demonstrating that an attacker can reach it. It might reproduce a vulnerability but propose a repair that breaks legitimate requests. Each transition needs its own evidence.
The SEC-bench Pro project illustrates how much the execution environment matters. Its tasks include vulnerability discovery and proof-of-concept generation, with containers, metadata, prompts, timeouts, and collected artifacts. This is an evaluation of an agent operating inside a defined setup. A production integration needs similarly explicit boundaries before its results become comparable.
Google also states that its reported external cybersecurity evaluations ran without guardrails. That condition matters when estimating what an accessible deployment will do under its actual restrictions. A capability result does not establish the behavior of every product configuration. Source: Google’s announcement.
CWE-bench offers a concrete acceptance standard
The independently operated CWE-bench leaderboard lists Carbon, using Antigravity, at 71% programmatic pass@1 and 81% pass@4. Its v1 evaluation gives each model four rollouts on each of 120 tasks, with a one-hour limit per rollout.
Those details change the interpretation of the headline number. The result belongs to a model-and-harness combination, under a specified time budget. Giving a system multiple attempts also changes the resources required to obtain an acceptable result.
CWE-bench’s deterministic checks require the exploit to stop working while existing tests continue to pass. That provides a useful starting point for an internal acceptance rule. A generated patch should arrive with a reproducer, a test showing the repaired behavior, and regression results. Reviewers can then assess what the change actually establishes.
Consider an illustrative authorization bug in a campaign management service. A proposed repair could reject the malicious request by disabling an endpoint entirely. The exploit would disappear, but so would a legitimate customer workflow. A useful verification environment must exercise both the forbidden operation and the permitted one. This example is a design scenario, not a reported Carbon result.
The engineering lesson is to invest in executable evidence before increasing the number of patches an agent can generate. Otherwise, greater throughput can simply move more uncertainty into the review queue.
Long-running work needs a durable record
Google reports that its internal teams have seen Carbon persist for days on complex engineering tasks. That is a vendor observation, rather than an independently established completion guarantee. Source: Google’s announcement.
For an integration, the practical question is what survives such a long run. I would require the workflow to retain the repository revision, investigation hypothesis, reproduction steps, changes attempted, test outputs, and unresolved questions. Without that record, restarting or reviewing the work becomes expensive even when the proposed patch is good.
A sensible division of responsibilities would let the agent investigate an isolated checkout and propose changes, while a separate verification stage tests the resulting artifact. Approval should depend on those results and the scope of the change. The agent’s own description of success is useful context, but it cannot replace the evidence.
This also gives developers a better failure outcome. A run that produces no acceptable patch can still return a reproducible defect and a clear account of unsuccessful approaches. That is more useful than an unsupported completion message.
Price the completed repair
Google announces introductory pricing of $2 per million input tokens and $10 per million output tokens, with cached input discounted by 95%. Its footnote gives subsequent prices of $4 and $20 respectively. These are announced rates for the planned launch. Source: Google’s announcement.
For a security workflow, token prices are one component of the decision. A more useful operating metric is total cost per accepted repair: inference, execution infrastructure, repeated attempts, and review effort divided by the number of patches that satisfy the acceptance criteria.
CWE-bench currently provides no average rollout cost for Carbon. Its accuracy result therefore does not establish a cost advantage. Source: CWE-bench.
Growth teams evaluating a security product should connect these engineering measurements to their product claims. Faster patch generation supports a narrow claim about generation speed. Claims about faster remediation require evidence that fixes also clear verification and review. The distinction affects what a customer can reasonably expect after enabling the feature.
For developers, the next useful step is to assemble a small set of reproducible security defects with regression tests and known acceptance conditions. When Carbon access becomes available, compare it with the existing workflow under the same permissions and execution budget. Adopt it where it produces more accepted repairs, or reduces the effort needed to reach them. That is the point at which a promising benchmark becomes a useful engineering decision.
Top comments (0)