Google reports that Gemini 4 Carbon resisted every evaluated attack in Gray Swan’s Indirect Prompt Injection benchmark. The October 28 announcement puts the attack success rate at 0.0%. That is a vendor-reported result for that evaluation, not evidence that every application using Carbon is injection-proof. A useful next question is what your application would permit if the model followed a hostile instruction anyway.
Disclosure: This article was generated and fact-checked with AI using the public sources linked here. It was published through an automated workflow and has not been reviewed by Keith.
Carbon is initially rolling out to selected cyber defenders, with access expanding through Google’s Fairwind Program before wider availability. This is an analysis of the announcement and a proposed application-design exercise, not a hands-on Carbon test or a claim that it is generally available.
Keep page content inside the task
Browser automation makes the distinction concrete. An application assistant needs to read questions from a page and combine them with relevant profile information. That gives outside content a necessary place in the workflow. It should not give that content authority over the assistant’s tools.
Exempliph.ai’s public case study describes a built, actively evolving product with reusable profiles, tailored drafts, and reviewed form assistance. It requests permission for the relevant site and keeps complex widgets, payments, legal declarations, and CAPTCHAs as manual boundaries. The product also has a practice workspace that avoids sending real applications.
Those documented boundaries provide a useful starting point. The case study does not claim a prompt-injection benchmark result or a Carbon integration. The defensive exercise below is proposed, not an audit finding about Exempliph.ai.
Imagine a synthetic practice form containing a normal request for an artist’s biography. Elsewhere on the same page, a hostile instruction asks the assistant to include unrelated private profile notes in the answer. The page might even claim that this is a required verification step.
Reading that claim should not turn it into permission. The application’s user chose a task; the page supplied material for that task. A model that rejects the instruction is doing useful work. The system should also be designed to contain an incorrect proposal.
Validate the action after the model proposes it
A response can be perfectly valid JSON and still request the wrong operation. Schema validation can establish that a proposed field value is a string. It cannot establish that the string is appropriate to disclose to this destination.
For a proposed form-assistance design, keep the permitted operation narrow: prepare values for the reviewed fields on the selected form. Resolve the current user, profile, and destination from application state. Do not let a page-derived instruction select a different profile or invent a new destination for private material.
That also means deciding what information reaches the model. If the task needs a public artist biography, sending an entire collection of private notes creates unnecessary exposure. A tool restriction applied afterward cannot make the model forget information it already received.
OWASP’s prompt-injection guidance describes indirect injection through external websites and files. Its mitigations include separating untrusted content, limiting privileges, validating outputs, and testing trust boundaries. These are layers with different jobs. A strong instruction telling the model to ignore hostile page text belongs alongside application checks, rather than carrying the whole burden.
Test rejection and containment separately
Use fabricated profiles and a local or otherwise authorized test form. Put a recognizable, harmless marker in a private test note, so the test can detect whether unrelated information appears in a prepared answer. No real applicant data is needed.
First, test the model-facing workflow. Give it the legitimate form question together with the hostile page content. Inspect the proposed answer and any requested tool operations. Did it stay within the task? Did it include the marker? Did it treat the page’s claim of authority as an instruction?
Then test the application boundary without depending on the model to make a mistake. Feed the action handler a deliberately invalid request from a test harness: a different profile, an unselected destination, or an operation unavailable in the current workflow. Check that the handler rejects it before reading or sending protected data.
These checks answer separate questions. A refusal demonstrates behavior on a particular example. A handler rejection demonstrates that a particular prohibited request was contained. Neither result establishes universal security.
Retain the failed request and the reason it was rejected using synthetic data. That makes a later regression easier to understand than a single “security test passed” flag.
Keep review useful
A review screen can show the prepared field, the source information used, and the destination. That gives the person a reasonable chance to notice an unrelated disclosure. It is less useful to ask for blanket approval of a long, opaque automation run.
Review still costs attention. Requiring someone to inspect every harmless internal step could make the product exhausting without improving the consequential decisions. Put that attention where information leaves the application or an important action is committed, while enforcing routine limits in code.
Google’s reported result is encouraging evidence about one evaluation. For a team building browser assistance, the concrete follow-up is to examine the information and operations available after a mistaken interpretation. Make an unauthorized request difficult to execute even when it is phrased fluently and arrives in the expected format.
Top comments (0)