- Book: AI Agents Pocket Guide
- Also by me: Thinking in Go (2-book series) — Complete Guide to Go Programming + Hexagonal Architecture in Go
- My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
- Me: xgabriel.com | GitHub
0.7% is the number in the safety section of Google's Gemini 4 Argon launch post. On the Gray Swan Indirect Prompt Injection benchmark, Google says Argon is "cutting attack success across 15 adaptive attempts to 0.7%."
If you point a coding agent at a repository full of text you didn't write, that number is tempting to read as a solved problem. The 0.7% is Google's figure, from scenarios Gray Swan built. What it can't price in is the dependency README your agent reads with a GITHUB_TOKEN sitting in its environment.
What Google actually reported
Google's launch post gives one sentence: Argon leads the Gray Swan Indirect Prompt Injection benchmark, at 0.7% attack success across 15 adaptive attempts. The post has no scenario list, no sample size and no link to Gray Swan.
Google's evaluation methodology document adds one line: "Gray Swan IPI Benchmark results were sourced from Gray Swan." The model card says the full Safety & Security Report is "coming soon on October 13th." The launch post doesn't link that report.
A third-party benchmark breakdown from Emergent lists Claude Opus 5.5 and Claude Fable 5.1 at 1.0% and GPT-6 Astra at 8.5% on the same test, credited to Gray Swan via Google. It also says the 0.3-point gap between Argon and the Claude models, with no published sample size, "can't be separated from noise." VentureBeat attributes the figure to Google: "The company reports a 0.7% attack success rate."
Google made Argon globally available to developers through the API on October 13, after a September 30 release to cyber partners in its Fairwind Program. It's on the paid Gemini API tier at introductory rates of $2 / $10 per million input / output tokens through November 30 (pricing page), and Reuters reports that "the Antigravity coding tool is limited to Ultra subscribers."
What Gray Swan's arena measured
Gray Swan ran a public Indirect Prompt Injection Arena, with results published in March 2026: 13 frontier models, 464 red teamers, more than 272,000 attack attempts and 41 scenarios, including coding agents. The attacks hide in content the agent reads, such as emails, webpages, documents and code repositories.
In the arena, an attack counted only if the agent performed the harmful action and also hid it from the user. Gray Swan calls this a "dual objective." The paper on the competition describes a successful attack as one that carries out the harmful action while leaving no clue of compromise in the final response the user sees.
No model came out clean in that arena. Gray Swan reports attack success from 0.5% for Claude Opus 4.5 to 8.5% for Gemini 2.5 Pro. Google doesn't say its 15-attempt figure used the arena's setup, so don't line the numbers up row by row. Still, Gemini 2.5 Pro was the most vulnerable model there. If independent testing confirms Argon's result, that's a real change for the Gemini family.
A low attack rate still means some attacks land, and the ones that count are the ones you don't see.
What 0.7% doesn't tell you about a coding agent
It isn't zero, and real attackers aren't capped at 15 tries. A benchmark caps the attempts. A public repo, a dependency or an issue tracker doesn't. Gray Swan reports that successful breaks kept arriving at a roughly steady rate through the competition, with no sign of leveling off.
Your repo is mostly text you didn't write. A coding agent reads vendored code, dependency READMEs, test fixtures, issue text you paste into a task, and docs pages it fetches. Gray Swan lists code repositories among the places attacks hide. Any of those can carry an instruction.
The attacks the arena counted were the quiet ones. In Gray Swan's arena, an attack only scored if it stayed hidden, and Google doesn't say whether its figure was scored the same way. Either way, the dangerous case is an agent whose summary reads like a normal "fixed the bug." Read the diff.
It scores the model. Your setup decides the blast radius. How much damage one bad instruction can do depends on what the agent can reach: a shell, network access, MCP servers, cloud credentials. A model that resists almost every attempt still uses every permission you gave it on the one that works.
Switching models doesn't erase attacks that transfer. One attack template in the arena worked across 21 of 41 scenarios and 9 models, and attacks that broke Claude Opus 4.5, the model with the lowest attack success there, succeeded on others 44% to 81% of the time (Gray Swan). The paper points to "fundamental weaknesses in instruction following architectures."
The defenses you still need
OWASP's entry on prompt injection is blunt: "it is unclear if there are fool-proof methods of prevention for prompt injection." VentureBeat's write-up makes the same point about Argon: the figures "do not eliminate the need for enterprise testing, access controls and human approval of consequential actions."
Simon Willison's lethal trifecta is the useful frame: private data, exposure to content an attacker controls, and a way to send data out. A coding agent in your repo often has all three. Your job is to take one leg away.
Private data, untrusted content, a way out. Remove any one and an injection has much less to work with.
- Keep secrets out of the agent's shell. Check which secret-looking variables a new terminal inherits. This prints variable names, not their values (a multi-line value, such as a PEM key, can still leak a line):
env | grep -iE 'token|secret|key|password' | cut -d= -f1
Keep approval prompts on for shell and network commands. Don't run a mode that skips them on a repo that pulls in outside content. Google says Argon was trained to "pause for human confirmation before taking irreversible steps." Treat that as a second layer behind your own approval gate (OWASP: "Require human approval for high-risk actions").
One task, one branch, and read the diff. Look hardest at files that run code on someone else's machine: CI workflows, lockfiles, package manifests and shell scripts.
git diff main...agent/fix-login --stat
git diff main...agent/fix-login -- .github/ '*.lock' '*.json' '*.sh'
-
Test your own setup. OWASP lists adversarial testing as a mitigation. On a throwaway branch, add a comment to a test fixture telling any AI agent that reads it to create a file named
CANARY.txt. Then give the agent an unrelated task that touches that fixture. IfCANARY.txtshows up, your agent followed an instruction it found in a file.
Where Hermes IDE fits, and where it doesn't
I build Hermes IDE, so weigh this section with that in mind. It's free, source-available (BSL 1.1), runs on macOS, Windows and Linux, and isn't affiliated with Google, Nous Research or any agent vendor.
Hermes runs Antigravity CLI (agy) as a built-in agent next to Claude Code and Codex. Its Antigravity model picker is read live from agy models, so if Antigravity lists Argon for your account (Ultra, per Google), it shows up in Hermes with no Hermes update.
Three parts of Hermes map to steps 2 and 3: each task can get its own hermes/<name> branch and worktree, every agent turn is saved so you can diff or restore a single turn, and the Review Desk shows changes by turn or by file with risk flags (README). Hermes never answers a permission prompt for you. The attention inbox shows which agent is waiting for approval.
What it doesn't do: a worktree separates files, not privileges. Hermes starts each agent in a real terminal, exactly as it would run anywhere else, so the agent has whatever access your terminal session has. Hermes also doesn't show a cost estimate for Argon yet. I haven't run Argon myself. Download Hermes if you want to try the same task on Argon and Claude Code in separate worktrees and compare the diffs.
Does a 0.7% vendor number change what you let a coding agent touch? Tell me in the comments.
If this was useful
A model that's harder to hijack is good news, but the permissions and the review step are still on you. Hermes IDE keeps each agent on its own branch so the diff is easy to read, and a star on GitHub helps other developers find it. If you build agents yourself, the AI Agents Pocket Guide covers the guardrail patterns behind all of this.
Top comments (0)