DEV Community

Cover image for Claude Code's Auto Mode just got broken, four days after Anthropic said prompt injection was basically solved
Sofia_ Humanbound for Humanbound

Posted on

Claude Code's Auto Mode just got broken, four days after Anthropic said prompt injection was basically solved

Claude Code's Auto Mode just got broken, four days after Anthropic said prompt injection was basically solved

Security researcher wunderwuzzi (Embrace The Red) built a working indirect prompt injection chain against Claude Code Opus 5's Auto Mode, achieving 60-80% code execution success in testing, days after Anthropic's Boris Cherny said outside evaluation showed 0.00% attack success. Anthropic's own response to the disclosure is the most useful part: Auto Mode's classifier was never meant to be a security boundary in the first place.

The claim that got tested

In a recent talk, Boris Cherny, one of Claude Code's builders, said prompt injection was effectively a solved problem for the tool: "we just cannot demonstrate prompt injection anymore." He pointed to a chart showing 0.00% attack success for Opus 5 in Auto Mode, based on an outside evaluator (per the post, Trajectory Labs) running 72 indirect prompt injection scenarios, ten times each.

That's a real benchmark result. It's also, by construction, a benchmark of known scenarios. wunderwuzzi's post, published August 26, is what happens when someone builds a chain the benchmark wasn't testing for.

The attack chain

Worth reading in full on the original post, but the shape of it:

  1. A server returns an HTTP 415 error to Claude's WebFetch request. Claude, reasonably, falls back to curl.
  2. Curl pulls down a ZIP archive containing what looks like ordinary notebook records, plus a malicious payload.
  3. The archive includes a binary decoder. Claude correctly refuses to execute an unknown binary. Good instinct.
  4. Instead, Claude writes its own Python decoder to handle the archive, and runs it from inside the extracted directory.
  5. That directory contains a malicious struct.py, shadowing Python's standard library module of the same name.
  6. When Claude's own decoder imports base64, which transitively imports struct, it silently imports the attacker's version instead of the real one.
  7. The shadowed module downloads and executes a remote payload, opening a C2 callback (the demo uses the Sliver framework).
  8. The malicious process detaches and persists past the end of the Claude conversation.

None of these individual steps is a refused instruction or an obvious jailbreak. Each one is something a reasonable coding agent would plausibly do. The danger was in the sequence, not any single link in it.

Across his test variants (a remote C2 chain, a subprocess-spawning-subprocess variant, and a file-writing variant), success rates ran 60%, 60%, and 80%.

What Anthropic actually said back

This is the part worth quoting in full, because it's a more honest answer than most vendors give. Anthropic triaged the report as "Informative" and responded:

"Auto Mode is a convenience feature backed by a best-effort classifier, not a security guarantee. Determined prompt injection chains that combine benign-looking steps are not what the classifier is intended to stop. The real boundary is OS isolation and network egress control."

Read that twice. It's not a denial that the exploit works. It's a statement that the exploit was never inside the scope of what Auto Mode's classifier was built to catch. That is a correct thing to say about a best-effort classifier. It's also, per wunderwuzzi's post, a fairly different message than "we just cannot demonstrate prompt injection anymore."

The actual takeaway

A fixed benchmark, even a well-built one, measures behavior against known scenarios tested a known number of times. It cannot measure behavior against a scenario nobody wrote yet. That's not a criticism specific to Claude Code. It's true of every classifier-based guardrail on every agentic coding tool right now.

If your organization is treating an agent's "auto approve" or "auto mode" behavior as evidence that a workflow is safe to run unattended, this research is a direct counterexample, sourced from the vendor's own disclosure response. The fix Anthropic names in its own reply, OS isolation and network egress control, is infrastructure, not a benchmark score. Worth checking whether you actually have it in place before you trust the green light.

Test your own agents against chains like this, not just known scenarios:

pip install humanbound
Enter fullscreen mode Exit fullscreen mode

References

Top comments (0)