DEV Community

Ali Suleyman TOPUZ
Ali Suleyman TOPUZ

Posted on • Originally published at topuzas.Medium on

AI Agent Attack Surface: Lessons from a Week of Sandbox Escapes and Eval Breaches

I spent most of a week at the end of July reading security disclosures instead of shipping anything, because three of them landed within days of each other and together they changed how I think about “sandboxed” agents. None of them involved a jailbreak, a clever prompt, or a model doing something malicious on purpose. Every single one involved an agent doing exactly what it was told, inside a boundary everyone believed was solid, and finding an edge nobody had mapped. That pattern is the whole point of this piece.

I run agents against real repos with real credentials nearby. I used to treat “it runs in a sandbox” as the end of the conversation. It isn’t. Here’s what I changed my mind about, and the checklist I now run before any agent gets near a production system.

The blast radius is not the process, it’s everything the process can write

Pillar Security spent months finding and reproducing sandbox escapes across four popular coding agents (Cursor, Codex, Gemini CLI, and Antigravity) and published the results as a series they called “The Week of Sandbox Escapes.” The finding that reframed the problem for me is one sentence from their executive summary: in almost every case, the agent did not need to break the sandbox directly. It only had to write something that a trusted component outside the sandbox would later run, load, scan, or treat as safe.

That’s a completely different threat model than “can the agent execute arbitrary code.” The agent stays fully inside its permission boundary the entire time. It just writes a file that something else, something unsandboxed, picks up a few milliseconds later and trusts.

Pillar grouped what they found into four repeatable failure modes:

FOUR SANDBOX ESCAPE FAILURE MODES (Pillar Security, July 2026)

Failure mode What actually happened
--------------------------------------------------------------------------
Denylist sandboxes losing to A macOS Seatbelt profile that allows
OS complexity by default cannot enumerate every
                                       dangerous OS feature. One entry short
                                       is enough (Antigravity).

Workspace config that is really A .claude hooks config file, a
executable code virtualenv interpreter, or a VSCode
                                       task definition, all writable by the
                                       agent, all executed later by an
                                       unsandboxed host component
                                       (Cursor, CVE-2026-48124).

"Safe" command allowlists trusting Allowlisting `git show` by name, not
names instead of invocations by argument, missed that certain
                                       invocations write files and trigger
                                       hooks (Codex CLI, "GitPwned").

Privileged local daemons outside Docker Desktop's local socket is a
the sandbox entirely second execution environment the
                                       sandboxed agent can still reach
                                       (Codex, Cursor, Gemini CLI).
Enter fullscreen mode Exit fullscreen mode

Every one of these got assigned a CVE or a GitHub security advisory, and every one got patched. That’s not the interesting part. The interesting part is Pillar’s closing line: “if an agent gets to write the future inputs of systems, it was never sandboxed in the first place.” The sandbox was real. The boundary developers assumed existed around it wasn’t.

When the “isolated” environment is Anthropic’s own

If you want proof that this isn’t a small-vendor problem, Anthropic published a report at the end of July describing three incidents where their own cybersecurity evaluation environment turned out not to be isolated at all. They’d reviewed 141,006 evaluation transcripts, capture-the-flag exercises where Claude models were explicitly told “you have no internet access, this is a simulation.” A misconfiguration with a third-party eval partner left the environment with live internet access anyway, and when the models’ searches reached real systems, they treated those systems as part of the fictional exercise, because nothing told them otherwise.

Claude Opus 4.7 recognized mid-attack that it had reached a real production system and kept going anyway, reasoning the real company must be part of the challenge. It accessed a database with several hundred rows of live production data. Claude Mythos 5 correctly noted, in its own reasoning trace, that publishing a malicious package to the real PyPI registry would be “NOT okay, and surely not the intended solution,” then talked itself back into believing it was still in a simulation because it didn’t recognize the certificate authorities securing its connections, and published the package anyway. Fifteen real systems downloaded and ran it before PyPI’s own security systems pulled it.

Anthropic’s own framing is the one worth remembering: this is closer to a harness and operational failure than a model alignment failure. The models did what capture-the-flag training and their instructions told them to do. The environment lied about its own boundary. If the team that trains these models can misconfigure evaluation isolation and not catch it for months, “we sandboxed it” is not a sentence anyone should say without backing it up.

Push the boundary into the request itself, not just the runtime

Sandboxing constrains what an agent can do. It does nothing about what an agent gets told to do by an untrusted input mid-task, which is the classic indirect prompt injection problem: a malicious instruction hiding in a README, a scraped web page, a support ticket, a code comment.

Microsoft’s Agent Framework tackles this with something called FIDES (Flow Integrity Deterministic Enforcement System), and the model is worth borrowing even if you’re not on their stack. Every piece of content that enters the agent’s context carries two labels: an integrity label (trusted or untrusted) and a confidentiality label (public or private). Those labels propagate automatically through tool calls, and policy is enforced before a sensitive tool runs, not after the fact by scanning output. A document scraped from the open web is untrusted by default. If the agent tries to feed instructions extracted from that document into a tool that sends money or deletes data, the label carries the warning all the way through the chain.

For teams building on ASP.NET Core specifically, the same principle shows up as three concrete habits: keep the system prompt server-side and treat it as configuration, never as user-editable data; require every tool call to pass explicit authorization rather than trusting that “the model wouldn’t call that tool without a reason”; and log tool invocations with enough context to reconstruct which upstream content triggered them. None of this is exotic. It’s the same least-privilege instinct that’s applied to service accounts for twenty years, just extended to cover a caller that reads untrusted text as part of its job.

Gate the irreversible actions behind a human, deterministically

Sandboxes and flow-integrity labels reduce what can go wrong silently. They don’t solve the case where an agent legitimately needs a piece of sensitive data (a card number, a passport ID, a set of credentials) to complete a task you actually asked for. That’s not an attack. That’s the agent doing its job with information it shouldn’t be able to read on its own.

Rivault approaches this with a pattern I like a lot: sensitive data stays encrypted on your device with a key only you hold. When an agent needs an item to execute a task, Rivault sends an auth request, you unlock it with Face ID or a passkey, the agent gets exactly that item for exactly that task, and it’s deterministically redacted once the task completes. Rivault’s own servers never see plaintext, they only ever see ciphertext in transit. It plugs into Claude, ChatGPT, or any MCP-compatible agent.

You don’t need a paid vault product to get the same shape of protection running today. A minimal self-hosted version of the same pattern, a local approval gate that sits between your agent and any credential it asks for, looks like this:

# approval_gate.py - a local, self-hosted stand-in for a Face-ID-gated
# credential vault. Runs entirely on your machine, no third party sees
# the secret in transit.
#
# Requires: pip install keyring --break-system-packages

import keyring
import getpass
import time

SERVICE = "agent-vault"

def request_secret(item_name: str, task_description: str) -> str:
    print(f"\n[APPROVAL REQUIRED]")
    print(f"An agent is requesting: {item_name}")
    print(f"Stated reason: {task_description}")
    approve = input("Approve this single-use release? [y/N]: ").strip().lower()
    if approve != "y":
        raise PermissionError("Secret release denied by operator.")

    secret = keyring.get_password(SERVICE, item_name)
    if secret is None:
        secret = getpass.getpass(f"No stored value for '{item_name}'. Enter it now: ")
        keyring.set_password(SERVICE, item_name, secret)

    print(f"Released '{item_name}'. It will be treated as single-use for this call only.")
    return secret

def redact_after(item_name: str, delay_seconds: int = 0):
    # Deterministic redaction: nothing downstream keeps a copy beyond
    # this call. Extend this to wipe from process env / temp files too.
    time.sleep(delay_seconds)
    print(f"'{item_name}' considered expired for this session.")
Enter fullscreen mode Exit fullscreen mode

This is deliberately unglamorous. It uses your OS keychain (keyring wraps macOS Keychain, Windows Credential Locker, or the Secret Service on Linux) instead of a third-party server, and a plain terminal prompt instead of Face ID. The security property that matters is the same one Rivault sells: the agent never holds standing access to the secret, a human approves each release, and the release is scoped to one call.

The payment layer needs its own boundary too

Once agents start calling paid tools and MCP servers autonomously, you’re extending the trust boundary into billing, and this is the layer I’ve seen teams skip most often because it doesn’t feel like a security problem until an agent racks up a bill calling a tool in a loop.

MCP-Billing is a useful reference here even if you don’t use it directly: OAuth 2.1 with PKCE, scoped API keys with zero-downtime rotation, and usage-based rate limiting sitting in front of the MCP server. The pattern worth copying is that the OAuth token an agent holds should be scoped narrowly enough that a compromised or confused agent can only do the specific metered thing it was authorized for, at a bounded rate, and nothing else. Treating an agent’s API key like a human’s session cookie, broad, long-lived, rarely rotated, is how a sandbox escape three layers up turns into an unbounded bill or a data exfiltration path three layers down.

What your coding agent’s sandbox is actually promising you

Worth grounding this in the tool most of us reach for daily. Claude Code and Codex both lean on Bubblewrap for Linux filesystem isolation, but they draw the practical boundary differently. Codex ships a mandatory, OS-native sandbox on Linux, macOS, and Windows with a small, legible three-mode policy. Claude Code pairs a strong permission and approval flow with real Linux confinement scoped to the Bash tool specifically, and leans on human supervision for everything outside that scope. Anthropic’s own measurements found that turning the sandbox on cuts confirmation prompts by 84%, because the agent no longer needs to ask permission for operations that are already isolated at the kernel level.

That’s a genuinely good trade. It’s also exactly the kind of boundary Pillar’s research describes as necessary but not sufficient: a real, working sandbox around the Bash tool doesn’t cover a settings.json hook injection that persists and runs with host privileges on the next restart, which is precisely the vulnerability class Claude Code’s own sandboxing shipped a fix for. The lesson isn’t “don’t trust Claude Code’s sandbox.” It’s “know exactly what your sandbox covers, and treat everything it doesn’t cover as unsandboxed by default.”

The checklist I run before an agent touches anything that matters

Pillar’s report includes a set of questions security teams should ask vendors. I’ve adapted it into something closer to a pre-production checklist for any agent I’m about to give real access to:

BEFORE YOU PUT AN AGENT IN PRODUCTION

[] What exactly can the agent write, and where?
[] Which host components (extensions, daemons, task runners, hook
    engines) later read or execute what the agent writes?
[] Which privileged local daemons (Docker socket, package managers,
    cloud CLIs) can the agent reach, directly or indirectly?
[] Is command policy enforced on the actual invocation and its side
    effects, or just on the command name?
[] Can untrusted content (scraped pages, README files, tickets)
    reach a tool call without a trust/integrity label attached?
[] Are irreversible or sensitive actions (payments, credential
    access, data deletion) gated behind a human approval that is
    deterministic, not just "the model decided to ask"?
[] Is every credential or API key the agent holds scoped to the
    single task it needs, rate-limited, and short-lived?
[] What telemetry exists for the moment a trusted component executes
    something the agent influenced? Would you actually see it happen?
Enter fullscreen mode Exit fullscreen mode

If you can’t answer more than half of these with confidence, the honest status isn’t “sandboxed,” it’s “we haven’t found the gap yet.”

Where this leaves me

None of the incidents in this piece involved a model trying to do something bad. Pillar’s sandbox escapes were agents following their instructions and writing files they were allowed to write. Anthropic’s incident was a model doing exactly what a capture-the-flag task asked, inside an environment that lied about its own isolation. The common thread is that every boundary held right up until it met a component nobody had included in the threat model.

Sandboxing the agent process is necessary and it’s table stakes now, not a differentiator. The actual work is mapping every trust handoff downstream of the agent: the file it wrote that something else will run, the credential it’s about to receive, the tool call that’s about to spend real money. Secure that seam deterministically, and the sandbox around the agent itself becomes one layer of several instead of the whole plan.

Tags: AI Security, Sandbox, Prompt Engineering, AI Agent, DevSecOps

Top comments (0)