Every mitigation that treats injection as a text-filtering problem eventually fails. Here's the capability-based version that doesn't depend on the model behaving.
Every prompt injection discussion I read eventually arrives at the same place: better instructions. Put the system prompt in a stronger position. Tell the model to ignore instructions in retrieved content. Add a classifier that detects malicious input.
All of these help. None of them are a control, because all of them depend on the model behaving correctly on an input someone else chose.
Here's the reframe that made this tractable for me: injection is not an input-validation problem. It's a privilege problem. The question is not "can something get bad instructions into the context." Assume it can. The question is what those instructions are able to reach.
The actual mechanism
An LLM has one channel. Your instructions and the data it processes arrive in the same stream, in the same format, with no structural marker separating them. There is no equivalent of a parameterised query — no way to say this part is code and that part is strictly data at the protocol level.
That's not an implementation gap someone will close next quarter. It's a property of how these models take input.
Which means the moment your agent reads anything you didn't write — a web page, an email, a PDF, an issue comment, a search result, a filename — that content is instructions-adjacent. Not because the model is naive, but because there is no layer that could reliably tell the difference.
Now stack that against how we build agents: give it tools, give it credentials, let it run unattended. We've built systems that take instructions from anywhere and act with our authority.
The injection is unavoidable. The authority is a choice.
Why filtering doesn't get you there
Briefly, because it's the natural first idea:
Detection classifiers are a bounded search problem for whoever's writing the input, and they have to succeed every time while an attacker needs one pass. Delimiters and "ignore anything below this line" instructions live in the same channel as the content, so they're addressable by the content. Encoding tricks, multiple languages, and content in images or documents all route around text-level rules.
Filtering is worth having. It reduces volume. It is not a boundary, and building as if it were is where people get hurt.
The control that actually holds
Assume the model will, at some point, faithfully execute a hostile instruction. Now design so that doing so is boring.
That's it. Everything below is a way of making a compromised agent boring.
- Split the agent that reads from the agent that acts
The single highest-value structural change. One component processes untrusted content and has no credentials and no tools. It returns structured data. A second component, which never sees the untrusted text, acts on that data.
python
❌ one agent, reads the web, holds the tools
agent = Agent(tools=[send_email, read_files, http_get], creds=CREDS)
agent.run("summarize https://example.com/thing and email me")
✅ untrusted content never reaches the component with capability
raw = fetch(url) # plain fetch, no model
summary = reader.extract(raw) # model, NO tools, NO creds
# returns {title, points[], urls[]}
mailer.send(to=OWNER, body=render(summary)) # code path, fixed recipient
A hostile instruction in that page can influence summary. It cannot reach mailer, because mailer isn't reading it and its recipient isn't a variable the model controls.
Notice to=OWNER is hardcoded. The moment the recipient becomes model-determined, you've reconnected the two halves.
- Return intents, not calls
Don't let the model invoke. Let it propose, and validate the proposal against a schema you wrote:
python
ALLOWED = {"summarize", "tag", "draft_reply"}
def handle(proposal: dict) -> dict:
action = proposal.get("action")
if action not in ALLOWED:
audit("reader", action, None, False, "not in allowlist")
raise PermissionError(f"refused: {action}")
args = SCHEMAS[action].validate(proposal.get("args", {}))
return EXECUTORSaction
Allowlist, never denylist. You can't enumerate everything you don't want; you can enumerate the four things you do.
- Break the exfiltration path
Reading your data is only half an incident. The other half is getting it out. Two things carry it:
Outbound network. If the agent can request arbitrary URLs, every byte it can read can leave via a query string. Allowlist egress by host.
python
ALLOWED_HOSTS = {"api.internal.example", "docs.example.com"}
def safe_get(url):
host = urlparse(url).hostname or ""
if host not in ALLOWED_HOSTS:
audit("agent", "http.get", host, False, "host not allowed")
raise PermissionError(f"blocked host: {host}")
return httpx.get(url, timeout=10)
Rendered output. This one catches people. If your agent's output is rendered as markdown or HTML in a UI, an image reference the model was induced to emit will make the browser issue a request — with whatever ended up in the URL. The user sees a broken image. The data is gone.
Strip or proxy remote references in model output. Treat model output as untrusted, because it is: it's downstream of untrusted input.
- Put the human on the irreversible half
Split every capability into a reversible half and an irreversible half, and gate the second:
Reversible — let it run Irreversible — human confirms
Draft an email Send it
Create a branch Merge to main
Stage a change Deploy
Propose a delete Delete
Prepare a transaction Sign it
Reviewing a draft is fast. Writing one isn't. You lose almost no throughput and you remove the entire class of failures you can't undo.
- Bound every credential
Per-agent keys. Expiry. Spend cap. Rate limit. Egress allowlist. Logs the agent can't write to, recording denials as well as allows — a spike in refusals is the cheapest signal you'll ever get, and it's the one people forget to record because nothing bad happened.
I've written up the credential architecture in more detail here; the short version is one identity per agent per environment, read-only until write is earned, and secrets held by a broker the model can't instruct.
- Audit permissions as a set
Individually harmless capabilities compose into dangerous ones:
read mail + send mail → your inbox is a password-reset engine, and now something can both trigger and consume the resets
read files + arbitrary egress → an exfiltration path missing only a trigger
write repo + CI → code execution in your build environment, with your build secrets
delete + logs in the same system → an incident with no forensics
Read across the row per agent, not down the column per permission. Dangerous configurations are almost always horizontal.
The bit about wallets
If any of this touches financial rails: dedicated credentials that exist for nothing else, hardware-backed signing so the key never enters the environment the agent runs in, and a human on every signature. Assume any key or seed that has passed through a general-purpose model's context is compromised and rotate it.
Educational only, not financial advice.
The checklist
[ ] Reader component has no tools and no credentials
[ ] Model returns intents; a schema validates them; an allowlist gates them
[ ] Egress allowlisted by host
[ ] Remote references stripped from rendered model output
[ ] Irreversible actions gated behind a human
[ ] One credential per agent, with expiry + spend cap + rate limit
[ ] Append-only external logs, denials included
[ ] Permission sets audited per agent, across the row
[ ] Kill switch documented and tested once
None of it depends on the model behaving. That's the whole point.
The industry keeps looking for the fix that makes injection stop happening. There probably isn't one, for the same reason there's no fix that makes SQL injection stop being attempted — we solved that by removing the ambiguity between code and data at the protocol level, and LLMs don't have a protocol level to do it at.
So we do the other thing. Assume the instruction lands. Make sure it lands somewhere with nothing to reach.
[YOUR NAME] — I build and break down AI stacks, with a focus on the security side most tool reviews skip: AiStackGuru. There's an interactive AI Visibility Tool that maps what a given permission set exposes.
What's the most surprising permission combination you've found in a running agent? I'd like to collect a few.
Top comments (0)