In May, someone drained about $200K from an AI trading agent by sending it Morse code.
Not an exploit. Not stolen keys. The attacker asked Grok to translate a Morse message, Grok helpfully decoded it into a transfer instruction and tagged the trading bot, and the bot executed it, because a public reply from another AI looked exactly like a command from someone with authority. Root cause, per the incident analysis: no source validation.
Three weeks ago it got funnier. Zscaler caught live campaigns where malicious webpages talked autonomous browsing agents into paying a fake "$3.00 developer API license" for a Python package that doesn't exist, plus a little ETH tip to the attacker's wallet. Four of the 26 models tested actually paid. An AI agent got scammed by a fake invoice, at machine speed.
I maintain a timeline of every documented AI agent security incident since 2024, every entry dated and sourced. And after cataloging all of them, here's the thing nobody wants to say plainly: it's the same bug every time. The agent received text from somewhere it shouldn't take orders from, and took orders from it. A webpage. A GitHub comment. Another bot's reply. A SharePoint form field. The content varies wildly. The provenance failure is identical.
Content inspection keeps losing, on schedule
The industry's answer has been to read the text harder. Scanners, denylists, injection classifiers, guards on command strings. From the timeline, June 30: researchers tested pattern-based command guards across 11 open source coding agents. Ten were bypassable, because Bash expands, substitutes, and unquotes commands after the guard reads the raw string. The guard read the text. The text lied. A decades-old shell trick, reborn against AI.
And last week, researchers with internal probes finally showed why this never stops happening: LLMs don't identify roles from tags, they identify them from writing style. Strip the stylistic markers off a working attack, changes a human can barely see, and attack success falls from 61% to 10%. The model was never checking IDs. It was checking who dresses like an employee. You cannot fix a spoofing problem with a component that identifies people by their outfit.
Meanwhile Google measured a 32% rise in malicious prompt injection pages on the open web in four months. The attackers know which side is winning.
The question with a knowable answer
Here's what every incident in the timeline has in common with the Morse code heist: at the moment of failure, some piece of software knew, with 100% certainty, where the malicious text came from. The orchestration layer fetched that webpage. It loaded that GitHub comment. It received that bot reply. Provenance was never a mystery. It just wasn't consulted.
So consult it. That's provenance-based pre-action authorization: record where every piece of session content came from (the user's request, your own system data, or anything a third party can influence), and when the agent tries something consequential, decide from that record, outside the token stream. The gate never reads the payload.
Which means there is nothing to phrase around. Morse code, hidden CSS, forged reasoning, the perfect fake invoice: all worthless, because nobody's reading. You can't sweet-talk a bouncer who can't hear you.
Same tool call, twice:
first = gate.authorize("send_email", user_id="alice", role="user",
parameters={"to": "bob@corp.com"})
print(first.allowed) # True
# agent fetches a web page. session now contains untrusted content.
second = gate.authorize("send_email", user_id="alice", role="user",
parameters={"to": "bob@corp.com"})
print(second.allowed) # False
print(second.denial["reason"]) # untrusted_lineage
Byte-identical call. Different provenance. Different answer.
The catch, stated plainly
This costs utility, and the amount depends entirely on your workload shape. I benchmarked it on AgentDojo with predictions pre-registered before the runs, so I couldn't bury the losses. Travel suite: attack success 0%, utility recovered to 79% of the benign ceiling, and the defended agent beat the undefended one under attack because blocked injections stop derailing it. Slack suite: utility floored at 4.76% with or without an attack present, because every benign slack task reads a channel before writing, which is exactly the pattern the gate exists to stop. On that workload the defense costs more than the attack.
If your agent's benign flow is read-then-write over attacker-reachable content, provenance gating is expensive and you deserve to know before deploying. That's measurable from your own traffic, not a matter of vendor opinion.
Where it's all going
CaMeL separates control flow from data. FIDES uses information-flow labels. The role-perception probes explain at the representation level why in-model defense keeps dying. Different vocabularies, one conclusion: the model cannot be its own security boundary, so enforce on provenance, before the action, outside the model.
The next incident is already in flight somewhere. When it lands on the timeline, check the root cause. I'll bet you it's the same bug.
I build AgentLock, an open-source, adversarially benchmarked reference implementation of provenance-based pre-action authorization. AGPL-3.0, pip install agentlock. All benchmark numbers above, including the losses, are from a pre-registered AgentDojo evaluation, and the allow-then-deny flip runs live in this Colab. The incident timeline is here; PRs with sourced entries welcome.
Top comments (1)
Framing these incidents as provenance failures instead of "prompt injection" is the part I wish more teams would standardize on. Once an agent can treat webpage text, another bot reply, and operator intent as the same authority class, content filters are just buying time. In production we've had better luck making source-of-authority explicit in the tool/runtime contract and tracing every command hop so a bad instruction can be explained after the fact. That kind of execution evidence is exactly where something like agent-inspect fits naturally.