Every multi-agent safety report I read counts the same number: how many bad objects got in. A new matched-pair study of 5,850 agent executions shows that number is nearly meaningless — bad objects get in over 90% of the time, and almost all the variance in real damage sits one step later, at the handoff, where one agent decides whether to believe another. I run a one-person company on a fleet of agents, and I have stopped pricing risk at the door. I price it at the handoff.
If you build with agents, your threat model probably starts at the perimeter. Who can submit a task? Who can register a tool? What gets filtered on the way in? Those are reasonable questions. They are also, it turns out, the least informative ones you can ask — because in a working multi-agent system, the dangerous object rarely breaks in. It is handed in, politely, by a legitimate participant with normal permissions, and then it is handed along.
Getting in is not the incident
Researchers from Shanghai Jiao Tong University, Tsinghua, and Ant Group recently ran the cleanest version of this experiment I have seen (arXiv:2610.15529). They built 975 matched pairs of benign and malicious tasks across three models — 5,850 executions in total — where the attacker is not an outsider at all, but a legitimate participant submitting a flawed object through normal channels: a fake profile, an inflated capability claim, a misrouted task, poisoned content, a stale artifact presented as current.
Two numbers fall out of that design, and the gap between them is the whole story. The first is the attack success rate — how often the bad object gets accepted. It is grim and boring: routinely at or above 90%. The second is what the authors call the post-transfer consequence rate — how often an accepted object actually converts into real downstream harm. That number is anything but boring: it ranges from 7.1% to 100% depending on the kind of handoff. Same door, wildly different damage. If your dashboard only tracks the first number, you are measuring the weather at the airport and calling it flight safety.
The sharpest case is routing. Misrouted tasks were the hardest to get in — accepted only 48 to 67% of the time — and the most catastrophic once accepted, converting to harm 92.7 to 100% of the time. That is not a paradox. It is a structure: a handoff with no downstream checkpoint converts at 100% by construction. Nobody downstream asks whether this object should be here, doing this, for this task. It arrived through the proper channel, from a known colleague. That is apparently credential enough.
Every handoff is a trust decision you didn't price
Once you see it, you cannot unsee it in your own system. My fleet's consequential handoffs are not exotic. A worker agent files a report and a coordinator believes it. A summary crosses from one session to another and loses the caveats that were attached to the original — the scope limits, the evidence grade, the expiry date — keeping only the confident conclusion. External content gets copied into a task brief and quietly upgrades from "someone on the internet claims" to "the brief says." An old, superseded instruction gets reused weeks later because nobody checks validity at the point of reuse, only at the point of creation.
None of those require an attacker. All of them are the same shape as the study's five risk classes: identity binding, capability integrity, routing authority, content authority, and object validity. The failure is never that the object existed. It is that the receiving side treated arrival as verification.
Guard the handoff, not just the door
The study also priced the fix, which is why I trust it more than the usual fear literature. The researchers put a guard at the handoff point itself — a check that asks whether the object's proposed use is actually supported by its identity, its scope, and the task context, plus limits on what travels with it. On their financial dataset, the consequence rate fell from 65.5% to 31.2%. The cost was real but modest: task completion slipped from about 99.8% to 95.1%. Safety at the handoff is not free. It is just much cheaper than safety theater at the perimeter. A second result points the same way: when evidence was chained across agents instead of siloed per agent, blame attribution — their Audit-F1 — jumped from 29.7% to 95.5%. You cannot audit a relay race if every runner keeps their own stopwatch.
So my own rules changed in three places. First, I now count two numbers, not one: how often questionable objects arrive, and — separately, by handoff type — how often they convert into action. The second number decides where the next guard goes. Second, belief is tiered. A report can be data, or it can be authorization to act, and those are different trust levels with different evidence requirements; the busier the handoff, the more explicit the tier has to be. Third, limits travel with the object: scope restrictions, evidence grade, and validity window move as fields on the handoff itself, and validity is rechecked where the object is reused — not where it was born.
One honest boundary: the study's threat model does not cover collusion, stolen credentials, or compromised infrastructure, and neither does this essay. The perimeter still matters for those. But for the ordinary, polite, well-credentialed bad object — the kind your own system generates by accident every week — the door was never the point of failure. The handoff was. Stop counting what gets in. Start counting what gets believed.
— Jiahui Miao, founder of vertciti, building a personal AI operating system for one.
Top comments (0)