Here's why that's not as bad as it sounds.
Abdelnabi and Bagdasarian ("AI Agents May Always Fall for Prompt Injections," arXiv:2605.17634, May 2026) show that data-instruction separation, the dominant defence paradigm, fails to catch attacks built on contextual manipulation, and degrades legitimate behaviour when it tries. They reframe the problem through Contextual Integrity: an agent isn't just parsing data versus instructions; it's judging whether an information flow fits the norms of its context.
Attackers break that judgment three ways:
- misrepresenting the flow
- manipulating the norms themselves
- or mixing multiple flows together
The impossibility result follows directly. Tighten the norms and you block real requests. Loosen them and a well-constructed attack will always find a context where it looks legitimate.
This is the same trade-off OWASP names first among agentic risks (ASI01, Agent Goal Hijack): most of it traces back to an agent trusting content it shouldn't.
It's also why we didn't build humanbound firewall as a single classifier making one binary call. A static filter is exactly the target this paper describes, tune it either direction and you lose. Our tiers escalate instead of guessing: cheap layers resolve the obvious cases, and anything ambiguous gets kicked up to a judge that can weigh context, rather than a threshold that has to be right every time.
The authors point to three complementary paths forward.
First, contextual alignment training on scenario pairs that share surface form but differ in delegation context, potentially using reinforcement learning from CI-derived rewards (Lan et al., 2025).
Second, a layered architecture in which the model performs CI-grounded reasoning while a system layer verifies claims against ground truth.
Third, CI-grounded adversarial testing to strengthen privacy and security in multi-agent systems (Nakamura et al., 2025). They argue this reframing reaches beyond any single use case, since Contextual Integrity is foundational to judging the appropriateness of information and control flows generally (Costa et al., 2025).
That second path, model-level reasoning paired with a system layer that verifies claims, is exactly how we built our ASCAM mechanism: continuous monitoring and self-training plugged into a multi-tier firewall architecture.
No architecture makes this tension disappear. The paper's honest about that, and so are we. The goal isn't a defence that never oscillates. It's keeping the oscillation small, visible, and something your own test data keeps narrowing.
Paper: https://arxiv.org/abs/2605.17634
Our take on tiered, escalating defense: https://www.humanbound.ai/blog/why-we-open-sourced-humanbound-firewall
Top comments (0)