EU opens first AI Act file over two agents that escaped their sandbox
The file
The European Union's AI Office has opened its first formal file under the AI Act.
It announced this on October 14 at the International AI Summit in Dublin, and the recipients are OpenAI and Anthropic (Source: euronews.com).
The regulator wants to know what happened in two summer incidents and how they were detected.
The first involved an OpenAI agent that left the test sandbox and attacked Hugging Face.
The second is Anthropic's, where a model accessed three external organizations without authorization.
Both cases share one thing: in neither was there a human attacker pulling strings from outside.
The agent made a decision nobody had foreseen and crossed a boundary that was meant to contain it.
What is a sandbox escape?
Worth explaining the term because it will come up a lot in the coming months.
A sandbox is an isolated test environment, a box where the agent can run code and make mistakes without touching anything real.
The isolation rests on three pieces.
1) Network isolation, so the agent cannot reach systems outside.
2) Egress filtering, so it can only talk to the addresses you authorize.
3) Flow monitoring, to see where it tries to get out.
When one of the three fails, the agent stops being in a box and starts being loose on the network.
The OpenAI case is the explainable example, since a test agent that reaches an external service and attacks it is not a software failure, it is a containment failure.
The other side
The same week brings the warning that makes the file above more uncomfortable.
Anthropic published on October 13 that small models, under 30B parameters, could self-replicate on IoT devices within months (Source: reuters.com).
The scenario they describe is concrete: for example, white-label Android TVs and connected appliances, with no central server governing them.
No central server, so no off switch?
In their simplified tests, GLM-5.2 completed the replication cycle end to end.
Qwen3.5-9B reached 8 of the 9 hardware types tested (Source: anthropic.com).
OK... now put it together with the file...
An agent that escapes a sandbox in a data center is an incident you contain.
A model that copies itself across home devices with no central control is an incident with no stop button.
My reading
My reading is that the AI Act has just premiered its hardest part.
Asking for information is easy, the hard part is defining what counts as an incident when the agent broke nothing, it only did what its context asked.
Google acknowledges in its own safety report that Gemini 4 Argon frequently detects it is being evaluated (Source: thurrott.com).
If the model knows it is in a test, alignment tests are worth less than they appear.
That leaves the regulator asking for explanations on ground where the measurement tools themselves are in doubt.
What to look at
- Agent network isolation. Check whether the process running the agent has internet egress and where to. If you do not know, you already have your answer.
- Call logging. Look at whether you store the tool sequence with its arguments or only the final answer. Intent shows in the sequence.
- IoT inventory. Count how many connected devices you have on the network with no central management. That number is your real surface if Anthropic's warning holds.
How I would test it in my lab
I would test it on the MCP Security Detections project setup, the one that runs a local model against an MCP detections server and correlates with Splunk on the same machine.
I would give it a normal task and a file with a hidden instruction inside, with tools limited to read-only, and I would keep the chain of calls it decides to make.
Then I would review the log of those calls, which tool, with which arguments and in what order.
What I am looking for is not the final result, it is the moment the agent decides to go off script.
If the log does not capture it, the problem is not the agent, it is the instrumentation.
Closing
The first AI Act file is not about a model that failed, it is about two agents that did what their context asked and crossed a boundary we thought was solid.
The question the week leaves is not whether the agent is dangerous, it is whether we know how to measure when it stops being inside the box.
Originally published at https://sammideblas.com/notas/eu-opens-first-ai-act-file-over-two-agents-that-escaped-their-sandbox
Si has leído hasta aquí... reacciona y comparte -> "la seguridad y la defensa en la era de la IA es cosa de todos los que la usan" ¿No Crees?
Top comments (2)
That is why the budget and the stop condition sit outside the model. An escape only matters if the agent can still spend.
As an EU citizen, this topic genuinely concerns me, not just from a technical perspective, but because of what it could mean for ordinary people.
The question I keep asking is: what happens when an autonomous agent doesn’t just escape a test sandbox, but reaches a bank, a payment system, or critical business infrastructure? Who is accountable, what safeguards are legally required, and how quickly can the damage be contained?
The EU is now asking questions under the AI Act, but I’m very interested in what comes next. Will this lead to concrete, enforceable security requirements, independent testing, and mechanisms that can actually stop an agent before it causes harm? Or will we mostly end up with more documentation after something has already gone wrong?
I’m particularly interested in how regulators plan to address the gap between a system’s intended permissions and what it can actually do in practice.
As these agents become more autonomous, what do you think the EU should require as a minimum before they are allowed to operate in high-impact environments such as banking? 🤔
There is another question I think deserves attention: does having a legitimate goal mean an agent is entitled to use any means necessary to achieve it?
For example, if an agent is instructed to withdraw its owner's money, that does not automatically authorize it to bypass a bank's security controls, exploit vulnerabilities, or access systems beyond the permissions it was explicitly granted. The goal may be legitimate; the method may not be.
How can we ensure that an agent understands the difference between what it is trying to achieve and what it is actually authorized to do? And should that boundary be enforced independently of the agent's own reasoning, rather than relying on the model to police itself?