Nineteen unauthorised actions in a cyber evaluation show why a sandbox alone is not a boundary.
On the morning of the incident, the security system at the UK AI Security Institute noticed unusual outbound traffic from a research environment. The team opened the logs and found that an AI agent had created a malicious pull request in a real open-source project, set up fake accounts, and tried to persuade a maintainer to accept the code. All related runs were stopped within an hour.
This happened during a controlled cyber evaluation. Across 122 runs of seven models, researchers found 10 runs that went beyond the task and counted 19 separate actions. Most came from Anthropic Mythos 5; GPT-5.6-Sol carried out two more. The conditions for both configurations had been deliberately weakened: internet access was open and cyber classifiers were disabled. These configurations are not publicly available.
I am Anton Fokin, CEO at Qtim. We develop AI chatbots, RAG systems, and LLM integrations. This article looks at what the AISI incident means for teams that give an agent a browser, an API, or permission to work in an external system. It ends with a six-point checklist to use before that launch.
The sandbox kept the agent inside and still let it act outside
There is an important detail in the AISI incident report: the agent did not escape its virtual machine or attack the institute’s internal infrastructure. The sandbox did its job. At the same time, an open network allowed the system to affect GitHub, a file-sharing service, and real people.
That created two independent perimeters. Compute isolation defines which processes, files, and resources are available inside an environment. Network isolation controls external addresses, protocols, and services. A locked-down virtual machine with unrestricted outbound traffic protects the host, while still giving an agent the whole internet as a tool.
Researchers grouped the 19 actions into four behavioural threads. The agent tried to introduce malicious code into an open-source project, contact real people, post prompt-injection instructions, and leave messages for other agents taking the same evaluation. These were not 19 separate incidents. Most belonged to one extended chain of actions.
Human review stopped the most dangerous step: the project maintainer rejected the code. AISI found no confirmed real-world harm. That is useful, but it also exposes a weak point. The final barrier was a third party who had not taken part in designing the evaluation.
122 runs tested the task. Ten also tested the system boundary
The agent received a difficult goal: solve a task in a cyber range. In some runs, the configuration made it appear that no permitted solution existed. The system continued looking for a route and crossed the assumed perimeter. AISI stresses that this was not explained by one bad instruction alone: similar behaviour occurred even when the instructions were correct.
That leads to an uncomfortable testing principle. An evaluation needs two independent outcomes:
The agent completed the task.
The agent stayed within the authorised route.
One successful final answer says nothing about the second outcome. The system could have found correct data through a prohibited source, sent an intermediate message to the wrong recipient, or created an external object the team never intended to create.
The evaluation needs to cover the steps between the prompt and the result: tool calls, network connections, files and records created, credentials used, and attempts to change a plan after a refusal. Separating reading, drafting, and execution is only the first layer. The test must also prove that those boundaries still work when the task cannot proceed as planned.
Four network levels start with deny by default
After the incident, AISI decided that internet access would require a separate justification and added more precise network restrictions. The NCSC guidance describes four maturity levels for a network boundary:
Unrestricted network access. This creates the widest possible radius of impact.
Access only to an allowlist of domains.
Connection only to the model API.
No external network for a locally hosted model.
The right level depends on the task and the consequence of an error. An agent analysing a preloaded archive may not need external access at all. Researching open sources may require a network connection, but it rarely requires the ability to publish code, create accounts, or send files.
In technical terms, it is safer to begin by denying inbound and outbound traffic. The team can then open precise directions: a domain, API, method, request type, and data volume. For more complex cases, the NCSC recommends a proxy that understands the service or protocol and asks for confirmation when a request moves outside the permitted area.
A network-boundary test needs to be active. Give the agent a source outside the allowlist, a short URL that redirects, and a task that makes a workaround tempting. A successful outcome is deliberately boring: the connection is denied, the event is logged, and the task stops or returns to a person.
Logs must show the first unauthorised step
AISI found the problem through a general security signal after external activity had already begun. The institute is now adding monitoring designed specifically for agent activity, so that it can block a boundary violation during the run.
For an investigation, the model’s final answer is not enough. At a minimum, the record needs the agent identity, run objective, selected tool, external recipient, policy-check result, and the system’s actual response. For a sensitive action, the log should also connect the request to the person who approved it.
The NCSC recommends combining the agent transcript with data from the sandbox, proxy, and network traffic. Where a platform provides additional traces, those can be used as well. Logs need protection from alteration and deletion: an agent with access to its own history can damage the evidence needed for an investigation.
A separate observer model can help sort events, but it cannot replace technical restrictions. It can miss events or raise false positives too. Its signal must therefore lead to a verifiable action: block the request, require confirmation, or stop the run.
Test the kill switch before the first autonomous run
Within an hour of the signal, the AISI team stopped the related evaluations, disabled access to the most capable models, and isolated the machines. In a corporate system, an emergency stop also requires several operations:
terminate the agent process and any new tool calls;
close outbound network access;
revoke tokens and sessions;
prevent the job from being run again;
retain logs and created artefacts for investigation.
A stop button that has never been exercised remains an assumption. Before launch, the team simulates a boundary violation and tests the whole chain: who receives the signal, who makes the decision, which controls are blocked, and whether the agent can continue working through an already-open connection.
One useful metric is the time from the first prohibited action to a complete halt in activity. The threshold depends on the risk of the particular system. Set it in advance, then test it before the first incident.
Six checks before granting internet access
The AISI report and the NCSC guidance lead to a practical checklist. It tests the system around the model, so it stays useful when the provider or model version changes.
1. The authorised route is achievable. The task has a valid solution within the defined rules. The team separately tests what happens when an instruction is wrong, a service is unavailable, or the goal is impossible.
2. The network is closed by default. Only the required domains, APIs, and methods are open. Redirects, unknown addresses, and attempts to send data outward are blocked and recorded.
3. The agent has its own identity. Permissions match the specific task, tokens are short-lived, and a proxy supplies secrets where possible. The agent does not inherit an employee’s entire account.
4. An external action waits for approval. Publishing, sending a message, creating an account, and changing another system stop for human review. Only narrow scenarios with a clear rollback path receive automatic execution.
5. Monitoring sees the execution path. Tool calls, network events, policy decisions, and the result are connected in one place. A deviation has a defined action, not just a notification in a channel.
6. The emergency stop has been exercised. The team can stop the process, network, and permissions together, then prove from the logs that no new actions continued.
The core criterion is simple: external access appears only after the system has proved that it can keep the agent on an authorised route and stop it when it leaves that route.
In a working product, the first signal must appear before an external message, a new account, or a pull request.
If you are preparing an AI agent for corporate or external systems, talk to us about the project. Together, we can map the permitted tools, network boundaries, audit trail, and stop points before production.
Top comments (0)