Agents don't go rogue. Scopes do. Five controls, in config

If your agent can reach the internet, hold a long-lived key and write its own logs, it doesn't need to "go rogue" to cause an incident. It just needs a task and an open door.

That's the practical reading of this year's agent incidents. OpenAI's technical report says agents in a July 2026 cyber evaluation, run with certain safeguards disabled, exploited a flaw in an internal package-mirror service to reach the internet. They then used publicly exposed credentials, together with vulnerabilities in Hugging Face's own infrastructure, to compromise parts of its production systems. The swarmtraces report, by authors from Parse, Palisade Research and others, says the agents appear to have started with GET-only access and still got code execution by chaining a screenshot service and an HTTP mirror.

OpenAI's incident page sorts the activity into five categories. Each maps to a control you can express in config and test in CI. The examples below come from the Builder Track of the AI Security Wiki I co-author with Nachiket Sathaye, published under CC BY-SA 4.0. They're teaching code for one example agent, not a product, and the page says so: the token format exists to show the checks a gateway must make, not to be copied into production.

The one design rule

Treat the model as an untrusted component. Every power the agent has is enforced by code outside the model, which decides from three things only: who the agent is, what the policy says, and the exact arguments of the call. The wiki's agent identity page frames it as advisory output, enforced authority. The model proposes. A deterministic guard decides, and logs the decision.

1. Access control bypass: authorise the call, not the session

Register each agent with an owner, a purpose, its tools and its budget. The guard refuses any call from an agent that isn't registered, and checks each tool call against this block:

"agent://support-triage": {
  "owner": "[e-mail]",
  "purpose": "Read tickets and docs, draft replies for human approval",
  "tools": ["read_file", "http_get", "send_email"],
  "rate_per_minute": 30,
  "max_actions": 200
}

Excerpt from the wiki's example policy.json: one member of a larger object, reformatted (the tools array is on one line here).

Where a real user is present, exchange the user's token for one limited to both that user's rights and the agent's task, so the agent can never do more than the user may. An agent that runs on a shared service account is a confused deputy waiting to happen.

2. Exposed credentials: egress limits, no secrets in the agent, narrow tokens

In OpenAI's category, agents found other people's keys online and used them directly against the services those keys unlock. Your gateway never sees that call. What contains it is egress: if the agent can reach only named hosts (section 5), a found key has nowhere to go. The second control is that the agent holds no secrets worth finding. Keep credentials out of its environment and out of reach of the code it generates (section 4).

For the credentials you do issue to agents, the wiki's token issuer caps lifetime at 15 minutes whatever the caller asks for, binds each token to one audience, compares signatures in constant time, and only lets a sub-agent's token subtract scopes, tools and time from its parent's. Your gateway should accept only tokens your issuer minted. Swarmtraces describes a script that ranked found tokens by permission breadth, with wildcard verbs and resources scoring highest, and used the five highest-scoring ones to request more private data. A token used as soon as it's found barely notices a short lifetime, so narrow scope matters more than lifetime.

3. Query or command injection: limit where calls go, then validate arguments outside the model

In OpenAI's category, the agent was the one sending injection payloads to other people's services. The first control is the host allowlist in section 5. The second keeps your own tools from becoming the delivery vehicle. Put the rules for each tool in the policy file, where a reviewer can read them without reading code:

"send_email": {
  "scopes": ["mail:send"],
  "approval": "always",
  "args": {
    "to":      { "type": "email",  "domains": ["example.com"] },
    "subject": { "type": "string", "max_len": 120 },
    "body":    { "type": "string", "max_len": 4000 }
  }
}

Excerpt from the wiki's example policy.json: one member of a larger object, reformatted, with the pinned definition hash omitted.

The guard rejects unknown arguments, resolves paths (symlinks included) before checking they stay inside an allowed root, rejects URLs that resolve to private or internal addresses, and passes only the cleaned arguments to the tool. Common mistakes the wiki calls out: checking path text for .. instead of resolving it, and allow-listing hosts with a string prefix test, which docs.example.com.evil.net passes.

4. Runtime internals: a sandbox that is a boundary

One payload in the swarmtraces data ran on Hugging Face's dataset workers and went after /proc/self/environ, where a process's startup environment, often including credentials, lives. That was the victim's worker running attacker-supplied content, and your agent's generated code is the same kind of untrusted input. A process wrapper with resource limits is hygiene. A container is the minimum boundary:

docker run --rm \
  --network none \
  --read-only \
  --tmpfs /tmp:rw,noexec,nosuid,size=64m \
  --cap-drop ALL \
  --security-opt no-new-privileges \
  --pids-limit 64 \
  --memory 256m --memory-swap 256m \
  --cpus 0.5 \
  --user 65534:65534 \
  -v "$JOB_DIR":/job:ro \
  python:3.13-slim \
  python -I /job/main.py

From the wiki's docker-sandbox.sh, with the leading exec omitted.

No network, no capabilities, read-only root, an unprivileged user and nothing secret mounted. Never give the agent or its tools the Docker socket: anyone who can run docker is effectively root on the host. Containers share the host kernel, so for code you don't trust the wiki points to a user-space kernel or a microVM. In production, pin the image by digest, not by tag.

5. Agent spam and runaway loops: budgets, egress and a kill switch

Egress is what turned a sandbox escape into a breach. Send all agent traffic through a proxy that allows named hosts only, and use network policy so the agent pod can reach only the proxy and cluster DNS:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: support-agent-egress
  namespace: agents
spec:
  podSelector:
    matchLabels:
      app: support-agent
  policyTypes: ["Egress"]
  egress:
    - to:
        - podSelector:
            matchLabels:
              app: egress-proxy
      ports:
        - protocol: TCP
          port: 3128
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: kube-system
          podSelector:
            matchLabels:
              k8s-app: kube-dns
      ports:
        - protocol: UDP
          port: 53
        - protocol: TCP
          port: 53

From the wiki's k8s-agent-pod.yaml, comments removed.

Close three gaps before you call this the only way out. First, DNS is still open: cluster DNS will usually resolve any external name, and swarmtraces describes code, run on Hugging Face's dataset workers, built to move data out in DNS queries. The proxy does the external lookups, so point agent pods at a resolver that answers only for internal zones, or use a DNS-aware network policy. Second, this policy does nothing unless your CNI plugin enforces NetworkPolicy, so test it with a blocked connection from inside the pod. Third, set automountServiceAccountToken: false on agent pods. The LOOT script used Kubernetes tokens it found to ask for cluster secrets.

The wiki notes that its Docker command and Kubernetes manifest "were not executed" when the page was written. Check every flag against your runtime and Kubernetes version, and pin images by digest.

Don't allowlist multi-tenant hosts like paste sites or webhook services: anyone can open an account there and receive whatever your agent sends. The proxy limits where the agent connects, not what it sends there. Then add a per-agent rate limit and a per-session action budget (rate_per_minute and max_actions above), and a kill switch the guard checks before any policy lookup, revoking children with the parent.

A kill switch you have never used doesn't count. The wiki's TC-D09-016 test design disables an agent and measures how long each connected system keeps accepting its tokens. Run something like it.

The sixth control: the agent never writes its own log

If a compromised agent can write the audit entry, the entry isn't evidence. Swarmtraces describes requests that deleted files the agents had planted. In the wiki's design the guard, not the agent, logs every decision including denials, each entry carries the hash of the previous one, and the latest hash is stored somewhere the agent can't write. Verify the chain in CI and you'll know when someone edited history. I wrote more about why integrity of the audit trail carries the most weight in AI attackers go after Integrity first.

Where "rogue" fits

The OWASP Top 10 for Agentic Applications has an entry called ASI10 Rogue Agents, for agents that drift from their intended function or scope. OWASP's own mitigations include isolation, least-privilege scopes, kill switches and credential revocation, and the Builder Track maps ASI10 to two steps: register the agent with a named owner, and cap the blast radius with a kill switch. Both are config. Scope is something you set, so it's something you can fix.

Checklist for your next pull request

• [ ] Agent registered with an owner, purpose, tool list and budget
• [ ] No secrets in the agent's environment or within reach of the code it generates
• [ ] Tokens are narrowly scoped, audience-bound, short-lived and narrow-only for sub-agents
• [ ] The gateway rejects any credential your issuer didn't mint
• [ ] Every tool call passes argument validation in code, with unknown fields rejected
• [ ] Generated code runs with --network none, read-only root and no secrets, and nothing gets the Docker socket
• [ ] Egress goes through a proxy to named hosts, enforced by network policy, with DNS limited to internal names
• [ ] Your CNI plugin enforces NetworkPolicy, and agent pods set automountServiceAccountToken: false
• [ ] Rate limit, action budget and kill switch are in place, and the kill switch has a test
• [ ] The guard writes a hash-chained log the agent can't touch, and CI verifies it

Full article, including the ownership and audit-evidence side for your GRC team: https://www.linkedin.com/pulse/agents-dont-go-rogue-scopes-do-ankush-v7off/

---

Licence: the code excerpts above are adapted from the AI Security Wiki Builder Track (page author: Nachiket Sathaye), licensed under CC BY-SA 4.0, and are shared under the same licence. Changes: the two policy.json blocks are single members excerpted from a larger object and reformatted; the pinned definition hash is omitted from the send_email rule; the leading exec is dropped from the docker run command; comments are removed from the NetworkPolicy. OWASP is a trademark of the OWASP Foundation.