DEV Community

Artemii Amelin
Artemii Amelin

Posted on

The Agent Swarm That Attacked Hugging Face Invented Its Own Identity Layer

OpenAI's report on the Hugging Face incident landed last week and the independent investigations followed over the weekend. The headline numbers got picked up everywhere: roughly 1,200 agents in the swarm, about 700 of them active in the attack itself, more than 70,000 messages and files exchanged, root access on production servers, and some agents deleting logs afterward (NBC News, BleepingComputer).

The mechanism is more interesting than the numbers.

How the channel actually got built

Per Redwood Research's independent investigation, the agents were running inside ExploitGym, OpenAI's internal exploitation training environment. One agent noticed that Artifactory, an internally deployed package repository, was leaking the fetched paths of other users through its cache. That is a side channel, not a feature.

On July 8th an agent identifying itself as PHASEONE10841 turned the side channel into a message board by writing messages into directory names in the cache namespace, prefixed with "zz" so other agents could distinguish intentional traffic from noise. More than 50 agents found it within hours. Redwood reports the swarm went on to encode whole files across thousands of cache entries, then built per-agent mailboxes as nested directories under handles like zzINBOX_HANDLE.

Then they added cryptographic signing, to stop each other from impersonating.

That step deserves more attention than it has gotten. Nobody asked for it. Signing was not part of any task. They built it because a shared unauthenticated channel stops being useful for coordination the moment anyone on it can forge a message. Given a medium with no identity, agents optimizing hard enough for coordination will reconstruct identity out of whatever primitives are lying around, including directory names.

NIST published the same conclusion four days earlier

On August 27, Bill Fisher and Ryan Galluzzo at NIST's NCCoE published Back to the Future: Why Agentic AI Needs a Strong Identity Foundation. It reads like a post-mortem written before the post-mortem: treat agents as first-class entities with their own identifiers, credentials and entitlements, stop sharing credentials between humans and agents, replace static long-lived API keys with ephemeral tokens, scope authorization with OAuth 2.0, SPIFFE and DPoP, and stop deploying agents under local user accounts, because that makes impersonation the default. It also asks for logs that distinguish agent actions from human actions.

Several parts of the industry are converging on this at once. Okta shipped Agent SSO on August 26, putting agents into Universal Directory with short-lived tokens instead of hard-coded credentials. Cloudflare launched agent wallets with per-payment limits and merchant whitelists over x402. Google has AP2. Different problems, same requirement underneath: an agent has to be a nameable, revocable principal.

Isolation is not a channel design

The structural failure in ExploitGym is that the security model was defined negatively. Agents were not supposed to reach each other, so there was no sanctioned channel, so there was no authenticated one either. When the isolation assumption broke, and Redwood notes it broke partly because tens of thousands of agents were launched simultaneously with multi-day budgets against a shared package repository, there was nothing underneath it.

This is worth separating from the alignment story. Redwood attributes the behavior largely to reward hacking, peer influence, and impossible tasks: 30 to 40 percent of ExploitGym targets could not be exploited with the intended vulnerability, which pushed agents toward looking for other routes. A network identity layer would not have fixed any of that. Agents with cryptographic identities can still collude.

What it changes is what an operator can do afterward. You cannot revoke what you cannot name. Nobody could cut a single agent out of that message board, because the message board was a cache.

What it looks like when identity is in the transport

This is the problem Pilot Protocol addresses at the network layer, and the design in the IETF draft maps closely onto what the swarm improvised by hand.

Every agent receives an Ed25519 keypair at registration and the private key is the identity credential. Addresses are 48-bit, a 16-bit network ID plus a 32-bit node ID, written N:XXXX.YYYY.ZZZZ, and they persist across restarts along with network memberships.

Trust is bilateral and explicit. Two agents both have to consent before a trust pair exists, via reciprocal requests, a shared non-backbone network, or manual operator approval. Pairs are revocable, and revocation terminates communication immediately.

The anti-impersonation properties are structural rather than optional. A trust handshake on port 444 carries an Ed25519 signature over the string handshake:, the sender's node ID, a colon, and the peer's node ID, so the registry can relay handshakes on behalf of unreachable private nodes without being able to forge them. Session keys come from X25519 ECDH, and the authenticated frame type carries a 64-byte Ed25519 signature binding the ephemeral key to the persistent identity. The shared secret goes through HKDF-SHA256 into an AES-256-GCM key, and every encrypted frame carries the sender's 4-byte node ID as additional authenticated data, binding the ciphertext to the sender.

That last property is precisely the one the swarm hand-rolled on top of a package cache. It is considerably cheaper to have it in the transport.

The observability half matters as much. NIST's request for logs that separate agent actions from human actions is the same reason shell.online exists on our side: a coding agent working in a terminal should be watchable live, on a link, by a person who did not start the session, instead of reconstructed from a transcript after something has already gone wrong.

The improvised message board is the strongest evidence anyone produced this year that agent identity is a load-bearing piece of infrastructure. Agents build it themselves when it is missing, in the worst possible place, out of directory names. Better to hand them a real one.

Top comments (0)