DEV Community: Poxek AI

Hugging Face Breached Through a Malicious Dataset: What the Patches Reveal

Poxek AI — Mon, 20 Jul 2026 11:51:02 +0000

On July 16, Hugging Face disclosed a compromise affecting part of its production infrastructure. The initial entry point was a malicious dataset, while the subsequent intrusion was, according to the company, carried out by an autonomous agent framework.

Attack Chain

The dataset abused two code-execution paths: a remote-code dataset loader and template injection in a dataset configuration. Code ran on a processing worker, after which the attacker obtained node-level access, harvested cloud and cluster credentials, and moved laterally into several internal clusters over a weekend.

The agent framework executed thousands of operations across a swarm of short-lived environments, while its command-and-control infrastructure migrated between public services. The underlying LLM remains unknown. There is also no independent confirmation that a human operator was absent from every stage of the campaign.

What the Patches Reveal

Hugging Face has not published a CVE, payload, or exact exploit map. Changes in the public dataset-viewer repository, however, allow for a cautious partial reconstruction.

On July 13, developers updated fsspec and introduced an allowlist. Workers now accept only hf, s3, zip, file, and local; reference, simplecache, data, and every other implementation are removed from the registry. The previously available fsspec.ReferenceFileSystem processed configuration values through unsandboxed jinja2.Template(...).render(...). This is technically consistent with the disclosed template-injection path and a possible SSTI-to-RCE chain, but Hugging Face has not officially tied this code to the intrusion.

That same day, the company hardened the worker pods that process untrusted datasets. The patch disabled Kubernetes ServiceAccount token mounting, enabled seccompProfile: RuntimeDefault, and dropped all additional Linux capabilities. This reduces the blast radius of a compromised worker, but the exact path to node-level access remains undisclosed.

The next changes are consistent with credential rotation. On July 14, a transition to IRSA reached production, removing static S3 keys from the environment. On July 15, MONGO_URL moved from a static password to MONGODB-AWS authentication through IRSA, followed by support for JWT key rotation. These pull requests match the timing and scope of the response, but the company has not presented them as an official postmortem.

Scope and Attribution

Hugging Face confirmed unauthorized access to a limited set of internal datasets and several service credentials. The company has not disclosed their number, privileges, exposure window, or the volume of any possible exfiltration. Its assessment of potential impact on customers and partners remains ongoing.

No evidence of tampering was found in public models, datasets, or Spaces. Hugging Face also verified its container images and published packages as clean.

No threat group has made a verifiable claim of responsibility or produced data samples or other proof of access. No public indicators of compromise—C2 domains, IP addresses, or hashes—have been released either.

Claims linking the incident to JADEPUFFER, naming OpenAI or Anthropic as the providers used during the investigation, calling it the “first fully autonomous breach,” or citing 4,200 stolen tokens and 1,800 private models remain unsupported by the available sources.

How Hugging Face Investigated the Intrusion

Telemetry correlation supported by LLM-based triage initially surfaced the attack. Analysis agents then processed more than 17,000 events, reconstructed the timeline, extracted indicators for the internal investigation, mapped affected credentials, and separated genuine attacker activity from decoys.

Commercial models blocked requests containing real attacker commands, exploit payloads, and C2 artifacts. Hugging Face moved the forensic analysis to a locally deployed GLM 5.2 model from Z.ai, keeping logs and exposed secrets inside its own infrastructure. GLM was used by the defenders; the attacker's model has not been identified.

Hugging Face closed both code-execution paths, rebuilt compromised nodes, revoked affected tokens, began a broader secret rotation, and strengthened cluster admission controls. The company recommends that users rotate their Hugging Face tokens and review recent account activity.

Sources

The Supply Chain Attack Vector Everyone Is Ignoring in AI Agents

Poxek AI — Tue, 30 Jun 2026 13:03:09 +0000

Most conversations about securing AI agents still revolve around prompt injection as if it’s purely a model problem. “Sanitize the input.” “Add better guardrails.” “Use a stronger system prompt.”

This framing misses where some of the most effective attacks are actually happening.

In recent demonstrations, autonomous agents were compromised through poisoned configuration files and code in repositories. Malicious instructions placed in what the agent treats as trusted source material caused it to harvest cloud credentials, enumerate internal infrastructure, and extract CI/CD keys — all without any direct manipulation of the model’s reasoning through user input. The agent simply did what it was built to do: read the code/config in its environment and act on it.

This is indirect prompt injection delivered through the supply chain.

Why This Is Different
Traditional prompt injection assumes the attacker has to reach the model through the “user” channel. The poisoned repository approach bypasses that entirely.

The agent has legitimate permission (often necessary for its function) to read from repositories, configuration files, or dependency manifests. Once those sources are compromised, the agent becomes an unwitting executor of attacker instructions.

This is not a new class of bug. It’s the same supply chain and trust issues that have plagued software development for years, now weaponized against systems that can act autonomously.

We saw similar patterns in 2025 with incidents like:
• Cline: a crafted GitHub issue title turned an authenticated coding session into a package installer affecting ~4,000 machines.
• LiteLLM: a backdoored release on PyPI that was pulled ~47,000 times in three hours.
• MCP servers: ~200,000 exposed with no authentication by design.
In each case, the compromise didn’t require breaking the AI model. It required abusing the authority the agent already possessed because of how the surrounding system was designed.
The Guardrail Blind Spot

Current defensive tooling for agents largely focuses on the prompt layer and tool-use restrictions. These are useful, but they assume the data the agent consumes is relatively clean or at least auditable in real time.

When the poison lives in a Git repository, a config file the agent
is expected to load, or a dependency it autonomously pulls, those assumptions collapse.

Many teams still treat “our repo” as a trusted boundary. That boundary is disappearing the moment agents start making decisions based on what they read there.

Practical Reality Check

If your agent can:
• Read code/config from external or even internal repositories
• Execute or act on what it reads
• Trigger pipelines, modify files, or call APIs

…then you have a supply chain attack surface that traditional application security controls were never designed to protect against autonomous execution.

Signing commits helps. Pinning dependencies helps. But these are partial measures. An agent operating at scale will eventually encounter poisoned or malicious content that looks legitimate enough to act on.

What Actually Moves the Needle

From an offensive security perspective, the teams making progress are treating every external (and many internal) sources an agent reads as untrusted by default. They are:
• Implementing provenance and integrity checks before agents act on code or config
• Severely limiting what an agent can do even when operating on “trusted” sources
• Monitoring for behavioral anomalies when agents interact with repositories or dependencies
• Designing workflows where high-impact actions require explicit confirmation rather than autonomous execution

The uncomfortable truth is that many current agent architectures were built by teams optimizing for capability first and security second. That order is now creating exactly the conditions for supply chain attacks to succeed at machine speed.

The question isn’t whether poisoned repositories will become a standard attack vector against agents. They already are.

The real question is whether your agent design assumes the code it consumes is safe — or whether it assumes the opposite.