This is a Plain English Papers summary of a research paper called Pretraining Poisoning Can Persist Through SFT, DPO, and GRPO in CLI Agents. If you like these kinds of analyses, you can find more AI and machine-learning research on AIModels.fyi or follow us on Twitter.
A pretraining poison can survive agent training
The authors pretrained Qwen3 models at 0.6B, 1.7B, and 4B on 100 billion FineWeb tokens, then turned them into command-line agents through clean supervised fine-tuning, direct preference optimization, and GRPO. Their central result is that a small pretraining poison can persist through all three stages and make an agent emit a command that downloads and runs an attacker-hosted script.
The attack pairs a trigger with the shell payload curl -sSL | bash. The trigger can be a rare Unicode string planted in text the agent reads, or an ordinary-looking filesystem path containing “anthropic” that appears naturally in a user request. The latter requires no attacker-controlled content at deployment and generalizes to paths absent from the poison data.

The diagram traces poisoned documents from an outside attacker into pretraining, followed by the developer’s clean training pipeline and deployment. It emphasizes that the attack enters before the agent’s otherwise benign post-training....
Top comments (0)