In August 2024, researchers demonstrated a silent attack against Microsoft 365 Copilot. Any user with write access to an indexed SharePoint folder could alter the system's answers across the entire organization. No changes to the model, system prompt, or any runtime component were required.
This is not prompt injection. The adversarial content was indexed and embedded before any query existed. The taxonomy matters because it determines where the defense must be built, and right now it is being built in the wrong place.
The injection point is indexing time, not query time
Prompt injection and RAG poisoning share a surface-level similarity: both manipulate what the LLM processes. The structural difference is where this happens in the pipeline.
Prompt injection targets the context window at inference time. RAG poisoning targets the knowledge store at ingestion time. The adversarial content is already embedded in the vector database before any query exists.
Greshake et al. (arXiv:2302.12173, 2023) classified indirect prompt injection as adversarial content retrieved and acted upon by an LLM. RAG poisoning extends that model: the attacker controls what gets indexed, not just what gets retrieved. OWASP formalized this as LLM08:2025 (Vector and Embedding Weaknesses), a distinct category absent from the 2023 Top 10.
The poisoned chunk survives model updates, system prompt changes, and API key rotations. It persists until the vector store is explicitly cleaned.
Five documents can hijack a million-document knowledge base
PoisonedRAG (Wei Zou et al., USENIX Security 2025, arXiv:2402.07867) demonstrated a fundamental property of vector similarity search. A small number of optimized adversarial documents can dominate retrieval results even in very large corpora.
The attack achieved 90% success rate injecting just 5 malicious texts into a corpus of millions. Separate research showed that poisoning 0.04% of a corpus produces 98.2% attack success and 74.6% system failure rate. Existing defenses including perplexity filtering, paraphrasing, and LLM-based detection were evaluated and found insufficient.
AgentPoison (NeurIPS 2024) achieved at least 80% attack success rate across 3 classes of real-world LLM agents. The poison rate was under 0.1% and no model fine-tuning was required. The attack is formulated as an optimization problem to maximize cosine similarity with target queries, with both black-box and white-box variants demonstrated.
Three attack surfaces in enterprise RAG deployments
Enterprise RAG deployments have 3 distinct injection surfaces, each requiring specific mitigations.
Direct document upload: ConfusedPilot (arXiv:2408.04870, DEF CON AI Village 2024) demonstrated the direct upload vector. Write access to any indexed folder is sufficient to poison Copilot responses organization-wide. The study noted that 65% of Fortune 500 companies use or plan to use RAG-based AI systems. The effect persists after malicious content removal due to index caching.
Web crawl: RAG systems that index external content can be poisoned by planting adversarial text on crawled pages. The effect lasts until the next re-crawl. The ChatGPT browsing exploitation in May 2024 documented this vector in a production environment.
Embedding collision: The Snyk RAGPoison technique adds vectors that surround a legitimate document in embedding space. Any query that would have retrieved the legitimate document now retrieves the poisoned content. In November 2024, Lasso Security discovered that deleted GitHub repositories from IBM, Google, Tencent, and PayPal persisted in Bing's RAG cache. These repositories surfaced through Copilot months after removal, showing that knowledge base cleanup is non-trivial.
AgentPoison: when the memory store is the attack surface
For autonomous LLM agents, the knowledge base is not just a reference source. It is also the episodic memory that informs future actions.
AgentPoison (NeurIPS 2024) planted backdoor trigger tokens mapped to a unique region in embedding space. Queries containing the trigger retrieve the poisoned demonstration; benign queries return normal results. The attack was tested across 3 agent classes: a RAG-based autonomous driving agent, a knowledge-intensive QA agent, and a healthcare EHRAgent. All were compromised.
The result is not information corruption. It is behavioral control: the attacker determines which actions the agent takes when the trigger appears in a future query. A 2026 study (arXiv:2606.04329) extended this model to episodic and procedural memory modules beyond vector retrieval.
A worm vector follows directly from this. The agent reads the poisoned document and takes an action that updates another document. The next agent that reads the updated document is also compromised.
Output filtering and system prompt hardening operate at the wrong layer
The standard AI security toolkit provides zero defense against RAG poisoning. This includes output sanitization, system prompt hardening, content classifiers, and RLHF safety training. All of these controls execute after retrieval has already occurred.
Poisoned content reaches the context window as retrieved facts, not as user instructions. Safety training and system prompt restrictions do not classify it as adversarial. Output classifiers cannot distinguish between the LLM accurately summarizing poisoned retrieved documents and the LLM hallucinating.
EchoLeak (CVE-2025-32711) was the first zero-click exploit in a production AI system. The attack exfiltrated OneDrive, SharePoint, and Teams data from Microsoft 365 Copilot without user interaction. Runtime defenses were insufficient. The persistence property is structurally identical to dependency poisoning in software supply chains. Like a compromised npm package, a poisoned RAG chunk is distributed automatically to all downstream consumers.
Detection lives at the ingestion pipeline, not the query pipeline
Effective RAG poisoning defense requires pre-indexing controls. Controls at the query pipeline arrive too late.
The TrustRAG approach uses multi-source consensus scoring at retrieval time. A retrieved claim requires corroboration from N independent sources before it is accepted. The MAGO Intel tool (intel.mago.team) monitors enterprise AI knowledge bases for anomalous retrieval patterns, flagging queries where low-credibility or recently-added sources consistently rank above established authoritative documents.
Cordon-MAS (arXiv:2605.26754, 2025) treats knowledge base writes as privileged operations requiring attestation, applying information-flow control in multi-agent systems. MemAudit (arXiv:2605.23723, 2025) provides causal attribution and structural anomaly detection for poisoned agent memory, usable for post-incident forensics.
The control that prevents RAG poisoning is not a model safety feature or a runtime filter. It is write-access governance on the indexing pipeline, the same primitive that prevents supply chain compromise in software. If the vector store has the same access controls as a public S3 bucket, the model's safety training is irrelevant.
Top comments (0)