Everyone says RAG fixed hallucination. The data tells a different story. In a preregistered study of more than 200 legal queries, three commercial retrieval-augmented research tools hallucinated between 17% and 33% of the time (Source: Magesh et al., Journal of Empirical Legal Studies, 2025). In the Philippines, the National Privacy Commission already ruled in Advisory No. 2024-04 that the Data Privacy Act of 2012 applies across the full lifecycle of AI systems that process personal data, from dataset collection onward (Source: NPC Advisory No. 2024-04, 2024).
RAG Reduces Hallucination. It Does Not Eliminate It
The same study ran three vendors on one dataset of more than 200 preregistered legal queries. Lexis+ AI answered 65% of queries accurately, Westlaw AI-Assisted Research was accurate 42% of the time, and Ask Practical Law AI gave incomplete answers on more than 60% of queries (Source: Magesh et al., Journal of Empirical Legal Studies, 2025).
Same category, three failure shapes: one under-answers, one invents, one stalls. None leaves a visible marker that lets a reviewer separate a grounded answer from a confident invention.
The study documented a case where Westlaw claimed a paragraph of the Federal Rules of Bankruptcy Procedure states that deadlines are jurisdictional, and no such paragraph exists (Source: Magesh et al., Journal of Empirical Legal Studies, 2025). Chief Justice John Roberts named hallucinations as a barrier to AI in legal practice in his 2023 year-end report (Source: Magesh et al., 2025).
Three Ways a Grounded Answer Goes Wrong
Retrieval-augmented generation does not fail loudly, it fails plausibly. Three paths account for most of it: the system retrieves the wrong evidence, it retrieves the right evidence and ignores it, or it retrieves nothing useful and answers anyway (Source: DevTo, 2025).
The second path gets missed, because the correct passage is sitting right there in the context window, retrieved and unused.
At the 15-million-document scale the write-up designs for (Source: DevTo, 2025), the first path stops looking like an error. A decoy passage that reads more authoritative than the true one is simply the passage the retriever scores highest. Hallucination rate becomes a function of how many plausible wrong answers are available, not how good the model is.
The Evidence You Retrieved Is Not the Evidence You Used
Retrieval is not usage. Models were evaluated on two tasks that require locating relevant information inside a long input: multi-document question answering and key-value retrieval (Source: Liu et al., TACL, 2023). Performance was highest when the relevant information sat at the beginning or the end of the context, and degraded significantly when it sat in the middle, even for models built explicitly for long context (Source: Liu et al., TACL, 2023).
Chroma widened the test across 18 LLMs at 8 input lengths and 11 needle positions, running 194,480 total LLM calls with a refusal rate of 0.035% (Source: Chroma, 2025). As needle-question similarity drops, degradation accelerates with input length (Source: Chroma, 2025).
The worrying part is the refusal rate. A model that knew it was lost would refuse. At 0.035% across 194,480 calls, essentially none did, so the decay is silent and the output looks like a confident answer (Source: Chroma, 2025).
The Corpus Is Regulated Data
A corpus assembled by scraping is not a neutral pile of text. NPC Advisory No. 2026-01, dated 13 April 2026, governs scraping of publicly available personal data and requires a valid lawful basis, adequate and necessary processing, privacy notices, a privacy impact assessment, and appropriate security measures (Source: NPC Advisory No. 2026-01, 2026). Public availability is not consent (Source: Baker McKenzie, 2026).
That attaches a legal dependency chain to the retrieval layer: lawful basis at collection, a defined and legitimate purpose, a documented privacy impact assessment, and provenance that tells you which document a claim came from. The National Privacy Commission put it plainly, the use of AI does not remove the obligations imposed by the Data Privacy Act (Source: NPC Advisory No. 2024-04, 2024). A corpus you cannot prove you had the right to assemble is a corpus you cannot defend when a data subject asks.
Build the Pipeline, Not the Prompt
Telling a model to answer only from the provided context is a wish, not a control (Source: DevTo, 2025). What works is a set of gates that run before a token reaches the user.
- Retrieve. Pull candidate passages and keep their identifiers attached.
- Constrain. Make the model answer only from the retrieved set, never from memory.
- Verify. Check every claim against the specific passage it cites.
- Abstain. Return "not supported by the evidence" when no passage carries the claim.
Near-zero hallucination is a pipeline property, not a prompt (Source: DevTo, 2025). That is why we treat this as a measurement and cognition problem rather than a model-size problem, and it is the framing we use at Yano.AI: a system that can prove its evidence and is allowed to abstain answers a different question than one that generates well.
FAQ
Q: Does retrieval-augmented generation still hallucinate?
A: Yes. Three commercial legal research tools hallucinated between 17% and 33% of the time on a preregistered dataset of more than 200 queries (Source: Magesh et al., Journal of Empirical Legal Studies, 2025).
Q: Can a better prompt remove the problem?
A: No. An instruction to stay inside the provided context is a request, not an enforcement mechanism. Verification and abstention need their own stages outside the model (Source: DevTo, 2025).
Q: Is a scraped document corpus a privacy problem in the Philippines?
A: Yes. NPC Advisory No. 2026-01 requires a lawful basis, adequate and necessary processing, privacy notices, a privacy impact assessment, and security measures for scraping publicly available personal data (Source: NPC Advisory No. 2026-01, 2026).
Q: What does context rot mean for a large document set?
A: Performance degrades as input length grows, and it degrades unevenly. Chroma found consistent decay across 18 LLMs at 8 input lengths and 11 needle positions (Source: Chroma, 2025).
Key Takeaway
RAG moved the failure point. It did not remove it. The systems that survive a corpus the size of a country's public record can name the document, the page, and the paragraph behind every claim, and say no when the evidence is not there.
So the next time an assistant answers, ask the one question no prompt can absorb: which document did this come from, and what would you have said if that document said something different?
Sources
- Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools (PDF)
- Hallucination-Free? - Stanford Law Publications
- Lost in the Middle: How Language Models Use Long Contexts
- Context Rot: How Increasing Input Tokens Impacts LLM Performance
- NPC Advisory No. 2024-04: Guidelines on the Application of the Data Privacy Act of 2012 to AI Systems Processing Personal Data (PDF)
- NPC Advisories and Circulars index
- Philippines: NPC Tightens Rules on Data Scraping - Baker McKenzie
- A Correct Answer and a Confident Lie Look Identical (DevTo)
Top comments (0)