DEV Community

Shekhar Kadyan
Shekhar Kadyan

Posted on

Why RAG agents confidently cite outdated docs (and 4 retrieval changes we shipped this week to cut token burn)

One of the most common failure modes in production RAG isn't a bad embedding model—it's retrieving a chunk that is semantically relevant, 100% real, and completely obsolete.

A policy doc from June gets amended in a separate memo, or a Confluence design page links to a Jira ticket that was later marked Won't Fix. Standard vector search retrieves the paragraph from the design doc, sees no status conflict inside that 300-token chunk, and answers with full confidence.

At the same time, if you try to fix this by giving an agent raw MCP tools to search every app on the fly, two things break:

  1. Token and latency bloat: The agent rewrites queries multiple times, pulls entire files into its context window, and burns $0.50+ per question.
  2. Cascading permission risks: As VentureBeat highlighted this week (citing Unit 42's MCP study), chaining 5+ standalone MCP servers into one agent creates a 78.3% cascading-trust attack surface when identity and ACLs aren't enforced at a unified gateway layer.

While working on PipesHub (our Apache 2.0, self-hosted enterprise context layer using Qdrant + a Neo4j/ArangoDB knowledge graph), we shipped four concrete changes over the last 48 hours to tackle both problems:

  • 1. Surfacing linked-record statuses and dates at retrieval time (feat(chat): 4dd9f58): A chunk rarely tells the whole story on its own. Because we maintain a knowledge graph of cross-app links (e.g., a Slack thread or Confluence doc linking to a Jira issue or GitHub PR), retrieval now returns each record's date and status alongside the current status of the records it links to. If a spec links to a ticket that was closed or superseded, the model sees that state immediately without an extra tool hop.
  • 2. Chronological period-by-period agent search (feat(agents): f4aaa41): When an agent needs to reconstruct a timeline ("what changed in our deployment policy between Q2 and Q4?"), single-shot top-K similarity mixes eras together. Stepping through time spans period by period in date order keeps temporal boundaries clean.
  • 3. "Find with less, then read the right part" (feat(retrieval): TARGETED_RETRIEVAL, 9794024): Instead of stuffing full parent documents into the prompt on the first pass, targeted retrieval identifies candidate blocks with minimal token footprint first, then reads only the specific section required to verify the answer.
  • 4. Caller-controlled query rewrite bypass (feat(search): 0c4d7f1): RAG pipelines usually run an internal LLM step to rewrite and expand user queries. That's great for a human typing a 3-word search in a UI, but when an upstream AI agent calls your search endpoint via MCP or SDK, the agent has already expanded the query. Letting the caller skip redundant query rewriting cuts an entire LLM call off the hot path.

Combined with Docling (2.132.0) and pdf.js 4 for page-and-bounding-box visual citations, this architecture reached 93.24 (#1 overall) on EnterpriseRAG-Bench and 92.7% on Google FRAMES.

If you're building self-hosted RAG or MCP infrastructure, all of the code is Apache 2.0:

How are you handling superseded documents and cross-record status checks in your own retrieval pipelines?

Top comments (0)