DEV Community

Cover image for A Six-Stage Evidence Pipeline for Enterprise Agentic RAG: Prepare, Plan, Route, Select, Assemble, Answer
shakti mishra
shakti mishra

Posted on AI-assisted

A Six-Stage Evidence Pipeline for Enterprise Agentic RAG: Prepare, Plan, Route, Select, Assemble, Answer

#rag #llm #ai #architecture

Before you blame the model

Maya is a solutions architect. She asks her team's assistant: "Summarize Northstar's latest RFP requirements and show this account's current revenue."

The assistant returns a polished brief. It cites an RFP. It includes a revenue figure. Both citations resolve to real documents.

The RFP is an older revision. The revenue number came from a slide deck, not the finance system.

The answer was grounded in something. Just not the right thing.

The reflex is to spend a week comparing models. A better first question: did the retrieval architecture give any model a fair chance? Check which evidence the model was allowed to see, where it looked, and when that evidence was last updated.

Retrieval is a research desk, not a filing cabinet

A filing cabinet answers one kind of question: "find me documents like this." A research desk knows that some answers live in documents, some in a database, and some only in the system of record at this exact second.

Agentic RAG, in practical terms, means the model can work out what information it needs, choose tools to get it, and try again when the evidence is weak. The application still owns security, access control, validation, execution limits and stopping conditions. That last sentence is what separates it from retrieve-then-generate with a loop bolted on.

You do not need a multi-agent system for this. You need a clear pipeline.

The six stages 

 BACKGROUND (continuous)
 ┌───────────────────────────────────────────────┐
 │ 1. PREPARE  sync, version, permission-tag data│
 └───────────────────────────────────────────────┘
                        │ ready to retrieve
 PER QUESTION           ▼
 ┌────────┐    ┌────────┐    ┌────────┐
 │2. PLAN │──▶│3. ROUTE│──▶ │4.SELECT│
 └────────┘    └────────┘    └───┬────┘
      ▲                          │
      │ insufficient evidence    ▼
      │ (bounded retries)   ┌─────────┐   ┌─────────┐
      └────────────────────│5.ASSEMBLE│──▶│6.ANSWER │
                            └─────────┘   │ + verify │
                                          └─────────┘
Enter fullscreen mode Exit fullscreen mode 
flowchart LR
  P[1 Prepare\nbackground] -.-> R
  Q[Question] --> PL[2 Plan] --> R[3 Route] --> S[4 Select] --> A[5 Assemble] --> AN[6 Answer + verify]
  AN -- gaps found, retries left --> PL

Stage 1 runs continuously. Stages 2 to 6 run per question, and if the evidence is not sufficient the system loops back within defined limits.

Indexed vs. live retrieval, and why MCP does not settle it

Two retrieval modes matter more than any vector database choice:

  • Indexed retrieval searches a maintained copy of source content.
  • Live retrieval queries the authoritative system at request time.

Neither wins automatically. Indexes are fast and good for fuzzy semantic search across documents. Live calls are the only honest answer for "current revenue."

One common confusion: MCP is an interface, not a freshness guarantee. An MCP tool can search a stale index or call a live service, and so can an ordinary REST API. What matters is the data path behind the tool. Document that path per tool.

Stage 1: Prepare

Before embeddings, get the foundation right. For Maya's RFP, track document version, owner, modified date and access permissions as first-class metadata.

{
  "doc_id": "rfp-northstar-2026",
  "revision": "R3",
  "supersedes": "R2",
  "modified": "2026-10-02T09:14:00Z",
  "owner": "bids-team",
  "acl": ["grp:northstar-account", "grp:sales-eng"]
}
Enter fullscreen mode Exit fullscreen mode

Sync tools such as Airweave can keep enterprise sources flowing into a searchable layer, but you still have to verify how permission changes, updates and deletions propagate. A deleted or re-permissioned document that lingers in the index is a data leak with a nice UI.

Technique choice: start with structure-aware chunking plus hybrid (keyword + vector) search, for example on Azure AI Search. Measure on real business questions and add heavier techniques only when the measurements demand it.

Stage 2: Plan

Understand the question before searching. Maya's question hides two retrievals with different data paths, plus an ambiguity: "current revenue" could mean quarterly recognized revenue, ARR, or something else. The agent should ask rather than guess.

For this question, decomposition is enough:

Q: "Northstar's latest RFP requirements and current revenue"
 ├─ sub-query A: RFP requirements | account=Northstar | version=latest
 └─ sub-query B: revenue          | account=Northstar | metric=? (clarify)
Enter fullscreen mode Exit fullscreen mode

Preserve the load-bearing terms ("Northstar", "latest"). If you use HyDE (hypothetical document embeddings), the generated text exists only to improve search. It is never evidence.

The goal of planning is to make the question easier to answer without changing what the user asked.

Stage 3: Route

Not every question belongs in a vector store. Route each sub-query by where the authoritative answer lives:

Sub-query Source Mode
Latest RFP requirements Document store / SharePoint index Indexed, filtered to latest revision
Current recognized revenue Finance system Live

Do not hand the agent unrestricted SQL. Expose a narrow read-only tool:

@tool
def get_recognized_revenue(account_id: str, period: str, currency: str) -> Revenue:
    """Read-only. Returns approved revenue figures for one account."""
    principal = require_identity()                 # who is asking
    authorize(principal, "finance:read", account_id)
    validate_period(period); validate_currency(currency)
    return finance_api.recognized_revenue(account_id, period, currency)
Enter fullscreen mode Exit fullscreen mode

The application validates inputs, verifies identity and permissions, and returns only approved data. With the account and metric settled, the two retrievals can run in parallel.

The key rule: enforce authorization before retrieved content reaches the model or the reranker. Filtering after the fact means the model has already seen what the user should not.

Stage 4: Select

Merge results, drop noise, and rank the most useful evidence the user is allowed to see. I would add three selection checks that directly target Maya's failure:

  1. Version check: if two revisions of the same document are present, keep the newest and flag the rest as superseded.
  2. Authority check: prefer a system of record over a derivative (a slide deck quoting revenue loses to the finance API).
  3. Freshness check: attach a retrieved-at and source-updated-at timestamp to every item.

Stage 5: Assemble

Turn selected evidence into a compact context without losing meaning. Keep provenance attached to each claim so the answer stage can cite it.

<evidence id="e1" source="rfp-northstar-2026@R3" kind="indexed" updated="2026-10-02">
  ...requirement text...
</evidence>
<evidence id="e2" source="finance-api/recognized_revenue" kind="live" retrieved="2026-10-24T12:10Z">
  account=Northstar period=FY26-Q3 value=...
</evidence>
Enter fullscreen mode Exit fullscreen mode

Summarize only when you must, and never strip the metadata that lets a reader audit the answer.

Stage 6: Answer and verify

Generate the response, then check that each claim is supported by assembled evidence, and report gaps explicitly: "I could not find a finance record for the requested period."

The loop-back rule needs hard limits:

MAX_ROUNDS, MAX_TOOL_CALLS, MAX_COST = 3, 8, 0.15
for round in range(MAX_ROUNDS):
    answer, gaps = generate_and_verify(evidence)
    if not gaps or budget_exhausted(): break
    evidence += retrieve_more(gaps)   # re-enters Plan/Route within the same ACL
return answer_with_open_gaps(answer, gaps)
Enter fullscreen mode Exit fullscreen mode

Bounded retries beat an agent that searches forever, and an honest "gap" beats a confident guess.

Where teams go wrong

  • Treating "has a citation" as "is correct."
  • Calling an MCP tool "live" without checking what is behind it.
  • Applying permission filters after retrieval or rerank.
  • Never storing document versions, so "latest" is unanswerable.
  • Letting the loop run without round, call and cost caps.

Key takeaways

  • A grounded answer can still be wrong: the failure is often which evidence was retrieved, not which model generated.
  • Split retrieval into six stages: Prepare, Plan, Route, Select, Assemble, Answer. Prepare runs continuously, the rest per question.
  • Choose indexed vs. live retrieval per question; MCP is an interface, not a freshness guarantee.
  • Enforce authorization before evidence reaches the model or reranker, and expose narrow read-only tools instead of raw SQL.
  • Bound the retry loop and make the system report evidence gaps instead of papering over them.

Your turn

When your RAG system gave a confident wrong answer last, was it a model problem, or could you have even said which source, version and timestamp the evidence came from?

Top comments (0)