From PDF to Evidence: Building a Production-Ready RAG Pipeline

Sagar — Tue, 29 Sep 2026 03:06:25 +0000

From PDF to Evidence: Designing a Production-Ready RAG Pipeline

RAG is often described as a simple architecture:

Documents → Embeddings → Vector Database → LLM

That diagram is useful for understanding the concept, but it is far from enough for a production document-processing system.

When the source documents are PDFs containing tables, scanned pages, headings, footnotes, forms, and complex layouts, the real challenge is not simply retrieving similar text.

The real challenge is:

Can the system generate an answer that can be traced back to the correct evidence in the original document?

This article describes the architecture and engineering considerations behind a production-oriented RAG pipeline for document processing.

1. The Real Problem Starts Before RAG

Consider a system where users upload contracts, invoices, reports, or other business documents.

A typical workflow looks like this:

PDF
 ↓
Document Intelligence / OCR
 ↓
Layout Analysis
 ↓
Page-aware Document Structure
 ↓
Chunking + Metadata
 ↓
Search Index
 ↓
Hybrid Retrieval
 ↓
LLM
 ↓
Structured Finding
 ↓
Evidence + Citation

The important observation is that retrieval quality depends heavily on document processing quality.

If the original PDF is poorly converted into text, even the best embedding model cannot recover information that was lost during extraction.

2. PDF Extraction Is More Than OCR

A PDF is not necessarily a collection of plain text.

It can contain:

Text
Tables
Images
Scanned pages
Headers and footers
Multiple columns
Forms
Signatures
Page numbers
Different visual sections

Simply extracting all text and splitting it every 500 tokens can destroy important relationships.

For example:

Invoice Amount: ¥12,500,000

Tax: ¥1,250,000

Total: ¥13,750,000

If these values are separated incorrectly during chunking, the LLM may retrieve only part of the information.

Therefore, the ingestion pipeline should preserve document structure whenever possible.

3. Keep Page Information

One of the most important pieces of metadata in a document RAG system is the source page.

Instead of storing only:

{
  "text": "The contract amount is..."
}

I prefer a structure closer to:

{
  "text": "The contract amount is...",
  "document_id": "contract-001",
  "page_number": 14,
  "section": "Contract Amount",
  "chunk_id": "contract-001-page14-chunk03"
}

This gives the retrieval system additional context and, more importantly, allows the generated result to point back to the original source.

For reviewed business documents, this distinction is extremely important.

A generated statement without evidence is difficult to trust.

A generated statement with:

Finding:
The contract amount exceeds the configured threshold.

Evidence:
Contract.pdf — Page 14

is much easier for a human reviewer to validate.

4. Chunking Should Follow Meaning

A common approach is to split documents by a fixed number of tokens.

For example:

Every 500 tokens → one chunk

This is simple, but document structure can be more important than chunk size.

A better strategy can combine:

Headings
Paragraph boundaries
Table boundaries
Sections
Page boundaries
Semantic relationships

For example:

Page 14
 ├── Contract Overview
 │    ├── Contractor
 │    ├── Contract Number
 │    └── Contract Amount
 │
 └── Payment Terms
      ├── Payment Schedule
      └── Conditions

The resulting chunks carry both the content and its context.

5. Hybrid Search Is Often More Practical

Pure vector search is powerful for semantic similarity.

However, business documents often contain exact identifiers:

ABC-2026-00125
¥13,750,000
Project ID: PJ-10293

These values may be better handled by keyword or exact matching.

This is where hybrid search becomes useful.

Conceptually:

User Query
     │
     ├───────────────┐
     ↓               ↓
Keyword Search   Vector Search
     │               │
     └───────┬───────┘
             ↓
       Result Ranking
             ↓
        Top Evidence

Keyword search can capture exact terms.

Vector search can capture semantic meaning.

Combining them gives the system more flexibility across different document types and query patterns.

6. Retrieval Is Not the End

A common mistake is to evaluate a RAG system only by asking:

“Does the LLM produce a good answer?”

That is too late in the pipeline.

We should separately evaluate retrieval.

For example:

Retrieval metrics

Precision

How many retrieved documents are actually relevant?

Recall

How many of the relevant documents did we successfully retrieve?

MRR (Mean Reciprocal Rank)

How high does the first relevant result appear?

These metrics help identify whether a problem comes from:

Retrieval
    ↓
Context construction
    ↓
Prompt
    ↓
LLM generation

Without separating these stages, it becomes difficult to determine why the system is failing.

7. Structured Output Makes Validation Easier

For business applications, free-form LLM responses can be difficult to validate.

Instead of asking the model to return:

I found that the amount appears to exceed...

we can define a structured schema:

{
  "finding": "Contract amount exceeds threshold",
  "severity": "warning",
  "value": 13750000,
  "threshold": 10000000,
  "evidence": [
    {
      "document": "contract.pdf",
      "page": 14
    }
  ]
}

Now the application can validate:

Required fields
Data types
Allowed values
Evidence presence
Citation format

before displaying the result to the user.

This creates a much stronger boundary between the probabilistic LLM layer and the deterministic application layer.

8. Evidence Grounding Should Be a First-Class Feature

One of the most important design decisions is to treat evidence as part of the generated result—not as an optional UI feature.

For example:

Finding
   │
   ├── Generated statement
   │
   ├── Supporting evidence
   │
   ├── Source document
   │
   └── Source page

The application can then allow a reviewer to move directly from the finding to the relevant page.

This creates a human verification loop:

AI Finding
    ↓
Evidence
    ↓
Human Review
    ↓
Approve / Reject / Correct

For systems used in document review, this workflow can be more valuable than simply trying to maximize the amount of text generated by the model.

9. Separate Finding Generation From Evidence Retrieval

Another useful architectural principle is to distinguish between:

What the model thinks is happening

and

What evidence supports that conclusion

For example:

Finding Generation
        ↓
"Amount exceeds threshold"
        ↓
Evidence Retrieval
        ↓
Page 14
        ↓
Source text / table
        ↓
Validation

This allows the application to verify that an AI-generated finding actually has supporting evidence.

It also makes debugging easier.

If a finding is incorrect, we can ask:

Was the document extracted incorrectly?
Was the wrong chunk retrieved?
Was the correct evidence retrieved but ignored?
Did the LLM interpret the evidence incorrectly?
Did schema or validation fail?

That is much more actionable than simply saying:

“The AI hallucinated.”

10. A Production Architecture

Putting these ideas together:

                 ┌─────────────────┐
                 │   PDF Upload    │
                 └────────┬────────┘
                          ↓
                ┌───────────────────┐
                │ OCR + Layout      │
                │ Understanding     │
                └─────────┬─────────┘
                          ↓
                ┌───────────────────┐
                │ Page-aware        │
                │ Document Model    │
                └─────────┬─────────┘
                          ↓
                ┌───────────────────┐
                │ Chunking +        │
                │ Metadata          │
                └─────────┬─────────┘
                          ↓
                ┌───────────────────┐
                │ Search Index      │
                │ Keyword + Vector  │
                └─────────┬─────────┘
                          ↓
                    User Query
                          ↓
                ┌───────────────────┐
                │ Hybrid Retrieval  │
                └─────────┬─────────┘
                          ↓
                ┌───────────────────┐
                │ LLM Generation    │
                │ Structured Output  │
                └─────────┬─────────┘
                          ↓
                ┌───────────────────┐
                │ Evidence +        │
                │ Citation Validation│
                └─────────┬─────────┘
                          ↓
                    Human Review

The LLM is only one component of the system.

The surrounding engineering determines whether the system is reliable enough for production.

11. What I Have Learned

Building AI systems around real-world documents has changed how I think about RAG.

The difficult part is rarely:

“How do I call an LLM?”

The difficult part is designing the entire pipeline around it.

A production system needs to consider:

Document extraction
Layout preservation
Chunking
Metadata
Retrieval
Ranking
Prompt design
Structured output
Validation
Evidence grounding
Evaluation
Monitoring
Human review

Most importantly, the system should make it easy to verify what the AI is saying.

For many enterprise AI applications, trust does not come from the model alone.

It comes from the combination of:

Good retrieval + structured generation + verifiable evidence + human review.

That is the foundation I would use when designing a production RAG system for document-heavy applications.

Final Thought

RAG should not be viewed simply as:

“Give documents to an LLM.”

A better mental model is:

“Build a reliable evidence pipeline around an LLM.”

Once you think about RAG this way, many engineering decisions—from page-aware ingestion to hybrid search and citation validation—become much clearer.

DEV Community: Sagar

From PDF to Evidence: Building a Production-Ready RAG Pipeline

From PDF to Evidence: Designing a Production-Ready RAG Pipeline

1. The Real Problem Starts Before RAG

2. PDF Extraction Is More Than OCR

3. Keep Page Information

4. Chunking Should Follow Meaning

5. Hybrid Search Is Often More Practical

6. Retrieval Is Not the End

Retrieval metrics

7. Structured Output Makes Validation Easier

8. Evidence Grounding Should Be a First-Class Feature

9. Separate Finding Generation From Evidence Retrieval

10. A Production Architecture

11. What I Have Learned

Final Thought