PDF Parsing Is the Hidden Bottleneck in Your RAG Pipeline

When a RAG system returns hallucinatory or plain wrong answers, our first instinct is usually to blame the retriever or swap the LLM for a bigger one.

However, while stress-testing an end-to-end RAG pipeline on table-heavy Korean documents, I realized the real culprit was located one stage earlier: the PDF parser.

If your parser shreds a table into a stream of flat text, no amount of prompt engineering or high-dimensional embeddings can recover the lost structural context.

---

How Bad Parsing Destroys Retrieval

1. Context Pollution: When basic parsers inject broken encoding, missing whitespace, or garbage metadata into text chunks, embedding models end up vectorizing noise. Retrieval precision drops immediately.
2. Structure Collapse: This is the ultimate killer for table-heavy PDFs. Flattening a multi-column table destroys the explicit relationships between headers, cells, and values. The LLM then hallucinates connections between unrelated numbers.

In my test cases, every failure regarding numeric thresholds traced directly back to a table that had been flattened at ingestion.

---

Benchmarking Local PDF Parsers (Korean + Tables)

I benchmarked standard open-source tools against complex Korean PDF documents containing merged cells and multi-line headers:

Parser / Library	Table Structure Preservation	Korean Encoding	Security / Local	Notes
PyPDFLoader (LangChain)	⭐	❌ Frequent corruption	✅ Fully local	Fast, but treats table text as a raw line-by-line dump. Structure is completely lost.
pdfplumber	⭐⭐⭐⭐	✅ Clean	✅ Fully local	Pure Python. Native extract_tables(), allows custom spatial layout & coordinate-based extraction.
LlamaParse / Commercial APIs	⭐⭐⭐⭐⭐	✅ Clean	❌ External API	Incredible extraction accuracy, but sending sensitive documents to third-party APIs was a non-starter for security.

Why I Built a pdfplumber Pipeline

• Structure-Aware: It preserves row/column coordinate boundaries, allowing us to reconstruct markdown tables accurately.
• Local & Lightweight: No external API dependency, zero risk of data exposure.

---

Overcoming pdfplumber Limitations with Post-Processing

While pdfplumber is far better than raw text dumps, it isn't magic out of the box. I had to implement a few custom post-processing rules to make the output LLM-ready:

import pdfplumber

def extract_and_clean_tables(pdf_path):
    cleaned_tables = []
    with pdfplumber.open(pdf_path) as pdf:
        for page in pdf.pages:
            tables = page.extract_tables()
            for table in tables:
                # 1. Handle merged cells returning as None
                # 2. Normalize in-cell line breaks (\n -> " ")
                cleaned_table = []
                for row in table:
                    cleaned_row = [
                        (cell.replace("\n", " ") if cell else "")
                        for cell in row
                    ]
                    cleaned_table.append(cleaned_row)
                cleaned_tables.append(cleaned_table)
    return cleaned_tables

• Merged Cell Handling: Blank-fill None values left behind by merged spans.
• In-cell Newlines: Replace internal \n with spaces so multi-line text cells don't fake a new row boundary.

---

Results: Golden Set Evaluation

I re-ran a test set of domain-specific queries against both pipelines:

Query Type	Default PyPDFLoader Pipeline	Custom pdfplumber Pipeline
Numeric Thresholds (Reference tables)	❌ Failed (retrieved wrong row)	✅ Exact Match
Formula / Term Definitions	❌ Failed (text mashed together)	✅ Exact Match
Recurring Schedule Values	⚠️ Partial / Hallucinated	✅ Exact Match

---

Key Takeaways

• Parsing quality IS retrieval quality: Don't waste weeks tuning chunk sizes or fine-tuning embeddings if your input text is already corrupted at parsing time.
• Minimize context pollution: Preserve table boundaries (e.g., convert to clean Markdown tables before chunking) to give the retriever and LLM the best chance of success.

---

Next Steps

Exploring local open-source Vision Language Models (e.g., Qwen2-VL or Florence-2) to parse extremely complex, nested image-based tables without sending data off-premise.