Why Naive Text Extraction Destroys RAG on Multi-Column PDFs (and How to Fix Reading Order)

When an AI retrieval-augmented generation (RAG) pipeline hallucinates on a question sourced from a scientific paper or financial report, the debugging process almost invariably looks at the wrong layer. Engineers test alternative embedding models, tweak chunk token budgets, or introduce expensive cross-encoder rerankers.

Yet in a surprising fraction of failure cases, the corruption happens before embedding: in the text extraction layer where the parser attempts to reconstruct human reading order from a format that was never designed to store it.

What a PDF File Actually Stores

A PDF is essentially a set of rendering commands designed to place ink or pixels consistently across any screen or printer. Under the hood, text lives inside stream operators bracketed by BT (Begin Text) and ET (End Text):

BT
  /F1 11 Tf
  72 700 Td
  [(Retrieval) -250 (augmented) -250 (generation)] TJ
  0 -14 Td
  (depends on document layout.) Tj
ET

Here, Td moves the coordinate cursor, Tj outputs raw glyph codes, and TJ outputs strings separated by numerical kerning shifts. Crucially:

1. There is no concept of a "sentence" or "paragraph" in an untagged PDF stream.
2. The order of rendering commands in the file reflects the sequence in which the generator tool drew elements—not necessarily how a human reads them.
3. Kerning offsets (like the -250 in the example above) are horizontal spacing nudges, leaving the parser to guess where word spaces belong.

When you pass this document to an automated parser, the tool must guess which glyphs form words, which words form lines, and in what order those lines should be concatenated.

The Multi-Column Braiding Problem

The most damaging extraction error occurs in two-column or multi-column layouts—the standard format for arXiv papers, IEEE publications, and corporate annual reports.

When an extraction library attempts to "fix" raw stream order by sorting text visually by vertical position ($y$-coordinate), it reads across both columns simultaneously. Line 1 of column A is followed immediately by line 1 of column B, then line 2 of column A, then line 2 of column B:

Column A:                          Column B:
Rats were fed a                    All authors declare
standard chow diet                 no competing
for fourteen days.                 financial interests.

Result of naive Y-coordinate sorting:
"Rats were fed a All authors declare standard chow diet no competing for fourteen days. financial interests."

This resulting text is insidious because it never throws a Python exception, passes character-count heuristics, and retains plausible English vocabulary. But when this chunk is sliced into a 512-token window and passed to an embedding model, the vector represents a semantic blend of laboratory protocol and financial disclosures. The vector sits in embedding space near neither topic, making accurate retrieval mathematically impossible.

Three Practical Tiers to Recover Reading Order

Depending on your pipeline constraints and throughput requirements, here are three practical engineering approaches for handling multi-column documents:

Tier 1: Geometric Block Partitioning with Vertical Banding

If you run lightweight CPU-only ingestion using libraries like PyMuPDF (fitz), you can operate on layout bounding boxes (blocks) rather than flat character dumps.

A common naive heuristic splits the page at the horizontal midpoint, but that fails when full-width titles, abstracts, or wide tables appear in the middle of a page. A more robust approach groups blocks into vertical bands separated by spanning elements:

# Practical 2-column reading order parser with vertical band guards
def extract_two_column_page(page, span_threshold=0.75):
    """
    Extracts text from a two-column page while preserving reading sequence.
    Groups text into vertical bands whenever a full-width block appears
    (e.g., page title, abstract, or full-width table/figure), preventing
    mid-page spanning elements from corrupting the sequence.
    """
    # PyMuPDF block format: (x0, y0, x1, y1, text, block_no, block_type)
    # block_type == 0 represents text blocks
    text_blocks = [b for b in page.get_text("blocks") if b[6] == 0]
    if not text_blocks:
        return ""

    page_width = page.rect.width
    midpoint = page_width / 2.0

    # Sort blocks by top coordinate to process page bands top-to-bottom
    text_blocks.sort(key=lambda b: b[1])

    ordered_blocks = []
    current_left = []
    current_right = []

    def flush_columns():
        nonlocal current_left, current_right, ordered_blocks
        if current_left:
            ordered_blocks.extend(sorted(current_left, key=lambda b: b[1]))
            current_left = []
        if current_right:
            ordered_blocks.extend(sorted(current_right, key=lambda b: b[1]))
            current_right = []

    for b in text_blocks:
        x0, y0, x1, y1 = b[0], b[1], b[2], b[3]
        box_width = x1 - x0

        # Spanning elements act as horizontal barriers between column bands
        if box_width >= page_width * span_threshold:
            flush_columns()
            ordered_blocks.append(b)
        elif x0 < midpoint:
            current_left.append(b)
        else:
            current_right.append(b)

    flush_columns()

    return "\n\n".join(b[4].strip() for b in ordered_blocks if b[4].strip())

Note: In production environments with varying margins or 3-column formats, dynamic gutter detection via horizontal projection profiles replaces the static midpoint.

Tier 2: Layout-Aware Vision and Segmentation Models

When documents include irregular multi-column grids, floating callout boxes, and complex multi-row tables, geometric heuristics begin to accumulate edge-case failures.

Layout-aware vision models (such as Docling, Marker, or layout segmentation vision transformers) treat the rendered page as a visual artifact. These architectures predict semantic regions (headings, captions, body paragraphs, tables) and explicitly infer the reading order graph between bounding boxes before converting elements into structured Markdown.

This approach requires significantly higher compute (typically GPU acceleration), but it prevents table rows from collapsing into unstructured text strings.

Tier 3: Producer-Side Tagged PDFs

If your team produces the documents entering the RAG corpus (e.g., internal research memos or documentation exports), the cleanest long-term solution is emitting Tagged PDF (PDF/UA or ISO 32000 tagged structures).

Tagged PDFs embed a structural logical tree directly into the file container, explicitly specifying reading sequence regardless of visual layout positioning. This eliminates heuristic guesswork for both screen readers and downstream AI extractors.

Audit Checklist Before Benchmarking Embedding Models

Before spending budget on new embedding endpoints or vector database re-indexing, run this quick quality audit on your raw text chunks:

1. Spot-check multi-column transitions: Extract 20 random pages from multi-column documents and check whether sentences jump horizontally across columns.
2. Inspect table headers: Confirm whether numbers in parsed output remain linked to their column and row descriptors or have flattened into meaningless lists of floats.
3. Verify formula integrity: Check whether mathematical symbols and sub/superscripts maintain syntactic continuity or fragment across line breaks.
4. Compare coordinate-sorted vs. raw output: A substantial divergence in text sequence often highlights where layout heuristics are struggling.

Clean document structure upstream is the most reliable foundation for downstream retrieval accuracy.