If your RAG pipeline answers questions about PDFs badly, look at the chunks before you tune the embeddings or the prompt. A lot of retrieval problems start at extraction: two-column pages read straight across, tables turned into a stream of numbers, and headings that become ordinary lines of text. Then you spend an afternoon cleaning text by hand before you can even chunk it.
This post covers what goes wrong, why converting to Markdown (not plain text) fixes most of it, how the common open-source libraries compare, and a small Python script you can run locally to see the difference.
Three things plain-text extraction breaks
1. Reading order in two-column layouts. Most extractors work line by line, top to bottom. On a two-column paper, line 1 of the left column is followed by line 1 of the right column. Every chunk then mixes two unrelated passages, and the embedding ends up halfway between them.
2. Tables. A table flattened into text keeps the values and loses the structure. "Q1 Q2 Q3 / 12 15 9" no longer tells the model which number belongs to which column. A Markdown table (| Q1 | Q2 | Q3 |) keeps the header attached to each row, and LLMs read that format well.
3. Headings. Headings are the best chunk boundaries you have, and they're also useful metadata ("this chunk is from 3.2 Limitations"). In plain text they look like any other line. In Markdown they're #/## lines, so a splitter such as LangChain's MarkdownHeaderTextSplitter or a 10-line function can cut on them.
CJK documents add one more problem: justified Chinese text is often stored with a space between every character, and line breaks land mid-sentence. Both get in the way of keyword search and chunk-size math.
The options, briefly
All of these are real, maintained projects. Check licenses yourself before you ship; the ones below are from PyPI/GitHub metadata as of October 2026.
| Library | Approach | Good at | Watch out for | License |
|---|---|---|---|---|
| pypdf | Pure-Python text extraction | Zero dependencies, fast | No layout, tables or headings | BSD-3 |
| pdfminer.six | Layout analysis (chars → lines → boxes) | Fine-grained positions, fonts, sizes | You build structure yourself | MIT |
| pdfplumber | Built on pdfminer.six | Ruled-table detection, easy word/char access | Borderless tables need tuning | MIT |
| PyMuPDF / pymupdf4llm | MuPDF engine, Markdown output | Fast; Markdown output ready to use | AGPL-3.0 or commercial license | AGPL / commercial |
| Docling (IBM) | ML layout + table models | Complex layouts, many formats | Heavier install, model downloads | MIT |
| Marker | ML pipeline, optional LLM | Hard layouts, equations | GPU helps; check model-weight terms | Apache-2.0 (code) |
| MinerU | ML pipeline, OCR | Scanned docs, Chinese | Heavy; custom license | MinerU license |
| unstructured | Element partitioning | Many file types, element metadata | Hi-res mode pulls in a lot | Apache-2.0 |
| MarkItDown (Microsoft) | Converts many formats to Markdown | Office files, quick conversions | PDF handling is lighter than PDF-specific tools (uses pdfminer/pdfplumber) | MIT |
A rough rule of thumb: if your PDFs have a text layer (born-digital reports, papers, government documents), a layout-aware rules approach on pdfminer.six/pdfplumber is light, deterministic and cheap to run. If they're scanned or very visual, you need OCR or an ML pipeline (Docling, Marker, MinerU), and you pay for it in compute.
A runnable example: two columns + a table → Markdown → chunks
We'll generate a small sample PDF so you don't need any files, then compare extract_text() with a ~90-line converter.
python -m venv .venv && source .venv/bin/activate
pip install pdfplumber==0.11.10 fpdf2==2.8.9
make_sample.py builds a page with a title, two columns, and a ruled table:
# make_sample.py - build a small two-column PDF with headings and a ruled table
from fpdf import FPDF
LEFT = ("Retrieval quality depends on chunk boundaries. When one chunk mixes "
"two sections, its embedding blurs both topics. Splitting at headings "
"keeps each chunk about one thing.")
RIGHT = ("Tables are worse. Flattened into plain text, a row of values "
"loses its column headers, so the model cannot tell which number "
"belongs to which metric.")
pdf = FPDF(format="A4")
pdf.add_page()
pdf.set_font("Helvetica", "B", 20)
pdf.cell(0, 12, "Chunking PDFs for RAG", new_x="LMARGIN", new_y="NEXT")
y = pdf.get_y() + 4
for x, title, body in [(10, "Why boundaries matter", LEFT), (110, "Why tables break", RIGHT)]:
pdf.set_xy(x, y)
pdf.set_font("Helvetica", "B", 14)
pdf.multi_cell(90, 8, title)
pdf.set_x(x)
pdf.set_font("Helvetica", "", 10)
pdf.multi_cell(90, 5, body)
pdf.set_y(150)
pdf.set_font("Helvetica", "B", 14)
pdf.cell(0, 8, "Formats compared", new_x="LMARGIN", new_y="NEXT")
pdf.set_font("Helvetica", "", 10)
with pdf.table(col_widths=(60, 40, 40)) as table: # fpdf2 draws ruled borders
for row in [("Output", "Headings", "Tables"), ("Plain text", "lost", "flattened"), ("Markdown", "# / ##", "pipe table")]:
r = table.row()
for cell in row:
r.cell(cell)
pdf.output("sample.pdf")
print("wrote sample.pdf")
pdf_to_md.py does four things: groups words into line fragments (splitting at the column gutter), orders them with a recursive XY-cut (split at a vertical gutter if one exists, otherwise at the widest horizontal gap), marks headings by font size relative to the page's median size, and swaps ruled tables for Markdown tables using pdfplumber's find_tables().
# pdf_to_md.py - minimal text-layer PDF -> Markdown -> heading-aware chunks
import statistics, sys
import pdfplumber
def lines_of(words, gap=12):
"""Group words into line fragments; a wide horizontal gap (a column gutter) starts a new fragment."""
rows = {}
for w in words:
rows.setdefault(round(w["top"]), []).append(w)
frags = []
for ws in rows.values():
ws.sort(key=lambda w: w["x0"])
cur = [ws[0]]
for w in ws[1:]:
if w["x0"] - cur[-1]["x1"] > gap:
frags.append(cur); cur = []
cur.append(w)
frags.append(cur)
return [{"x0": f[0]["x0"], "x1": f[-1]["x1"], "top": min(w["top"] for w in f),
"bottom": max(w["bottom"] for w in f), "size": f[0]["size"],
"text": " ".join(w["text"] for w in f)} for f in frags]
def gaps(intervals):
intervals = sorted(intervals); out = []; end = intervals[0][1]
for a, b in intervals[1:]:
if a > end:
out.append((end, a))
end = max(end, b)
return out
def xy_cut(items, min_gutter=7):
"""Recursive XY-cut: split at a column gutter if there is one, else at the widest horizontal gap."""
if len(items) <= 1:
return items
width = max(i["x1"] for i in items) - min(i["x0"] for i in items)
cols = [g for g in gaps([(i["x0"], i["x1"]) for i in items]) if g[1] - g[0] >= min_gutter]
for a, b in sorted(cols, key=lambda g: g[0] - g[1]): # widest gutter first
left = [i for i in items if i["x1"] <= a]; right = [i for i in items if i["x0"] >= b]
if min(max(i["x1"] for i in left) - min(i["x0"] for i in left),
max(i["x1"] for i in right) - min(i["x0"] for i in right)) > 0.2 * width:
return xy_cut(left) + xy_cut(right)
rows = gaps([(i["top"], i["bottom"]) for i in items])
if not rows:
return sorted(items, key=lambda i: (i["top"], i["x0"]))
a, b = max(rows, key=lambda g: g[1] - g[0])
return xy_cut([i for i in items if i["bottom"] <= a]) + xy_cut([i for i in items if i["top"] >= b])
def page_to_markdown(page):
tables = [t for t in page.find_tables() if len(t.extract()) >= 2] # ruled tables only
def in_table(w):
return any(t.bbox[0] <= w["x0"] and w["x1"] <= t.bbox[2] and
t.bbox[1] <= w["top"] and w["bottom"] <= t.bbox[3] for t in tables)
words = [w for w in page.extract_words(extra_attrs=["size"]) if not in_table(w)]
body = statistics.median(w["size"] for w in words)
items = lines_of(words)
for t in tables:
rows = [[(c or "").replace("\n", " ").strip() for c in r] for r in t.extract()]
md = ["| " + " | ".join(rows[0]) + " |", "|" + "---|" * len(rows[0])]
md += ["| " + " | ".join(r) + " |" for r in rows[1:]]
x0, top, x1, bottom = t.bbox
items.append({"x0": x0, "x1": x1, "top": top, "bottom": bottom, "size": body,
"text": "\n".join(md), "table": True})
out = []
for it in xy_cut(items):
if it.get("table"):
out.append(it["text"])
elif it["size"] >= body * 1.8:
out.append("# " + it["text"])
elif it["size"] >= body * 1.3:
out.append("## " + it["text"])
elif out and not out[-1].startswith(("#", "|")):
out[-1] += " " + it["text"] # re-join wrapped lines into one paragraph
else:
out.append(it["text"])
return "\n\n".join(out)
def chunk_by_heading(md):
chunks, current = [], []
for block in md.split("\n\n"):
if block.startswith("#") and current:
chunks.append("\n\n".join(current)); current = []
current.append(block)
return chunks + ["\n\n".join(current)]
if __name__ == "__main__":
with pdfplumber.open(sys.argv[1] if len(sys.argv) > 1 else "sample.pdf") as pdf:
page = pdf.pages[0]
print("=== naive extract_text() ===")
print(page.extract_text()[:240], "...\n")
md = page_to_markdown(page)
print("=== markdown ===")
print(md, "\n")
for i, c in enumerate(chunk_by_heading(md)):
print(f"--- chunk {i}: {c.splitlines()[0]!r} ({len(c)} chars)")
Run it:
python make_sample.py && python pdf_to_md.py
Here's what extract_text() gives you. Look at lines 3-5: each one runs straight across both columns.
Chunking PDFs for RAG
Why boundaries matter Why tables break
Retrieval quality depends on chunk boundaries. When Tables are worse. Flattened into plain text, a row of
one chunk mixes two sections, its embedding blurs both values loses its column headers, so the model cannot
topics. Splitting at headings keeps each chunk about tell which number belongs to which metric.
one thing.
Formats compared
Output Headings Tables
Plain text lost flattened
Markdown # / ## pipe table
And here's the converter output (tested with Python 3.13, pdfplumber 0.11.10, fpdf2 2.8.9):
# Chunking PDFs for RAG
## Why boundaries matter
Retrieval quality depends on chunk boundaries. When one chunk mixes two sections, its embedding blurs both topics. Splitting at headings keeps each chunk about one thing.
## Why tables break
Tables are worse. Flattened into plain text, a row of values loses its column headers, so the model cannot tell which number belongs to which metric.
## Formats compared
| Output | Headings | Tables |
|---|---|---|
| Plain text | lost | flattened |
| Markdown | # / ## | pipe table |
--- chunk 0: '# Chunking PDFs for RAG' (23 chars)
--- chunk 1: '## Why boundaries matter' (196 chars)
--- chunk 2: '## Why tables break' (170 chars)
--- chunk 3: '## Formats compared' (134 chars)
Each column comes out as one paragraph under its own heading, the table keeps its header row, and the chunker splits on headings. In a real pipeline you'd merge tiny chunks (like the title-only chunk 0) into the next one and add the heading path as metadata.
Where this toy version breaks
It works on this page, but real documents will break it quickly:
- Paragraphs: it joins every non-heading line until the next heading. Real code needs to detect paragraph breaks (indents, vertical spacing) and hyphenation.
- Borderless "three-line" academic tables aren't detected by
find_tables()with default settings. - Rotated margin text (arXiv side stamps), headers/footers and page numbers end up in the body.
- Symbol fonts map glyphs into Unicode's Private Use Area, and some fonts give
(cid:NN)garbage. You want to detect those pages, not embed them. - Scanned pages have no text layer at all, and you should flag them rather than index empty strings.
Wrapping up
For born-digital PDFs, most of the RAG quality you lose at extraction comes back once you keep reading order, headings and tables. Start with a small rules-based converter like the one above, look at your chunks, and only move to a heavier ML pipeline when your documents actually need it.
This article was written with AI assistance. The code was run and the output above is copied from that run.
Top comments (0)