I thought my PDF parser was done — then I ran it on 80 real resumes

#opensource #ai #rust #python

Building RAG pipelines, the step that kept letting me down was the very first one: turning a PDF into text. Two runs of the same file could give slightly different output, tables collapsed into garbled lines, and Chinese PDFs were mostly a coin flip. When your chunks change between runs, so do your embeddings and your eval numbers — and you stop trusting the index.

So I wrote pdfmuse: a PDF/DOCX parser with one goal — be a precise, boring pre-layer for RAG. Same file in, same output out, every time. No probabilistic models in the core path.

This post is partly about how it works, and partly about the humbling bug I only found by dogfooding it on real data. That second part is the one worth your time.

The design

One Rust core, three bindings (Python / Node / WASM) that are byte-identical. They all serialize through one path; a CI gate fails if their output diverges by a single byte. Prototype in Python, ship in Node/WASM, get the exact same chunks.
Fast. On a public, reproducible benchmark (65 arXiv papers — the fetch script ships in the repo, so you can rerun it) it's ~7.7× faster than PyMuPDF and wins every file, and ~150× faster than pdfplumber, at 100% character coverage. That came from an O(1)-per-object parser and a zero-allocation content-stream tokenizer instead of leaning on the general-purpose library underneath.
Deterministic + zero ML in the core. Scanned/image-only pages don't get guessed at — they return a NeedsOcr warning, and OCR/layout inference are a pluggable backend.
CJK is first-class. CID/Type0 fonts and CMap/ToUnicode are in the main path, not bolted on.

You can try it in your browser (WASM, nothing gets uploaded — your file never leaves the tab): https://casperkwok.github.io/pdfmuse/ — drag a PDF and watch it come apart into text-with-coordinates, tables, and Markdown.

pip install pdfmuse        # or: npm i @pdfmuse/node  /  cargo add pdfmuse-core

import pdfmuse
data = open("report.pdf", "rb").read()
text = pdfmuse.to_text(data)       # plain reading-order text
md   = pdfmuse.to_markdown(data)   # headings + tables
doc  = pdfmuse.parse(data)         # full IR: chars/blocks with exact bboxes

The part where I was wrong

I had a test suite. Snapshot tests, a fuzzer, cross-binding parity, a CJK suite — all green. I ran it on a handful of sample resumes and the output looked great. I was ready to call the extraction layer done.

Then I ran it against 80 real resumes pulled from a product database (I build a resume tool on the side). The plan was a boring "shadow comparison" vs PyMuPDF to confirm parity before switching anything.

8 of the 80 came back with zero characters. Not garbled — empty. And it failed silently: no warning, no error, just an empty document. PyMuPDF read all 8 fine.

That's a 10% total-failure rate, invisible to my entire test suite, because my test files happened to be clean.

Chasing it down

The 8 failing files all shared traits:

All used Type0 / Identity-H subset fonts — and they had valid ToUnicode maps. So it wasn't a missing-CMap problem.
7 of 8 were generated by Canva, the 8th by PDFium.
The tell: the failing files had 2–15 form XObjects each; every file that worked had zero.

There it was. Canva (and Chrome's PDFium, and plenty of design tools) draw the page's text inside a form XObject, invoked by a Do operator. My content-stream interpreter didn't handle Do at all — and, worse, a lazy-loading optimization I'd added skipped /XObject bodies for speed. So the actual page text was never even parsed. On a clean PDF where text lives in the page content stream directly, everything worked. On a Canva export, I silently got nothing.

The fix was to make the interpreter recurse into form XObjects (decode the stream, apply its /Matrix, build its own font map, recurse with a depth guard) and stop skipping them in the loader. After that:

zero-char failures: 8/80 → 0/80. The 10 form-XObject files now match PyMuPDF ~100%.

The lesson isn't "I wrote a bug." It's that a green test suite on synthetic fixtures told me nothing about the real distribution. Canva is one of the most popular resume builders on earth; "10% of resumes" is not an edge case. I only found it because I ran it on real data before trusting it. If you're swapping any parser into a pipeline, do the shadow-comparison on your corpus first — the demo files lie.

A performance truth, too

While I was in there: I'd been claiming "4× faster than PyMuPDF." For a Python user, that turned out to be… not quite true. The Rust core parses in ~1.3 ms, but if you json.loads the full intermediate representation (every char with its bbox) back into Python, that deserialization costs ~3.5 ms and eats the win — you end up roughly on par with PyMuPDF.

The fix was obvious once measured: if you only want text, don't materialize the whole IR. to_text() / to_markdown() now return a string straight from Rust, and the Python text path is back to full core speed. Measure the path your users actually take, not the one that flatters your benchmark. (Profiling that same corpus later turned up two more hotspots — a table detector that mistook a dense figure for a 1000-cell table, and a form XObject re-parsed 18,000 times on a single page — and fixing those is what got it to winning every file.)

Using it in RAG

There are drop-in loaders for the three big frameworks, all emitting chunks with heading_path + bbox metadata (which section, where on the page):

# LangChain
from langchain_pdfmuse import PdfmuseLoader
docs = PdfmuseLoader("report.pdf", mode="elements").load()

# LlamaIndex
from llama_index.readers.pdfmuse import PdfmuseReader
docs = PdfmuseReader(mode="elements").load_data("report.pdf")

# Haystack
from pdfmuse_haystack import PdfmuseConverter
docs = PdfmuseConverter(mode="markdown").run(sources=["report.pdf"])["documents"]

Honest limitations

No OCR (by design). Scanned pages return NeedsOcr; OCR is a backend, kept out of the deterministic core.
Reading order on dense two-column academic layouts. Single-column, tables, and clean two-column read correctly, but papers with very tight gutters (think IEEE/arXiv, where equations bleed into the column gap) can still interleave the two columns. It's a hard geometric-layout problem I'm deliberately leaving to a future vision backend rather than faking with brittle heuristics.
Young project (v0.1.x). The core tests are strong, but see the story above about sample size.