If you've built any pipeline that ingests PDFs, you already know the demo is misleading.
You test on ten clean, well-formatted invoices. Extraction works perfectly. You ship it. Then invoice #47 arrives from a vendor who updated their template last quarter, and suddenly your "working" pipeline is silently returning garbage, or worse, returning incorrect but plausible-looking data that nobody notices until it causes a real problem downstream.
This is the default failure mode of almost every PDF extraction approach, and it's worth understanding why.
The core assumption almost everything makes
Most PDF parsing tools, whether you're using PyPDF, pdfplumber, Camelot, or even a lot of the "AI-powered" extraction tools, quietly assume the document's structure is stable. They're looking for a table in roughly the same place, a field label followed by a value in roughly the same layout, headers and footers in roughly the same positions.
This works great until it doesn't. And "doesn't" happens constantly in the real world:
A vendor updates their invoice template
A scanned document has slightly different margins than the last batch
A multi-column layout reflows differently depending on how much text is in a section
A table spans two pages instead of one
Someone exports the same "kind" of document from a different piece of software entirely
None of these are edge cases if you're processing documents at any real volume. They're Tuesday.
Why this is worse than it looks
The frustrating part isn't that the extraction fails, it's how it fails. A parser built on positional or template assumptions doesn't usually throw an error when the layout shifts. It just extracts the wrong thing, confidently. A field gets pulled from the wrong location. A table gets truncated. Text from a header ends up concatenated into the body.
This is a much worse failure mode than a crash, because nothing tells you it happened. If you're feeding this into a RAG pipeline or a downstream automation, you don't find out until someone notices the numbers don't add up, or the answer to a query is subtly wrong.
What actually helps
A few practical patterns, from lowest to highest effort:
Separate structure detection from content extraction. Instead of extracting "the value at position X, Y," extract based on what the content actually looks like, a labeled field, a table with a header row, a block of prose. This is more robust to layout shifts because you're not hardcoding coordinates.
Validate against the source, not just the schema. It's not enough to check that your output JSON is well-formed. Spot-check a sample of extractions against the actual source document. A field can be perfectly valid JSON and still be pulled from the wrong place.
Treat format drift as the norm, not the exception, when picking tools. If you're evaluating extraction tools, don't just test them on your cleanest documents. Test them on your messiest ones, the scanned copy, the vendor who does things differently, the one with a table that spans pages. That's where the real difference between tools shows up, and it's invisible in a clean demo.
Don't assume "AI-powered" means format-agnostic. A lot of tools market themselves as AI-powered extraction, but are still fundamentally doing template matching or fixed-schema extraction under the hood. Worth actually testing rather than assuming the label solves the problem for you.
This is genuinely the problem we've been working on at Corvic, building an extraction that's designed around inconsistent, real-world document structure rather than assuming a fixed layout, so it holds up when a vendor changes their invoice template for the third time this year.
The takeaway
If your PDF pipeline works great in testing and falls apart in production, the problem usually isn't your code; it's the assumption baked into whatever extraction approach you're using. Documents in the real world don't hold still. Whatever you build (or buy) needs to expect that from day one, not treat it as an edge case to patch later.
Curious what others have run into here, especially around scanned documents or tables that span pages. What's broken on you the most?
Top comments (1)
PDF parsing is a perfect example of AI needing boring structure around it. The model can help interpret content, but layout drift, tables, headers, footnotes, and scanned pages need a pipeline that exposes uncertainty instead of pretending every extracted string is equal.