Every RAG project hits the same wall on day two: the PDFs.
You have a folder of them — contracts, manuals, invoices, scanned reports — and you need the text inside. So you reach for a PDF library, run it, and get back a wall of characters with the tables flattened into gibberish, the headings gone, and the two-column pages interleaved line by line. Feed that to an embedding model and your retrieval quality is ruined before you even start.
Then you hit the scanned ones. No text layer at all. Now you're wiring up an OCR engine, tuning it, and gluing the two paths together.
I got tired of rebuilding this for every project, so I turned it into an API. Send a PDF, get back clean Markdown that keeps its structure.
What "clean" actually means
The goal isn't just extracting characters — it's preserving the shape of the document so a language model can use it:
- Headings stay headings, so you can chunk on them.
- Tables come back as Markdown tables, not scrambled rows.
- Lists stay lists.
- Reading order is respected on multi-column pages.
- Scanned pages are OCR'd automatically, in the same call.
That last point matters: you send the file and you don't care whether it's digital or a photo of a page. The API figures it out.
The two endpoints
- pdf-to-markdown — the workhorse. PDF in (as a file or a URL), clean Markdown out, plus the page count and whether OCR kicked in. This is what you chunk and embed.
- extract — for the structured case. Point it at an invoice or a receipt and get back fields — totals, dates, vendor, line items — as JSON, so you skip the regex entirely.
Why an API instead of a local library
Two reasons. First, the "digital PDF plus scanned PDF plus tables plus reading order" combination is genuinely fiddly to get right, and it's the kind of plumbing that has nothing to do with your actual product. Second, it's deterministic and stateless: same file in, same Markdown out, in one request, with nothing to install or keep updated.
If your RAG retrieval is only as good as your ingestion — and it is — this is the cheapest place to buy a big quality jump.
It's live on RapidAPI with a free tier: https://rapidapi.com/cesaricf79/api/document-intelligence3
How are you handling PDF ingestion in your RAG stack right now? I'd love to hear what's working and what still hurts.
Top comments (2)
The part I'd actually stress-test is reading order on multi-column pages, that's where most pipelines fail quietly, until you start seeing answers that mix two sections that were never meant to go together.
And the "same file in, same result every time" promise makes me curious with OCR in the mix, worth clarifying which part is actually deterministic and which isn't.
Great questions — both land on exactly where this gets hard.
On multi-column reading order: you're right that this is where pipelines fail quietly. Under the hood it leans on PyMuPDF's layout/column detection to reconstruct reading order instead of dumping the raw text stream, so standard two-column papers and reports come out in the right order and tables stay intact. Where I'd still tell you to test your own worst cases: pages with mixed column counts, floating sidebars/callouts, or captions wedged between columns — those can still interleave. It won't silently merge two unrelated sections on a clean 2-column layout, but I won't promise perfection on magazine-style chaos, so the honest advice is to throw your gnarliest page at the free tier first.
On "same in, same out" + OCR: fair to push on that, because it means two different things depending on the page.
Digital PDFs (real text layer): fully deterministic — same bytes in, same Markdown out, every time. No model, no sampling.
Scanned pages: there's no text layer, so OCR runs. That path is deterministic in the "no randomness/temperature, same image → same output" sense — it's not an LLM guessing differently each call — but it's best-effort recognition, not a guaranteed-correct transform: output quality tracks scan quality (DPI, skew, noise). So the promise I'd actually stand behind is "no nondeterministic AI in the loop," not "OCR is always right."
The response also tells you whether OCR was used, so you can flag those files for a closer look downstream.
Appreciate the sharp read — exactly the kind of thing worth being precise about.