I built the PDF-to-Markdown API expecting layout to be a solved problem. It isn't.
The failure mode was subtle and brutal: academic papers with two-column layouts came back as interleaved garbage. Line one of column A, line one of column B, line two of column A... The Markdown looked fine at a glance — headings, paragraphs, lists all present — but the reading order was destroyed. I fed one into a RAG pipeline and the retriever started quoting sentences that spanned two unrelated columns. The chunks were technically valid; semantically they were nonsense.
My first fix was naive: split the page down the middle by x-coordinate. Worked great until a figure spanned both columns and the caption got shredded. Then a wide table. Then a single-column abstract followed by two-column body. The geometry was lying to me in a new way every time.
What actually worked was a hybrid: detect column boundaries from the whitespace gutters — the vertical runs of empty space between blocks — then fall back to single-column flow when no stable gutter exists. And always respect explicit reading-order hints when the PDF has them (tagged PDFs, which are rarer than they should be).
The lesson I keep relearning: layout is semantic. A PDF isn't text with coordinates; it's a visual argument about how ideas relate. When you flatten it to Markdown, you're not extracting text — you're reconstructing the author's intent. Every column, every sidebar, every pull-quote is a decision about what matters together.
If you're building document pipelines, test on papers with figures spanning columns, tables with merged cells, and multi-column footnotes. The pretty PDFs are easy. The messy ones are the spec.
I ended up shipping it as part of the PDF-to-Markdown converter: https://x402.freeq.one/tools/pdf_to_markdown.html
Top comments (1)
The RAG symptom is the part worth pinning down, because it's the only failure here that's invisible to the converter itself. The Markdown looked fine; the damage showed up three stages later in retrieval. A fixture that catches it: take a two-column page where column A is method and column B is results, convert, then score every output chunk for cross-column mixing - adjacent sentences whose source spans sit in different x-bands on the same lines. Presence of headings and paragraphs can't detect this; only tracing sentences back to their source rectangles can. And one precedence fixture for the tagged-PDF path: build a file where the tag reading order deliberately disagrees with the visual geometry (sidebar quoted mid-page). Assert the tags win. The day a tagging bug or an over-eager gutter detector flips that precedence, you want a red test, not a quietly reordered paragraph in someone's embedding store.