Turn any PDF into clean Markdown for your RAG pipeline

César Freitas — Tue, 29 Sep 2026 09:57:14 +0000

Every RAG project hits the same wall on day two: the PDFs.

You have a folder of them — contracts, manuals, invoices, scanned reports — and you need the text inside. So you reach for a PDF library, run it, and get back a wall of characters with the tables flattened into gibberish, the headings gone, and the two-column pages interleaved line by line. Feed that to an embedding model and your retrieval quality is ruined before you even start.

Then you hit the scanned ones. No text layer at all. Now you're wiring up an OCR engine, tuning it, and gluing the two paths together.

I got tired of rebuilding this for every project, so I turned it into an API. Send a PDF, get back clean Markdown that keeps its structure.

What "clean" actually means

The goal isn't just extracting characters — it's preserving the shape of the document so a language model can use it:

Headings stay headings, so you can chunk on them.
Tables come back as Markdown tables, not scrambled rows.
Lists stay lists.
Reading order is respected on multi-column pages.
Scanned pages are OCR'd automatically, in the same call.

That last point matters: you send the file and you don't care whether it's digital or a photo of a page. The API figures it out.

The two endpoints

pdf-to-markdown — the workhorse. PDF in (as a file or a URL), clean Markdown out, plus the page count and whether OCR kicked in. This is what you chunk and embed.
extract — for the structured case. Point it at an invoice or a receipt and get back fields — totals, dates, vendor, line items — as JSON, so you skip the regex entirely.

Why an API instead of a local library

Two reasons. First, the "digital PDF plus scanned PDF plus tables plus reading order" combination is genuinely fiddly to get right, and it's the kind of plumbing that has nothing to do with your actual product. Second, it's deterministic and stateless: same file in, same Markdown out, in one request, with nothing to install or keep updated.

If your RAG retrieval is only as good as your ingestion — and it is — this is the cheapest place to buy a big quality jump.

It's live on RapidAPI with a free tier: https://rapidapi.com/cesaricf79/api/document-intelligence3

How are you handling PDF ingestion in your RAG stack right now? I'd love to hear what's working and what still hurts.

Stop wrestling with broken JSON from your LLM

César Freitas — Tue, 29 Sep 2026 09:13:51 +0000

If you've shipped anything on top of an LLM, you know this pain: you ask the model for JSON, and you get back JSON wrapped in a code fence, with a friendly "Sure! Here's your data:" in front, single quotes instead of double, a trailing comma, and — on a bad day — the last brace missing because the response got cut off.

So you write a regex. Then another. Then a try/except that strips fences. Then you discover the model sometimes emits True instead of true. Six months later your "JSON cleaner" is 200 lines of defensive string surgery that everyone is afraid to touch.

I got tired of copy-pasting that file between projects, so I turned it into a small API. Here's what it handles and how it works.

The failure modes

Real LLM output that should be JSON tends to break in a predictable handful of ways:

Markdown code fences around the JSON.
Prose around it — "Here is the result: { ... } Hope this helps!"
Single quotes instead of double.
Unquoted keys.
Trailing commas.
Python/JS literals — None, True, False, NaN, undefined.
Inline comments.
Truncation — the response hit the token limit and just... stops.

The first seven are cosmetic. The eighth is the nasty one: you have to walk the string, track the open braces and brackets and whether you're inside a string, and close everything back up in the right order.

One call

Send the broken string; get back valid parsed JSON, a normalized text version, and the list of fixes that were applied. The truncated case is closed automatically — a fragment that starts an object with an unclosed array comes back as a complete, valid object.

Why deterministic, not "ask another model to fix it"

You could send the broken JSON to a second LLM call and ask it to fix it. But that's slower, costs tokens, is non-deterministic, and can hallucinate values that were never there. Repairing JSON is a parsing problem, not a reasoning problem — so it should be solved with a parser. Same input, same output, every time, in milliseconds.

It's part of a small toolbox

The same API also has the endpoints I kept needing next to JSON repair:

context-compress — trim text to a token budget before sending it to a model (removes HTML and boilerplate, dedupes, cuts on sentence boundaries).
srt-clean — turn SRT/VTT captions into clean text for prompting, and tell you how many tokens you saved.
log-triage — paste raw Docker/systemd/stack-trace logs and get the exception type, message, and likely cause.

If this saves you from maintaining yet another clean_json.py, it's live on RapidAPI with a free tier: https://rapidapi.com/cesaricf79/api/llm-dev-utilities

What's your worst "the model returned almost-JSON" story? I'm curious how many of these failure modes I'm still missing.

DEV Community: César Freitas