Last Tuesday, I was setting up a quick RAG pipeline for our internal team documentation. The task seemed harmless: ingest about 30 PDFs into a vector store and let the team chat with them.
Two hours in, the bot started giving completely hallucinated answers.
When I checked the vector database chunks, I realized why: the PDF text extractors had completely butchered the tables. Column headers were floating three paragraphs away, footnote numbers were injected into the middle of API parameters, and H2 headers were flattened into plain text.
If you've ever tried to feed messy raw text into an LLM, you already know the pain.
🤦♂️ The Reality of "Quick" PDF Extraction
Most of us usually default to something like pypdf, pdfplumber, or heavy OCR wrappers. They work fine for simple single-column text, but the moment you throw a real-world document at them—say, a technical spec with tables and code blocks—things break down fast:
[PDF Visual Layout] ──► (Naive Text Extraction) ──► Plain Text Soup
│
▼
LLM has zero clue what belongs to what ◄───┘
You spend half your day writing regex filters just to stitch broken tables back together or stripping out random page numbers.
It's tedious, fragile, and honestly, a waste of sprint time.
💡 The Pivot: Markdown First, Chunks Later
What LLMs actually understand well is Markdown.
Markdown gives your context window semantic boundaries without heavy HTML bloat:
Headers (#, ##) tell the embedding model where a new topic begins.
Native pipes (| col |) preserve tabular relationships.
Backticks keep code and parameters isolated.
Instead of writing a custom parsing script every time a non-standard PDF came along, I started testing out lightweight conversion tools to turn PDFs into clean .md files before doing any chunking.
While testing a few options, I came across this online PDF to Markdown converter.
What made me stick with it for quick prototype runs:
1. Tables actually survive: It maps complex multi-row tables into standard Markdown grid tables without scrambling columns.
2. Clean typography: It respects font sizes and translates them into proper Markdown header levels (H1, H2, H3) rather than just spitting out uppercase words.
3. No environment bloat: You don't have to battle Tesseract binaries or Docker OCR dependencies locally when you just need to inspect and clean a document quickly.
🛠️ The Workflow That Saved My Pipeline
Here is how simple chunking becomes once you feed your pipeline clean Markdown instead of raw extracted strings:
code
from langchain_text_splitters import MarkdownHeaderTextSplitter
# 1. Load the clean Markdown file you exported
with open("api_documentation.md", "r", encoding="utf-8") as f:
clean_markdown = f.read()
# 2. Tell the splitter to preserve structural hierarchy
headers_to_split = [
("#", "Module"),
("##", "Endpoint"),
("###", "Parameters")
]
splitter = MarkdownHeaderTextSplitter(headers_to_split_on=headers_to_split)
sections = splitter.split_text(clean_markdown)
# Each chunk now has its parent headers attached as metadata!
print(f"Total structured chunks: {len(sections)}")
print(f"Sample chunk metadata: {sections[0].metadata}")
Notice the difference? Because the document was converted to Markdown first, the splitter automatically knows where endpoints start and stop. No orphaned text, no mangled tables.
💭 Bottom Line
We often overcomplicate pipelines by writing custom parsers for things that already have simple fixes. If your LLM or documentation migrations are choking on messy PDFs, stop feeding them raw strings.
Give the md-convert.org PDF to Markdown tool a spin on one of your trickiest PDF documents and see if it saves you the regex headache.
How do you guys handle PDF ingestion in your stacks? Are you sticking to self-hosted OCR pipelines, or using pre-processing utilities? Drop your setup below!
Top comments (0)