Why Markdown Is Better Than HTML for LLM Context Windows

XSron Hou — Wed, 05 Aug 2026 15:43:09 +0000

Originally posted on the Scrapio blog — sharing here too.

When you're building a RAG pipeline or an LLM agent that reads web pages, the format you put into the context window matters more than most people realize. Raw HTML and clean markdown can represent the same information, but one costs 10× more in tokens and adds substantial noise that degrades model performance.

The problem with HTML

A typical product page might have 3,000 words of actual content. As raw HTML, that page is often 50,000–80,000 characters — mostly tags, class names, JavaScript, tracking pixels, and navigation menus.

LLMs tokenize all of it. You pay for every <div class="product-carousel__item--highlighted"> even though it carries no information your model needs.

Measured comparison

We ran 100 pages through both approaches and measured:

Metric	Raw HTML	Clean Markdown
Avg tokens per page	18,400	1,850
Tokens that are content	~12%	~94%
GPT-4o cost per 1,000 pages	~$91	~$9
Extraction accuracy (structured tasks)	71%	89%

The accuracy difference is real: LLMs perform better when they're not trying to reason through a wall of HTML noise.

What clean markdown looks like

Raw HTML snippet:

<div class="pdp-price-container" data-testid="price-wrapper">
  <span class="price--sale" aria-label="Sale price">
    <span class="price__currency">$</span>
    <span class="price__amount">279</span>
    <span class="price__cents">.99</span>
  </span>
</div>

Same content as markdown:

**Price:** $279.99

The markdown version is 22 tokens. The HTML version is 89 tokens — and that's a small excerpt. An entire page compounds this by 50–100×.

Getting clean markdown from any URL

curl -X POST https://api.scrapio.dev/v1/fetch \
  -H "Authorization: Bearer sk-..." \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/article", "output": ["markdown"]}'

Response:

{
  "request_id": "req_abc123",
  "mode": "inline",
  "status": "completed",
  "outputs": {
    "markdown": "# Article Title\n\nContent here..."
  },
  "usage": { "credits": 1 }
}

The markdown output strips navigation, ads, scripts, and layout elements. What remains is the readable content — headings, paragraphs, lists, tables, and code blocks.

JavaScript-rendered pages

SPAs and dynamic pages need a render step:

{
  "url": "https://example.com/dashboard",
  "output": ["markdown"],
  "render_js": true
}

Without render_js, you'd get the pre-hydration HTML — often an empty shell.

For RAG pipelines

When crawling large sites for a vector database, markdown is especially important because:

Chunking is cleaner — markdown structure (headings, paragraphs) provides natural split points
Embeddings are better — semantic models trained on clean text produce better vectors when the input isn't polluted with HTML syntax
Storage is cheaper — 10× smaller documents mean lower storage and fewer API calls to retrieve them

Next steps

Try the Webpage to Markdown template for a copy-paste implementation
See Batch Article Extractor for crawling and indexing entire sites
Read How to Feed Live Website Data into an LLM Agent

DEV Community: XSron Hou