How many tokens does a web page cost your LLM? We measured 20 real pages

Leif Albrand — Tue, 13 Oct 2026 20:59:21 +0000

Raw HTML vs. full-page text vs. main-content Markdown — median, range, and the cases where the saving is small.

If you build RAG pipelines or agents that read the web, every page you fetch ends up as input tokens: in the embedding step, in the context window, on every query. So what you hand the model matters. We measured it on 20 real pages instead of guessing.

TL;DR

The median raw HTML page in our sample was ~128,000 tokens. One page would fill a 128k context window.
LLM-oriented Markdown of the same pages needed a median 98 % fewer tokens than raw HTML and 63 % fewer than the full page converted to text.
Most of that second saving comes from main-content extraction (dropping navigation, footers, cookie notices, teasers): median 51 % on its own. Cleanup on top added a median ~9 %.
The range is wide: 18 % to 90 % against full-page text. Pages that are already lean save little.

Setup

20 public pages, German-language sites, picked to mix two kinds: 6 start/overview pages (lots of menus and teasers) and 14 articles (news, encyclopedia, documentation, statute text, recipes, guides). Sources include Wikipedia, MDN, tagesschau.de, heise online, the German Environment Agency, gesetze-im-internet.de, Chefkoch, DHL, Siemens, Verbraucherzentrale, n-tv, t-online and MSN.
Each page was fetched fresh (no cache) and saved in four variants:
1. Raw HTML — the rendered page source.
2. Full-page text — the whole page converted to Markdown, menus and footer included. This is roughly what a plain "fetch this URL" tool hands a model.
3. Main-content Markdown — only the main content.
4. LLM Markdown — main content, further cleaned for model input, plus a short source header.
Tokenizer: o200k_base via tiktoken (the GPT-4o tokenizer). Other tokenizers give different absolute counts; the ratios move much less.
Statistic: median of the per-page savings, so a handful of huge portal pages can't drive the result.
Measured on 1 October 2026. Pages change; a re-run will give somewhat different numbers.

Results

	Total tokens, 20 pages	Median page
Raw HTML	3,713,862	128,051
Full-page text	242,852	10,917
Main-content Markdown	109,929	3,241
LLM Markdown	94,987	2,948

Median saving of LLM Markdown:

compared with	all 20 pages	start pages (6)	articles (14)
raw HTML	98 %	99 %	98 %
full-page text	63 %	64 %	59 %

Per page, sorted by saving against full-page text

Page	Kind	Raw HTML	Full-page text	LLM Markdown	vs. HTML	vs. full text
Verbraucherzentrale: guide	article	404,544	5,952	4,902	98.8 %	17.6 %
German copyright act § 44b	article	1,265	480	355	71.9 %	26.0 %
nord24: heat pump article	article	27,875	7,261	5,365	80.8 %	26.1 %
Verbraucherzentrale: start page	start	562,566	5,573	3,608	99.4 %	35.3 %
Wikipedia: Big Data	article	94,483	25,259	15,713	83.4 %	37.8 %
heise online: start page	start	359,135	29,770	16,135	95.5 %	45.8 %
Environment Agency: emissions	article	121,354	20,064	10,354	91.5 %	48.4 %
Chefkoch: recipe	article	140,381	4,831	2,459	98.2 %	49.1 %
n-tv: heat pump guide	article	119,333	18,372	8,648	92.8 %	52.9 %
tagesschau.de: business	start	478,580	12,501	5,072	98.9 %	59.4 %
heise online: news item	article	91,736	5,404	1,858	98.0 %	65.6 %
Siemens: start page	start	134,748	6,449	2,059	98.5 %	68.1 %
tagesschau.de: fuel prices article	article	261,767	7,339	1,960	99.3 %	73.3 %
MDN: HTML docs	article	56,651	12,837	3,280	94.2 %	74.4 %
t-online: heat pump article	article	150,606	10,486	2,616	98.3 %	75.1 %
MSN: heat pump article	article	244,653	20,174	4,347	98.2 %	78.5 %
DHL: private customers	start	59,085	11,416	2,155	96.4 %	81.1 %
Environment Agency: climate change	article	99,713	11,347	1,337	98.7 %	88.2 %
DHL: online postage	article	114,545	10,319	1,119	99.0 %	89.2 %
Chefkoch: start page	start	190,842	17,018	1,645	99.1 %	90.3 %

What we take from it

1. Never send raw HTML to a model. Even the leanest page in the set (a statute text, 1,265 tokens of HTML) lost 72 %. Typical pages are 100k–500k tokens of HTML for a few thousand tokens of content — mostly scripts, inline styles, tracking and markup.

2. "Full page as text" is not lean either. Converting HTML to text removes the markup but keeps every menu, footer link, cookie notice and teaser. On the median page that is still more than three times the tokens of the actual content.

3. Main-content extraction is the big lever. Dropping boilerplate saved a median 51 % against full-page text; extra cleanup for LLM input added a median ~9 %. This is the honest part: much of the gain is available from any decent main-content extractor. Whichever tool you use, check that it actually does this, and spot-check its output — an extractor that cuts too much is worse than one that cuts too little.

4. The saving depends on the page. Portal and overview pages save the most; long, clean articles the least (Wikipedia: 38 %; a consumer guide: 18 %). A single median number hides that; plan with the range.

5. Fewer tokens is not only cheaper. Less noise in the context means less for the model to wade through, and less irrelevant text in your vector index competing with the passages you want to retrieve.

A rough cost sense

On the median page, full-page text is ~10,900 tokens and LLM Markdown ~2,900: about 8,000 input tokens saved per page and pass. At an illustrative $1 per million input tokens, that is about $8 per 1,000 pages per pass — and RAG pipelines pay it at embedding time, while agents pay it again every time the page lands in a context window. Plug in your own model price.

Limitations

20 pages is a small sample, and all are German-language sites. English sites may differ.
Tokenizer-specific absolute numbers (o200k_base).
Pages change; this is a snapshot from 1 October 2026.
"Full-page text" here is our own whole-page Markdown conversion, not any specific third-party tool.
We did not benchmark other extraction services or libraries. The comparison is between page representations, not between products.
Token count is not answer quality. We measured retrieval separately (headings-based chunks vs. fixed-length chunks, 58 questions, one embedding model) — that is a different, smaller study.

Disclosure and method

I'm the founder of Ragfeed, a Firecrawl-compatible web-data API hosted in Germany; the four variants above were produced with it, and "LLM Markdown" is its llmMarkdown format. The script, page list and raw numbers are public at https://github.com/LeifAlbrand/ragfeed-beispiele, so you can rerun the comparison yourself — including against another Firecrawl-compatible API (--ohne-llm). If you want to try it on your own pages, there is a free tier (1,000 pages a month): https://ragfeed.de

DEV Community: Leif Albrand