Append .md to Any of Our Blog URLs. Here Is Why That Matters

When an AI agent reads one of our blog articles, the HTML version weighs 466 KB. The words in that article weigh 18 KB. Everything else is markup, styling hooks and framework payload that a language model pays for in tokens and throws away.

This week we shipped the fix across our entire blog: every one of our 208 articles, in all three languages, now has a raw Markdown twin. Take any article URL, append .md, and you get the clean text. That is 624 static files, generated at build time, served at zero runtime cost. The Markdown version of the article I measured is 96 percent smaller than its HTML sibling.

This post explains why we did it, how the layer works, why we deliberately broke with the pattern Vercel and Cloudflare use, and what this realistically does and does not do for AI visibility.

The Forgotten Half of the llms.txt Standard

Most people know llms.txt as "robots.txt for AI": a Markdown file at your domain root that tells language models what your site is about. Jeremy Howard proposed the standard in September 2024, and adoption studies in 2026 put it at roughly one in ten domains.

What almost everyone skips is the second half of the same proposal. The spec explicitly recommends that pages "provide a clean markdown version of those pages at the same URL as the original page, but with .md appended."

That second half is where the real value sits. An llms.txt file is a table of contents. The .md pages are the actual meal. A table of contents with no readable chapters behind it is exactly why several independent analyses this year found that an llms.txt file alone does not measurably move citation rates.

Documentation platforms figured this out first. Stripe, Anthropic, Neon and FastAPI serve clean Markdown when you append .md to a docs URL. Mintlify built content negotiation into every docs site they host. Sentry serves Markdown to agents at the same URLs humans use. In February 2026, Vercel confirmed that Claude Code and OpenCode already send Accept: text/markdown headers with their requests, and in March, Cloudflare launched Markdown for Agents, converting HTML to Markdown at the edge.

So the convergence among infrastructure companies is done. What we have not seen is regular business sites and blogs following. That gap is the opportunity.

Why Token Economics Decide This

The case is not aesthetic, it is arithmetic.

Our measured example: 466,398 bytes of HTML versus 18,028 bytes of Markdown for a 2,900-word article. A factor of 26. Vercel reports around 90 percent size reduction for their negotiated pages; our static twins land at 96 percent because there is no boilerplate left at all.

For an agent, that difference is not cosmetic. Smaller payloads mean fewer tokens per page, which means an agent researching a topic can afford to read twenty of your articles instead of one. It also means fewer parsing errors: no cookie banners, no navigation, no script tags that HTML-to-text converters mangle mid-sentence.

The traffic side makes this urgent. Cloudflare Radar data from June 2026 shows automated requests now make up more than half of HTML traffic on the web, with AI crawlers alone at about a fifth of verified bot traffic. And the crawl-to-referral ratios are brutal: Anthropic's crawlers fetch thousands of pages for every visit they send back. Those fetches happen whether you serve them well or badly. Serving them badly just means you pay the bandwidth and get misread anyway.

What We Actually Built

The layer has four parts, and all of them are static. Nothing runs at request time, no API is called, no model is involved.

Markdown twins. Every article URL accepts .md: for example /en/blog/claude-guide-2026.md. Each file starts with YAML frontmatter carrying the title, description, author, publish and update dates, tags, the canonical HTML URL, and the URLs of all three language versions. Below that sits the full article, with every relative link rewritten to an absolute one, and a short read-more block linking related articles from the same topic cluster, also as .md links. An agent that lands on one article can keep reading without ever touching HTML.

Feeds that carry the content. Our RSS feeds now include the complete article HTML for the newest twenty posts, not just teaser descriptions. There is a JSON Feed 1.1 endpoint, and each of our 26 topic clusters has its own feed, so a subscriber, human or machine, can follow just "Claude and Claude Code" or just "AI visibility".

Discovery. Every article's HTML head links its Markdown twin via <link rel="alternate" type="text/markdown">, the same way you would advertise an RSS feed. Our llms.txt lists all 26 reading paths and states the .md convention once, prominently. The JSON Feed carries the Markdown URL per item as an extension field.

Consolidation. Each .md response sends an HTTP Link header declaring the HTML page as canonical. No noindex games: search engines consolidate signals onto the HTML version, while agents are free to consume the Markdown.

Because the blog metadata lives in one typed source file, the whole layer regenerates on every build. A new article gets its three twins, its feed entries and its llms.txt listing automatically.

Why We Chose Explicit URLs Over Content Negotiation

Vercel and Cloudflare both champion content negotiation: one URL, and the server inspects the Accept header to decide whether you get HTML or Markdown. It is elegant, and for their infrastructure it is the right call.

We deliberately did not do that, and the reason is caching.

Any site behind a CDN edge cache has to answer one question honestly: does your cache key include the Accept header? On common CDN plans the practical answer is no. The edge stores one variant per URL. Now imagine an agent requests your article with Accept: text/markdown at the exact moment the cache is cold. The Markdown response gets cached under the article URL, and the next thousand human visitors get served raw Markdown in their browsers. Content negotiation behind an edge cache that ignores Vary is a cache poisoning incident waiting for a trigger.

Explicit .md URLs make the problem structurally impossible. Two URLs, two cache entries, no ambiguity. It costs a tiny bit of elegance and buys total predictability. If you control your edge configuration end to end, negotiate away. If you are on a standard CDN plan, explicit URLs are the safer architecture, and they happen to be exactly what the llms.txt spec recommended in the first place.

The Honest Part: What This Does Not Do

I want to be precise here, because this space is full of inflated promises.

Serving Markdown will not, by itself, make ChatGPT cite you. The 2026 data is clear that llms.txt adoption alone shows no measurable citation lift, and I have no reason to believe .md twins alone behave differently. Citations follow from content that answers real questions, structured data that machines can verify, consistent entities, and actual indexing. Our own numbers, over 14,100 Bing Copilot citations in a rolling 30-day window as of early July 2026, come from that whole stack working together over months. The Markdown layer is one brick, not the building.

What the layer does deliver, today: agents that already speak this dialect read your site at a fraction of the cost. Claude Code sends the Markdown Accept header now. Cursor, Windsurf and other IDE agents fetch llms.txt routinely when pointed at a site. Every one of those interactions works better when there is clean Markdown behind the front door. And when an assistant reads your article cleanly instead of parsing it out of 466 KB of markup, the odds that it represents your content correctly go up. That is the realistic promise: cheaper to read, harder to misquote, ready for the traffic mix the web is visibly moving toward.

How to Ship This on Your Own Site

The pattern is portable to any stack that knows its content at build time.

1. Generate the twins at build time. If your content lives in Markdown already, you are 80 percent done: emit one .md file per page per language, with frontmatter carrying title, canonical URL and translations. Rewrite relative links to absolute ones so the file survives being read out of context.
2. Set the canonical Link header. Each Markdown response should declare its HTML sibling as canonical. That keeps your search setup untouched.
3. Advertise it. Add the alternate link tag to your HTML heads, and state the convention once in your llms.txt. Discovery is half the feature; an undiscoverable endpoint does not exist.
4. Use the shortcuts your platform gives you. On Cloudflare, Markdown for Agents is a zone setting. On WordPress, plugins now handle Markdown serving. If you run a documentation platform like Mintlify, you already have it.

A working example is one click away: this very article, like every article on this blog, has its twin at this URL with .md appended. If you want the same layer, plus the structured data and measurement stack around it, built for your site, that is literally what we do.

Where This Is Heading

The web already crossed the line where machines request more HTML than humans do. Every trend line I can see says the gap widens: agent traffic grows, human pageviews per task shrink, and the sites that answer machine requests cleanly get read more, misquoted less, and cited when it counts.

Markdown twins will look obvious in two years, the way responsive design looks obvious now. The standard already asks for them, the tooling exists on every stack, and the cost is a build step. The only genuinely surprising thing is how few sites outside the documentation world have shipped it. For once, being early is cheap.

---

This article was originally published on studiomeyer.io.