DEV Community: Andriy Pyvovarchuk

llms.txt v2 is out — here's what actually changed

Andriy Pyvovarchuk — Fri, 18 Sep 2026 20:30:56 +0000

The llms.txt proposal was quietly revised in August 2026,
nearly two years after the original. If you already publish an llms.txt, the
good news is short: the file format didn't change. A v1 file is still a valid
v2 file. Nothing to migrate.

What changed is everything around the file — how agents find it, what a
subpath file means, and which tooling the spec expects. I maintain an llms.txt
validator, so I had to go through the diff line by line and change real code.
Here's the list, and what each item costs you.

The format is untouched

Still exactly this, in this order: an optional BOM, an # H1 with the project
name (still the only required element), a > blockquote summary, optional
heading-free prose, then zero or more ## sections holding Markdown link lists.

If your file scored well against v1, it scores the same against v2. You can stop
reading here if all you wanted was "do I need to rewrite my file". You don't.

1. Link relations — the actual new thing

This is the substantial addition, and it answers the most common complaint from
two years of adoption: given a page, how does an agent find its Markdown
version, or the llms.txt that covers it, without guessing at URLs?

v2 answers with two standard link relations:

rel="describedby" → the llms.txt that covers this page
rel="alternate" type="text/markdown" → this page's Markdown version

Either can be an HTML element:

<link rel="describedby" href="/llms.txt">
<link rel="alternate" type="text/markdown" href="/docs/page.html.md">

Or an HTTP response header — which is the only option for non-HTML resources
like the Markdown files themselves, and which you can set in your CDN config
without touching a single page:

Link: </docs/page.html.md>; rel="alternate"; type="text/markdown", </docs/llms.txt>; rel="describedby"

If you followed the older advice — including advice I published myself — to
advertise the file with rel="alternate" type="text/plain", swap it for
rel="describedby". That was a reasonable convention while the spec had none.
Now it has one.

Cost to you: one line in your layout, or one CDN header rule.

2. Both `.md` URL forms are now legal

v1 specified one shape for a page's Markdown twin: .md appended to the full
page URL, so page.html → page.html.md. In practice several publishing tools
replaced the extension instead, producing page.md. v2 blesses both. For URLs
without a filename, append index.html.md or index.md.

Cost to you: zero. Whichever form you already emit is now correct.

3. Subpath files are finally defined

v1 permitted an llms.txt outside the root but never said what it meant. v2
defines it:

A file covers the URLs under its path, and where more than one file applies,
agents should use the most specific one.

So /docs/llms.txt covers everything under /docs/, and an agent reading a docs
page prefers it over the root file.

This matters most if you control a path but not a host — a GitHub Pages project
site, a docs section on a shared domain. It's also the spec's stated reason for
sticking with a conventional filename instead of /.well-known/, which only ever
exists at the origin root.

This is the one that bit me. My validator normalized any input to
https://host/llms.txt — it threw the path away. So fastht.ml/docs came back
as "no llms.txt found" even though fastht.ml/docs/llms.txt has existed for two
years. That's the spec's own reference example, and I was reporting it as
missing. The fix is to target the path's file and walk up to the root only if
there's nothing there:

example.com/docs/tutorials  →  /docs/tutorials/llms.txt
                            →  /docs/llms.txt          ← found, stop
                            →  /llms.txt

One detail worth getting right: only walk up on a definitive 404/410. A 403
or 429 is a WAF blocking you, and a 5xx is a broken server — neither tells you
anything about the parent path, so treating them as "not here" invents a wrong
answer.

Cost to you: nothing, unless you build tooling. If you do, check your URL
normalization today.

4. The context-expansion tooling is gone (and `## Optional` lost its teeth)

v1 described llms_txt2ctx, a tool that expanded an llms.txt into a single LLM
context. v2 drops it and states the expectation directly instead: agents view or
search the llms.txt, then follow the links they need. Which means the links
should point at LLM-friendly content, and the file itself stays small enough to
sit in context.

With that tooling goes the mechanical meaning of the ## Optional section, which
existed to tell those tools what to omit. Optional sections are still allowed and
still a useful convention for secondary links an agent can skip — they just don't
drive anything automatic any more.

Worth saying out loud since it comes up constantly: llms-full.txt was never
in the spec, in either version. It's a widely adopted industry practice, and a
good one for large documentation sets. Just don't call it a requirement.

What I'd actually do this week

Add rel="describedby" pointing at your llms.txt. One line.
If you publish Markdown twins, advertise them with rel="alternate" type="text/markdown".
Replace any rel="alternate" type="text/plain" discovery hint.
If your docs are the part agents care about, consider scoping a /docs/llms.txt to them — that's now a first-class choice, not a grey area.

Why there was a revision at all

Because the premise stopped being speculative. When llms.txt was proposed in
September 2024, "agents will read your website" was a prediction. Today
documentation platforms generate the file automatically, Chrome's Lighthouse
audits sites for one as part of its agentic-browsing checks, and the AI labs
publish llms.txt files for their own developer docs. v2 is what a proposal looks
like after real adoption tests its assumptions. The
changes page is short and worth reading in
full.

If you want to check where your own site stands against v2, my
llms.txt validator covers both halves of this
post. It follows subpath files the way v2 says agents should — paste
example.com/docs and it targets /docs/llms.txt, walking up to the root only
on a real 404 — and it reports which of the two link relations your pages
actually advertise, reading both the <link> tags and the Link: header, so a
CDN-level header counts the same as markup.

How to add llms.txt to WordPress (3 ways, pick the one that fits)

Andriy Pyvovarchuk — Thu, 10 Sep 2026 09:02:14 +0000

WordPress powers something like 40% of the web, so "how do I add an llms.txt to WordPress?" is a question a lot of people are about to ask. Good news: the file just has to be served at https://yoursite.com/llms.txt as plain text — and there are three ways to get there depending on how much access you have.

(New to the format? It's a small Markdown file that tells AI models what your site is and which pages matter — start with a quick primer, then come back.)

Option 1 — Upload the file (simplest, no plugin)

If you can reach your files (host file manager or SFTP), this is the cleanest path:

Write your llms.txt (one # title, a > summary, ## sections of links).
Upload it to the web root — usually public_html/ — so it lands at yoursite.com/llms.txt.

Done. No plugin, nothing to maintain. This is the right answer for most self- or managed-hosted WordPress sites.

Option 2 — Serve it with a snippet (no file access)

Can't upload files? Add a tiny route. Drop this in your theme's functions.php (or a code-snippets plugin):

add_action('init', function () {
  if (trim($_SERVER['REQUEST_URI'], '/') === 'llms.txt') {
    header('Content-Type: text/plain; charset=utf-8');
    echo "# Example\n\n";
    echo "> Example is a WordPress site about X. These links cover the essentials.\n\n";
    echo "## Guides\n- [Start here](https://example.com/start): The 5-minute intro\n";
    exit;
  }
});

It intercepts the request for /llms.txt and returns plain text directly — no file on disk needed.

Option 3 — Use a plugin

Several SEO/AI plugins now generate an llms.txt from your pages automatically. Handy if you don't want to touch code — but review the output. Auto-generators love to dump every page and wrap in marketing copy; llms.txt is a curated index, so trim it to the pages that actually matter.

The WordPress-specific gotchas

These trip people up more than the file itself:

Purge your cache/CDN after publishing — WordPress caching plugins and CDNs will happily serve a stale 404 for /llms.txt until you clear them.
Root, not a subfolder — it must be yoursite.com/llms.txt, not yoursite.com/blog/llms.txt.
Plain text, not HTML — make sure it's served as text/plain, not rendered through a page template as text/html.

Verify it actually works

The failure you won't see on your own: it loads for you but returns a cached 404 to everyone else, or the linked pages 404. Run your domain through an llms.txt validator — it confirms the file is served correctly as plain text and that every linked page actually loads. Aim for 100/100, then you're done.

Pick the option that matches your access, mind the cache, validate, ship. Ten minutes on the world's most common CMS.

Shopify already made you an llms.txt — and it might not be working for you

Andriy Pyvovarchuk — Thu, 10 Sep 2026 08:58:28 +0000

If you run a Shopify store, open https://yourstore.com/llms.txt right now. There's almost certainly a file there. You didn't write it — Shopify generates one for every store as part of its Agentic Commerce push.

Here's the part worth knowing: the default file often steers AI shopping agents toward Shopify's own Shop app rather than your store and your pages. It's a sensible default for Shopify. It's not necessarily the one you want.

So on Shopify the job isn't "add an llms.txt" — it's take control of the one you already have.

First, look at what you're serving

Open /llms.txt on your own domain and read it as an agent would. Ask:

Does it point at my collections, products and policies?
Or does it mostly route to Shopify infrastructure?
Is it a curated shortlist, or a dump?

That tells you whether you need to do anything at all.

Option 1 — override it with a theme template (recommended)

The cleanest way to take control. In Online Store → Themes → ⋯ → Edit code, open the Templates folder and create a new template named:

llms.txt.liquid

Shopify serves that template at /llms.txt, so its content replaces the default. Keep it short and curated:

# Example Store

> Example sells handmade ceramics. This file points AI shopping agents at the
> pages that matter.

## Catalog
- [Best sellers](https://example.com/collections/best-sellers): Top products
- [New arrivals](https://example.com/collections/new): Latest additions

## Help
- [Shipping & returns](https://example.com/policies/shipping-policy): Policies
- [Contact](https://example.com/pages/contact): Support

Because it's a Liquid template you can generate parts of it dynamically — but resist the urge to loop over every product (see below).

Option 2 — use an app

The Shopify App Store has several llms.txt apps that generate the file from your products, collections and pages, and refresh it as the catalog changes. Good if you don't want to touch theme code — but review what it outputs, because the default behaviour of most generators is "include everything."

The mistake almost everyone makes: dumping the catalog

llms.txt is a curated index, not a sitemap. A 60-line file beats a 5,000-line one, because an agent reading it has a limited context budget and every junk entry spends it.

List collections, best sellers and key policies — not all 2,000 SKUs. The agent can follow a collection link if it needs depth; it can't recover from drowning in your entire product feed.

One thing you can stop doing

If you set this up in 2024–2025, you may have uploaded a file to Shopify's CDN and added a URL redirect to reach it. That workaround is obsolete — Shopify serves /llms.txt natively now, so use a theme template or an app instead of maintaining a redirect.

Then verify it

The failure you won't notice: a file that looks right but links to collections you renamed or products you retired. Paste your domain into an llms.txt validator — it confirms the file is reachable and well-formed, and fetches every linked URL so dead links surface before an agent hits them. Re-check after big catalog changes.

Shopify handed you a head start on this one. Ten minutes of curation turns it from a file that exists into a file that sells for you.

llms-full.txt explained: the companion file to llms.txt

Andriy Pyvovarchuk — Fri, 04 Sep 2026 13:04:39 +0000

Once you've published an llms.txt, you'll run into its lesser-known sibling: llms-full.txt. Same idea, different job — and knowing when to publish which saves an AI a lot of round-trips. Here's the short version.

llms.txt is the table of contents. llms-full.txt is the whole book.

llms.txt is a compact, linked overview. A model reads it, then has to fetch each link to actually read the content.
llms-full.txt embeds the complete content of those pages directly in one file. A model ingests everything in a single request — no follow-up fetches.

That's the entire distinction. One points; the other contains.

When to publish llms-full.txt

It's optional — only llms.txt (really, just its H1) is required. Reach for llms-full.txt when:

You have documentation that's genuinely useful read in full — API refs, SDK guides, tutorials.
You want agents to work without crawling dozens of pages one by one.
You already keep Markdown sources you can concatenate, so generating it is cheap.

If your site is a handful of marketing pages, skip it — the index is enough.

Don't forget per-page `.md` versions

The spec also suggests serving a clean Markdown version of each page at the same URL with .md appended (and index.html.md for directory URLs). That gives a model a parse-friendly copy of any single page without the nav, cookie banner, and ads wrapped around your HTML. Three levels, increasing granularity:

llms.txt — the curated index
llms-full.txt — everything, in one file
page.md — one clean page on demand

Who actually does this

The llms.txt + llms-full.txt pairing shows up on documentation-heavy sites — Anthropic and Vercel among them: the index for orientation, the full dump for deep ingestion.

Before you ship either

Both files are plain text served at your root, and both fail the same quiet way: they look fine but point at (or contain links to) dead pages. Run your domain through an llms.txt validator — it fetches every linked URL and, when you check a domain, reports whether a sibling llms-full.txt exists. Start with llms.txt; add llms-full.txt when your content is worth reading end to end.

How to create your first llms.txt (a 10-minute, copy-paste guide)

Andriy Pyvovarchuk — Fri, 04 Sep 2026 12:53:06 +0000

llms.txt sounds like a spec you need to study. It isn't — it's a small Markdown file, and you can write a good one in about ten minutes. Here's every part in order, then a complete file you can paste and adapt.

1. One H1 title

The first line is a single # heading — your project or site name. This is the only strictly required part of the whole format.

# Example Docs

2. A blockquote summary

Right under the title, add a one- or two-sentence > blockquote. It's the first context a model reads, so be specific: what the thing is and who it's for.

> Example is a payments API. This file links to the docs an LLM needs to
> integrate and debug Example.

3. (Optional) a line or two of prose

You may add plain paragraphs before any section — a bit more orientation. No headings here; keep it to a sentence or two.

4. Group links under `##` sections

Each section is an ## heading followed by a Markdown list. Every item is a link with an optional description after a colon.

## Docs

- [Quickstart](https://example.com/docs/quickstart): Integrate in five minutes
- [API reference](https://example.com/docs/api): Every endpoint and parameter

Two rules that matter: use absolute https:// URLs, and write descriptions for a developer or model — not marketing copy.

5. Push nice-to-haves into `## Optional`

A section literally named ## Optional is the convention for secondary stuff an agent can skip when it needs a shorter context — changelogs, blog posts, extra references.

6. Host it at the root (or the path it covers)

Serve it at https://yourdomain.com/llms.txt as Content-Type: text/plain. Spec v2 also lets it live at a subpath that covers everything beneath it — /docs/llms.txt describes /docs/, and agents prefer the most specific file that applies. Big docs sites often also publish an llms-full.txt with the full expanded content.

7. Let your pages point at it

So agents don't have to guess the URL, advertise it from your pages:

<link rel="describedby" href="/llms.txt">

…or set the equivalent Link: response header once in your CDN config.

The complete file (paste & adapt)

# Example — Payments API

> Example is a payments API. This file links to the docs an LLM needs to
> integrate, test, and debug Example.

## Docs
- [Quickstart](https://example.com/docs/quickstart): Integrate in five minutes
- [Authentication](https://example.com/docs/auth): API keys and OAuth
- [API reference](https://example.com/docs/api): Every endpoint and parameter

## Examples
- [Sample app](https://example.com/examples/app): A working end-to-end integration

## Optional
- [Changelog](https://example.com/changelog): Release notes
- [Blog](https://example.com/blog): Background and deep dives

Swap in your own name, summary, and links. Keep it curated — the best few pages, not every page.

8. Validate before you ship

The one step people skip: actually checking it resolves. A file can be perfect Markdown and still point at 404s, which wastes a crawler's budget. Paste your domain into an llms.txt validator — it checks the structure, fetches every linked URL, and lists exactly what to fix. Aim for 100/100, then ship it.

That's the whole thing. Ten minutes, one small file, and your site finally tells AI models what it is instead of making them guess.

How to test your llms.txt before you ship it

Andriy Pyvovarchuk — Thu, 06 Aug 2026 13:29:41 +0000

An llms.txt is easy to write and easy to get subtly wrong. The file can look perfectly fine in your editor and still be malformed, or point at pages that 404 — and unlike a broken web page, nothing visibly breaks, so you never notice. Here's exactly what to check before (and after) you publish.

1. Structure

Confirm the spec basics:

exactly one # H1 title,
a > blockquote summary right under it,
## sections that each contain a Markdown list of links.

Multiple H1s and empty sections are the two most common structural mistakes. If your file has two # lines, demote one to ##.

2. Link reachability (the check almost everyone skips)

This is the big one. Every URL you link should actually resolve. A flawless-looking file that points at 404s wastes a crawler's budget and quietly erodes trust — and most "llms.txt checkers" only lint the syntax, so dead links sail straight through them.

You have to actually request each linked URL and look for anything that's broken or redirecting. This is where doing it by hand falls apart: eyeballing structure is fine, but clicking twenty links to check status codes is tedious and easy to get wrong.

3. Best practices

Absolute https:// URLs — not relative paths, not http://.
A short : description after each link so the model knows what it is.
Served as Content-Type: text/plain; charset=utf-8 (a lot of servers wrongly return text/html).

Manual vs automated

You can verify all of the above by hand. Structure, sure. But link reachability across dozens of URLs, checking content type, re-checking every time you edit the file — that's exactly the kind of tedious, error-prone work worth automating.

The fast way

Paste your domain (or the raw text) into an llms.txt validator. It checks the structure, fetches every linked URL concurrently, applies the best-practice rules above, and returns a 0-100 score with a prioritized, fixable list of findings. Re-run it whenever you change the file.

What a clean result looks like: 100/100, every link fetched, with the HTTP status, size, content type and link count broken out. A single broken link drags the score down even when the Markdown is perfect.

The 30-second checklist

[ ] One # H1, one > summary, ## sections with links
[ ] Every linked URL returns 200 (no 404s, no redirect chains)
[ ] All links absolute https://
[ ] A : description on each link
[ ] Served as text/plain; charset=utf-8
[ ] Re-checked after your latest edit

Get those six right and your llms.txt is in better shape than most on the web today. Test it once, fix what's flagged, ship it.

Which AI crawlers actually read your llms.txt? (GPTBot, ClaudeBot & friends)

Andriy Pyvovarchuk — Thu, 06 Aug 2026 05:55:44 +0000

You added an llms.txt. Reasonable next question: who's actually reading it? The honest answer surprises people — so let's clear it up, and more importantly sort out which bots you should care about and how you actually control them (spoiler: not with llms.txt).

First, split the bots into two piles

Lumping all "AI crawlers" together is where people get confused. There are two kinds, and you treat them very differently:

Answer / search crawlers — they fetch a page to answer a user's question right now, and they're the ones that get you cited in AI answers:

OpenAI: OAI-SearchBot, ChatGPT-User
Anthropic: Claude-SearchBot, Claude-User
Perplexity: PerplexityBot, Perplexity-User

You generally want these.

Training crawlers — they collect data to train future models:

GPTBot, ClaudeBot, Google-Extended, Applebot-Extended, CCBot, Bytespider, Meta-ExternalAgent

Whether to allow these is a separate, opt-out decision — and blocking them does not hurt your AI-search visibility.

So which of them read llms.txt today?

Straight answer: adoption is still early. llms.txt is a young, voluntary convention. It's picking up support in AI developer tools and agent frameworks, but most of the big crawlers above do not yet fetch it automatically. No bot is required to read it.

That sounds like a reason to skip it. It isn't — for two reasons:

The clean pages it points to help models right now. Even a crawler that ignores the file itself benefits from the well-structured, Markdown-friendly pages you curated for it.
It's cheap readiness. A correct llms.txt costs ten minutes and positions you for the moment adoption tips — which, given how fast this space moves, won't be long.

The part people get wrong: llms.txt doesn't control access

This is the big misconception. llms.txt never blocks or grants anything. It only curates — a suggested reading list. If you want to control which AI crawlers may fetch your pages, that's robots.txt's job.

A visibility-friendly setup allows the answer crawlers and makes a deliberate call on the training ones:

User-agent: OAI-SearchBot
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: GPTBot
Disallow: /

That example says "let AI search engines cite me, but keep my content out of model training." Your call on that last line — just know it's a training opt-out, not a visibility switch.

The short checklist

Allow the answer/search crawlers (OAI-SearchBot, Claude-SearchBot, PerplexityBot, …) so AI tools can cite you.
Decide training opt-outs (GPTBot, Google-Extended) deliberately — blocking them doesn't touch AI-search visibility.
Publish a curated llms.txt and then actually check it — the most common failure is a valid-looking file pointing at dead links. Run it through an llms.txt validator that fetches every URL.

llms.txt is the map; robots.txt is the gate. Keep the map honest and the gate deliberate, and you're ahead of nearly everyone.

Does llms.txt help SEO? An honest answer

Andriy Pyvovarchuk — Tue, 04 Aug 2026 06:22:19 +0000

If you searched this, you've probably also seen the other kind of post — "add llms.txt and watch your rankings climb." Let me save you the disappointment and then tell you why the file is still worth ten minutes of your time.

The short answer: no, not for classic SEO

llms.txt is not a Google ranking signal. Google has said Googlebot doesn't use it for search ranking. It's not a meta tag, not a sitemap replacement, not a backlink, and no, it won't move you up the results page. Anyone selling "llms.txt → #1 on Google" is selling snake oil.

So if your only goal is blue-link rankings, llms.txt isn't your lever. Close the tab, go fix your Core Web Vitals.

The longer answer: it helps a different kind of visibility

Still here? Good — because the question is slightly out of date. More and more, people don't get an answer about your product from a ranked list of links. They ask ChatGPT, Claude, Perplexity, or Google's AI overview, and read whatever those models say about you.

Being cleanly readable by those models is its own discipline now — people call it GEO (generative engine optimization), the AI-answer cousin of SEO. And that's exactly what llms.txt is for. A tidy one:

points models at your best, most current pages instead of letting them guess,
gives them clean context so answers about you are accurate rather than hallucinated,
pairs with Markdown versions of your pages so the model reads content, not your nav bar, cookie banner, and ads.

It won't rank you. It changes what the AI says when someone asks about you — which, increasingly, is the thing that actually drives the click.

So is it worth it?

Simple test: does your audience ask AI tools about your domain — your docs, your API, your product? If yes, llms.txt is a small file with real upside and basically no downside. Write it once, keep it curated, done.

If you're a purely local business with no documentation and nobody's asking an LLM about you, the payoff is smaller — spend the time elsewhere.

What to do instead of chasing rankings with it

Treat llms.txt as one piece of AI-readiness, alongside the things that genuinely help discovery:

a clean robots.txt that actually lets the AI crawlers you want in,
schema.org structured data,
a real sitemap,
and, yes, a curated llms.txt on top.

Then make sure the file is actually correct — the most common failure is a perfectly-formatted llms.txt that points at 404s, which does the opposite of building trust. Run it through an llms.txt validator that fetches every linked URL, and fix whatever it flags.

The honest pitch: llms.txt won't help your SEO. It'll help the thing SEO is slowly turning into. That's a better bet anyway.

I made my website callable by another AI agent — here's the actual JSON-RPC

Andriy Pyvovarchuk — Tue, 04 Aug 2026 06:01:22 +0000

Most "AI integration" still means a human reads your docs and writes a client. A2A flips that: another agent reads a small machine description of what your service does and calls it directly — no browser, no scraping, no human in the loop. I wired it up on a small tool I run (a free llms.txt validator) and it's simpler than the acronyms suggest. Here's the whole thing, with the real requests.

What A2A actually is

A2A (Agent2Agent) is an open protocol — now under the Linux Foundation — that lets AI agents discover and call each other over plain HTTP. Think of it as the agent-to-agent counterpart of a public API: instead of publishing OpenAPI docs for humans, you publish a machine description another agent can consume and invoke.

It has exactly two moving parts:

The agent card — a discovery document.
The skill — a callable JSON-RPC endpoint.

That's it. You can ship a useful A2A surface with one skill and a single method.

Part 1 — the agent card (discovery)

You serve a JSON file at /.well-known/agent-card.json describing who you are and what you can do. The required fields are name, description, version, url (your endpoint) and at least one skill:

{
  "protocolVersion": "0.3.0",
  "name": "llms.txt Validator",
  "description": "Validate a website's llms.txt and return a score with findings.",
  "url": "https://llms-txt-validator.dev/a2a",
  "skills": [{
    "id": "validate_llms_txt",
    "name": "Validate llms.txt",
    "description": "Given a domain or URL, fetch and validate its llms.txt."
  }]
}

The card is a contract, not a meta tag. An agent fetches it, sees a validate_llms_txt skill, and knows both what you offer and where to call.

Part 2 — the endpoint (a callable skill)

The card's url points at a JSON-RPC 2.0 endpoint. An agent invokes the message/send method with a message; you do the work and return a Task. Here's a real call to my agent:

POST /a2a
{ "jsonrpc": "2.0", "id": "1", "method": "message/send",
  "params": { "message": { "role": "user",
    "parts": [{ "kind": "text", "text": "validate llmstxt.org" }] } } }

…and the reply — a completed task carrying a human-readable summary and structured data the calling agent can use directly:

{ "result": { "kind": "task", "status": { "state": "completed" },
  "artifacts": [{ "parts": [
    { "kind": "text", "text": "Validated llmstxt.org: score 100/100..." },
    { "kind": "data", "data": { "ok": true, "report": { "scores": { "overall": 100 } } } }
  ] }] } }

No HTML, no parsing, no guessing. The agent asked a question in natural language and got back exactly the data it needed. You can curl this yourself.

The minimum viable A2A

You don't need the whole spec to start. Pick one real thing your service does and:

Serve a valid agent card at /.well-known/agent-card.json with one skill.
Implement the synchronous message/send method wrapping that capability.
Return a completed Task with the result as an artifact.

Streaming, task history, and push notifications are all optional — set capabilities.streaming to false and add them only when you actually need them.

A2A vs WebMCP vs MCP (they're not competitors)

A2A — networked agents calling your service over HTTP (this article).
WebMCP — an agent running inside the browser calling tools on your open page.
MCP — connecting tools to a single model or app, usually locally.

They stack rather than compete: A2A reaches networked agents, WebMCP reaches in-browser ones, MCP wires tools into one model.

The one rule: don't publish a card you can't back

It's tempting to drop an agent card to look modern and leave the endpoint returning 501. Don't. An agent that fetches your card and calls a dead url trusts you less afterward — you've spent its call for nothing. Every skill you advertise should resolve to real, working behavior. The validator I built reports whether your agent card is there and parses as valid JSON — it deliberately stops short of calling the url inside it, because firing message/send at someone's agent just because they were scanned is not a thing a checker should do. That last mile is yours: call your own endpoint and confirm it answers.

If you ship one, drop your agent card in the comments — I'd like to call it.

7 llms.txt mistakes I see constantly (and the 30-second fix for each)

Andriy Pyvovarchuk — Sun, 26 Jul 2026 06:57:50 +0000

llms.txt is deceptively simple — it's just Markdown — so most people write one in five minutes and move on. The problem: a file can look perfectly valid and still be broken in ways that quietly waste an AI crawler's time or get it ignored. Here are the seven mistakes I run into most, and the quick fix for each.

1. More than one H1

The spec wants exactly one top-level # Title. Extra # headings scramble the structure a model relies on. Fix: keep a single #, demote every other heading to ##.

2. No blockquote summary

The one- or two-sentence > blockquote right under the title is the first context a model reads. Skipping it is the single most common miss. Fix: add one line.

# Acme

> Acme turns CSV files into charts. These links cover setup, the API, and pricing.

3. Sections with no links (or no sections at all)

Empty ## sections add nothing; a wall of links with no structure is just as bad. Fix: group your key pages under real headings (## Docs, ## Guides) with a Markdown bullet list of links under each.

4. Broken links

This is the big one. A file can be flawless Markdown and still point at 404s — which burns crawl budget and erodes trust. Most linters check syntax and never actually fetch the URLs, so broken links sail straight through. Fix: verify every link resolves. (This is the check I built my validator around — it fetches each one.)

5. Relative or `http://` links

Models may not resolve relative paths like /docs/start, and plain http:// links look untrustworthy and can break under HTTPS-only crawling. Fix: use absolute https:// URLs everywhere.

6. The wrong content type

Serve the file as text/plain; charset=utf-8. A surprising number of servers return text/html for it (a default misconfiguration), and some consumers then won't treat it as llms.txt at all. Fix: set the content type explicitly.

7. A bloated, uncurated dump

llms.txt is a curated index, not a sitemap. Hundreds of undescribed links bury the signal you're trying to send. Fix: keep the best pages, add a short : description after each, and push secondary links to an ## Optional section.

Catch all seven at once

You don't have to eyeball these by hand. Paste your domain into an llms.txt validator — it scores structure, fetches every linked URL, and tells you exactly which of the above to fix, in plain language.

A clean report: 100/100 with every link fetched. Broken links drag the score down even when the Markdown is perfect — which is exactly the failure mode syntax-only checkers miss.

Fix these seven and you're ahead of most llms.txt files on the web today. The bar is genuinely low right now — which is the opportunity.

llms.txt vs robots.txt vs sitemap.xml: what actually does what

Andriy Pyvovarchuk — Sun, 26 Jul 2026 06:26:18 +0000

If you run a website in 2026 you now have three plain-text files that can live at your root and all look suspiciously similar: robots.txt, sitemap.xml, and the newcomer, llms.txt. They get confused constantly — I've seen people add llms.txt thinking it blocks AI crawlers (it doesn't) or skip it thinking robots.txt already covers it (it doesn't). Here's the clean mental model.

The one-line version

robots.txt → controls crawler access: which paths bots may or may not fetch.
sitemap.xml → lists every indexable page so search engines miss nothing.
llms.txt → curates a shortlist of high-signal pages for AI models at answer time.

Access, coverage, curation. Three different jobs.

robots.txt is about permission

robots.txt is an access-control convention: "Googlebot may crawl /blog but not /admin." It's a bouncer at the door. It doesn't describe your content, doesn't rank it, and isn't written for language models in particular. Blocking or allowing a path there says nothing about what the page is — only whether a bot may request it.

Crucially: llms.txt is not an access-control file. It can't stop an AI crawler from reading anything. If that's your goal, robots.txt (and server-side rules) is still the tool.

sitemap.xml is about coverage

A sitemap tries to be exhaustive: every URL you want indexed, so search engines don't overlook a page. It's machine-oriented XML for crawlers, not a human-readable summary, and it makes no attempt to say which pages matter most — they're all just listed.

llms.txt is about curation

llms.txt is the opposite of exhaustive. It's a hand-picked, Markdown list of the few pages that best help a model understand and use your site, each with a short description. It exists for inference — the moment a user asks ChatGPT or Claude "how do I do X with your product" — not for training datasets and not for access control.

# Acme

> Acme turns CSV files into charts. These links cover setup, the API, and pricing.

## Docs
- [Quickstart](https://example.com/quickstart): first chart in 5 minutes
- [API reference](https://example.com/api): endpoints and auth

Where a sitemap says "here is everything," llms.txt says "here is what's worth your limited context window."

You don't choose — you use all three

This is the part people miss. A strong 2026 setup has all three, each doing its job:

File	Question it answers	Audience
`robots.txt`	May you fetch this?	all crawlers
`sitemap.xml`	What should you index?	search engines
`llms.txt`	What matters most here?	AI models

robots.txt governs access, sitemap.xml ensures coverage, and llms.txt curates signal for AI. They complement each other; none replaces another.

Getting llms.txt right

The catch with llms.txt is that it's easy to write one that looks valid but points at dead links or ships as the wrong content type — and unlike the other two, almost nobody checks it. After you write yours, run it through an llms.txt validator that fetches every linked URL and confirms it resolves, then fix whatever it flags. Aim for no broken links first; structure second.

Three files, three jobs. Once that clicks, llms.txt stops being mysterious — it's just the curation layer the web didn't have until AI needed one.

I made my website callable by AI agents — the three layers (with the code I shipped)

Andriy Pyvovarchuk — Sun, 26 Jul 2026 05:26:06 +0000

"Make your site agent-ready" gets thrown around a lot, but it actually means three different things to three different kinds of AI agent — and they stack. I built a small tool (a free llms.txt validator) and wired up all three layers on it, so this is the copy-the-code version, not the hand-wavy one.

But before the three layers, let's start at the bottom — because the whole stack rests on one small file most sites still don't have.

First: what is llms.txt, and why should you care?

llms.txt is a plain-text file you put at yoursite.com/llms.txt that tells AI models what your site is about and which pages actually matter. That's it. It's written in Markdown, and it's read by AI assistants and crawlers — ChatGPT, Claude, Perplexity, and the growing crowd of AI agents — when they try to understand your site.

Here's the problem it solves. When an AI tries to "read" your website today, it gets a mess: HTML wrappers, navigation menus, cookie banners, ads, JavaScript that may not even run. It has to guess what's important and burns a big chunk of its limited context window doing it. Often it guesses wrong, or gives up.

llms.txt hands the AI a clean, curated map instead — you decide what your site is and which pages are the good ones, in priority order:

# Example — Project Management for Teams

> Example is a project-management app. This file points AI assistants
> at our most useful docs and product pages.

## Docs
- [Quickstart](https://example.com/quickstart): get set up in 5 minutes
- [API reference](https://example.com/api): every endpoint, with examples

## Product
- [Pricing](https://example.com/pricing): plans and limits

Two useful mental models:

It's the AI counterpart to sitemap.xml. A sitemap helps search engines find every page; llms.txt helps AI models understand the few pages that matter, in the format they read most cheaply.
It's the flip side of robots.txt. robots.txt says what crawlers may not touch; llms.txt says what your site is and where the value is.

Why it matters now: people increasingly get answers about your product from an AI assistant instead of clicking to your site. If the AI can't cheaply and correctly understand you, it either skips you or describes you wrong. llms.txt is your chance to feed it an accurate, curated summary — so you're represented the way you intend. It's a context layer, not a ranking trick: you're not gaming anything, you're just making your site legible to machines.

(There's an optional companion, llms-full.txt, that inlines the full text of those pages so an AI can ingest everything in one fetch. Start with llms.txt.)

That's the foundation. Now the three layers that build on it.

The tool this article is about: paste a domain, it fetches /llms.txt, checks it against the spec, and tests every link.

Here are the three independent layers. You can adopt any one on its own:

llms.txt — context for crawlers reading your site.
WebMCP — tools for an agent driving your UI inside the browser.
A2A — a callable endpoint for another agent hitting you over the network.

Crawler → browser agent → networked agent. Different agents, different mechanisms, zero overlap.

Layer 1 — llms.txt: context for crawlers

The baseline, and the highest-value one for most sites. A plain-text llms.txt at your root tells an AI crawler what your site is and which pages matter, in priority order. It's read at fetch time — no JavaScript, no protocol, no SDK. It's literally Markdown:

# Example Docs

> Developer documentation for the Example API.

## Core
- [Quickstart](https://example.com/quickstart): 5-minute setup
- [Authentication](https://example.com/auth): API keys and OAuth

## Reference
- [API reference](https://example.com/api): every endpoint

Rules that actually matter in practice:

One # H1 title, one > blockquote summary.
## sections with real Markdown links.
Absolute HTTPS URLs — relative links are the single most common failure.
Serve it as text/plain.

That's the whole layer. If you do nothing else, do this one.

Layer 2 — WebMCP: tools for in-browser agents

WebMCP lets a page expose callable tools to an AI agent running inside the browser — so instead of the agent guessing at your form fields and clicking around, it invokes a typed action you defined. There are two ways to expose a tool, and I shipped both.

Declarative — annotate a real form and the browser synthesizes a tool from it. Zero JavaScript:

<form toolname="open_llms_txt_report"
      tooldescription="Validate a site's llms.txt and open its report.">
  <input name="url" toolparamdescription="Domain or URL to validate">
</form>

Imperative — register a tool in JavaScript that returns structured data directly:

navigator.modelContext?.registerTool({
  name: "validate_llms_txt",
  description: "\"Validate a website's llms.txt; returns a 0-100 score and findings.\","
  inputSchema: {
    type: "object",
    properties: { url: { type: "string" } },
    required: ["url"]
  },
  execute: (input) =>
    fetch("/api/validate?url=" + encodeURIComponent(input.url))
      .then(r => r.json())
      .then(data => ({ content: [{ type: "text", text: JSON.stringify(data) }] }))
});

Guard the imperative call behind if (navigator.modelContext) so it's a no-op where WebMCP isn't present. Reality check: it's a Chrome origin trial today, so actual reach is tiny. Treat it as a forward signal and a clean reference, not a traffic source.

Layer 3 — A2A: a callable agent for other agents

The Agent2Agent (A2A) protocol — now under the Linux Foundation — lets other agents discover and call your service over plain HTTP. No browser, no scraping. It's the agent-to-agent counterpart of a public API.

Two parts: a discovery document (the agent card) and a callable endpoint (the skill).

The agent card — a JSON file at /.well-known/agent-card.json:

{
  "protocolVersion": "0.3.0",
  "name": "llms.txt Validator",
  "description": "Validate a website's llms.txt and return a score with findings.",
  "url": "https://llms-txt-validator.dev/a2a",
  "skills": [{
    "id": "validate_llms_txt",
    "name": "Validate llms.txt",
    "description": "Given a domain or URL, fetch and validate its llms.txt."
  }]
}

The endpoint — the card's url points at a JSON-RPC 2.0 endpoint. An agent calls message/send; you do the work and return a Task. A real call:

POST /a2a
{ "jsonrpc": "2.0", "id": "1", "method": "message/send",
  "params": { "message": { "role": "user",
    "parts": [{ "kind": "text", "text": "validate llmstxt.org" }] } } }

…and the reply — a completed task with a human-readable summary and structured data:

{ "result": { "kind": "task", "status": { "state": "completed" },
  "artifacts": [{ "parts": [
    { "kind": "text", "text": "Validated llmstxt.org: score 100/100..." },
    { "kind": "data", "data": { "ok": true, "report": { "scores": { "overall": 100 } } } }
  ] }] } }

The agent gets exactly the data it asked for. You don't need the whole protocol to start — pick one real capability, serve a card with one skill, implement the synchronous message/send, return a completed Task. Streaming, task history and push notifications are optional (capabilities.streaming: false) until you need them.

The one rule: don't publish a signal you can't back

It's tempting to drop an empty agent card or fake form annotations to make some audit go green. Don't. An agent that fetches your card and calls a dead endpoint — or invokes a tool that does nothing — trusts you less afterward. Every signal should resolve to something real — and be honest about what a checker can actually see. The validator I built reports an A2A card as present when it's there and parses; it doesn't call the endpoint behind it. For WebMCP it can only see declarative <form toolname=…> markup, because an imperative navigator.modelContext.registerTool runs in the browser and is invisible to a server-side fetch — so it says that in the result instead of reporting a blank as "absent". Whether the thing behind the signal works is still on you to test.

How to verify each layer

llms.txt — run your domain through an llms.txt validator; aim for 100/100 with no broken links.

A clean report: 100/100, with Structure / Links / Best practices scored separately. The tool fetches every linked URL, so a broken link drags the score down even when the Markdown is perfect.

WebMCP — in Chrome 149+, enable chrome://flags/#enable-webmcp-testing and run Lighthouse's Agentic Browsing audit; your tools should be listed.
A2A — fetch your /.well-known/agent-card.json, then POST a message/send call to its url and confirm you get a completed task back.

That's the whole stack: llms.txt for crawlers, WebMCP for in-browser agents, A2A for networked ones. Start with llms.txt (real value today), add WebMCP and A2A as forward signals when you have a genuine action or service to expose.

If you build one, I'd love to see it — drop your agent card in the comments.