Once you've published an llms.txt, you'll run into its lesser-known sibling: llms-full.txt. Same idea, different job — and knowing when to publish which saves an AI a lot of round-trips. Here's the short version.
llms.txt is the table of contents. llms-full.txt is the whole book.
-
llms.txtis a compact, linked overview. A model reads it, then has to fetch each link to actually read the content. -
llms-full.txtembeds the complete content of those pages directly in one file. A model ingests everything in a single request — no follow-up fetches.
That's the entire distinction. One points; the other contains.
When to publish llms-full.txt
It's optional — only llms.txt (really, just its H1) is required. Reach for llms-full.txt when:
- You have documentation that's genuinely useful read in full — API refs, SDK guides, tutorials.
- You want agents to work without crawling dozens of pages one by one.
- You already keep Markdown sources you can concatenate, so generating it is cheap.
If your site is a handful of marketing pages, skip it — the index is enough.
Don't forget per-page .md versions
The spec also suggests serving a clean Markdown version of each page at the same URL with .md appended (and index.html.md for directory URLs). That gives a model a parse-friendly copy of any single page without the nav, cookie banner, and ads wrapped around your HTML. Three levels, increasing granularity:
-
llms.txt— the curated index -
llms-full.txt— everything, in one file -
page.md— one clean page on demand
Who actually does this
The llms.txt + llms-full.txt pairing shows up on documentation-heavy sites — Anthropic and Vercel among them: the index for orientation, the full dump for deep ingestion.
Before you ship either
Both files are plain text served at your root, and both fail the same quiet way: they look fine but point at (or contain links to) dead pages. Run your domain through an llms.txt validator — it fetches every linked URL and, when you check a domain, reports whether a sibling llms-full.txt exists. Start with llms.txt; add llms-full.txt when your content is worth reading end to end.
Top comments (0)