In my last post I scored major websites on AI readiness, and the most common failure by far was the simplest one: no llms.txt file. A national newspaper scored 33/100 largely because of it. This post is the fix.
What the file is
llms.txt is a plain-text file at yoursite.com/llms.txt that tells AI assistants what your site is and where the important pages live. Think robots.txt, but as a welcome mat instead of a bouncer. Shopify has one. Stripe's docs have one. WordPress.org has one. The AI systems reading the web increasingly expect one.
Writing it by hand
The format is deliberately simple Markdown. A minimal correct file:
`# Your Site Name
One sentence describing what your site is and who it serves.
`## Docs
- [Getting started](https://yoursite.com/docs/start): How to set up in five minutes
- [API reference](https://yoursite.com/docs/api): Full endpoint documentation
Blog
- [Why we built X](https://yoursite.com/blog/why-x): The problem and our approach``
Rules of thumb: one H1 with your site name, a blockquote summary right under it, then sections (H2) grouping your most important pages, each as a Markdown link with a one-line description. Skip login pages, legal boilerplate and pagination; include the pages you'd want an AI to quote. Save it as plain text, upload it to your web root, done.
For a five-page site, honestly, write it by hand; it takes ten minutes and you know your content best.
Generating it at scale
The manual approach stops being fun at 50 pages, and stops being possible across a portfolio of client sites. That's the gap I built for: llms.txt Generator crawls a site (sitemap-first), reads each page's own title and meta description, groups pages into sections by URL structure, and outputs a spec-correct llms.txt ready to upload. About $0.02 per site.
One honest design choice worth knowing: it does NOT use AI to invent descriptions. Everything in the generated file comes from your site's own metadata. If your meta descriptions are bad, the file will show you that too, which is its own kind of useful.
The check-and-fix loop
The workflow I'd recommend for agencies: run an AI visibility audit across your client list (half a cent per site), sort by score, generate llms.txt files for the ones missing them, upload, and re-run the audit to show the before/after. That's a deliverable clients understand, built from about ten minutes of actual work.
Top comments (0)