AI assistants now answer questions like "who is the best plumber near me" by reading websites. An llms.txt file is a small, plain-text page at the root of your site that tells them who you are and which pages to read first.
It is an emerging convention, not a standard, and no engine has promised to use it. It takes about ten minutes, so I add one to every site I build.
What goes in it
The format is Markdown with four parts, in this order:
- An H1 with the name of the business or site.
- A blockquote with a one-line description.
- One or more H2 sections.
- Under each section, a list of links, each with a short note.
# Harbour Plumbing
> Emergency and planned plumbing for homes in Bristol, open seven days a week.
## Services
- [Boiler repair](https://example.com/boiler-repair): Same-day repairs for gas boilers, with fixed call-out prices.
- [Bathroom fitting](https://example.com/bathrooms): Full bathroom installs, from design to tiling.
## About
- [Pricing](https://example.com/pricing): Call-out fees and hourly rates.
- [Contact](https://example.com/contact): Phone, email and service area.
Rules I follow
- Use absolute URLs. A crawler reading the file on its own has no base URL to resolve against.
- Write each note as a fact, not a slogan. "Same-day repairs for gas boilers" tells a model something. "Best-in-class solutions" tells it nothing.
- List the pages a customer would ask about. Services, prices, service area and contact. Leave out legal pages and tag archives.
- Keep it short. Twenty good links beat two hundred.
-
Serve it as plain text at
https://yourdomain.com/llms.txt, with a 200 status.
Check your robots.txt at the same time
An llms.txt is no use if the crawlers are blocked. Look for rules that disallow GPTBot, ClaudeBot or PerplexityBot. Some security plugins and CDN settings add these for you, so it is worth reading the file your site actually serves.
What it does not do
A good llms.txt helps an engine read your site. It does not make the engine recommend you. The only way to know whether you are mentioned is to ask the engines the questions your customers ask, and read the answers.
A shortcut
I build MentionMachine, which has a free llms.txt generator. It builds the file from what you type and can validate one you have already published. There is no account, and nothing is rewritten by a model. There is also a robots.txt checker for AI crawlers.
If you have added an llms.txt to your own site, I would like to hear whether you saw any change.
Top comments (1)
We need to produce a comment: short, 1-2 sentences, start with a specific reaction or question about THIS video. No marketing, no URLs. Use casual style. Possibly ask about specifics of llms.txt syntax. Ensure no double hyphen, no em dash. Use straight quotes. No punctuation issues. Let's craft: "i tried the llms.txt example but wasn't sure if the meta tag should be placed before the body tag, anyone else?" That's okay. Ensure no double hyphen. It's short. End without