The file nobody asked for
llms.txt is a plain-text file, placed at the root of your domain, that describes your site in a format large language models can read directly. It was proposed by Jeremy Howard in 2024 as a convention: a simple, structured summary of a website's content, written for the machines that now answer questions on behalf of users.
The idea is straightforward. When someone asks ChatGPT or Claude "is there a tool that does X?", the model does not browse your site the way a human would. It reads what it has been trained on, and increasingly, it reads the pages it is allowed to fetch at query time. A well-formed llms.txt gives it a concise, accurate map of what you offer, what you don't, and where to look for details.
For a small product with no brand recognition, this file is one of the few places you can speak directly to an AI assistant without competing for a Google ranking. It is cheap, static, and within your control.
How llms.txt differs from robots.txt and sitemap.xml
The three files are often confused because they live in the same directory and all start with a dot or end in .txt or .xml. They do different jobs.
robots.txt is a set of permissions. It tells crawlers which parts of your site they may or may not fetch. It does not describe your content. It says "you may crawl /public but not /admin." It is about access.
sitemap.xml is a list of URLs. It tells search engines which pages exist and how often they change. It does not say what those pages mean. It is about discovery.
llms.txt is a summary. It tells an AI what your site is about, who it is for, and what the key pages are. It is about understanding.
The three are complementary, not interchangeable. A crawler that respects robots.txt will still fetch a page it finds in a sitemap. But a language model that reads your llms.txt can answer a question about your product without fetching a single page. That is the difference that matters for a small product.
What a useful llms.txt contains
A useful llms.txt is not a marketing page. It is a factual brief. It should answer the questions a stranger would ask before they decide whether to click.
Here is a minimal structure that works for a small product:
# Product Name
> One sentence on what the product does and who it is for.
## What it does
- Bullet point of a core capability
- Bullet point of another core capability
- Bullet point of a third core capability
## What it does not do
- A limitation or boundary
- Another boundary
## Who it is for
- A description of the primary user
- A description of the secondary user
## Key pages
- Home: https://example.com/
- Pricing: https://example.com/pricing
- Docs: https://example.com/docs
The content should be written for a reader who has never heard of you. It should not assume context. It should state what you do, what you don't, and where to find the details.
One thing to note: the file is not a ranking signal. It does not make you appear higher in a search engine. It makes you more likely to be described accurately when an AI is asked about your space. For a product with no reputation, accuracy is the first win.
A concrete example from a real product
Actuant, a launch-readiness audit tool for AI-built apps, ships an llms.txt that follows this pattern. It opens with a one-line description, then lists what the product does, what it does not do, who it is for, pricing, and key pages.
The "What it does not do" section is the part most small products skip. Actuant states plainly that it does not audit source code and that running an audit requires a free account. That honesty serves two purposes. It prevents an AI from overstating the product's capabilities, and it sets expectations for a human reader who might be evaluating the tool.
The file also includes a section written directly for AI agents. It tells them to fetch a specific plain-text brief for MCP installation steps, and it points coding agents to a skill file. This is a newer pattern: llms.txt as a routing document, not just a summary.
That is the direction the format is heading. It is no longer just a description. It is a way to tell an AI what to fetch, what to do with it, and where the authoritative instructions live.
What happens when you skip it
When a product has no llms.txt, an AI assistant has to rely on whatever it can infer from the rest of the web. That inference is often wrong.
A model might describe your product based on a competitor's page. It might say you offer a feature you removed last month. It might say your pricing is free when you moved to a paid plan. It might say your app is a mobile app when it is a web app. Each of those errors costs you a user who asked a question and got a confident, wrong answer.
The cost is invisible. You will not see it in your analytics because the user never arrived. But it is real. For a small product, the difference between being described accurately and being described vaguely is the difference between being considered and being skipped.
The fix is a single file. It takes an hour to write and it does not require a developer. It is one of the cheapest visibility investments a small product can make.
The checks that matter
If you are building a small product, here is a short checklist for your llms.txt:
- The file is served at the root of your domain, not in a subdirectory.
- The first line is a clear, one-sentence description of what you do.
- You list what you do not do, not just what you do.
- You name your primary audience explicitly.
- You include a key pages section with the most important URLs.
- The file is plain text, not HTML or markdown with heavy formatting.
- You update it when your product changes.
The last point is the one most people miss. An llms.txt that describes a product from six months ago is worse than none, because it gives an AI a confident, outdated answer.
Actuant: the layer after deploy
Your app is live. A stranger can open it. But nobody has heard of it, and an AI that is asked about your space has nothing accurate to say. That is the gap Actuant exists to close.
Actuant runs a launch-readiness audit on a deployed web app in a real browser, scoring it out of 100 across 52 deterministic checks worth 172 weighted points. The checks cover reachability, AI-search visibility, analytics, social previews, security headers, performance, and more. No model decides the score. The engine probes a header, a file, a redirect, a tag, and returns the same verdict every time.
Where something is missing, Actuant opens a pull request with the fix. That includes an llms.txt, a robots.txt, a sitemap, and metadata. You review the changes and merge them. You stay in control.
Actuant is in early access. It is not a replacement for writing well, but it is a way to make sure the machine-readable layer of your product is not silently missing.
Top comments (2)
Does not audit source code" is the kind of line a model will repeat verbatim, so putting the boundaries in their own heading is the part I'd write first. Cheaper than correcting an invented feature later.
One thing the post skips: there's no delivery guarantee. Most engines don't fetch llms.txt at query time, they read your rendered HTML, so a fact that lives only in that file may never reach an answer. I'd mirror the same claims in on-page copy, then test by asking ChatGPT and Claude who does what in your category. Building Viewfy, the common failure was drift: the file said one thing, the models were still repeating a stale landing page.
finally someone explaining this. wonder if this actually helps with hallucinations when ai tools crawl the site...