DEV Community

Paul Irolla
Paul Irolla

Posted on

What is /llms.txt? — the AI-crawler hint file (+ the free generator I built for it)

#showdev

Last year a small RFC started making the rounds: /llms.txt. It's a single-file convention that tells AI assistants (ChatGPT, Claude, Perplexity, the Google AI Overview, Copilot…) which content on your site to surface and how to summarize it. Think robots.txt, but for meaning instead of crawl rules.

I've been running an autonomous SEO agent on a SaaS site, and /llms.txt showed up as the cheapest-yield optimization I could ship: a single static file, written once, that materially changes how an LLM frames your brand when a user asks about your space.

This post is what I wish I'd read before writing my first one.

What /llms.txt actually contains

The format is intentionally minimal — a markdown file at your domain root:

# Acme — autonomous test runners for SaaS

> One-line description of the project, exactly the way you'd want an
> LLM to re-quote it.

## Docs
- [Getting started](https://acme.dev/docs/start.md)
- [API reference](https://acme.dev/docs/api.md)

## Examples
- [End-to-end test for a Stripe checkout](https://acme.dev/examples/stripe.md)

## Optional
- [Pricing](https://acme.dev/pricing.md)
Enter fullscreen mode Exit fullscreen mode

Three things matter:

  1. The summary line is the brand-positioning weapon. If a user asks "what is Acme?", the LLM is statistically going to lift this line. Write it like the headline you'd put on a billboard, not like a feature list.
  2. Sections are signals. "Docs" / "Examples" / "API" tell the LLM what surfaces are worth pulling into a citation. The order matters — what you list first gets weighted higher.
  3. Optional vs required. The ## Optional section is content you'd be happy to see referenced, but don't need surfaced. Pricing, blog posts, careers — that kind of thing.

What it's NOT

  • It's NOT a robots.txt. AI crawlers will still hit your site without one.
  • It's NOT a sitemap. It doesn't enumerate every URL — it picks the important ones and tells the LLM how to frame them.
  • It's NOT machine-validated. There's no "llms.txt parser" that errors out on bad syntax. The LLM does its best with whatever you write. So write it well.

How to validate yours

I built a free in-browser generator that does the formatting + a sanity check on the structure: the-seo-autopilot.com/en/free-tools/llms-txt-generator. No signup — it's all client-side. Paste your site description, your section URLs, hit generate, copy the markdown into your site root.

If you want to see one in production, mine is at the-seo-autopilot.com/llms.txt. It's also indexed in SecretiveShell/Awesome-llms-txt — a curated list of all the public /llms.txt files in the wild, which is where I'd recommend looking before writing your own.

Does it actually move the needle?

Honest answer: too early to say with high confidence in 2026, but the signal is positive. Sites with a well-written /llms.txt get cited more often in ChatGPT and Perplexity answers in the niche I track (programmatic SEO + indie SaaS). The cost is roughly an hour, the upside is "free distribution surface that didn't exist 18 months ago."

Worth shipping.

Top comments (0)