llms.txt: the evidence says nobody is reading it

⏰ The 30-second version

• llms.txt is a proposed Markdown file that lists your important pages for AI systems. It is a proposal, not a standard anyone has adopted.
• Ahrefs analysed 137,000 sites and found 97% of llms.txt files received zero traffic in May 2026. The bots are not fetching them.
• SE Ranking looked at 300,000 domains and found no link between having llms.txt and being cited in AI answers.
• Google has said repeatedly that it does not use the file, and OpenAI, Anthropic and Perplexity document robots.txt and their crawler user-agents instead.
• Writing one takes an hour and costs nothing. Believing it will make AI recommend you costs a quarter.

What the file is

Put a Markdown file at yourdomain.com/llms.txt containing a short description of your site and a curated list of links to the pages you consider important. The pitch is that AI crawlers, which struggle with JavaScript-heavy pages and sprawling navigation, could read the curated list instead of guessing.

It's often compared to robots.txt, which causes most of the confusion. robots.txt controls access. llms.txt controls nothing — it's a suggestion. Search Engine Land's framing is the clearest one going: it isn't a crawl-blocker, it's a treasure map.

The idea is reasonable. The question is whether anyone is holding the map.

What the evidence shows

This is the part most articles skip, so here it is with sources.

Server logs, at scale. Ahrefs analysed 137,000 sites and found 97% of llms.txt files received zero traffic in May 2026. Not "low engagement" — zero requests.

Citations, at scale. SE Ranking's analysis of 300,000 domains found no clear relationship between llms.txt adoption and how often a domain was cited in LLM answers.

Semrush's own logs. Semrush checked their server logs from mid-August to late October 2025 and recorded zero visits to their llms.txt from Google-Extended, GPTBot, PerplexityBot or ClaudeBot. Googlebot and Bingbot fetched it a handful of times and treated it as an ordinary file.

Google, on the record. John Mueller: "AFAIK none of the AI services have said they're using LLMs.TXT (and you can tell when you look at your server logs that they don't even check for it). To me, it's comparable to the keywords meta tag." Gary Illyes added that Google has no plans to support it. When someone spotted an llms.txt on a Google developer domain and asked whether that was an endorsement, Mueller answered "no".

The deeper objection. On the Search Off the Record podcast, Mueller made a structural argument: even a perfectly written llms.txt is self-reported. If every site claims to be the best source, the file gives an LLM no way to tell them apart. That's the meta-keywords story again, and it ended the same way.

OpenAI. GPTBot has been observed fetching /llms.txt on some sites, which reads like exploration rather than endorsement. OpenAI's published bot documentation covers OAI-SearchBot, GPTBot and ChatGPT-User, and manages access through robots.txt. llms.txt is not in it.

97% — of llms.txt files got zero bot traffic (Ahrefs, 137,000 sites, May 2026)

So should you make one?

Make one if you want. It takes an hour, it can't hurt, and if the standard is ever adopted you're already done. Anthropic, Cloudflare and Vercel publish one, mostly for documentation sites where an agent already landed and needs a map.

Just be precise about what you're buying: an option on a future, not a visibility lever today. Two failure modes to avoid.

Don't let it substitute for the work. An hour on llms.txt feels like progress. It isn't the thing that gets you named when a customer asks ChatGPT which brand to buy.

Don't let an LLM write it for you and call it strategy. Mueller's aside is hard to argue with: if an LLM can generate the file by reading your site, then an LLM can read your site.

What actually moves AI visibility

The signals with evidence behind them are unglamorous:

1. Crawlable text. If your answer only exists inside an image, a PDF or a JavaScript tab, it does not exist. This is still the single most common blocker we find in scans.
2. A direct answer to a real question. Conclusion first, in a paragraph that stands on its own without the three paragraphs above it.
3. Third-party corroboration. What review sites, forums and retailers say about you often carries more weight in an answer than what your own homepage claims.
4. Structure that survives extraction. Question-shaped subheads, FAQ blocks, comparison tables — formats that quote cleanly.

Google's own AI-optimization guidance says the same thing in fewer words: you do not need special AI files to appear in generative search.

The hour spent better

• [ ] Find one buyer question your site answers only in an image
• [ ] Rewrite it as text with the conclusion first
• [ ] Add three FAQ entries in the words customers use
• [ ] Re-check the same question in 30 days Want to see which questions your site currently fails? The free scan tests one of your pages against 30 real buyer questions and shows the gaps, no sign-up needed. If you'd rather check by hand first, the GEO audit checklist takes about 20 minutes.

FAQ

Q. Does llms.txt help with AI citations?
There's no evidence that it does. Two large studies (Ahrefs, 137,000 sites; SE Ranking, 300,000 domains) found zero traffic to the files and no citation effect, and no major AI provider documents it as a signal.

Q. Is llms.txt a Google ranking factor?
No. Google has stated it does not use the file, and that you do not need any special AI text file to appear in AI Overviews or AI Mode.

Q. Why do tools like Yoast and Lighthouse mention llms.txt then?
Because it's cheap to support and popular to ask about. Chrome's developer docs describe it as an "emerging convention" that may help an agent already on your site. That's a hedge, not a finding.

Q. If I already have one, should I remove it?
No need. It costs nothing to keep. Some people noindex it so it doesn't compete with real pages in search. Just don't count it as work done on visibility.

---

Originally published at en.bultibulti.com. We run a free GEO audit there — it checks one of your pages against 30 buyer questions, no signup.