AI systems are answering questions about your business right now. When someone asks ChatGPT, Claude, or Gemini about your services, they either get it right — or they hallucinate. Most of the time, they hallucinate.
The problem isn't that your site lacks content. It's that AI systems have no structured way to discover who you are, what you do, or how to refer to you.
robots.txt tells crawlers what to ignore. There's nothing equivalent that tells AI systems what to get right.
Until now.
AI Discovery Files
AI Discovery Files are a set of machine-readable files you place at the root of your website. They give AI systems a single, authoritative source of truth about your business.
Think of them as robots.txt for AI — except instead of blocking crawlers, you're telling AI systems exactly who you are.
The AI Discovery Files Specification defines 10 file types across three tiers:
Tier 1 — Essential (Start Here)
| File | Purpose |
|---|---|
llms.txt |
Plain-text project summary for LLMs |
llms.html |
Human-readable version (same content) |
ai.txt |
Permissions and crawling preferences |
Tier 2 — Identity & Brand
| File | Purpose |
|---|---|
ai.json |
Machine-parseable permissions (JSON) |
identity.json |
Structured business identity |
brand.txt |
Naming conventions, abbreviations, terminology |
faq-ai.txt |
Pre-answered questions for AI systems |
Tier 3 — Technical
| File | Purpose |
|---|---|
developer-ai.txt |
Technical context (stack, APIs, architecture) |
robots-ai.txt |
AI-specific crawler access rules |
llm.txt |
Per-page context (like robots.txt but for individual URLs) |
What They Look Like
Here's a minimal llms.txt:
# Acme Web Design
> Acme Web Design is a UK-based web design agency specialising in
> WordPress development for small businesses.
## Services
- WordPress website design
- E-commerce development
- Website maintenance and support
## Contact
- Website: https://www.acme-webdesign.co.uk
- Email: hello@acme-webdesign.co.uk
- Location: Manchester, UK
And the ai.txt block separately:
# AI Access Policy for acme-webdesign.co.uk
User-Agent: *
Allow-Training: no
Allow-Synthesis: yes
Allow-Citation: yes
Citation-Required: yes
Citation-Format: "Acme Web Design (https://www.acme-webdesign.co.uk)"
These are plain text files. No framework, no build step, no dependencies.
Why This Matters
Without these files, AI systems:
- Guess your business name (often wrong)
- Fabricate services you don't offer
- Confuse you with competitors
- Omit you entirely from responses
With them, every major AI system has a single source of truth. You get mentioned. You get cited. You get recommended — accurately.
This isn't SEO. This is AI visibility — making sure AI systems can discover, interpret, and correctly represent your website.
How to Implement
Option 1: Manual
Create the files by hand and drop them in your web root. The spec has examples for every file type.
Option 2: WordPress Plugin
If you're on WordPress, there's a plugin that generates all 10 files from your existing site content:
AI Discovery Files on WordPress.org
It pulls your site title, tagline, pages, and posts to auto-generate each file. You review and publish — no coding required.
Option 3: Build Your Own
The spec is open and documented. The file formats are deliberately simple — plain text and JSON. You could generate them from any CMS, static site generator, or build pipeline.
Checking Your AI Visibility
Want to see how your site scores? The AI Visibility Checker scans your domain and reports which files are present, whether they're valid, and what's missing.
Every result is deterministic — no opaque scores or prompt variance. It checks your infrastructure, not your popularity.
The Bigger Picture
We're in the early days of a shift. Search engines index pages. AI systems need to understand businesses. The sites that provide structured, machine-readable identity now will be the ones AI systems cite accurately later.
robots.txt became universal because it solved a real problem simply. AI Discovery Files solve the next one.
The AI Discovery Files Specification is open and published under CC BY 4.0. The WordPress plugin is free and open source.
Top comments (0)