llms.txt vs mcp.json: Why Your Site Needs Both (And What They Actually Do)

TL;DR: llms.txt helps AI assistants read your site by mapping your best content. mcp.json lets AI agents call your site by exposing structured tools. They solve different problems, and your site needs both if you want to show up in agent-driven answers instead of getting skipped.

I updated our website generator last week to auto-deploy llms.txt files in every new site. Two days later, a founder asked me: "So what exactly does this do that my sitemap doesn't?" Fair question. I realized I'd been lumping llms.txt and mcp.json together as generic "AI stuff" when they're actually solving completely different problems.

Here's what I should have said from the start: we're building websites for two audiences now. The first audience is human visitors who click through your navigation and read your pages. The second audience is AI agents that crawl your site, extract information, and either cite your content in chat responses or call your site's APIs to take actions on behalf of users.

Those agents need different signals than humans do. Your homepage hero section might look great to a visitor, but an AI assistant reading your HTML sees twenty CSS classes, three analytics scripts, and a pile of nested divs before it finds your actual value proposition. That's the gap llms.txt and mcp.json are filling.

I check this on PageSpeed Insights for every site we ship now. A perfect 3/3 Agentic Browsing audit score requires:

They're not competing standards. They're complementary files that tell AI agents two fundamentally different things about your site. One is a map. The other is a menu of actions.

llms.txt: A Content Map for AI Reading

Think of llms.txt as a table of contents written specifically for AI assistants. It's a Markdown file that lists your most important pages with brief descriptions, so when ChatGPT or Perplexity encounters your site, it knows exactly where the valuable content lives.

Instead of scraping your homepage HTML and guessing which links matter, the AI reads your llms.txt and finds your pricing page, your feature comparison, your case studies, your best blog posts. No guessing. No hallucinated URLs from cached training data.

llms.txt gives AI assistants a clean, structured summary of your most important pages, so they cite canonical content instead of scraping cluttered HTML.

I built this into version 183.71 of our website generator. Every site that runs the sitemap agent now gets an auto-generated llms.txt that includes:

• The homepage with your site's core value proposition
• Key product or feature pages with descriptions
• Your most important blog posts or content
• Contact and about pages

What it doesn't include: privacy policies, terms of service, legal disclaimers. Those aren't "important content" per the emerging standard. AI assistants don't need to cite your cookie policy when someone asks what your product does.

The practical impact: when someone asks ChatGPT "What does [your product] do?" or "How does [your company] approach X?", the AI can pull from your llms.txt map instead of guessing which page to read. It's the difference between an assistant citing your actual pricing page versus hallucinating a price from a cached blog post written two years ago.

Writing llms.txt by Hand

The format is dead simple. Here's a real example from one of our sites:

``markdown

My Product Name

https://example. com/
The homepage. We help developers ship faster by automating repetitive setup tasks.

https://example. com/pricing
Pricing page. Three tiers: Free, Pro ($29/mo), Team ($99/mo).

https://example. com/blog/how-we-built-this
Case study: how we reduced deployment time from 4 hours to 12 minutes.

https://example. com/docs
Documentation hub. Getting started guides and API reference.

https://example. com/contact
Contact page. Email support@example. com or use the form.
``

That's it. One URL per line, followed by a one-sentence description. No XML. No schema markup. Just plain Markdown that both humans and AI can read. Save it as llms.txt at your site root.

Retroactive llms.txt for Existing Sites

If you're adding llms.txt to a site that's already live, you need to decide which pages deserve to be in the map. Here's the heuristic I use:

1. Start with your sitemap.xml and filter for pages with real content (not legal pages, not redirects)
2. Rank pages by how often you want them cited when someone asks about your product
3. Keep the top 10-15 pages, write a one-line description for each
4. Deploy the file and add it to your robots.txt

The robots.txt part matters. Add this line so AI crawlers know the file exists:

``txt
User-agent: *
Allow: /llms.txt

Sitemap: https://yoursite.com/sitemap.xml
``

Most AI agents don't actively look for llms.txt yet. But the ones that do (including some custom research agents and newer Perplexity iterations) respect it immediately. You're getting in early, which means your content gets cited while competitors' sites get skipped.

mcp.json: An Action API for AI Interaction

Now here's where mcp.json (also called WebMCP) does something entirely different. It's not about reading your content. It's about calling your site. It's a machine-readable manifest that defines specific tools an AI agent can invoke to get structured data back.

Think of it as an API specification, but designed for AI agents instead of developers. A typical mcp.json file might define tools like:

• get_pricing → returns your current pricing tiers with exact dollar amounts
• get_contact → returns your support email and contact methods
• get_faq → returns your frequently asked questions with answers
• check_availability → returns whether a product or service slot is available

When an AI agent reads your mcp.json, it sees these tools and can call them during a conversation. A user asks "How much does the pro plan cost?" and instead of reading your pricing page HTML, the agent calls your get_pricing tool and gets back clean JSON with the exact answer. No scraping. No guessing. No stale cache.

mcp.json defines callable tools that give AI agents exact, structured answers: your real prices, your real email, your real product availability.

Here's the key architectural difference: llms.txt is passive. It's a map. The AI reads it, then reads your pages. mcp.json is active. It's a set of endpoints. The AI calls it and gets immediate structured responses.

As we explain in our structured data tips for AI-ready websites, these agent-facing files complement traditional SEO signals like sitemaps and JSON-LD rather than replacing them.

Writing a Basic mcp.json Manifest

The WebMCP spec is still evolving, but here's a minimal working example that defines two tools:

json
{
"name": "My Product API",
"version": "1.0.0",
"description": "Tools for interacting with My Product",
"tools": [
{
"name": "get_pricing",
"description": "Returns current pricing tiers and features",
"inputSchema": {
"type": "object",
"properties": {}
}
},
{
"name": "get_contact",
"description": "Returns contact email and support options",
"inputSchema": {
"type": "object",
"properties": {}
}
}
],
"endpoints": {
"get_pricing": "https://example.com/api/pricing",
"get_contact": "https://example.com/api/contact"
}
}


Each tool has a name, a description, and an endpoint URL. When an AI agent calls get_pricing, it hits your /api/pricing endpoint and expects JSON back. You control what data gets returned. The agent gets structured facts instead of parsing HTML.

Deploying mcp.json to Three Locations

Here's the annoying part: the WebMCP standard is new, and different AI agents expect the manifest in different places. To maximize compatibility, deploy your mcp.json file to three locations:

1. /.well-known/webmcp.json (the official spec location)
2. /mcp.json (root fallback for agents that skip .well-known)
3. /agent.json (some tools look for this name)

Yes, it's redundant. Yes, it's a pain. But until the standard solidifies, this triple deployment gives you the best chance of being discovered by agent frameworks that are experimenting with different conventions.

On most hosting platforms, you can set up a single source file and three symlinks or redirects. Here's the setup for a static site:

``bash

Create the master file

cat > .well-known/webmcp.json << EOF
{
"name": "My Product API",
"version": "1.0.0",
...
}
EOF

Create fallback copies

cp .well-known/webmcp.json mcp.json
cp .well-known/webmcp.json agent.json
``

If you're on Windows and using PowerShell, the workflow is similar but with different commands:

``powershell

Create the directory if needed

New-Item -ItemType Directory -Force -Path .well-known

Create the master file (use your actual JSON content here)

@"
{
"name": "My Product API",
"version": "1.0.0",
...
}
"@ | Out-File -FilePath .well-known\webmcp.json -Encoding utf8

Create fallback copies

Copy-Item .well-known\webmcp.json mcp.json
Copy-Item .well-known\webmcp.json agent.json
``

Deploy all three files. When the standard settles, you can drop two of them. Until then, redundancy wins.

What Makes a Good WebMCP Tool

Not every API endpoint should be a WebMCP tool. The best tools return facts that change or actions that matter. Here's what works:

Good tools:

• get_pricing → returns current prices (changes when you update pricing)
• check_availability → returns real-time availability (changes as inventory moves)
• get_latest_post → returns your most recent blog post (changes when you publish)
• search_docs → searches your documentation (user-specific results)

Bad tools:

• get_about_page → just returns static HTML (llms.txt already maps this)
• get_logo → returns an image URL (not structured data)
• subscribe_newsletter → writes data (WebMCP is read-only for now)

The pattern: if an AI agent calling your tool gets back structured data that's either dynamic or user-specific, it's a good tool. If it's just returning static content that could be read from a page, llms.txt is the better fit.

I built AI Agent Ready to solve exactly this gap. It generates both files automatically, deploys them to the right locations, and validates that your site passes the Agentic Browsing checks. If you're shipping multiple sites or managing a portfolio, automation is the only way to keep up.

Why Deploy Both (Even Though Adoption Is Still Low)

Neither llms.txt nor mcp.json is widely adopted yet. Most AI crawlers don't fully honor either one. So why am I building both into every site we ship?

Because the cost is near zero and the positioning advantage is real.

Here's my current ranking of what actually moves the needle for AI visibility today:

1. Sitemap + robots.txt that allow AI bots (most impactful right now, every major AI crawler respects this, non-negotiable)
2. JSON-LD structured data (actively used by Google today, immediate SEO benefit)
3. llms.txt (emerging standard with growing adoption for AI-search citation, low cost, future-facing)
4. mcp.json / WebMCP (most forward-looking, least adopted, but the agent-actions layer that could matter enormously as browsing agents mature)

The first two are table stakes. If your site doesn't have a sitemap or blocks AI bots in robots.txt, you're invisible. The second two are bets on where agent behavior is heading in the next 6-12 months.

Google added a fifth category to PageSpeed Insights in May 2026 called Agentic Browsing. It doesn't give a score out of 100; instead it shows a ratio like 3/3.

• llms.txt present and properly formatted
• mcp.json present with at least one callable tool
• sitemap.xml with all key pages included
• JSON-LD structured data on major pages
• passing accessibility checks (agents rely on semantic HTML)

Most sites still score 0/3 or 1/3. Getting to 3/3 takes an afternoon of work. That gap is your positioning advantage right now.

The Four-Layer Agent-Ready Stack

Here's how I think about the full stack for agent-ready websites. You need all four layers:

1. Discoverable (Sitemap + Robots.txt)

AI crawlers need to know your site exists and which pages to index. This is the foundation. Without a sitemap, you're invisible.

2. Understandable (JSON-LD Structured Data)

Your pages need machine-readable context. JSON-LD tells AI agents what kind of page they're looking at (article, product, organization) and extracts key fields (author, price, date). This is how Google's Knowledge Graph works, and it's how AI search engines will cite you.

For implementation patterns, the JSON prompt guide for AI agents walks through specific schema markup for different page types.

3. Readable (llms.txt)

Once an AI agent lands on your site, llms.txt tells it which pages matter most. This is the content map layer. It answers: "If you can only read five pages, read these five."

4. Interactive (mcp.json)

Finally, mcp.json exposes callable tools that let AI agents get exact answers without scraping. This is the actions layer. It answers: "If you need real-time data or want to verify a fact, call these endpoints."

Each layer builds on the previous one. You can't skip straight to mcp.json if your sitemap blocks AI bots. But once you have layers 1 and 2 in place, adding layers 3 and 4 is straightforward.

What Happens If You Skip This

Here's the risk: your competitors deploy llms.txt and mcp.json, and their content starts getting cited in AI-driven search results while yours gets skipped. A user asks "What are the best tools for X?" and ChatGPT cites three competitors but not you, because their sites gave the AI a clean map and yours made it scrape HTML.

Or worse: a user asks "How much does [your product] cost?" and the AI hallucinates a price from a cached blog post written two years ago, because you didn't expose a get_pricing tool that returns your current pricing.

Agent-ready sites will win citations. Sites that ignore this layer will become invisible to the next generation of search behavior. That's not speculation. It's already happening in vertical AI search engines that prioritize structured data over keyword optimization.

Start With llms.txt, Add mcp.json Later

If you're resource-constrained, start with llms.txt. It's a single Markdown file. You can write it by hand in ten minutes. Deploy it to your site root, add it to robots.txt, and you're done.

Then, when you have time to build API endpoints, add mcp.json. Start with one or two simple tools like get_pricing or get_contact. Deploy the manifest to all three locations. Test it by checking the Agentic Browsing score on PageSpeed Insights.

That's the pragmatic path: llms.txt this week, mcp.json next month. Both files are small. Both are future-facing. Both give you positioning advantage while most sites are still ignoring this layer entirely.

But even if you're doing this manually for a single site, the work is worth it. We're not optimizing for today's AI agents. We're optimizing for the ones that launch six months from now, when agent-driven search is the default and sites without llms.txt and mcp.json are invisible.

---

📦 Publishing Kit — Dev.to

Title Options (5)

Selected: llms.txt vs mcp.json: Why Your Site Needs Both (And What They Actually Do)

Alternates:

1. AI Agents Need Two Things From Your Site: A Map and a Menu
2. llms.txt and mcp.json Aren't Competing—They're Solving Different Problems
3. How to Make Your Site Readable and Actionable for AI Assistants
4. The Real Difference Between llms.txt and mcp.json (With Implementation Examples)

Slug

llms-txt-vs-mcp-json-why-your-site-needs-both

Tags

webdev, ai, tutorial, llms