DEV Community

Cover image for Making your content AI friendly in 2026
Prashant 🗽
Prashant 🗽

Posted on • Originally published at strategicnerds.com

Making your content AI friendly in 2026

#ai #webdev #tutorial #documentation

Your documentation has a new primary audience. While you've been optimizing for human readers, AI coding assistants have quietly become the biggest consumers of your content. Claude Desktop, Cursor, GitHub Copilot, and a dozen other tools are reading your docs, parsing your examples, and generating code based on what they find.

I've written about how this changes developer relations and how I use AI tools in my own marketing workflow. This post is the technical playbook. Everything you need to do to make your website, documentation, and blog content fully consumable by AI systems.

tl;dr A Priority checklist for making your content AI-friendly

High impact, proven techniques:

  • Write complete, runnable code examples (most important)
  • Show full request/response pairs for APIs
  • Document every error code with resolution steps
  • Publish OpenAPI specification at conventional URLs
  • Use semantic HTML with proper heading hierarchy
  • Add JSON-LD schema to all pages
  • Include language tags on all code blocks

Medium impact:

  • Configure robots.txt for AI crawlers (training vs. retrieval)
  • Maintain comprehensive sitemap with priorities
  • Implement IndexNow for content freshness
  • Provide RSS/Atom feeds with full content
  • Serve Markdown versions of documentation pages
  • Add copy buttons to code examples
  • Write scenario-based documentation

Emerging/speculative:

  • Create llms.txt with site guidance (adoption unproven)
  • Build MCP server for direct AI integration (requires security review)
  • Test documentation regularly with AI assistants

The first tier is table stakes. The second tier provides meaningful advantages. The third tier is worth experimenting with but has less proven ROI.

Read on for more descriptions of each item and why they matter.

Understanding GEO and AEO

Before diving into implementation, it helps to understand the terminology. Two concepts have emerged to describe optimization for AI systems:

Generative Engine Optimization (GEO) is the practice of optimizing content so that large language models like ChatGPT, Claude, Gemini, and Perplexity cite it as a trusted source in their responses. Instead of competing for blue links on a search results page, you're influencing what AI models generate when users ask questions.

Answer Engine Optimization (AEO) focuses specifically on AI-powered search features like Google's AI Overviews, Bing Copilot, and Perplexity's instant answers. These are answer-first surfaces that users increasingly rely on instead of clicking through to websites.

In practice, GEO and AEO overlap significantly. Both are built on the idea that AI engines need optimized content. The strategies in this guide serve both purposes.

The stakes are real: Ahrefs found that AI overviews reduced click-through rates for top-ranking Google content by 34.5% in one year. Meanwhile, AI referrals to top websites surged 357% year-over-year. Gartner predicts a 25% drop in traditional search engine volume by 2026. Being cited by AI systems is becoming as important as ranking in search results.

How AI systems consume your content

Different AI systems consume documentation in different ways. Understanding this helps prioritize your optimization efforts:

RAG (Retrieval-Augmented Generation): Tools like Perplexity and ChatGPT's browsing mode search the web, retrieve relevant pages, and use that content to generate responses. They rely on traditional crawling and indexing.

Web search: AI coding assistants like Cursor and GitHub Copilot can search documentation sites directly. They parse HTML, extract content, and use it for context.

MCP (Model Context Protocol): Some AI assistants can connect directly to your API through MCP servers, bypassing documentation entirely. This is the most direct integration.

Direct context loading: Tools like Claude Desktop and Cursor can load files directly into their context window. This is where llms-full.txt and markdown versions of documentation become valuable.

The techniques in this guide optimize for all of these consumption patterns.

The llms.txt standard

The llms.txt file is a proposed standard for helping AI systems understand your site. Created by Jeremy Howard of Answer.AI in 2024, it provides explicit guidance on how to navigate and prioritize your content.

Important caveat: As of early 2026, no major AI provider (OpenAI, Anthropic, Google) has confirmed they actually read llms.txt files during crawling. Server log analysis shows AI crawlers are not requesting these files. The standard remains aspirational. However, Anthropic has published an llms.txt on their own site, suggesting openness to the idea, and the format is useful for AI coding assistants that fetch documentation on demand.

Create a file at your domain root called llms.txt. The format uses Markdown with a specific structure:

# Your Product Name

> Brief summary of your product. This blockquote should contain key information
> that helps AI systems understand what your product does and who it's for.

Your Product is a [category] that helps developers [value prop]. This section
can contain additional context paragraphs.

## Documentation
- [Getting Started](https://example.com/docs/getting-started): Complete setup instructions with runnable examples
- [API Reference](https://example.com/docs/api-reference): Full API documentation with request/response pairs
- [Examples](https://example.com/docs/examples): Working code examples for common use cases
- [Changelog](https://example.com/docs/changelog): Recent changes and version history

## Guides
- [Authentication](https://example.com/docs/auth): API key and OAuth setup
- [Error Handling](https://example.com/docs/errors): Complete error code reference

## Optional
- [Blog](https://example.com/blog): Technical articles and tutorials
- [Community](https://example.com/community): Forums and discussions
Enter fullscreen mode Exit fullscreen mode

The specification requires:

  • An H1 with the project name (the only required element)
  • A blockquote summary with key information
  • Zero or more body paragraphs (no headings)
  • H2 sections organizing links to documentation
  • Links formatted as [Title](URL): Description
  • An "Optional" section for content that can be skipped in shorter contexts

You can also create an llms-full.txt that includes complete documentation in Markdown. This is useful for AI coding assistants that can load full context:

# Your Product - Complete Documentation

> Complete documentation for Your Product, a [category] for developers.

## Getting Started

[Full markdown content of getting started guide]

## Authentication

[Full markdown content of authentication docs]

## API Reference

[Full markdown content of API reference]
Enter fullscreen mode Exit fullscreen mode

Keep llms-full.txt to a reasonable size. One analysis found that Turborepo's llms.txt consumed 116,000 tokens, making it essentially unusable. Aim for under 100,000 tokens when possible.

Token optimization for AI consumption

AI systems have limited context windows. Even with models supporting 100k+ tokens, efficiency matters. Large context windows increase latency and cost.

Serve Markdown alongside HTML

Many documentation platforms (Mintlify, GitBook, Docusaurus) now serve .md versions of pages alongside HTML. This is often more valuable than llms.txt because it provides clean, parseable content for each page.

If your docs framework supports it, expose markdown versions at predictable URLs:

https://docs.company.com/getting-started        -> HTML for humans
https://docs.company.com/getting-started.md     -> Markdown for AI
Enter fullscreen mode Exit fullscreen mode

For Next.js or custom documentation sites, add an API route:

// app/docs/[...slug]/markdown/route.ts
import { getDocContent } from '@/lib/docs';

export async function GET(
  request: Request,
  { params }: { params: { slug: string[] } }
) {
  const content = await getDocContent(params.slug.join('/'));

  if (!content) {
    return new Response('Not found', { status: 404 });
  }

  return new Response(content.markdown, {
    headers: {
      'Content-Type': 'text/markdown; charset=utf-8',
    },
  });
}
Enter fullscreen mode Exit fullscreen mode

Markdown vs HTML token efficiency

Markdown can reduce token usage by up to 10x compared to HTML. A page that consumes 5,000 tokens as HTML might only need 500 tokens as Markdown.

<!-- HTML: ~150 tokens -->
<section class="content-section">
  <h2 id="authentication" class="section-heading">
    <a href="#authentication">Authentication</a>
  </h2>
  <p class="section-content">
    All API requests require authentication using an API key.
  </p>
</section>
Enter fullscreen mode Exit fullscreen mode 
<!-- Markdown: ~30 tokens -->
## Authentication

All API requests require authentication using an API key.
Enter fullscreen mode Exit fullscreen mode

Content chunking for large documentation

For documentation sites with hundreds of pages, create tiered llms.txt files:

  • llms.txt - Overview with links to sections (under 10k tokens)
  • llms-quickstart.txt - Just getting started content (under 20k tokens)
  • llms-api.txt - API reference only (under 50k tokens)
  • llms-full.txt - Everything (keep under 100k tokens if possible)

This lets AI systems choose the appropriate level of detail for the task at hand.

Semantic HTML structure

AI systems parse HTML to understand content structure. Clean semantic HTML makes this parsing reliable.

Document structure 

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>Page Title - Site Name</title>
  <meta name="description" content="Clear description for both search engines and AI">
</head>
<body>
  <header role="banner">
    <nav role="navigation" aria-label="Main navigation">
      <!-- Navigation -->
    </nav>
  </header>

  <main role="main">
    <article>
      <header>
        <h1>Article Title</h1>
        <p class="excerpt">Brief summary of what this page covers</p>
      </header>

      <section aria-labelledby="section-1">
        <h2 id="section-1">Section Heading</h2>
        <!-- Content -->
      </section>

      <section aria-labelledby="section-2">
        <h2 id="section-2">Another Section</h2>
        <!-- Content -->
      </section>
    </article>

    <aside role="complementary" aria-label="Related content">
      <!-- Related links, table of contents -->
    </aside>
  </main>

  <footer role="contentinfo">
    <!-- Footer -->
  </footer>
</body>
</html>
Enter fullscreen mode Exit fullscreen mode

Heading hierarchy

AI systems use heading structure to understand document organization. Never skip levels:

<!-- Good -->
<h1>API Reference</h1>
  <h2>Authentication</h2>
    <h3>API Keys</h3>
    <h3>OAuth 2.0</h3>
  <h2>Endpoints</h2>
    <h3>Users</h3>
    <h3>Projects</h3>

<!-- Bad - skipped h2 -->
<h1>API Reference</h1>
  <h3>Authentication</h3>
  <h3>Endpoints</h3>
Enter fullscreen mode Exit fullscreen mode

Anchor links on headings

Make every heading linkable. This lets AI systems reference specific sections:

<h2 id="authentication">
  <a href="#authentication" aria-label="Link to this section">
    Authentication
  </a>
</h2>
Enter fullscreen mode Exit fullscreen mode

JSON-LD structured data

JSON-LD tells AI systems exactly what type of content they're looking at. This is different from helping them parse the content itself. It's metadata about the content.

Article schema for blog posts 

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "Article",
  "headline": "Article Title",
  "description": "Clear description of what this article covers",
  "datePublished": "2026-01-12T08:00:00Z",
  "dateModified": "2026-01-12T08:00:00Z",
  "author": {
    "@type": "Person",
    "name": "Author Name",
    "url": "https://example.com/about"
  },
  "publisher": {
    "@type": "Organization",
    "name": "Company Name",
    "logo": {
      "@type": "ImageObject",
      "url": "https://example.com/logo.png"
    }
  },
  "mainEntityOfPage": {
    "@type": "WebPage",
    "@id": "https://example.com/blog/article-slug"
  }
}
</script>
Enter fullscreen mode Exit fullscreen mode

TechArticle schema for documentation

For technical documentation like getting started guides, use TechArticle instead of Article:

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "TechArticle",
  "headline": "Getting Started with Authentication",
  "description": "Complete guide to implementing authentication",
  "proficiencyLevel": "Beginner",
  "dependencies": "Node.js 18+, npm",
  "datePublished": "2026-01-12T08:00:00Z",
  "author": {
    "@type": "Organization",
    "name": "Company Name"
  }
}
</script>
Enter fullscreen mode Exit fullscreen mode

HowTo schema for tutorials

Tutorials benefit from HowTo schema that breaks down the steps:

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "HowTo",
  "name": "How to Set Up API Authentication",
  "description": "Step-by-step guide to implementing API key authentication",
  "totalTime": "PT15M",
  "step": [
    {
      "@type": "HowToStep",
      "name": "Install the SDK",
      "text": "Run npm install @company/sdk",
      "url": "https://example.com/docs/setup#install"
    },
    {
      "@type": "HowToStep",
      "name": "Configure API Keys",
      "text": "Add your API key to the configuration file",
      "url": "https://example.com/docs/setup#configure"
    },
    {
      "@type": "HowToStep",
      "name": "Make Your First Request",
      "text": "Call the API using the authenticated client",
      "url": "https://example.com/docs/setup#first-request"
    }
  ]
}
</script>
Enter fullscreen mode Exit fullscreen mode

FAQPage schema

FAQ schema helps AI systems understand question-answer pairs:

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "FAQPage",
  "mainEntity": [
    {
      "@type": "Question",
      "name": "What authentication methods do you support?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "We support API keys and OAuth 2.0. API keys are recommended for server-side applications. OAuth is required for applications that access user data."
      }
    },
    {
      "@type": "Question",
      "name": "What are the rate limits?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "Free tier: 100 requests per minute. Pro tier: 1,000 requests per minute. Enterprise: custom limits."
      }
    }
  ]
}
</script>
Enter fullscreen mode Exit fullscreen mode

SoftwareApplication schema for your product

Help AI systems understand your product itself:

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "SoftwareApplication",
  "name": "Product Name",
  "applicationCategory": "DeveloperApplication",
  "operatingSystem": "Cross-platform",
  "offers": {
    "@type": "Offer",
    "price": "0",
    "priceCurrency": "USD"
  },
  "aggregateRating": {
    "@type": "AggregateRating",
    "ratingValue": "4.8",
    "ratingCount": "1250"
  }
}
</script>
Enter fullscreen mode Exit fullscreen mode

Code blocks

Code blocks are where AI assistance happens. Make them excellent. If you're scaling content production, ensure your content production process includes these standards.

Language identification

Always specify the language. This enables syntax highlighting for humans and accurate parsing for AI:

<pre><code class="language-typescript">
const client = new Client({
  apiKey: process.env.API_KEY
});

const response = await client.users.list();
</code></pre>
Enter fullscreen mode Exit fullscreen mode

In Markdown:

(three backticks)markdown
const client = new Client({
  apiKey: process.env.API_KEY
});

const response = await client.users.list();
(three backticks)
[Sorry, the Markdown renderer in Dev.to is incomplete]
Enter fullscreen mode Exit fullscreen mode

Copy buttons

Copy buttons reduce friction. When a developer (or AI) finds the right code, copying should be instant:

function CodeBlock({ code, language }: { code: string; language: string }) {
  const [copied, setCopied] = useState(false);

  const copyToClipboard = async () => {
    await navigator.clipboard.writeText(code);
    setCopied(true);
    setTimeout(() => setCopied(false), 2000);
  };

  return (
    <div className="relative">
      <button
        onClick={copyToClipboard}
        className="absolute top-2 right-2"
        aria-label="Copy code to clipboard"
      >
        {copied ? 'Copied!' : 'Copy'}
      </button>
      <pre>
        <code className={`language-${language}`}>{code}</code>
      </pre>
    </div>
  );
}
Enter fullscreen mode Exit fullscreen mode

Complete examples

This is the most important principle. AI systems need complete, runnable code. They cannot fill in the gaps the way experienced developers can.

Bad (incomplete):

// Initialize the client
const client = new Client(/* config */);

// Make a request
const result = await client.query(/* params */);
Enter fullscreen mode Exit fullscreen mode

Good (complete):

import { Client } from '@company/sdk';

// Initialize with your API key
const client = new Client({
  apiKey: process.env.COMPANY_API_KEY,
  baseUrl: 'https://api.company.com/v1'
});

// Query all users with pagination
async function getAllUsers() {
  const users = [];
  let cursor: string | undefined;

  do {
    const response = await client.users.list({
      limit: 100,
      cursor
    });
    users.push(...response.data);
    cursor = response.nextCursor;
  } while (cursor);

  return users;
}

// Usage
const users = await getAllUsers();
console.log(`Found ${users.length} users`);
Enter fullscreen mode Exit fullscreen mode

Request and response pairs

For API documentation, always show the complete request AND response:

### Create a user

**Request:**

(three backticks)bash
curl -X POST https://api.company.com/v1/users \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "email": "user@example.com",
    "name": "Jane Developer"
  }'
(three backticks)

**Response (201 Created):**

(three backticks)json
{
  "id": "usr_abc123",
  "email": "user@example.com",
  "name": "Jane Developer",
  "created_at": "2026-01-12T08:00:00Z",
  "updated_at": "2026-01-12T08:00:00Z"
}
(three backticks)

**Error Response (400 Bad Request):**

(three backticks)json
{
  "error": {
    "code": "invalid_email",
    "message": "The email address format is invalid",
    "details": {
      "field": "email",
      "value": "not-an-email"
    }
  }
}
(three backticks)
Enter fullscreen mode Exit fullscreen mode

Exhaustive error documentation

AI systems hallucinate when they encounter gaps in documentation. Error documentation is often the biggest gap.

Document every error code your API can return:

## Error Codes

| Code | HTTP Status | Description | Resolution |
|------|-------------|-------------|------------|
| `invalid_api_key` | 401 | API key is missing or invalid | Check your API key in the dashboard |
| `rate_limited` | 429 | Too many requests | Wait and retry with exponential backoff |
| `invalid_request` | 400 | Request body is malformed | Validate JSON syntax |
| `resource_not_found` | 404 | Requested resource doesn't exist | Check the resource ID |
| `permission_denied` | 403 | API key lacks required permissions | Update API key scopes in dashboard |
| `internal_error` | 500 | Server-side error | Retry or contact support |

### Error Response Format

All errors return a consistent JSON structure:

(three backticks)json
{
  "error": {
    "code": "error_code",
    "message": "Human-readable error description",
    "details": {
      // Additional context when available
    },
    "request_id": "req_xyz789"
  }
}
(three backticks)
Enter fullscreen mode Exit fullscreen mode

Meta tags

Standard meta tags remain important for both search engines and AI systems.

<head>
  <!-- Basic meta -->
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Page Title - Site Name</title>
  <meta name="description" content="Clear, specific description under 160 characters">

  <!-- Canonical URL -->
  <link rel="canonical" href="https://example.com/current-page">

  <!-- Open Graph for social and some AI systems -->
  <meta property="og:title" content="Page Title">
  <meta property="og:description" content="Description for social sharing">
  <meta property="og:type" content="article">
  <meta property="og:url" content="https://example.com/current-page">
  <meta property="og:image" content="https://example.com/og-image.png">

  <!-- Twitter/X Cards -->
  <meta name="twitter:card" content="summary_large_image">
  <meta name="twitter:title" content="Page Title">
  <meta name="twitter:description" content="Description for Twitter">

  <!-- Article-specific meta -->
  <meta property="article:published_time" content="2026-01-12T08:00:00Z">
  <meta property="article:modified_time" content="2026-01-12T08:00:00Z">
  <meta property="article:author" content="Author Name">

  <!-- RSS/Atom feed -->
  <link rel="alternate" type="application/rss+xml" title="RSS Feed" href="/rss.xml">
  <link rel="alternate" type="application/atom+xml" title="Atom Feed" href="/atom.xml">
</head>
Enter fullscreen mode Exit fullscreen mode

robots.txt for AI crawlers

The robots.txt file controls which crawlers can access your content. Understanding the AI crawler landscape is critical because there are two distinct types:

Training crawlers scrape content to train or fine-tune AI models:

  • GPTBot (OpenAI)
  • ClaudeBot (Anthropic)
  • Google-Extended (Google/Gemini)
  • Meta-ExternalAgent (Meta/Llama)
  • Bytespider (ByteDance)
  • CCBot (Common Crawl)

Retrieval crawlers fetch content in real-time to answer user queries:

  • OAI-SearchBot (OpenAI search indexing)
  • ChatGPT-User (ChatGPT browsing)
  • Claude-User (Claude browsing)
  • PerplexityBot (Perplexity indexing)
  • Perplexity-User (Perplexity browsing)

Many sites want to be cited in AI responses but don't want their content used for model training. Here's how to configure that:

# robots.txt

User-agent: *
Allow: /

Sitemap: https://example.com/sitemap.xml

# ===========================================
# ALLOW: AI search and retrieval crawlers
# These fetch content to answer user queries
# ===========================================

User-agent: OAI-SearchBot
Allow: /

User-agent: ChatGPT-User
Allow: /

User-agent: Claude-User
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Perplexity-User
Allow: /

# ===========================================
# BLOCK: AI training crawlers
# These scrape content for model training
# ===========================================

User-agent: GPTBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: Google-Extended
Disallow: /

User-agent: Meta-ExternalAgent
Disallow: /

User-agent: Bytespider
Disallow: /

User-agent: CCBot
Disallow: /

# ===========================================
# Additional Anthropic crawlers
# ===========================================

User-agent: anthropic-ai
Disallow: /

User-agent: Claude-Web
Disallow: /

# ===========================================
# Block admin and private areas from all
# ===========================================

User-agent: *
Disallow: /admin/
Disallow: /api/internal/
Enter fullscreen mode Exit fullscreen mode

Important caveats:

  1. Compliance varies. ChatGPT-User and some retrieval crawlers don't always respect robots.txt directives. They're designed to fetch content users explicitly request.

  2. Spoofing is common. Fake crawlers can impersonate legitimate user agents. The most reliable verification is checking request IPs against officially published ranges from OpenAI, Anthropic, and others.

  3. The landscape shifts fast. GPTBot's share of AI crawler traffic surged from 5% to 30% between May 2024 and May 2025. Meta-ExternalAgent appeared at 19% in 2025. Monitor your server logs.

  4. Cloudflare changed defaults. Since July 2025, Cloudflare blocks AI bots by default. Site owners must explicitly allow AI crawlers if desired.

Sitemap optimization

Your sitemap helps AI systems discover and prioritize content:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <!-- High priority: Getting started and core docs -->
  <url>
    <loc>https://example.com/docs/getting-started</loc>
    <lastmod>2026-01-12</lastmod>
    <changefreq>weekly</changefreq>
    <priority>1.0</priority>
  </url>

  <url>
    <loc>https://example.com/docs/api-reference</loc>
    <lastmod>2026-01-10</lastmod>
    <changefreq>weekly</changefreq>
    <priority>0.9</priority>
  </url>

  <!-- Medium priority: Guides and tutorials -->
  <url>
    <loc>https://example.com/docs/guides/authentication</loc>
    <lastmod>2026-01-08</lastmod>
    <changefreq>monthly</changefreq>
    <priority>0.7</priority>
  </url>

  <!-- Lower priority: Blog posts -->
  <url>
    <loc>https://example.com/blog/some-article</loc>
    <lastmod>2026-01-05</lastmod>
    <changefreq>yearly</changefreq>
    <priority>0.5</priority>
  </url>
</urlset>
Enter fullscreen mode Exit fullscreen mode

IndexNow for content freshness

IndexNow is an open protocol created by Microsoft and Yandex that notifies search engines immediately when content changes. In an AI-powered search landscape, real-time updates matter more than ever.

When ChatGPT browses the web, it relies on Bing's infrastructure. The faster your site is indexed in Bing, the more likely your content will be found and cited in ChatGPT responses.

Implement IndexNow to push updates instantly:

// lib/indexnow.ts
const INDEXNOW_KEY = process.env.INDEXNOW_KEY;
const SITE_HOST = 'example.com';

export async function notifyIndexNow(urls: string[]) {
  if (!INDEXNOW_KEY) return;

  const response = await fetch('https://api.indexnow.org/indexnow', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      host: SITE_HOST,
      key: INDEXNOW_KEY,
      urlList: urls,
    }),
  });

  return response.ok;
}

// Call after publishing or updating content
await notifyIndexNow([
  'https://example.com/docs/new-feature',
  'https://example.com/blog/announcement',
]);
Enter fullscreen mode Exit fullscreen mode

Host your key file at https://example.com/{your-key}.txt containing just the key value.

IndexNow is supported by Bing, Yandex, Naver, Seznam.cz, and Yep. It's integrated into major SEO plugins (Yoast, Rank Math) and platforms (Cloudflare, Shopify). The Home Depot reported new pages indexed within hours instead of days after implementing IndexNow.

RSS and Atom feeds

Feeds provide a structured stream of your content. Include full content, not just excerpts:

<?xml version="1.0" encoding="UTF-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
  <title>Company Blog</title>
  <link href="https://example.com/blog" rel="alternate"/>
  <link href="https://example.com/atom.xml" rel="self"/>
  <updated>2026-01-12T08:00:00Z</updated>
  <id>https://example.com/blog</id>

  <entry>
    <title>Article Title</title>
    <link href="https://example.com/blog/article-slug"/>
    <id>https://example.com/blog/article-slug</id>
    <updated>2026-01-12T08:00:00Z</updated>
    <summary>Brief excerpt</summary>
    <content type="html">
      <!-- Full HTML content of the article -->
      <![CDATA[
        <p>Full article content here...</p>
      ]]>
    </content>
    <author>
      <name>Author Name</name>
    </author>
  </entry>
</feed>
Enter fullscreen mode Exit fullscreen mode

OpenAPI specification

If you have an API, publish an OpenAPI (Swagger) specification. This is the single most valuable artifact for AI systems trying to help developers integrate with your product:

openapi: 3.0.3
info:
  title: Company API
  description: Complete API for interacting with Company services
  version: 1.0.0
  contact:
    email: api@company.com
servers:
  - url: https://api.company.com/v1
    description: Production

paths:
  /users:
    get:
      summary: List all users
      description: Returns a paginated list of users in your organization
      operationId: listUsers
      parameters:
        - name: limit
          in: query
          description: Maximum number of users to return (1-100)
          schema:
            type: integer
            minimum: 1
            maximum: 100
            default: 20
        - name: cursor
          in: query
          description: Pagination cursor from previous response
          schema:
            type: string
      responses:
        '200':
          description: Successful response
          content:
            application/json:
              schema:
                type: object
                properties:
                  data:
                    type: array
                    items:
                      $ref: '#/components/schemas/User'
                  nextCursor:
                    type: string
                    nullable: true
              example:
                data:
                  - id: "usr_abc123"
                    email: "jane@example.com"
                    name: "Jane Developer"
                nextCursor: "cursor_xyz"
        '401':
          description: Invalid API key
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Error'

components:
  schemas:
    User:
      type: object
      required:
        - id
        - email
      properties:
        id:
          type: string
          description: Unique user identifier
          example: "usr_abc123"
        email:
          type: string
          format: email
          description: User's email address
          example: "jane@example.com"
        name:
          type: string
          description: User's display name
          example: "Jane Developer"

    Error:
      type: object
      required:
        - error
      properties:
        error:
          type: object
          required:
            - code
            - message
          properties:
            code:
              type: string
              description: Machine-readable error code
            message:
              type: string
              description: Human-readable error description
Enter fullscreen mode Exit fullscreen mode

Host your OpenAPI spec at predictable, conventional URLs. AI tools look for these locations:

  • https://api.company.com/openapi.yaml (API subdomain)
  • https://docs.company.com/openapi.json (docs root)
  • https://company.com/.well-known/openapi.yaml (well-known path)
  • https://company.com/api/openapi.json (api path)

Consider hosting both YAML and JSON formats. Link to the spec from your documentation homepage and robots.txt.

MCP compatibility

The Model Context Protocol (MCP) allows AI assistants to interact with external data sources directly. If your product has an API, consider building an MCP server.

MCP has become the universal standard for connecting AI models to tools, data, and applications. In December 2025, Anthropic donated MCP to the Agentic AI Foundation under the Linux Foundation, with OpenAI, Google, Microsoft, and Amazon as founding members. OpenAI officially adopted MCP in March 2025. There are now over 10,000 published MCP servers.

An MCP server exposes your API to AI assistants in a standardized way. The AI can then make API calls on behalf of the developer without writing integration code.

Security warning

MCP was not originally released with security as a priority. Research from Knostic in 2025 found 1,862 exposed MCP servers, all allowing unauthenticated access. These servers revealed internal tool listings, API tokens, and cloud credentials to anyone who knew how to query them.

If you build an MCP server:

  • Require authentication. Never expose tools without verifying the caller.
  • Use principle of least privilege. Don't give MCP servers service_role or admin access.
  • Bind to localhost only unless you specifically need remote access.
  • Audit tool permissions. Limit what actions each tool can perform.
  • Monitor usage. Log all MCP requests for security review.

Documentation template

Document your MCP server clearly:

## MCP Server

We provide an official MCP server for AI assistants.

### Installation

\`\`\`bash
npm install -g @company/mcp-server
\`\`\`

### Configuration (stdio transport)

Add to your Claude Desktop config (`~/Library/Application Support/Claude/claude_desktop_config.json`):

\`\`\`json
{
  "mcpServers": {
    "company": {
      "command": "company-mcp-server",
      "env": {
        "COMPANY_API_KEY": "your-api-key"
      }
    }
  }
}
\`\`\`

### Configuration (Streamable HTTP transport)

For remote MCP servers, use the HTTP transport:

\`\`\`json
{
  "mcpServers": {
    "company": {
      "url": "https://mcp.company.com/v1",
      "headers": {
        "Authorization": "Bearer your-api-key"
      }
    }
  }
}
\`\`\`

### Available Tools

The MCP server exposes these tools to AI assistants:

- \`list_users\`: List all users in your organization
- \`get_user\`: Get details for a specific user
- \`create_user\`: Create a new user
- \`update_user\`: Update an existing user
- \`delete_user\`: Delete a user

### Security

This server requires API key authentication. Never share your API key or commit it to version control. The server uses read-only database access by default; write operations require explicit scope elevation.

### Example Usage

Once configured, you can ask Claude:
- "List all users in my Company account"
- "Create a new user with email test@example.com"
- "Show me the details for user usr_abc123"
Enter fullscreen mode Exit fullscreen mode

Scenario-based documentation

AI systems need to understand not just how your product works, but what can be built with it. This aligns with the twelve types of content that work for developers and choosing what content to build. Document common scenarios:

## Common Scenarios

### Building a User Dashboard

This guide shows how to build a real-time user dashboard.

**What you'll learn:**
- Fetching user data with pagination
- Setting up webhooks for real-time updates
- Displaying user activity feeds

**Prerequisites:**
- Node.js 18+
- A Company API key
- Basic familiarity with React

**Time to complete:** 30 minutes

[Full implementation guide follows...]
Enter fullscreen mode Exit fullscreen mode 
### Implementing Role-Based Access Control

This guide demonstrates implementing RBAC using our permissions API.

**What you'll learn:**
- Creating roles and permissions
- Assigning roles to users
- Checking permissions in your application

**Use cases:**
- Multi-tenant SaaS applications
- Enterprise applications with complex permission hierarchies
- Applications with admin/user role separation

[Full implementation guide follows...]
Enter fullscreen mode Exit fullscreen mode

Testing AI consumption

Validate that AI can actually use your documentation:

  1. Test with Claude: Copy a section of your docs into Claude and ask it to generate integration code. Does the output work?

  2. Test with Cursor: Point Cursor at your docs directory. Can it answer questions about your API correctly?

  3. Monitor support tickets: Track tickets mentioning "AI generated code" or "Copilot suggested." These reveal documentation gaps.

  4. Run integration tests: Take AI-generated code from your docs and run it against your actual API. Does it work on first try?

The measurement problem

Traditional documentation metrics (page views, time on page) become less relevant when AI consumes your docs. This fundamentally changes how we measure developer marketing ROI. New metrics to track:

Integration quality metrics

  • Error rates on new integrations: If AI-generated code fails frequently, your docs have gaps
  • Time-to-first-successful-call: This should decrease as AI-assisted integration improves
  • Support tickets by topic: Spikes in basic questions indicate AI isn't finding answers

Traffic and attribution metrics

  • AI referral traffic: Track visits from AI user agents in your analytics. Filter server logs for GPTBot, ClaudeBot, PerplexityBot, ChatGPT-User, and similar agents.
  • API adoption vs. docs traffic ratio: Rising adoption with flat docs traffic means AI is working
  • llms.txt access logs: Check if AI crawlers even request your llms.txt file. Many don't.

Citation tracking

  • Brand mentions in AI responses: Tools like Qwairy and Profound track when AI systems cite your content
  • Competitor mention ratio: How often are you cited vs. competitors for relevant queries?
  • Accuracy of AI-generated code: Test whether AI assistants generate working code from your docs

Validation process

Run monthly validation checks:

  1. Ask Claude, ChatGPT, and Perplexity questions about your product
  2. Test the generated code against your actual API
  3. Track accuracy rates over time
  4. Identify gaps where AI hallucinates or gives outdated information

Putting it together

AI-mediated discovery and AI-friendly documentation is now a primary channel for reaching developers. The same principles that help AI systems, clear structure, complete examples, exhaustive error handling, also help human developers. For templates to systematize this work, see my developer marketing frameworks and templates. The companies that optimize for it will win. The ones that don't will watch their documentation work hard for diminishing returns.

For more on how AI is changing developer relations, read my piece on your docs are for AI now. For a complete overview of developer marketing strategy in 2026, see my complete developer marketing guide. And for the complete picture of developer marketing in the AI era, check out Picks and Shovels.

Top comments (0)