DEV Community: Everyday Dev

Integrate Any LLMs.txt into Your MCP (with Stripe as the Example)

Everyday Dev — Mon, 01 Sep 2025 11:50:13 +0000

If you’re building or using AI agents, you want answers that are current, source‑backed, and token‑efficient. That’s exactly what llms.txt plus the Model Context Protocol (MCP) delivers: a way to discover the right docs, fetch only what you need, and wire it straight into your IDE or agent host.

In this guide, you’ll learn:

What llms.txt is and why it matters
How to consume any llms.txt (we’ll use Stripe’s as an example)
How to integrate docs via an MCP server into Cursor, Windsurf, or Claude Desktop
Best practices for security, performance, and governance
Troubleshooting tips and advanced patterns

We’ll use Stripe’s public llms.txt at:

https://docs.stripe.com/llms.txt

What is llms.txt?

llms.txt is a human‑ and LLM‑readable Markdown file at the root of a docs site (for example, /llms.txt). It’s like a compact, curated “sitemap for AI” that lists the most important, LLM‑friendly pages—often with .md mirrors for clean parsing and minimal tokens.

Why it’s useful:

Curated: Points your agent at the best sources, not every page
Efficient: Markdown mirrors parse cleanly and compress well
Reliable: Reduces hallucinations by nudging agents to fetch real docs

Core structure (typical):

H1 title
Optional summary in a blockquote
H2 sections with bullet‑listed links: [Title](URL) — optional note
Optional section (commonly titled “Optional” or “Additional”) for nice‑to‑have pages

Try opening Stripe’s: https://docs.stripe.com/llms.txt

How llms.txt and MCP Work Together

llms.txt: Discovery layer. It tells agents where the “good stuff” lives.
MCP (Model Context Protocol): Execution layer. It standardizes how an agent host (like an IDE) talks to tools/servers that can read resources (docs), call tools (APIs), and apply prompts.

Put simply: Use an MCP “docs server” that knows how to read llms.txt and fetch the linked pages on demand. Your IDE/agent then calls MCP tools to grab only the pages it needs to answer a question.

We’ll use an off‑the‑shelf server (mcpdoc) to make this easy.

Quick Start: Consume Stripe’s llms.txt with an MCP Server

We’ll run a local MCP server that:

Registers a “Stripe Docs” source via its llms.txt
Exposes tools to list sources and fetch docs
Speaks stdio (for IDEs) or SSE (for browser tooling/inspection)

Prerequisites

Install uv (a fast Python package runner):

curl -LsSf https://astral.sh/uv/install.sh | sh

Verify:

uvx --version

Option A: Run Over SSE (Great for Inspection)

uvx --from mcpdoc mcpdoc \
  --urls "Stripe:https://docs.stripe.com/llms.txt" \
  --transport sse \
  --port 8082 \
  --host localhost

The server will fetch and index Stripe’s llms.txt.
You can point an MCP inspector at it to explore tools/resources.

Optional: Open the MCP Inspector in another terminal:

npx @modelcontextprotocol/inspector

Then connect to:

URL: http://localhost:8082

Option B: Run Over stdio (For IDE Integration)

Many IDEs/hosts expect stdio transport. Just switch the flag:

uvx --from mcpdoc mcpdoc \
  --urls "Stripe:https://docs.stripe.com/llms.txt" \
  --transport stdio

We’ll wire this into popular IDEs next.

Integrate with Your IDE (Cursor, Windsurf, Claude Desktop)

Below are minimal configurations that register the server and nudge the agent to fetch docs before answering.

Cursor

Edit or create ~/.cursor/mcp.json:

{
  "mcpServers": {
    "stripe-docs-mcp": {
      "command": "uvx",
      "args": [
        "--from",
        "mcpdoc",
        "mcpdoc",
        "--urls",
        "Stripe:https://docs.stripe.com/llms.txt",
        "--transport",
        "stdio"
      ]
    }
  }
}

Optional: Add a simple “use‑the‑docs‑first” rule in Cursor’s User Rules:

For any questions about Stripe, use the MCP server "stripe-docs-mcp":
1) call list_doc_sources
2) call fetch_docs for Stripe's llms.txt to see curated pages
3) select and fetch the most relevant .md pages
4) answer citing the fetched URLs

Windsurf (Codeium)

Edit ~/.codeium/windsurf/mcp_config.json:

{
  "mcpServers": {
    "stripe-docs-mcp": {
      "command": "uvx",
      "args": [
        "--from",
        "mcpdoc",
        "mcpdoc",
        "--urls",
        "Stripe:https://docs.stripe.com/llms.txt",
        "--transport",
        "stdio"
      ]
    }
  }
}

Add a corresponding “fetch before answer” instruction in Windsurf’s settings.

Claude Desktop

On macOS, edit:
~/Library/Application Support/Claude/claude_desktop_config.json

{
  "mcpServers": {
    "stripe-docs-mcp": {
      "command": "uvx",
      "args": [
        "--from",
        "mcpdoc",
        "mcpdoc",
        "--urls",
        "Stripe:https://docs.stripe.com/llms.txt",
        "--transport",
        "stdio"
      ]
    }
  }
}

Tip: If Python path issues arise, add an explicit interpreter:

"args": [
  "--python",
  "/usr/bin/python3",
  "--from",
  "mcpdoc",
  "mcpdoc",
  "--urls",
  "Stripe:https://docs.stripe.com/llms.txt",
  "--transport",
  "stdio"
]

How to Use It in Practice (Agent Flow)

Ask your IDE agent something like:

“How does Stripe recommend handling API version upgrades?”
“Show me how to verify webhook signatures with Stripe.”

The ideal flow:
1) list_doc_sources → confirms Stripe source is registered
2) fetch_docs on https://docs.stripe.com/llms.txt → returns curated links
3) fetch_docs on the most relevant .md pages (for example, api/versioning.md, upgrades.md, webhooks.md, webhooks/signature.md)
4) Agent answers and cites the exact URLs it fetched

This approach yields:

Current answers directly grounded in Stripe docs
Minimal token waste
Clear traceability via MCP tool logs

Useful Stripe Pages Typically Found via llms.txt

API overview: https://docs.stripe.com/api.md
Versioning: https://docs.stripe.com/api/versioning.md
Upgrades: https://docs.stripe.com/upgrades.md
Testing: https://docs.stripe.com/testing.md
Webhooks: https://docs.stripe.com/webhooks.md
Webhook signatures: https://docs.stripe.com/webhooks/signature.md
Connect: https://docs.stripe.com/connect.md

Note: Always rely on the live llms.txt to see the authoritative, current set.

Advanced: Combine Multiple llms.txt Sources

Your work might span multiple ecosystems (e.g., Stripe + LangChain + your internal docs). Point the server at several llms.txt files:

uvx --from mcpdoc mcpdoc \
  --urls \
  "Stripe:https://docs.stripe.com/llms.txt" \
  "LangChain:https://python.langchain.com/llms.txt" \
  "LangGraph:https://langchain-ai.github.io/langgraph/llms.txt" \
  --transport sse \
  --port 8082 \
  --host localhost

Your agent can now fetch across these sources, still grounded in curated pages.

Building Your Own Docs MCP Server (Optional)

If you prefer custom logic or internal/private docs:

Expose resources:
- One resource for the llms.txt itself
- One resource per linked .md page (on demand)
Expose tools:
- list_doc_sources to enumerate registered llms.txt endpoints
- fetch_docs(urls: string[]) to retrieve pages as needed
Add authentication for private sources (API keys, OAuth) and enforce domain allowlists.

Skeleton (Python) with a pseudo‑MCP server outline:

import asyncio
from typing import List
import httpx

ALLOWED_DOMAINS = {"docs.stripe.com"}

async def fetch_text(url: str) -> str:
    domain = url.split("/")[2]
    if domain not in ALLOWED_DOMAINS:
        raise ValueError(f"Domain not allowed: {domain}")
    async with httpx.AsyncClient(timeout=30) as client:
        r = await client.get(url, headers={"Accept": "text/markdown,text/plain,*/*"})
        r.raise_for_status()
        return r.text

# Pseudo-MCP handlers:
async def list_doc_sources():
    return [{"label": "Stripe", "url": "https://docs.stripe.com/llms.txt"}]

async def fetch_docs(urls: List[str]):
    results = []
    for url in urls:
        try:
            text = await fetch_text(url)
            results.append({"url": url, "ok": True, "content": text})
        except Exception as e:
            results.append({"url": url, "ok": False, "error": str(e)})
    return results

# Wire these into your MCP SDK of choice (FastMCP, custom, etc.)

Use an MCP SDK (for example, FastMCP) to register these as tools/resources with stdio/SSE transports.

Security, Performance, and Governance

Trust but verify: llms.txt is discovery, not trust. Enforce domain allowlists in your server and host.
Prefer .md mirrors: Faster, cleaner parsing. Many sites expose .md or content‑negotiated Markdown.
Rate limits & caching:
- Cache fetched pages by URL and ETag/Last‑Modified.
- Backoff on 429s; respect robots and publisher guidance if applicable.
Permissions:
- For private docs, authenticate and log all access.
- Keep IDE/host tool traces enabled for auditing.
Governance:
- If you publish docs, add /llms.txt and consider an “Optional” section for secondary content.
- Review and prune your llms.txt to keep it tight and useful.

Troubleshooting

Domain blocked errors:

Add --allowed-domains for off‑domain links, e.g.:

uvx --from mcpdoc mcpdoc \
  --urls "Stripe:https://docs.stripe.com/llms.txt" \
  --allowed-domains docs.stripe.com,anotherdomain.com \
  --transport stdio

Tools not visible in IDE:
- Ensure the process is running and using --transport stdio.
- Validate your JSON config paths and syntax.
Python/uv path issues:
- Add --python /path/to/python to uvx args (Claude Desktop often needs this).
Agent ignores the server:
- Add a short rule reminding it to use list_doc_sources and fetch_docs before answering Stripe questions.

Example: A Complete Developer Flow

1) Start server (stdio for IDE use):

uvx --from mcpdoc mcpdoc \
  --urls "Stripe:https://docs.stripe.com/llms.txt" \
  --transport stdio

2) Configure your IDE (Cursor/Windsurf/Claude Desktop) as shown above.

3) Ask: “What’s the recommended approach to handle Stripe webhook signature verification?”

Agent calls list_doc_sources
Agent fetches https://docs.stripe.com/llms.txt
Agent fetches https://docs.stripe.com/webhooks.md and https://docs.stripe.com/webhooks/signature.md
Agent answers with steps and includes links for verification

Result: An answer grounded in live Stripe docs, fetched just‑in‑time.

Wrap‑Up

llms.txt gives your agents a curated map; MCP turns that map into action. By plugging Stripe’s llms.txt (or any other) into an MCP docs server, you get:

Source‑backed answers
Token efficiency and faster runs
Auditable, composable workflows across multiple documentation sets

If you’d like, I can:

Generate a ready‑to‑paste IDE config for your team
Add rules that force “fetch before answer” for your key vendors
Scaffold a custom MCP server that merges Stripe docs with your internal guides behind auth

Happy building—and happy fetching.

llms.txt: From “Nice to Have” to Practical Advantage

Everyday Dev — Mon, 01 Sep 2025 07:37:41 +0000

Large Language Models (LLMs) don’t “browse” websites like humans. They fetch targeted context and synthesize answers. That’s where llms.txt comes in: a compact, human‑ and LLM‑readable map that highlights the most important, LLM‑friendly pages of your site.

Think of it as “sitemap for AI”—but curated. Where robots.txt tells crawlers what not to index, llms.txt tells LLMs what is most worth reading and how to read it efficiently.

What llms.txt Is (and What It Isn’t)

It is:
- A Markdown file at the root of your site (usually at /llms.txt).
- A curated index of high‑signal pages, ideally with Markdown mirrors (.md) for clean parsing.
- Structured to be easy for agents and tools to parse quickly.
It isn’t:
- A full sitemap clone or a complete crawl target.
- A ranking/SEO silver bullet.
- A magic switch that all LLMs use automatically today.

The practical value today: it gives your developers and AI tools a stable, predictable way to fetch the right docs, examples, and references—on demand, with fewer tokens and fewer hallucinations.

State of Adoption (Reality Check)

Many early implementations copy a sitemap into Markdown. This is not LLM‑friendly and defeats the purpose. Curate, don’t dump.
Major LLM providers vary in how they use llms.txt right now. Adoption is growing in the developer‑tools ecosystem (IDEs, MCP servers, agents) because it immediately improves answer quality and traceability.
Low effort, immediate benefits: Even a minimal, well‑curated llms.txt helps your team’s agents today, regardless of broad search engine adoption.

Bottom line: Don’t wait for a “universal mandate.” Make your docs better for your own agents and developer workflows now.

A Good llms.txt: Structure and Conventions

Aim for clarity and token efficiency. A typical llms.txt:

H1 title
Optional one‑sentence summary (blockquote)
H2 sections grouping related links
Bulleted links to the most helpful pages, preferably .md versions
An “Optional” or “Additional” section for nice‑to‑have pages

Example skeleton:

# Acme Docs

> Official developer documentation for Acme APIs, SDKs, and integrations.

## Getting Started
- [Overview](https://docs.acme.com/overview.md)
- [Quickstart](https://docs.acme.com/quickstart.md)

## API
- [API Overview](https://docs.acme.com/api.md)
- [Authentication](https://docs.acme.com/api/auth.md)
- [Errors](https://docs.acme.com/api/errors.md)

## Webhooks
- [Webhooks](https://docs.acme.com/webhooks.md)
- [Signatures](https://docs.acme.com/webhooks/signature.md)

## Optional
- [Changelog](https://docs.acme.com/changelog.md)

Key tips:

Prefer .md mirrors where possible.
Keep sections short and editorially selected (10–50 links beats 500).
Link canonical pages, not marketing LPs or UI‑heavy content.

The Fastest Path: Autogenerate llms.txt with llms.page

If you don’t have llms.txt yet, you can bootstrap one automatically.

Visit: llms.page
Or fetch directly from your server, on demand:
- https://get.llms.page/{yourdomain}/llms.txt

Example:

https://get.llms.page/github.com/llms.txt

How it works:

Send a GET request to https://get.llms.page/example.com/llms.txt.
The service analyzes your homepage/internal links and produces a curated llms.txt‑style Markdown file on the fly.
You can fetch and serve that response directly from your own domain, or cache it locally if you prefer. No scheduler required.

Note: Auto‑generated files are a great starting point. For best results, review and prune to emphasize the most useful pages and, when possible, swap links to .md mirrors.

Integration Options

You have two easy ways to publish:

1) Redirect your /llms.txt to llms.page

Pros: Zero backend changes, always fresh.
Cons: Depends on a third‑party service.

Nginx example:

location = /llms.txt {
  return 302 https://get.llms.page/$host/llms.txt;
}

Apache example:

Redirect 302 /llms.txt https://get.llms.page/%{HTTP_HOST}/llms.txt

2) Fetch and serve from your domain

Pros: Full control, can cache, works offline, stable.
Cons: You’re responsible for when/how you refresh.

Simple fetch‑and‑serve (Node, on request or during build/deploy):

# fetch_llms_txt.sh
node fetch_llms_txt.mjs

// fetch_llms_txt.mjs
import fs from "node:fs/promises";
import https from "node:https";

const domain = process.argv[2] || "example.com";
const url = `https://get.llms.page/${domain}/llms.txt`;

https.get(url, (res) => {
  let data = "";
  res.on("data", (chunk) => (data += chunk));
  res.on("end", async () => {
    if (res.statusCode === 200) {
      await fs.writeFile("./public/llms.txt", data, "utf8");
      console.log("Wrote ./public/llms.txt");
    } else {
      console.error("Failed:", res.statusCode, data.slice(0, 500));
      process.exit(1);
    }
  });
});

Serve ./public/llms.txt at https://yourdomain.com/llms.txt.
Optional: add a periodic refresh (cron) if you want it to stay fresh without redeploys:

0 2 * * * /usr/local/bin/bash /path/fetch_llms_txt.sh yourdomain.com

Why Bother Now? Practical Benefits

Faster, cleaner answers for your team’s AI tools and IDEs
Lower token usage by pointing agents to clean Markdown
Less hallucination, more citations
Easy to automate and keep up to date

Bonus: Plug llms.txt into Your MCP Workflow

If you already use the Model Context Protocol (MCP), you can let your IDE/agent fetch the right docs automatically.

Everyday Dev

Sep 1 '25

Integrate Any LLMs.txt into Your MCP (with Stripe as the Example)

#mcp #llm #ai #tutorial

Comments

6 min read

FAQ

Will this help SEO?
- Indirectly at best. The main benefit is better agent performance and more reliable, cited answers for your users and teams.
Do I need perfect Markdown mirrors?
- No, but they help a lot. Start with HTML if you must, then add .md mirrors over time.
What if my site structure changes often?
- Use the llms.page redirect, fetch‑and‑serve on demand, or add an optional scheduled refresh to keep /llms.txt fresh with minimal effort.

Conclusion

llms.txt is low‑effort leverage: a lightweight, curated map that makes your content easier for agents to consume accurately and efficiently. Autogenerate it today, polish the top sections, and wire it into your MCP workflow so your team gets faster, source‑backed answers—no heavy infrastructure required.

Automate Emails with AI: Building an Email Agent in n8n

Everyday Dev — Sun, 23 Mar 2025 15:15:39 +0000

In this tutorial, we'll walk through building an AI-powered email assistant using n8n, OpenAI, and Gmail. This agent will send, retrieve, and reply to emails—entirely driven by natural language.

⚙️ What You'll Build

A conversational email agent that can:

Send emails via AI prompts
Read your inbox
Reply to messages
Maintain context across chats

Let’s dive in.

🧠 Prerequisites

An n8n instance (self-hosted or cloud)
Gmail OAuth2 credentials set up in n8n
OpenAI API credentials
Basic familiarity with n8n workflows

🗺️ Workflow Blueprint

Here's the high-level flow:

Trigger – Waits for a chat message
AI Agent – Parses intent (send, get, reply)
OpenAI Chat Model – Powers the agent's brain
Gmail Tool – Performs email actions
Memory Buffer – Maintains chat context

🔧 Step-by-Step Setup

1. Trigger: Start When a Chat Message Is Received

Use the When chat message received node. This will simulate a user sending instructions to the agent, like:

Send an email to alice@example.com saying “Meeting moved to 3 PM.”

2. AI Agent: Interpret the Command

Add an AI Agent node from the Advanced AI collection.

Prompt Type: Define Below
Text: {{ $json.chatInput }}
Connect this to:
- Language Model (next step)
- Memory (Simple Memory node)
- Tools (Gmail nodes)

3. LLM: Add the OpenAI Chat Model

Use the OpenAI Chat Model node.

Model: gpt-4o-mini (or another OpenAI model)
Provide your OpenAI API credentials

Connect this to the AI Agent via Chat Model.

4. Memory: Store Chat Context

Add a Simple Memory node.

Connect it to the AI Agent via Memory.

This allows your agent to remember previous commands and carry conversation context.

📬 Gmail Tool Integration

You’ll need to authenticate with Gmail via OAuth2 in n8n.

a) Send Emails

Add a Gmail Tool node:

Operation: Send
To: {{ $fromAI('To') }}
Subject: {{ $fromAI('Subject') }}
Message: {{ $fromAI('Message') }}

Connect this to the AI Agent via Tool.

b) Get Recent Emails

Add another Gmail Tool node:

Operation: Get Many
Limit: 10
Same credentials, connected to the AI Agent.

c) Reply to Emails

Add one more Gmail Tool node:

Operation: Reply
Message ID: {{ $fromAI('Message_ID') }}
Message: {{ $fromAI('Message') }}

Same connection and credential setup.

🚀 Example Prompts

Try any of these in your chat interface:

Send an email to john@example.com with subject “Invoice” and say “Please find the attached invoice.”

Show me my latest 5 emails.

Reply to the email from Jane saying “Thanks, sounds good!”

🧪 Test & Deploy

Once everything is connected:

Activate the workflow
Send chat prompts to trigger the workflow
Watch your AI assistant handle your emails ✨

📦 Final Thoughts

This Email Agent leverages the power of LLMs and n8n's automation to streamline your inbox. Expand this further by integrating file attachments, smart filtering, or even calendar scheduling.

💡 Pro tip: You can extend this to other email platforms like Outlook with minimal changes.