agent-manifest.txt — a proposed web standard for AI agents (formerly agents.txt)

Jasper van Veen — Fri, 13 Mar 2026 22:46:14 +0000

Editor's note (March 2026): This proposal has been renamed from agents.txt to agent-manifest.txt. The original name was chosen for its direct analogy to robots.txt, but the agents.txt namespace became crowded: an independent IETF Internet-Draft (draft-srijal-agents-policy-00) had already claimed that filename, and multiple community projects were independently using it for different purposes. The new name more accurately reflects the document's purpose — a rich capability manifest — while keeping a clean path toward formal standardisation. The GitHub repository, spec, and all references have been updated. The article text below preserves the original framing.

The web has robots.txt. It's been around since 1994, and it answers one question well: can you look at this?

AI agents don't just look. They book flights, submit forms, call APIs, authenticate as users, and transact on behalf of people. And there's no standard for any of it.

I've been thinking about this gap for a while, and last week I drafted a proposal: agent-manifest.txt (originally agents.txt).

The idea

Place a file at https://yourdomain.com/agent-manifest.txt. It tells agents what they can do, how to do it, and under what terms:

Site-Name: ExampleShop
Site-Description: Online marketplace for sustainable home goods.

Allow-Training: no
Allow-RAG: yes
Allow-Actions: no
Preferred-Interface: rest
API-Docs: https://api.exampleshop.com/openapi.json
MCP-Server: https://mcp.exampleshop.com

[Agent: *]
Allow: /products/*
Allow: /search
Disallow: /checkout

[Agent: verified-purchasing-agent]
Allow: /checkout
Auth-Required: yes
Auth-Method: oauth2
Allow-Actions: yes

Why would agents comply?

Two reasons:

Self-interest. When a site advertises an MCP server or REST API, a well-built agent wants to use it - it's faster and more reliable than scraping HTML. Compliance isn't a favour; it's rational.

Legal posture. A published machine-readable policy makes ignoring it actionable. "You had a standard and ignored it" substantially strengthens CFAA and Computer Misuse Act arguments.

What it covers (that `robots.txt` does not)

Concern	robots.txt	agent-manifest.txt
Crawl permissions	Yes	-
Action permissions	-	Yes
API / MCP discovery	-	Yes
Training / RAG consent	-	Yes
Agent identity tiers	-	Yes
Auth methods	-	Yes

They're complementary - sites should have both.

Status

Draft v0.3.0, published under CC BY 4.0. I'd love feedback, pushback, and contributions.

GitHub: https://github.com/jaspervanveen/agents-txt

Three open questions I'm genuinely unsure about:

Agent identity verification - how do you prove an agent is who it claims to be?
Should capabilities use a controlled vocabulary, or free-form strings?
As MCP matures, how tightly should this integrate with it?

What do you think?

DEV Community: Jasper van Veen

agent-manifest.txt — a proposed web standard for AI agents (formerly agents.txt)

The idea

Why would agents comply?

What it covers (that robots.txt does not)

Status

What it covers (that `robots.txt` does not)