Stop Writing READMEs for Your AI Agent. Here's What Actually Works.

#ai #techwriting #webdev #ux

A team I read about last month shipped an AI agent with 40 tools, a 90-page README, and a support queue that filled up within a week. Not because the agent was broken. Because nobody could tell, from the docs, what it would actually do when things went sideways.

Here's what nobody tells you when you start documenting an agent: everything you learned about writing docs for software does not transfer. Traditional documentation describes a system that behaves the same way every time you run it. An agent doesn't. Same prompt, same tools, different outcome depending on temperature, context window, and what the model decided to reason about that pass. If your documentation reads like a README for a REST API, you've already lost.

Your Agent Doesn't Fail Like Your API Did

A REST API either returns a 200 or it throws an error you can catalog. You document the happy path, list the error codes, done. An agent doesn't have a fixed set of failure modes. It can call the right tool with the wrong arguments. It can call three tools when one would do. It can decide, reasonably, to take an approach you never anticipated and never tested for.

Most teams still copy-paste the deterministic-software template anyway: install steps, an API reference, a changelog. That template assumes predictability. An agent's documentation has to assume the opposite. You're not writing a manual for a machine that always does the same thing. You're writing a briefing for something that reasons, and reasoning means it will sometimes reason its way into a decision you didn't design for.

That doesn't mean documentation is pointless because "it's non-deterministic anyway." It means the job changes. You're not documenting outputs. You're documenting constraints, intent, and the boundaries the agent should reason inside of.

Think about the difference in practice. A payments API's docs say: "POST /charge returns 402 if the card is declined." Done, closed case. An agent that handles refunds needs docs that say: "Only issue a refund automatically if the order is under 30 days old and under $50. Anything else, draft the refund and flag it for a human." The first is a fact about the system. The second is a policy the agent has to apply under judgment, which means your documentation is now doing the work a manager would do with a new hire, not the work a spec sheet does with a compiler.

Write Tool Descriptions Like Contracts, Not Comments

Here's where most agent documentation actually breaks: the tool descriptions themselves. Anthropic's engineering team has been blunt about this in their guidance on writing tools for agents: a vague description like "processes data" is worthless to a model deciding whether to call it. A description like "searches customer databases by email and returns name and total purchases" tells the agent exactly when it applies and what it gets back.

Treat every tool description as a two-part contract. First, activation criteria: when should the agent reach for this tool, spelled out in concrete terms, not "for data tasks." Second, operational instructions: what should the agent do once it's decided to use it, including what NOT to do. The Model Context Protocol community has converged on roughly the same split: a "when to use" section and a "you should" section, because agents get confused fast when tools overlap in function or when the boundary between two similar tools is left implicit.

Namespacing helps too. If you've got a dozen tools that all touch "customer data" in some way, group them under a shared prefix and make the differences explicit in each description. You are not writing for a developer skimming a reference page. You're writing for a model that has to pick one tool out of many, in a few hundred milliseconds, based only on the words you gave it.

Every Tool You Document Costs Real Tokens

This part surprises people who come from traditional docs: your documentation has a budget, and it's not measured in pages. It's measured in tokens, and it comes out of the same context window the agent uses to reason.

The numbers are not subtle. An enterprise MCP server with around 400 tools can burn over 400,000 tokens just loading tool schemas before the agent has processed a single message from the user. A well-documented individual tool typically runs 500 to 1,500 tokens. Load them all up front and you've spent a huge chunk of the context budget on descriptions the agent won't even use in that turn.

The fix that's emerged is progressive disclosure: load a short name and description at startup, and only pull in the full documentation when the agent actually decides it's relevant. Teams doing this report reductions in the 85x to 100x range on token usage, without losing tool-selection accuracy. It works in tiers: a lightweight discovery layer (tool name plus a one-line description, maybe 100 tokens), an activation layer that loads the full spec once the agent commits to using it, and an execution layer that's only pulled in when the tool actually runs.

If you're writing agent docs today, this changes how you structure them. Don't write one giant reference file and expect it all to load every time. Write short, sharp top-level descriptions, and push the detailed behavior, edge cases, and examples into a layer the agent only reads when it needs to.

Document What Breaks, Not Just What Works

The last habit worth killing: documentation that only describes the happy path. A model card documents the trained model. A system card documents the deployed application end to end, including where it's known to fail. An emerging pattern called agent cards takes this further for operational agents specifically: a structured record of the agent's roles, its tool integrations, its memory taxonomy, its monitoring hooks, and the boundaries of its authority.

You don't need a formal card to borrow the idea. Write down, explicitly, what the agent is not allowed to decide on its own. Write down what happens when a tool call fails silently, or returns something malformed, or times out. Write down what "good enough" looks like versus what should escalate to a human. That's the part of the documentation that actually gets read when something goes wrong in production, and it's the part almost every team skips because it's less satisfying to write than the feature list.

I've seen this play out the same way twice now: a team ships an agent, the demo goes great because the demo only ever exercises the happy path, and the first real production incident happens because nobody wrote down what the agent should do when a downstream tool returns an empty result instead of an error. The agent doesn't know that's a failure. It just sees data, and it keeps going. That gap isn't a model problem. It's a documentation gap, and it's yours to close.

None of this is about writing more. Most agent documentation I've seen is too long in the wrong places (exhaustive parameter tables) and too short in the right ones (what happens when it's wrong). Cut the padding. Spend the words on activation criteria, token cost, and failure behavior. That's the documentation an agent, and the human debugging it at 2am, actually needs.