Most people install our tools through an agent now
We build a few developer products — an event-ledger database, an S3-compatible
object store, and others. Over the last few months we noticed a shift: most
people who try them don't start in a browser. They tell a coding agent “add this
to my project,” and the agent does the install and the wiring.
So we started watching agents work against our documentation.
Watching an agent guess
Most of the time it went fine. But every so often the agent would invent a
config flag that didn't exist, or reach for the wrong port, or call an endpoint
that was almost right. It wasn't the model being dumb — our docs were written
to persuade and onboard a human, and the agent was reconstructing facts from
prose three paragraphs apart. A fact an agent has to infer is a fact you failed
to state.
A second document, for a different reader
So per release we started shipping a second file: documentation.ai.md. Not a
terser translation of the human docs — a different document, for a different
reader, held to a stricter standard of precision.
It's English, dense, and self-sufficient: an agent that has read only that file
can install, configure and call the product. Fixed sections, in order:
- One-line identity
- Install / run — exact commands, exact image names and ports
- Configuration — every env var / key, with its default
- API quickstart — a real request/response, not pseudocode
- Admin surface — how the first credential is obtained, the common operations
- Architecture facts that affect integration
- Links
No marketing language. Absolute honesty about status — if something is early or
unaudited, the file says so, because an agent that recommends an unstable feature
to a user because the doc oversold it is a real failure.
“Why not just llms.txt?”
Fair question, and we get it a lot. We're not trying to replace
llms.txt. It's a great site-level index of your content.
documentation.ai.md is a per-product, per-release operational doc — closer in
spirit to llms-full.txt: complete, not an index. They compose cleanly.
We made it a standard
It worked well enough across our products that we wrote it up as an open standard
(CC BY 4.0), so anyone can adopt it:
- GitHub: https://github.com/iwasoftcom/ai-docs-standard
- Write-up: https://iwasoft.com/blog/ai-docs-standard
If you build developer tools, your users' agents are already reading your docs.
This is just writing the version they actually need.
Top comments (1)
"A fact an agent has to infer is a fact you failed to state" is the most useful sentence in the spec. The failure you describe, agents inventing config flags from persuasive prose, is exactly what we hit running catalog enrichment against vendor docs: the marketing page and the install reality drift apart, and the agent trusts the page.
The per-release binding is the strong design choice here. A site-level index starts going stale at the first breaking release, and a documentation.ai.md only avoids that if it ships in the same PR as the change it documents. Curious whether you gate that in CI, a release that changes an env var default but not the .ai.md file seems like the first regression this standard should catch.