I reviewed the APIs from a financial management tool that I use personally. The APIs provide access to all the data for a net worth dashboard, an asset portfolio with form-specific fields (bank, pension, crypto, RSU, etc.), and filtered cashflow transactions. They provide an openapi.json and llms.txt, plus documentation in a foreign language.
The openapi.json was clearly written with agents in mind: it defines the auth and describes the token lifecycle, and includes standard error shapes. Each operation also has a summary and a short description of what the call returns and when you'd use it. Responses carry inline JSON examples.
For agent-facing docs, that looks like a strong surface: paths, auth, errors, and copy-paste-shaped payloads all in the spec.
But there's a problem: one method returns one of 8 form variants with distinct nested arrays with lots of variant-specific data (think pension vs RSU vs crypto specific info), and the examples don't cover these. Those per-form fields are documented in llms.txt, not in the spec. Instead, the spec has two examples out of eight.
Without llms.txt in context, agents tend to generalize from the example shapes provided. It might read form and still not know whether to look for stocksData, cryptoData, or something else — so it builds a uniform model and misses most of the detail that actually makes the portfolio useful.
So the questions I'm left with are:
- How likely are agents to realize that they need to always keep the llms.txt in context?
- Will they do it even when explicitly told to?
- Or should the openapi.json have examples for every variant?
Top comments (3)
This is a real agent-design problem: OpenAPI tells the agent what it can call, but not always how the product wants to be used. I would treat llms.txt like operating context with freshness and precedence rules, especially when the API surface is technically complete but the business constraints live elsewhere.
That’s a useful way to put it @alexshev . I’m curious how you’d set precedence in practice when OpenAPI and llms.txt disagree — or when llms.txt is stale?
Exactly. The OpenAPI spec is the callable surface, not the full operating manual. For agents, the missing layer is often policy and interpretation: when not to call, what a field really means, and which examples are canonical when the schema is technically valid but ambiguous.