This week I audited two products that ship autonomous agents. Both had the same problem, and neither had caused it on purpose.
The pattern is calendar drift. The roadmap moved to autonomous actions — refunds, cancellations, purchases. The contract stayed where it was drafted: a cloud-SaaS world where software displays things and humans click buttons.
Finding 1: the contract that predates the product
Product A sells customer-service agents that, per its own homepage, handle "returns, refunds, cancellations, subscription changes end to end." Its Terms of Service are dated 2023 and describe the service as a tool that connects to your helpdesk. Every clause that would matter for an agent issuing a refund — what actions are permitted, who's liable when it's wrong, when a human must be in the loop — doesn't exist. What exists is a generic warranty and a cap of twelve months of fees, machinery built for downtime.
Finding 2: the safety net that lives in marketing
Product A's homepage makes a promise you'd want if agents moved your money: you set the confidence threshold, and whatever the agent can't handle goes to your team. That routing behavior is the entire risk control. It appears nowhere in the contract. If the vendor quietly changed the threshold logic, no term would be breached.
Finding 3: the machine buyer with no seat at the table
Product B is an insurance broker with the most machine-transparent surface I've ever audited — agents.txt, an MCP manifest, OpenAPI, even a documented "preferred purchase tool." Then I opened the developer portal: it links the consumer browsewrap. The Terms talk about "your credit card." Nothing about agent authority, attribution of prefilled data, or what it means when an autonomous buyer completes checkout. The best agent infrastructure in the industry, governed by a contract for humans browsing a website.
Why this happens (it's not sloppiness)
Terms get drafted at incorporation, when the product is a dashboard. Agents get shipped three pivots later. Nobody updates a 2023 contract to say "our software now executes financial transactions autonomously" because nobody is assigned to notice. The gap surfaces at the worst moment: the first enterprise security review, where the reviewer's job is to ask exactly this.
The fix fits in an afternoon
Three additions, none requiring a lawyer to invent language from scratch:
- An "Agent Actions" section — enumerate permitted autonomous actions with ceilings, make threshold routing and human escalation contractual commitments, and decide whether agent-action liability sits inside or outside the general cap.
- An AI-data addendum — if your security page promises an opt-out from model-improvement use, that promise belongs in the DPA, not just on the marketing page.
- A machine-buyer rider — one page: agent authority representations, attribution for prefill data, prohibited uses.
I keep two anonymized samples of this pattern online (report format, severity scale, fix language): sample 1 · sample 2. If you want your own docs checked against what your product actually does: hire page — $99, 72 hours, pay after delivery. Documentation review, not legal advice.
The uncomfortable version of this article is one sentence: if your agents can spend money and your ToS still describes a dashboard, your contract is lying about your product.
Top comments (0)