An agent loop re-sends its context on every step, so token consumption grows with
task length in a way a single chat turn never does. When that growth meets a hard
hourly ceiling the session degrades rather than ending — usually mid-refactor, which
is the worst possible moment.
A per-request allowance fixes the failure mode, because the budget becomes a count
you can plan against before you start rather than a number you watch slide.
What to check before switching
- Does the gateway speak the [OI] schema, including streaming and tool calls?
- Are request limits published per plan, not just token limits?
- Can you issue separate keys per client, so attribution is not a spreadsheet exercise?
- What happens at the ceiling — a clean 429 with Retry-After, or a mid-stream drop?
Why flat-rate helps agents specifically
Chat workloads are bursty but short. Agent workloads are long and recursive: a
fifty-step task can re-read the same context fifty times. Under token metering the
cost of a task is only knowable after it finishes, which makes budgeting a
post-mortem exercise. Under a request allowance the cost is bounded by the number of
steps you allow, which is a design decision you make up front.
Setup
The gateway is [OI]-compatible, so most agent frameworks need only a base-URL
change. Setup notes per tool are in the APIClaw documentation.
Top comments (0)