Cost-Weighted Rate Limiting — Why Request Counts Stopped Working

Your rate limit says 1,000 requests per hour per consumer. It has said that for three years and it worked fine.

Here is the assumption underneath it that nobody wrote down: callers distribute roughly evenly across your endpoints. Some hit the cheap list endpoint, some hit the expensive aggregation, and in aggregate 1,000 requests costs approximately what you modelled when you picked the number.

Human-driven clients honour that assumption reasonably well, because they are shaped by user interfaces that expose endpoints in proportion to how often people need them.

Machine callers do not. An agent optimising for task completion finds whichever endpoint yields the most useful information per call and concentrates there. If that happens to be your report aggregation endpoint — the one that scans a year of data and takes 800ms of database time — then 1,000 requests per hour now costs roughly two orders of magnitude more than the same budget spent on cheap reads.

The limit did not change. What it permits did.

The fix: weight the routes

Instead of counting requests, count cost units.

GET  /v1/items                 weight: 1
GET  /v1/items/{id}            weight: 1
POST /v1/items                 weight: 2
GET  /v1/reports/summary       weight: 40    # scans, aggregates
POST /v1/exports               weight: 100   # spawns a job
GET  /v1/search                weight: 15    # hits the search cluster

A consumer gets a budget in weighted units rather than requests. 1,000 units per hour buys 1,000 cheap reads, or 25 report calls, or 10 exports, or some mixture — and every mixture costs you approximately the same.

The weights do not need to be precise. They need to be ordinally correct and roughly proportional. Getting from "all routes equal" to "expensive routes cost 40x" captures nearly all the available benefit; refining 40 to 37 captures almost none.

Deriving the weights

Do not guess, and do not use response time alone — it correlates with cost but conflates queueing with actual work.

A workable approach from real traffic:

1. For each route, over a representative window, measure:
     - mean DB time (or downstream service time)
     - mean CPU time in your service
     - external API calls triggered, and their unit cost
     - bytes returned (if egress is billed)

2. Combine into a per-route cost estimate. Any reasonable
   weighting of these works; the ratios matter, not the units.

3. Normalise so your cheapest meaningful route = 1.

4. Round to something legible: 1, 2, 5, 10, 25, 50, 100.
   Legible weights are ones your consumers can reason about.

Round numbers matter more than they should, because these weights end up in your public documentation and a consumer building against them needs to predict their own consumption.

Implementation notes

Token bucket generalises cleanly. The standard algorithm already decrements by one per request; decrementing by weight instead is a small change. Refill rate becomes units per second rather than requests per second. Burst capacity becomes the bucket size in units.

Return the cost in the response headers. This is what lets well-behaved consumers self-regulate:

X-RateLimit-Limit:      1000
X-RateLimit-Remaining:  742
X-RateLimit-Reset:      1755561600
X-RateLimit-Cost:       40      ← what THIS call consumed

That last header is the one usually missing and the one that matters most for machine callers. An agent that can see a call cost 40 units can adapt; one that only sees a remaining balance dropping unpredictably cannot.

Reject with a useful body, not just a status. A 429 with an empty body tells an agent nothing except to try again, which it will, immediately. Say what the limit is, what this call would have cost, when the budget resets, and — where applicable — which cheaper endpoint achieves something similar. Descriptive errors are cheaper than retry storms.

Check gateway support before building it yourself. Support for weighted limiting varies enormously across API management tools and it is one of the few criteria in that category that genuinely discriminates. Some support it natively, some via a plugin, some not at all. If you are mid-selection, test this explicitly rather than reading the feature matrix.

The related problem: who gets the budget

Weighted limiting fixes what a request costs. It does not fix who pays.

When a call arrives from an agent, acting for a user, through an orchestration platform, on a service credential, there are at least three candidate identities. And you generally want different answers for different questions:

permissions  → the end user
quota        → the platform or tenant
billing      → the account holder

Tooling supporting only per-key attribution collapses these into one, which is wrong in a way that stays invisible until a billing dispute or an access incident. If your consumer mix is shifting toward agent-mediated traffic, check that your gateway can attribute on more than one dimension before you need it to.

Rolling it out without breaking consumers

Phase 1  Ship weights + X-RateLimit-Cost header.
         Enforce nothing. Log what WOULD have been rejected.

Phase 2  Read the logs. Any consumer that would break is a
         conversation, not a surprise 429 in production.

Phase 3  Enforce for new consumers only.

Phase 4  Migrate existing consumers with notice and a
         generous initial budget, tightened over time.

The shadow period in phase 1 is the part worth insisting on. It costs a few weeks and it converts every breakage into a scheduled conversation.

Full guide — the five capabilities that matter in API management tools, deployment topology, and the selection framework: API Management Tools in 2026. We also review API platform decisions.

Frequently Asked Questions

Why do request-per-minute limits fail with AI agents?

Because they assume callers spread evenly across endpoints. Agents concentrate on whichever route is most useful for their task, and if that route is expensive, a fixed request budget permits orders of magnitude more cost than intended.

How precise do route weights need to be?

Ordinally correct and roughly proportional is enough. Moving from all-routes-equal to expensive-routes-cost-40x captures nearly all the benefit. Round to legible values like 1, 2, 5, 10, 25, 50, 100 so consumers can predict their own consumption.

How do I derive weights from real traffic?

Measure per-route database or downstream time, CPU time, triggered external API calls and billed egress over a representative window. Combine them, normalise so the cheapest meaningful route equals 1, and round. Response time alone is a poor proxy because it conflates queueing with work.

What headers should a weighted limiter return?

Limit, remaining and reset as usual, plus a per-call cost header showing what that specific request consumed. The cost header is usually missing and is the one that lets machine callers adapt rather than probe blindly.

Does my API gateway support this?

Varies substantially — some natively, some by plugin, some not at all. It is one of the few genuinely discriminating criteria left in that category, so test it in a trial rather than trusting a feature matrix.

How do I roll this out safely?

Ship the weights and cost header in shadow mode first, enforcing nothing and logging what would have been rejected. Read those logs, talk to affected consumers, then enforce for new consumers before migrating existing ones with notice and a generous starting budget.