A 429 Too Many Requests from an API Gateway endpoint does not tell you which throttle produced it. There are four, they nest, and three of them are invisible from the response.
Four levels, not one
AWS documents four distinct types of throttling-related setting, and conflating them is the reason this is confusing:
- AWS Regional throttling limits — applied across all accounts and clients in a Region. Set by AWS, not visible, not changeable by any customer. Nothing you configure can exceed them.
- Per-account limits — applied to every API in your account in one Region. This is the one people mean when they say “the API Gateway limit”. Adjustable on request, but never above the Regional limit.
- Per-API, per-stage limits — applied at the method level for a stage. You can set one value for all methods or different values per method. Cannot exceed the AWS limits.
- Per-client limits — applied to callers identified by an API key attached to a usage plan. Cannot exceed the per-account limits.
Every level is a token bucket, which is why each has two numbers rather than one. The rate is how fast tokens are added, in requests per second. The burst is the bucket’s capacity — the target maximum number of concurrent submissions API Gateway will fulfil before it starts returning 429. A burst larger than your rate is what lets a spike through without shaping; a burst of zero would make the rate a hard ceiling with no tolerance at all. AWS is explicit that both throttles and quotas are applied on a best-effort basis and should be read as targets, not as guaranteed ceilings, so an occasional overrun in either direction is documented behaviour rather than a bug.
The order they are applied in
AWS documents the evaluation order precisely, and it is the reverse of what most people assume — the narrowest limit is checked first:
- Per-client or per-method throttling limits set for an API stage in a usage plan.
- Per-method throttling limits set for an API stage.
- Account-level throttling for the Region.
- AWS Regional throttling.
The practical reading: a per-method limit you set on one route protects the account limit from that route, but does nothing to protect that route from everything else in the account consuming the shared bucket first. This is exactly the asymmetry that catches an AI endpoint. Your model route is low-volume and expensive; some other API in the same account and Region is high-volume and cheap; the cheap one exhausts the account bucket and the expensive one starts returning 429 without having changed at all. The fix is not on the AI endpoint — it is a per-method limit on the noisy neighbour.
The account limit you share with everything
AWS documents the default account-level throttle as 10,000 requests per second per account per Region, shared across HTTP APIs, REST APIs, WebSocket APIs and WebSocket callback APIs, with a token bucket capacity of 5,000 requests. Two details in that sentence do real work. First, WebSocket callback traffic counts — every @connections post in a streaming relay draws on the same bucket as ordinary requests, which is a good reason to buffer tokens rather than post each one. Second, AWS states the burst quota is set by the service team based on your overall RPS quota and is not something a customer can control or request changes to, even though the rate is adjustable.
The default is also not uniform. AWS documents a reduced default of 2,500 RPS with a 1,250 burst in a specific list of Regions — including Africa (Cape Town), Europe (Milan), Asia Pacific (Jakarta), Middle East (UAE), Asia Pacific (Hyderabad), Asia Pacific (Melbourne), Europe (Spain), Europe (Zurich), Israel (Tel Aviv), Canada West (Calgary), Asia Pacific (Malaysia), Asia Pacific (Thailand) and Mexico (Central). A deployment that behaves in us-east-1 and throttles in eu-south-2 at a quarter of the load is not mysterious.
Both figures, and the Region list, are from the Amazon API Gateway quotas page at the time of writing. The Region list in particular grows as AWS launches Regions — check it rather than trusting this page for a Region that opened recently.
The trade nobody reads about
AWS documents the REST API integration timeout as 50 milliseconds to 29 seconds. For a synchronous model call that is a genuinely tight ceiling — a long generation, or a cold-started container behind the route, will exceed it and return 504 with the model still running and still billing.
Since June 2024 AWS allows the ceiling to be raised, via the Service Quotas entry named “Maximum integration timeout in milliseconds”, for Regional REST APIs and private REST APIs only — edge-optimized REST APIs and HTTP APIs are not eligible. AWS notes in the same announcement that raising it may require a reduction in your account-level throttle quota. That is the sentence to carry away from this page: on an account with a mix of API types, buying headroom for one slow AI route can lower the request ceiling for everything else. Where the work genuinely takes minutes, the cheaper answer is to stop making it synchronous — return a job id, do the work behind a Step Functions workflow, and stream or poll the result.
Working out which one hit you
The response body will not tell you. Access logging will, because API Gateway exposes the reason as a context variable. Enable access logs on the stage with a format that includes the gateway response type and message:
{
"requestId": "$context.requestId",
"ip": "$context.identity.sourceIp",
"apiKeyId": "$context.identity.apiKeyId",
"routeKey": "$context.routeKey",
"status": "$context.status",
"responseType": "$context.error.responseType",
"errorMessage": "$context.error.message",
"integrationLatency": "$context.integrationLatency"
}
A responseType of THROTTLED means a rate or burst limit was exceeded. QUOTA_EXCEEDED means a usage plan’s request quota — the per-day or per-month allowance, which is a different mechanism from the rate limit and is also returned as a 429. Those two look identical to the client and have unrelated fixes: one wants a higher rate, the other wants a different plan.
Alongside the logs, watch the stage-level CloudWatch metrics. The ratio of 4XXError to Count tells you how much of your traffic is being rejected, and IntegrationLatency versus Latency separates time spent in your backend from time spent in the gateway. If Latency is high while IntegrationLatency is flat, the queue is in front of your code, not inside it.
An AI endpoint has two independent throttles in series, and they return the same status code: API Gateway’s 429 in front, and the model provider’s own rate limit — Bedrock’s ThrottlingException, or an upstream 429 with its own reset headers — behind. Distinguishing them matters because the first wants backpressure and the second wants failover to a different model or Region. A gateway like Multigrid sits at the second boundary: it reads each provider’s rate-limit signalling, retries or routes elsewhere, and surfaces one consistent error so the front half can tell “you are sending too fast” from “the model is full”.
Top comments (0)