rate limiting exists to protect a system from abuse and overload, but poorly designed rate limits end up punishing legitimate users more than they deter actual bad actors. getting this right requires thinking about rate limiting as a product design problem, not just an infrastructure protection mechanism.
fixed windows create a thundering herd problem at the boundary
the simplest rate limiting approach, allow x requests per fixed time window, has a subtle flaw that shows up under real traffic patterns. a client that exhausts its limit near the end of one window can immediately make another full burst of requests the moment the next window opens, effectively doubling the intended rate right at the window boundary. under concentrated traffic, this creates a spike exactly at each window edge, which is the opposite of what rate limiting is supposed to prevent.
sliding window or token bucket approaches avoid this specific failure mode by smoothing the allowed rate continuously rather than resetting sharply at fixed intervals. the added implementation complexity is usually worth it for any api with meaningful traffic volume, since the fixed-window edge case tends to show up in production even when it looks like an unlikely corner case in design review.
different endpoints deserve different limits, not one global number
applying a single rate limit uniformly across every endpoint in an api treats a cheap, read-only lookup the same as an expensive, computation-heavy operation. this either sets the limit too low for cheap endpoints, frustrating users who are making reasonable use of lightweight calls, or too high for expensive endpoints, failing to actually protect the system from the load that matters most.
segmenting limits by the actual resource cost of each endpoint, sometimes expressed as a weighted cost system where different endpoints consume different amounts of a shared budget rather than counting as a flat one request each, more accurately reflects what the rate limit is actually trying to protect against.
communicate limits clearly, in the response, not just in documentation
a rate-limited response that returns a generic error with no indication of when the client can retry forces every integrating developer to either guess or read documentation that may be out of date relative to the actual configured limits. including standard rate limit headers, remaining quota, reset time, and the limit itself, directly in every response, not just the ones that hit the limit, lets client applications build sensible retry and backoff logic without needing to hardcode assumptions about the limit that could drift out of sync with the actual server configuration.
this small addition meaningfully reduces the volume of confused support requests from developers who hit a limit without understanding why or when they can retry.
distinguish between burst tolerance and sustained rate
real usage patterns are rarely perfectly smooth. a client might legitimately need to make a short burst of requests, loading a dashboard that fires several calls at once, for example, followed by long stretches of low activity. a rate limit that only accounts for sustained average rate, with no tolerance for reasonable bursts, ends up blocking exactly this kind of normal usage pattern.
token bucket algorithms handle this naturally, since they allow accumulated unused capacity to be spent in a burst up to a defined ceiling, while still enforcing a sustained average rate over time. designing the burst ceiling and refill rate around actual observed legitimate usage patterns, rather than an arbitrary round number, avoids penalizing normal client behavior while still catching genuinely abusive traffic.
rate limit by the right identity, not just ip address
ip-based rate limiting is the simplest to implement but breaks down in common real-world scenarios: multiple legitimate users behind a shared corporate nat can get lumped into a single limit meant for one client, while a single malicious actor can trivially rotate ip addresses to evade an ip-based limit entirely.
rate limiting by authenticated api key or account identity, where authentication is available, provides a much more accurate mapping between the limit and the actual entity it's meant to constrain. for unauthenticated endpoints where ip is the only available signal, combining it with other lightweight fingerprinting signals, while being mindful of privacy implications, produces a more resilient limit than ip alone.
provide a path for legitimate high-volume users
some of the most valuable api consumers are exactly the ones most likely to hit a standard rate limit, since heavy legitimate usage looks statistically similar to abuse from a pure request-volume perspective. having a clear, low-friction process for legitimate high-volume users to request an elevated limit, rather than forcing every heavy user to either work around the default limit or abandon the integration, keeps rate limiting from becoming an unintended ceiling on the api's most valuable use cases.
the underlying goal
rate limiting done well is close to invisible to legitimate users and genuinely restrictive to abusive ones. rate limiting done poorly inverts that: legitimate users hit confusing walls during normal usage spikes, while determined bad actors adapt around a rigid, uniformly applied limit. the difference between the two usually comes down to whether the limit was designed around actual observed usage patterns and communicated clearly, or set as a single arbitrary number applied uniformly across a system that has much more variation in legitimate use than a flat limit accounts for.
Top comments (0)