Most teams add rate limiting to their API as an afterthought — usually right after something has already gone wrong. A scraper hammers an endpoint, a client integration goes into a retry loop, or a single misbehaving user takes down a shared resource for everyone else. By then, you're not designing a rate limiter, you're firefighting.
This post walks through the rate limiting mistakes that show up most often in production systems, why they happen, and what a more resilient approach looks like.
The Problem With "Just Add A Limit"
The instinct is usually: cap requests at X per minute per API key, done. In practice, this single-number approach breaks down fast for a few reasons:
- Not all requests cost the same — a cached read and a heavy join are treated identically under a flat request-count limit.
- Bursts are normal, not exceptional — a dashboard loading 15 widgets fires 15 requests instantly, then goes quiet for minutes.
- Fixed windows create edge-of-window spikes — 100 requests at 0:59 and 100 more at 1:01 is 200 requests in two seconds, technically within "the rules."
What Actually Works Better
Sliding window or token bucket algorithms: instead of a hard reset every N seconds, a token bucket refills at a steady rate, and each request costs a token. This naturally allows small bursts while enforcing a steady average — matching real usage far better than a fixed window.
Cost-based limiting, not just count-based: weight your endpoints so an expensive search costs more "budget" than a simple lookup by ID. This prevents a handful of expensive calls from doing more damage than a thousand cheap ones.
Separate limits for authenticated vs. unauthenticated traffic: anonymous/IP-based traffic should have tighter limits than identified clients, so a shared office IP doesn't get incorrectly throttled.
Respond with the right signals: a bare 429 forces every integration to guess when it's safe to retry. Include a Retry-After header and expose limit, remaining, and reset time (X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset) so well-behaved clients back off correctly instead of retrying in a tight loop.
The Mistake That Causes the Most Damage
The biggest failure mode isn't a badly tuned number — it's rate limiting that isn't distributed correctly across multiple servers. If each instance keeps its own in-memory count, a client can multiply their effective limit by the number of instances behind your load balancer. A limit of "100 requests per minute" quietly becomes "100 times N servers" until traffic spikes and the database falls over anyway.
The fix is centralizing the counter — usually with Redis — so every instance checks and decrements against the same source of truth. It adds a small amount of latency per request, but it's the difference between a rate limiter that actually limits anything and one that only works on a good day.
A Practical Starting Point
If you're retrofitting rate limiting onto an existing API rather than designing it from scratch, a reasonable rollout looks like this:
- Start with logging and monitoring only — understand your actual traffic shapes before setting a real limit.
- Set limits per authenticated client, not per IP, wherever identity is available.
- Use a token bucket or sliding window, not a fixed reset window.
- Centralize your limit counters if running more than one instance.
- Return clear headers and a Retry-After value on every 429.
- Alert on clients consistently near their limit — often a sign of a bug, not malicious intent.
Rate limiting is invisible when it's working and very visible when it isn't. Getting the fundamentals right early avoids a class of production incidents that are annoying to diagnose precisely because everything "looks fine" on a request-count dashboard while the system is being overwhelmed underneath it.
This article draws on real-world patterns encountered while building and scaling backend systems for client projects. If you're working through API architecture decisions like this one, more on our approach is available at Web Matrix Lab.
Top comments (0)