Every API you expose to the internet will eventually be abused. Automated scrapers, credential stuffing bots, misbehaving integrations, and sometimes just a well-meaning client with a loop that runs too fast. Without rate limiting, a single bad actor can consume all your server resources and degrade the experience for every other user.
Rate limiting is one of those mechanisms that seems simple on the surface but reveals surprising depth when you implement it.
What Rate Limiting Actually Protects
Before discussing algorithms, it's worth being explicit about the threats:
- Resource protection: Preventing any single client from consuming a disproportionate share of CPU, memory, database connections, or bandwidth.
- Cost control: If your API calls AI inference APIs, SMS providers, or payment processors, an unconstrained client can rack up significant charges in minutes.
- Abuse prevention: Credential stuffing and enumeration attacks rely on volume. Rate limiting raises the cost for attackers significantly.
- Fair access: In multi-tenant systems, rate limiting ensures one tenant's spike doesn't degrade everyone else's experience.
The Four Algorithms That Matter
Fixed Window is the simplest: count requests per client in a fixed time interval, reject when the count exceeds the limit. Implementation in Redis is a single INCR with EXPIRE. The weakness is the boundary problem — a client can send the max at the end of one window and the max at the start of the next, effectively doubling their rate for a brief period.
Sliding Window Log eliminates the boundary problem by tracking timestamps of every request in the window. Accurate but memory-intensive: 1000 requests/minute × 10,000 clients = 10 million stored timestamps. Best suited for low-volume, high-value endpoints like login or password reset.
Sliding Window Counter is the recommended default. It maintains counters for the current and previous fixed windows, then computes a weighted count based on how far into the current window you are. Good balance of accuracy, memory efficiency, and implementation simplicity.
Token Bucket models rate limiting as a bucket that fills at a steady rate. Two parameters — refill rate and bucket capacity — independently control sustained throughput and burst tolerance. This is the algorithm used by most cloud providers and maps naturally to tiered pricing models.
Where to Rate Limit
Rate limiting should be layered:
- At the edge or load balancer (Nginx, Cloudflare, AWS API Gateway): protects application servers from receiving excessive traffic at all. Your first line of defense against volumetric abuse.
- At the API gateway or middleware: where you implement business-level limits by authenticated user, API key, subscription tier, or endpoint.
- At individual services: in microservice architectures, prevents a misbehaving upstream service from overwhelming a downstream dependency.
Each layer protects against different failure modes. Don't rely on just one.
Communicating Limits to Clients
Good rate limiting is transparent. Every response should carry headers like X-RateLimit-Limit, X-RateLimit-Remaining, and X-RateLimit-Reset. A 429 Too Many Requests response should include a Retry-After header that tells well-behaved clients exactly when to try again. This converts rate limiting from a blunt instrument into a collaboration between your API and your consumers.
Three Key Takeaways
- The sliding window counter is the right default for most APIs. It provides close-to-accurate rate limiting with minimal memory overhead, without the complexity of tracking individual request timestamps.
- Identify clients by API key, not IP address. IP-based limiting is increasingly unreliable — users behind corporate proxies share addresses, and attackers with botnets distribute requests across thousands of IPs.
- Rate limits are a product decision as much as a technical one. What you allow per tier, what gets a 429 versus a graceful degradation, and how generously you size burst capacity all affect your users' experience directly.
Read the full article at novvista.com for complete algorithm implementations, Redis code examples, and a guide to designing rate limit tiers for subscription-based APIs.
Originally published at NovVista
Top comments (0)