API rate limiting strategies: token bucket, leaky bucket, and sliding window

Rate limiting is essential for protecting your API from abuse, ensuring fair usage, and maintaining system stability. The algorithm you choose affects accuracy, memory usage, and burst behavior. Understanding the options helps you pick the right one for your use case.

The token bucket algorithm is the most popular choice. You maintain a bucket that fills with tokens at a steady rate. Each request consumes a token. If the bucket is empty, the request is denied. Burst requests can be served as long as there are accumulated tokens. This allows natural bursts while enforcing a long-term average rate. Token bucket is easy to implement and works well for most APIs.

The leaky bucket algorithm smooths out bursts by processing requests at a constant rate. Incoming requests fill a bucket that leaks at a fixed rate. If the bucket overflows, excess requests are rejected or queued. This creates a very consistent request flow but can reject requests during bursts even if your server could handle them. Leaky bucket is ideal when you need to protect downstream systems that cannot handle bursts.

Sliding window algorithms track requests within a moving time window. The simplest version maintains a counter per time bucket and approximates the sliding window by weighting the previous bucket. More accurate implementations maintain a sorted set of request timestamps per client. Sliding window is more memory-intensive than token bucket but provides precise control over request rates.

Redis is the standard backend for distributed rate limiting. Use INCR with TTL for simple counters, or sorted sets with ZREMRANGEBYSCORE for sliding windows. For very high-throughput systems, consider local rate limiting with synchronized configuration updates, or use a library like rate-limiter-flexible.

Return standard rate limit headers: X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset. These let clients adjust their request rate without hitting limits. Return a 429 Too Many Requests response with a Retry-After header when a client exceeds the limit.

Test your rate limiting under load. Verify that limits are enforced correctly with concurrent requests from the same client. Test the behavior when Redis is down your rate limiter should fail open or closed depending on your security requirements.

-

Rizwan Saleem | https://rizwansaleem.co