Rate limiting is one of those things every API needs but few developers think deeply about. You slap on a "100 requests per minute" limit and call it a day. But there are actually 7 distinct algorithms, each with wildly different behavior for the same configuration.
This post explains when each algorithm shines, when it fails, and shows the actual behavior difference with code you can run.
The Problem They All Solve
A client sends too many requests. You need to say "no" at some point. The question is how you count and when you say no - and that's where things get interesting.
Let's say your limit is 10 requests per minute. Depending on the algorithm, a client could:
- Send 20 requests in 2 seconds (boundary exploit)
- Send exactly 10 evenly-spaced requests (smooth)
- Send 10 in a burst, then wait (bursty but fair)
- Run 10 operations in parallel (concurrent)
Same limit. Completely different behavior.
1. Fixed Window
How it works: Divide time into fixed buckets (e.g., 0:00-1:00, 1:00-2:00). Count requests in the current bucket. Reset when a new bucket starts.
Window 1 (0:00-1:00) Window 2 (1:00-2:00)
|-- req req req -- limit --| |-- req -- resets --|
count: 3 count: 1
limit: 10 limit: 10
The catch: A client can send 10 requests at 0:59 and 10 more at 1:00 - effectively 20 requests in 2 seconds despite a "10 per minute" limit. This is the boundary burst problem.
import { rateLimit } from '@tzezar/throtto'
const limiter = rateLimit({ limit: 10, window: '1m', algorithm: 'fixed-window' })
// At 0:59 - sends 10 requests (all allowed, window nearly over)
// At 1:00 - sends 10 more (all allowed, new window!)
// Result: 20 requests in 2 seconds
When to use: Non-critical limits where simplicity matters. Dashboard rate limiting, analytics endpoints, internal services.
When NOT to use: Anything where a 2x burst matters - authentication endpoints, payment APIs, external rate-limited upstream services.
2. Sliding Window Counter
How it works: Combines the current and previous window with a weighted overlap. If you're 40% into the current window, count = (previous * 0.6) + current.
Previous Window Current Window
count: 8 count: 3
|--- 40% into window ---|
effectiveCount = 8 * 0.6 + 3 = 7.8
limit: 10
result: ALLOWED (7.8 < 10)
This smooths out the boundary burst. It's not perfect - it's an approximation - but it's close enough for virtually every use case while using only O(1) memory.
import { rateLimit } from '@tzezar/throtto'
// This is the default algorithm - you don't even need to specify it
const limiter = rateLimit('100/minute')
// Or explicitly:
const limiter = rateLimit({ limit: 100, window: '1m', algorithm: 'sliding-window-counter' })
When to use: Most API rate limiting. It's the best default.
When NOT to use: When you need exact per-request precision (use Sliding Window Log) or when you want to allow bursts (use Token Bucket).
3. Sliding Window Log
How it works: Stores a timestamp for every single request. On each check, prune expired timestamps, count what remains.
Window: 60s now
|--------------------------------------|
t1 t2 t3 t4 t5 t6 t7 t8
^^ expired, pruned
count = timestamps still in window = 6
limit: 10
result: ALLOWED
It's the most accurate algorithm - zero approximation. But it stores one entry per request, so memory grows with traffic.
import { rateLimit } from '@tzezar/throtto'
const limiter = rateLimit({ limit: 100, window: '1m', algorithm: 'sliding-window-log' })
// At 1000 req/sec with a 1-minute window, that's 60,000 entries per key.
// Fine for low-volume. Dangerous for high-throughput.
When to use: Financial APIs, compliance systems, anywhere exact enforcement is legally required. Low-volume endpoints where the memory cost is acceptable.
When NOT to use: High-throughput APIs. At scale, the memory cost makes this impractical.
4. Token Bucket
How it works: Imagine a bucket that holds tokens. It starts full. Each request consumes a token. Tokens refill at a constant rate. If the bucket is empty, the request is denied.
Capacity: 10 tokens
Refill: 5 tokens/second
t=0 [##########] 10 tokens (full)
burst of 8 requests
t=0 [##........] 2 tokens left
t=1 [#######...] 7 tokens (refilled 5)
t=2 [##########] 10 tokens (capped at capacity)
The key insight: Token Bucket explicitly allows bursts. A client that's been idle accumulates tokens and can spend them all at once. This is a feature, not a bug.
import { rateLimit } from '@tzezar/throtto'
// Simple: rateLimit maps limit -> capacity, derives refill from window
const limiter = rateLimit({ limit: 100, window: '1m', algorithm: 'token-bucket' })
// Fine-grained: control burst size vs sustained rate independently
import { createLimiter, tokenBucket } from '@tzezar/throtto'
const limiter = createLimiter({
algorithm: tokenBucket({
capacity: 20, // can burst up to 20
refillRate: 10, // but sustain only 10/sec
refillInterval: '1s',
}),
})
When to use: APIs that should tolerate bursts - chat apps, real-time features, CDNs. When you want to reward idle clients with accumulated "credit."
When NOT to use: When you need smooth, constant output (use Leaky Bucket). When bursts could overwhelm downstream services.
5. Leaky Bucket
How it works: The inverse of Token Bucket. Requests fill a bucket that drains at a constant rate. If the bucket overflows, requests are rejected. The output is always smooth.
Capacity: 10
Drain: 2/second
requests --> |##########| bucket fills
|########..|
|------+---|
|
v
2 req/sec output (constant)
Key difference from Token Bucket:
| Token Bucket | Leaky Bucket | |
|---|---|---|
| Starts | Full (allows burst) | Empty (no burst) |
| Burst | Up to capacity | None - constant drain |
| Controls | Input rate | Output rate |
import { rateLimit } from '@tzezar/throtto'
const limiter = rateLimit({ limit: 100, window: '1m', algorithm: 'leaky-bucket' })
When to use: Traffic shaping. Protecting upstream services from spikes. Queue-based systems where constant throughput matters.
When NOT to use: When clients expect immediate responses to bursts. When idle time should "earn" future capacity.
6. GCRA (Generic Cell Rate Algorithm)
How it works: Originally designed for ATM network cell-rate policing. Tracks only one number per key: the Theoretical Arrival Time (TAT) - when the next request "should" arrive assuming perfect spacing.
emission_interval = period / limit (ideal gap between requests)
delay_tolerance = emission_interval * burst
On each request:
new_tat = max(current_tat, now) + emission_interval
allow_at = new_tat - delay_tolerance
if allow_at <= now -> ALLOW
else -> DENY
One number. Per key. That's it. It's insanely memory-efficient.
import { rateLimit } from '@tzezar/throtto'
const limiter = rateLimit({ limit: 100, window: '1m', algorithm: 'gcra' })
When to use: High-cardinality keys (millions of unique users/IPs). When memory is precious. As a drop-in for Sliding Window Counter with lower memory footprint.
When NOT to use: When you need intuitive behavior for debugging - GCRA's "scheduling" mental model can be confusing compared to simple counting.
7. Concurrency
How it works: Unlike the other 6, this doesn't limit rate - it limits parallelism. Each allowed request gets a "slot." You must release the slot when done. If all slots are taken, new requests are denied.
maxConcurrent: 3
Slot 1: [====active====]
Slot 2: [====active===========]
Slot 3: [====active====]
Request 4: DENIED (all slots in use)
Slot 1 released:
Request 4: ALLOWED (slot available)
import { rateLimit } from '@tzezar/throtto'
const limiter = rateLimit({ limit: 5, window: '30s', algorithm: 'concurrency' })
const result = await limiter.check('user-1')
if (result.allowed) {
try {
await doExpensiveWork()
} finally {
await limiter.reset('user-1') // release the slot
}
}
When to use: Database connection pools, file upload limits, expensive computations (image processing, video transcoding). Any time you care about how many things run at once, not how fast they arrive.
When NOT to use: Normal API rate limiting. This is for resource protection, not request throttling.
Choosing the Right Algorithm
What do you need?
|
|-- Limiting concurrent operations? --> Concurrency
|
|-- Need burst tolerance?
| |-- High burst --> Token Bucket
| |-- Tunable --> GCRA
|
|-- Need smooth constant output? --> Leaky Bucket
|
|-- Need per-request precision? --> Sliding Window Log
|
|-- Want simplest/fastest? --> Fixed Window
|
|-- Best default? --> Sliding Window Counter
Real-World Combinations
In practice, you often want multiple algorithms working together:
import { createCompoundLimiter, rateLimit } from '@tzezar/throtto'
const limiter = createCompoundLimiter([
// Token bucket for burst tolerance (20 requests instantly)
{ name: 'burst', limiter: rateLimit({ limit: 20, window: '1s', algorithm: 'token-bucket' }) },
// Sliding window for accurate sustained rate
{ name: 'sustained', limiter: rateLimit({ limit: 100, window: '1m' }) },
// Fixed window for cheap hourly cap
{ name: 'hourly', limiter: rateLimit({ limit: 1000, window: '1h', algorithm: 'fixed-window' }) },
])
// All three are checked. If ANY denies, the request is denied.
const result = await limiter.check('user-123')
This is the pattern most production APIs should use: burst tolerance at the second level, accuracy at the minute level, and a hard cap at the hour level.
Performance
How much overhead do these add? Benchmarked on a memory store (Intel Core 7 240H, Node.js 22):
| Algorithm | ops/sec | avg latency |
|---|---|---|
| Fixed Window | 2.19M | 381 ns |
| Sliding Window Counter | 2.13M | 393 ns |
| Token Bucket | 2.16M | 388 ns |
| Leaky Bucket | 2.15M | 390 ns |
| GCRA | 2.16M | 388 ns |
| Sliding Window Log | 2.06M | 411 ns |
| Concurrency | 155K | 6.4 us |
All except Concurrency run in under 400 nanoseconds. The bottleneck in any real system is the store (Redis round-trip, database query), never the algorithm.
TL;DR
| Algorithm | One-liner | Use when |
|---|---|---|
| Fixed Window | Cheapest, least accurate | You don't care about boundary bursts |
| Sliding Window Counter | Best default | General API rate limiting |
| Sliding Window Log | Most accurate, most memory | Exact enforcement required |
| Token Bucket | Allows bursts | Bursty traffic is expected and OK |
| Leaky Bucket | Smooth output | Protecting upstream from spikes |
| GCRA | Minimal memory | Millions of keys, memory-constrained |
| Concurrency | Limits parallelism | Resource protection, not rate |
Code examples use @tzezar/throtto, a TypeScript rate limiting library I built. But the concepts apply regardless of what library you use.
Top comments (0)