Last month our /api/videos/trending endpoint started returning 502s at peak hours. Not because the database was slow, and not because LiteSpeed ran out of workers. The culprit was a handful of scrapers hammering the endpoint 40 times a second each, blowing past our SQLite FTS5 layer and pinning CPU on JSON serialization. Our old rate limiter — a plain requests_this_minute counter in a database column — was letting them through in bursts. A client could fire 60 requests in the first two seconds of a minute, get throttled, then do it again the moment the clock ticked over. That is the classic fixed-window failure, and at DailyWatch it was quietly degrading the experience for every legitimate user browsing free video discovery.
This post walks through how we replaced that counter with a proper sliding window rate limiter backed by Redis. I'll show the fixed-window bug in concrete terms, build a sliding window log with a sorted set, make it atomic with a Lua script so it survives concurrent requests, and then show a memory-cheaper sliding window counter variant for high-cardinality traffic. Every code block here runs — I've stripped the framework glue but kept the logic exactly as it ships.
Why the Fixed Window Lets Bursts Through
A fixed-window limiter buckets requests by wall-clock interval. "100 requests per minute" means: increment a counter keyed on the current minute, reset it when the minute rolls over. It's trivial to implement, which is why everyone starts there.
The problem is the boundary. Consider a limit of 100 requests per 60 seconds. A client sends 100 requests at 12:00:59, then another 100 at 12:01:00. Both windows individually respect the limit. But in the one-second span straddling the boundary, the server saw 200 requests — double the intended rate. For a video API where a single trending request fans out into thumbnail fetches and metadata lookups, that doubling is enough to trip Cloudflare's origin timeout.
Here is the buggy version we were running, roughly:
<?php
// The fixed-window limiter that let bursts through. Do not ship this.
declare(strict_types=1);
final class FixedWindowLimiter
{
public function __construct(
private readonly Redis $redis,
private readonly int $limit = 100,
private readonly int $windowSeconds = 60,
) {}
public function allow(string $clientId): bool
{
// Key changes every window: e.g. rl:api:203.0.113.7:29347186
$bucket = (int) floor(time() / $this->windowSeconds);
$key = "rl:api:{$clientId}:{$bucket}";
$count = $this->redis->incr($key);
if ($count === 1) {
// First hit in this bucket — set the TTL so it self-cleans.
$this->redis->expire($key, $this->windowSeconds);
}
return $count <= $this->limit;
}
}
The logic is correct for what it claims to do. The trouble is what it claims to do is wrong: it enforces "100 per fixed bucket," not "100 per any 60-second stretch." Those are different guarantees, and adversarial traffic finds the gap immediately.
The Sliding Window Log
The honest fix is to track the timestamp of every request and, on each new request, count how many happened in the trailing window. If the count is under the limit, allow and record the new timestamp; otherwise reject. This is the sliding window log, and Redis sorted sets (ZSET) are almost purpose-built for it.
Store each request as a member of a sorted set where the score is the request's timestamp. To evaluate a new request:
- Remove every member with a score older than
now - window(they've aged out). - Count remaining members with
ZCARD. - If under the limit, add the new request and set a TTL on the key.
<?php
declare(strict_types=1);
final class SlidingWindowLog
{
public function __construct(
private readonly Redis $redis,
private readonly int $limit = 100,
private readonly int $windowSeconds = 60,
) {}
public function allow(string $clientId): bool
{
$key = "rl:log:{$clientId}";
$nowMs = (int) (microtime(true) * 1000);
$windowStart = $nowMs - ($this->windowSeconds * 1000);
// 1. Drop entries that have aged out of the window.
$this->redis->zRemRangeByScore($key, 0, $windowStart);
// 2. How many requests are still inside the window?
$count = $this->redis->zCard($key);
if ($count >= $this->limit) {
return false;
}
// 3. Record this request. Use a unique member so identical
// timestamps don't collide and overwrite each other.
$member = $nowMs . ':' . bin2hex(random_bytes(6));
$this->redis->zAdd($key, $nowMs, $member);
$this->redis->expire($key, $this->windowSeconds + 1);
return true;
}
}
Two details matter here. First, the member string is timestamp:random, not just the timestamp. Two requests landing in the same millisecond would otherwise map to the same sorted-set member and the second zAdd would silently update the score instead of adding a row — undercounting your traffic. Second, the TTL is windowSeconds + 1. Once a client goes quiet, the whole key evaporates on its own, so idle clients cost nothing.
This is accurate to the millisecond. The cost is memory: one sorted-set entry per request in the window. For an endpoint doing 100 requests/minute/client across a few thousand clients that's fine. For a limit of 10,000/minute it starts to hurt, and we'll address that later.
Making It Atomic With Lua
The class above has a race condition, and it's the kind that only shows up under the exact load you built the limiter to handle. Between the zCard read and the zAdd write, another concurrent request for the same client can run the same check. Both see count = 99, both conclude they're under the limit of 100, both add. Now you have 101 requests in the window. Under heavy concurrency — which is precisely when scrapers attack — this leaks a meaningful fraction of extra traffic.
The fix is to make the whole check-and-record sequence atomic. Redis runs Lua scripts atomically: nothing else executes on that key while the script runs. We move the entire decision into a single EVAL.
-- sliding_window.lua
-- KEYS[1] = rate limit key
-- ARGV[1] = now in milliseconds
-- ARGV[2] = window size in milliseconds
-- ARGV[3] = limit (max requests per window)
-- ARGV[4] = a unique member id for this request
-- Returns: { allowed (1/0), remaining, retry_after_ms }
local key = KEYS[1]
local now = tonumber(ARGV[1])
local window = tonumber(ARGV[2])
local limit = tonumber(ARGV[3])
local member = ARGV[4]
local cutoff = now - window
-- Purge aged-out entries first.
redis.call('ZREMRANGEBYSCORE', key, 0, cutoff)
local count = redis.call('ZCARD', key)
if count >= limit then
-- Oldest surviving entry tells us when a slot frees up.
local oldest = redis.call('ZRANGE', key, 0, 0, 'WITHSCORES')
local retry = window
if oldest[2] then
retry = (tonumber(oldest[2]) + window) - now
if retry < 0 then retry = 0 end
end
return { 0, 0, retry }
end
redis.call('ZADD', key, now, member)
-- Expire in whole seconds, rounded up, plus a small cushion.
redis.call('PEXPIRE', key, window + 1000)
return { 1, limit - count - 1, 0 }
Because every branch — purge, count, decide, record — happens inside one atomic execution, the read/write race is gone. The script also does something the naive version couldn't: it computes a real Retry-After. The oldest entry in the window will age out at oldest_score + window, so that's exactly when the caller's next request would succeed. Returning that lets clients back off intelligently instead of blind-retrying.
The PHP wrapper loads the script once with SCRIPT LOAD and calls it by SHA to avoid shipping the source on every request:
<?php
declare(strict_types=1);
final class AtomicSlidingWindow
{
private string $sha;
public function __construct(
private readonly Redis $redis,
private readonly int $limit = 100,
private readonly int $windowSeconds = 60,
) {
$lua = file_get_contents(__DIR__ . '/sliding_window.lua');
$this->sha = $this->redis->script('load', $lua);
}
/**
* @return array{allowed: bool, remaining: int, retryAfterMs: int}
*/
public function check(string $clientId): array
{
$key = "rl:sw:{$clientId}";
$nowMs = (int) (microtime(true) * 1000);
$member = $nowMs . ':' . bin2hex(random_bytes(6));
// evalSha with 1 key; the rest are ARGV.
$res = $this->redis->evalSha($this->sha, [
$key,
(string) $nowMs,
(string) ($this->windowSeconds * 1000),
(string) $this->limit,
$member,
], 1);
// NOSCRIPT recovery: reload and retry once if Redis was flushed.
if ($res === false) {
$lua = file_get_contents(__DIR__ . '/sliding_window.lua');
$this->sha = $this->redis->script('load', $lua);
$res = $this->redis->evalSha($this->sha, [
$key,
(string) $nowMs,
(string) ($this->windowSeconds * 1000),
(string) $this->limit,
$member,
], 1);
}
return [
'allowed' => (bool) $res[0],
'remaining' => (int) $res[1],
'retryAfterMs' => (int) $res[2],
];
}
}
The NOSCRIPT recovery matters in production. If Redis restarts or someone runs SCRIPT FLUSH, your cached SHA becomes invalid and evalSha fails. Reloading and retrying once turns a hard outage into an invisible hiccup.
Wiring It Into the Request Path
With the limiter class in hand, the endpoint front controller becomes a short guard clause. We key on the real client IP — behind Cloudflare that lives in CF-Connecting-IP, not REMOTE_ADDR, which would show you Cloudflare's edge address and lump every visitor into one bucket.
<?php
declare(strict_types=1);
$redis = new Redis();
$redis->connect('127.0.0.1', 6379);
$limiter = new AtomicSlidingWindow($redis, limit: 100, windowSeconds: 60);
// Behind Cloudflare, trust CF-Connecting-IP. Never trust it if the
// request did not actually come through Cloudflare — verify the edge
// IP against Cloudflare's published ranges before believing this header.
$clientIp = $_SERVER['HTTP_CF_CONNECTING_IP'] ?? $_SERVER['REMOTE_ADDR'] ?? 'unknown';
$route = $_SERVER['REQUEST_URI'] ?? '/';
// Namespace the key by route so a burst on /search doesn't
// starve a client's budget on /trending.
$clientId = hash('xxh3', $clientIp . '|' . parse_url($route, PHP_URL_PATH));
$result = $limiter->check($clientId);
header('X-RateLimit-Limit: 100');
header('X-RateLimit-Remaining: ' . $result['remaining']);
if (!$result['allowed']) {
$retryAfter = (int) ceil($result['retryAfterMs'] / 1000);
header('Retry-After: ' . max(1, $retryAfter));
http_response_code(429);
header('Content-Type: application/json');
echo json_encode([
'error' => 'rate_limited',
'retry_after_seconds' => max(1, $retryAfter),
]);
exit;
}
// ... under the limit: serve the video payload from SQLite FTS5 ...
A word on the header trust issue, because it's a real vulnerability. If you blindly read CF-Connecting-IP, an attacker hitting your origin IP directly can spoof it and hand you a different fake IP on every request, defeating the limiter entirely. In production we bind LiteSpeed to only accept traffic from Cloudflare's IP ranges and reject direct origin hits, so by the time the header reaches PHP it's already trustworthy. If you can't lock down the origin, validate the connecting address against Cloudflare's published ranges before trusting the header.
Trading Accuracy for Memory — the Sliding Window Counter
The log is exact but stores one entry per request. When we applied it to our search autocomplete endpoint — thousands of requests per client per minute — memory climbed fast. The industry answer, popularized by Cloudflare's own rate limiter, is the sliding window counter: a weighted approximation that uses only two integers per client instead of a full log.
The idea: keep a plain fixed-window counter for the current window and the previous window. To estimate the rate over the trailing window, take the current window's count and add a fraction of the previous window's count, weighted by how far into the current window you are. If you're 25% into the current minute, you weight the previous minute at 75%.
import time
import redis
r = redis.Redis(host="127.0.0.1", port=6379, decode_responses=True)
def allow(client_id: str, limit: int = 100, window: int = 60) -> bool:
now = time.time()
current_bucket = int(now // window)
prev_bucket = current_bucket - 1
# Fraction of the current window that has NOT yet elapsed.
elapsed = (now % window) / window # 0.0 .. 1.0
prev_weight = 1.0 - elapsed # weight for previous bucket
cur_key = f"rl:swc:{client_id}:{current_bucket}"
prev_key = f"rl:swc:{client_id}:{prev_bucket}"
pipe = r.pipeline()
pipe.get(prev_key)
pipe.get(cur_key)
prev_count, cur_count = pipe.execute()
prev_count = int(prev_count or 0)
cur_count = int(cur_count or 0)
# Weighted estimate of requests in the trailing `window` seconds.
estimated = cur_count + prev_count * prev_weight
if estimated >= limit:
return False
# Under the limit: record the hit and refresh the TTL.
pipe = r.pipeline()
pipe.incr(cur_key)
pipe.expire(cur_key, window * 2)
pipe.execute()
return True
This uses two keys per client, each a single integer, regardless of request volume. The approximation error is small and bounded: it assumes the previous window's requests were spread evenly, which is slightly optimistic for bursty traffic but wrong by at most a few percent in practice. Cloudflare reported roughly 0.003% of requests wrongly allowed or denied across 400 million requests when they analyzed this approach — a rounding error compared to the burst leakage of a fixed window.
There's still a subtle race here: the get reads and the incr write aren't atomic together, so under extreme concurrency two requests can both read estimated = 99 and both increment. If that matters for your endpoint, fold the whole thing into a Lua script the same way we did for the log. For most APIs the counter's job is to shed abusive load, not to enforce an exact ledger, and the small slop is acceptable in exchange for flat memory.
Which One to Reach For
After running both in production, here's how we decide:
- Sliding window log (ZSET + Lua): Use when the limit is low-to-moderate (say, under a few hundred per window) and you need exactness — billing-adjacent endpoints, write APIs, anything where over-admitting one request is a real problem. Memory scales with requests-in-window, so keep the limit modest.
- Sliding window counter (two integers): Use for high-volume read endpoints where you're shedding abuse rather than counting precisely — search, autocomplete, trending feeds. Constant memory per client, negligible approximation error.
- Fixed window: Still fine for coarse, non-adversarial internal limits where the boundary burst genuinely doesn't matter. Just never expose it on a public endpoint that scrapers can find.
A few operational notes that saved us grief. Always set a TTL on every key so idle clients self-clean — a limiter that leaks keys becomes its own outage. Always compute a real Retry-After and return it; well-behaved clients will honor it and stop pounding you. And always run the check-and-record as one atomic unit under real concurrency, because the race between reading a count and writing a new entry is invisible in testing and obvious in production.
Conclusion
The fixed-window counter that started our 502s wasn't broken code — it was the wrong guarantee. "100 per bucket" and "100 per any 60 seconds" look identical until an adversary lines up on the boundary. Moving to a Redis sorted-set sliding window log gave us millisecond-accurate enforcement, and the Lua script closed the concurrency race that would otherwise have let a fraction of that abuse through anyway. For our highest-volume endpoints we fall back to the two-integer sliding window counter, trading a sliver of accuracy for constant memory. Since the switch, origin CPU at peak dropped by roughly a third and the 502s are gone. If you're running a public read API on a lean stack — ours is PHP 8.4, SQLite FTS5, and LiteSpeed behind Cloudflare — this pattern is a couple hundred lines of code and pays for itself the first time a scraper finds your endpoint.
Top comments (0)