Designing a Rate Limiter Like a Jedi's Lightsaber: Cutting Through Traffic with Redis

#systemdesign #architecture #backend #programming

The Quest Begins (The "Why")

Ever felt like your API is getting slammed by a horde of overeager clients, and you’re stuck watching latency spike like a dragon hoarding gold? I’ve been there. A few months ago, our micro‑service started getting pummelled by a misbehaving mobile app that decided to refresh its feed every 200 ms. The servers groaned, the DB cried, and our SLA looked like it had taken a lightsaber to the chest. We needed a way to say “no” politely, but fast, and we wanted it to work across dozens of instances without turning our system into a Rube Goldberg machine of locks and counters.

The obvious first try? A simple in‑memory counter per process. Increment, check against a limit, reset after a window. It worked… until we scaled out. Suddenly each node had its own view of the world, and the limit was effectively multiplied by the number of replicas. We tried a shared database table, but the round‑trip latency added ~5 ms per request—enough to turn our lightsaber into a blunt stick.

That’s when the quest for a distributed rate limiter truly began. The dragon we needed to slay wasn’t just traffic; it was the need for atomic, low‑latency decisions that could be shared across the cluster without choking the network.

The Revelation (The Insight)

The breakthrough came when I stopped thinking about “counters” and started thinking about time‑sorted sets. Redis already gives us O(log N) inserts and deletes, and crucially, it lets us run Lua scripts atomically. The insight was simple: store each request as a timestamp in a sorted set, trim the set to the window, and then check its size. If the size is below the threshold, we allow the request and add the new timestamp; otherwise we reject.

Why does this beat the classic fixed‑window counter approach?

No spikes at window boundaries – the sliding window smooths out bursts.
Exact counting – we know precisely how many requests happened in the last N seconds, not an approximation.
Atomicity via Lua – the whole check‑add‑trim operation is a single server‑side script, so no race conditions even under heavy concurrency.

The trade‑off? Slightly more memory (we keep every timestamp for the window) and a tiny CPU cost for the ZADD/ZREM operations. But compared to the network hammer of a DB‑based counter or the inaccuracy of a per‑process fixed window, it’s a bargain. Plus, Redis is already in our stack for caching, so we’re not adding a new dependency.

Here’s a quick ASCII diagram of what lives in Redis for a key ratelimit:<user_id>:

   Sorted Set (ZSET) for key "ratelimit:123"
   ----------------------------------------
   Score (timestamp)   Member (unique id)
   1725000000          req:abc123
   1725000001          req:def456
   1725000002          req:ghi789
   ...                 ...
   1725000050          req:xyz999   <-- newest
   ----------------------------------------
   ZREMRAYBYSCORE key 0 (now-window)   ; drop old
   ZCARD key                               ; count current
   if count < limit:
       ZADD key now unique-id
       return ALLOW
   else:
       return DENY

Wielding the Power (Code & Examples)

Let’s see the “before” – a naive per‑process fixed window in Go (pseudo‑code, just to show the pain):

// BEFORE: broken when scaled out
var (
    mu      sync.Mutex
    count   int
    lastReset time.Time
)

func allow() bool {
    mu.Lock()
    defer mu.Unlock()
    if time.Since(lastReset) > window {
        count = 0
        lastReset = time.Now()
    }
    if count >= limit {
        return false // reject
    }
    count++
    return true // allow
}

Scale this to three replicas and you instantly get a limit of 3 * limit. Not what we wanted.

Now the “after” – a Redis‑backed sliding window logger, wrapped in a helpful Go helper:

// ratelimiter.go
package ratelimiter

import (
    "context"
    "time"

    "github.com/go-redis/redis/v8"
)

var luaSlide = redis.NewScript(`
    local now = tonumber(ARGV[1])
    local window = tonumber(ARGV[2])
    local limit = tonumber(ARGV[3])
    local key = KEYS[1]

    -- remove outdated entries
    redis.call('ZREMRANGEBYSCORE', key, 0, now-window)
    -- count current
    local current = redis.call('ZCARD', key)
    if current < limit then
        -- add this request with a unique member (now + random)
        redis.call('ZADD', key, now, now..':'..math.random(1,10000))
        redis.call('PEXPIRE', key, window*2) // keep key alive a bit longer
        return 1
    end
    return 0
`)

func Allow(ctx context.Context, rdb *redis.Client, id string, limit int, window time.Duration) bool {
    now := float64(time.Now().UnixNano()) / 1e6 // milliseconds
    // KEYS[1] = ratelimit:<id>
    key := "ratelimit:" + id
    res, err := luaSlide.Run(ctx, rdb, []string{key}, now, float64(window.Milliseconds()), float64(limit)).Result()
    if err != nil {
        // fail open? or fail closed? Here we choose to allow on error to avoid outage.
        return true
    }
    return res.(int64) == 1
}

Traps to avoid (the “boss fights” on our quest):

Using a non‑atomic multi‑step approach – if you do ZCARD, then ZADD separately, a race condition can let two slips through when the set hovers at limit‑1. The Lua script guarantees the check‑and‑add is indivisible.
Forgetting to set an expiry – without a TTL, the key lives forever, wasting memory. We PEXPIRE the key for a little over twice the window so it survives bursts but gets cleaned automatically.
Choosing a poor member uniqueness – using just the timestamp as the member can cause collisions if two requests arrive in the same millisecond. Appending a random suffix (or a request ID) ensures each entry is distinct.

A quick sanity check: with a limit of 100 req/s and a 1‑second window, a burst of 150 requests will see the first 100 get true, the next 50 get false, and the sorted set will contain exactly 100 timestamps after the script runs—no more, no less.

Why This New Power Matters

Now that we’ve got this lightsaber in hand, the battlefield looks different. Our API can gracefully throttle abusive clients while letting legitimate traffic flow at full speed. Because the decision lives in Redis, a single network hop (often sub‑millisecond) replaces the costly DB round‑trip or the fuzzy math of per‑process counters.

The sliding window gives us a fair view of traffic: a client that sends 80 requests in the first half‑second and another 80 in the next half‑second still gets limited, preventing the “burst‑then‑silence” loophole that fixed windows miss. And because the script is atomic, we can safely run thousands of limiter checks per second on a modest Redis instance without worrying about lost updates.

What can you build with this?

Global API gateways that protect micro‑services across regions.
Per‑user or per‑API‑key quotas for SaaS platforms without sprinkling locks everywhere.
Dynamic bursting – just change the limit argument on the fly and the same logic adapts.

If you’re already running Redis for caching, you’re essentially getting a free, battle‑tested rate‑limiting engine for the cost of a few extra bytes of memory.

Your Turn: Embark on Your Own Quest

Grab your favorite language, spin up a Redis instance (Docker makes it trivial: docker run -p 6379:6379 redis:7), and drop the Lua script above into a helper. Try it with a simple HTTP middleware and watch the logs as excess requests get politely turned away.

Challenge: Implement a tiered limiter—different limits for authenticated vs. anonymous users—using the same Redis key pattern but with a prefix like ratelimit:anon:<id> vs. ratelimit:auth:<id>. Share your snippet in the comments; I’d love to see how you tweak the lightsaber for different combat scenarios.

May your code be swift, your limits be fair, and your servers stay unscathed. Happy hacking! 🚀

DEV Community