I run a gateway that sits in front of a handful of services, and rate limiting there is per-tenant: every customer gets a budget per window, and once several replicas are behind a load balancer, an in-process counter falls apart. Each replica hands out the full budget on its own, so three replicas means three times the traffic you thought you'd allow. The fix is a shared counter, and Redis is the obvious place to keep it. This is a walk through the actual code I pushed today, including the part that's easy to get subtly wrong.
The counter, in one round trip
Fixed-window counting is the boring, correct starting point: keep an integer per tenant per window, bump it on each request, and reject once it goes over the limit. In Redis that's INCR. The catch is that INCR on its own creates a key with no expiry, so you also have to set a TTL, and the order and atomicity of those two operations is the whole story.
Here's the Redis backend:
// Incr bumps the counter for key and returns the count after the increment.
func (rc *RedisCounter) Incr(ctx context.Context, key string, window time.Duration) (int64, error) {
pipe := rc.rdb.Pipeline()
incr := pipe.Incr(ctx, key)
pipe.ExpireNX(ctx, key, window)
if _, err := pipe.Exec(ctx); err != nil {
return 0, fmt.Errorf("increment rate limit key %s: %w", key, err)
}
return incr.Val(), nil
}
Two commands, one pipeline, one round trip. INCR creates the key at 1 or bumps an existing one. EXPIRE NX sets the window TTL only if the key doesn't already have one. So the first request in a window starts the clock, and every request after that leaves the clock alone. When the TTL fires, the key vanishes, the next INCR recreates it at 1, and the window has reset. That's the entire fixed-window mechanic.
The race you write by accident
The naive version of this is INCR, look at the result, and if it's the first request, call EXPIRE. It reads fine and it's wrong. Between the INCR and the EXPIRE you have a gap, and if the process dies in that gap, or the second command fails on its own, you're left with a key that has a count of 1 and no TTL. It never expires. It counts up forever. A day later that tenant is permanently over budget for a window that ended long ago, and nobody can explain why one customer is getting 429s at 3am. I've debugged the shape of this bug before, and it's miserable because it's rare and it doesn't reproduce on demand.
ExpireNX closes most of that. The two commands ride in the same pipeline, so they go out together, and EXPIRE NX is idempotent about the TTL: it sets one when there's none and does nothing when there already is. So a retry, a duplicate, a request that races with the very first one in the window all converge on the same state. The key that exists always ends up with a TTL. The failure mode where a key counts forever with no expiry needs a much narrower window to appear, essentially the pipeline itself failing after INCR landed but before EXPIRE NX did, which Exec surfaces as an error you can act on rather than silently swallow.
One honest caveat, which the code comment states plainly: EXPIRE NX is a Redis 7 feature. If you're on an older Redis it doesn't exist, and you'd fall back to a Lua script that does the increment-and-conditional-expire in one atomic server-side step. I pin Redis 7 in compose and CI, so ExpireNX is enough here, but if you can't pin your Redis version, reach for the Lua path instead.
The core doesn't import Redis
The part I'm happiest with is that none of the limiting logic knows Redis exists. The middleware talks to an interface:
// Counter is the shared store behind RateLimitShared. Incr bumps the counter
// for key and returns the count after the increment; the implementation is
// responsible for expiring the key at the window boundary, which is what makes
// the window fixed.
type Counter interface {
Incr(ctx context.Context, key string, window time.Duration) (int64, error)
}
That's the whole seam. Redis is one implementation of Counter. The middleware builds the key, calls Incr, and decides pass or reject based on a plain int64. It has no opinion about where the number came from:
key := "rl:unknown"
if t, ok := tenant.From(c); ok {
key = "rl:" + t.ID
}
count, err := counter.Incr(c.UserContext(), key, window)
if err != nil {
// Fail open, but log at most once per second per process. During
// a store outage every request takes this path, and one identical
// line per request would flood the log at exactly the wrong time.
now := time.Now().Unix()
if last := lastLogged.Load(); now != last && lastLogged.CompareAndSwap(last, now) {
log.Printf("rate limit store error, failing open: %v", err)
}
return c.Next()
}
if count > int64(limit) {
return fiber.NewError(fiber.StatusTooManyRequests, "rate limit exceeded")
}
return c.Next()
Two things worth pausing on. The key is "rl:" + tenant.ID, so each tenant spends from its own budget, and traffic that arrives with no resolved tenant falls into a shared rl:unknown bucket instead of skipping the limiter entirely. And on a store error the limiter fails open: the request passes and the error is logged, throttled to at most one line per second so a Redis outage doesn't also drown your logs. That's a deliberate call. A limiter that fails closed turns a Redis blip into a total outage, and for this gateway a window of unmetered traffic beats taking everything down. If your limiter is protecting something that genuinely cannot tolerate a burst, you'd want the opposite, and you'd make that choice on purpose.
Testing it without Redis, then with
Because the core only depends on Counter, the unit tests don't touch Redis at all. They use a scripted fake:
func (f *fakeCounter) Incr(_ context.Context, key string, window time.Duration) (int64, error) {
f.lastKey = key
f.lastWindow = window
if f.err != nil {
return 0, f.err
}
f.count++
return f.count, nil
}
That fake counts like a real store and records the last key and window, so the tests assert exactly what the middleware asked for: that it allows up to the budget then blocks, that it keys off rl:acme, that it falls back to rl:unknown, and that setting err drives it down the fail-open path without ever needing a store to actually be down. These run in milliseconds with no network.
Redis-specific behavior gets its own integration test, gated on TEST_REDIS_URL so it's skipped unless a real Redis is around. It stands up two independent Fiber apps over one RedisCounter to fake two replicas, and checks the thing that only a shared store can do:
// Two requests on replica A and one on replica B spend the whole budget of
// 3, even though no single replica saw more than two requests.
for i := 1; i <= 2; i++ {
if got := hitReplica(replicaA); got != http.StatusOK {
t.Fatalf("replica A request %d = %d, want %d", i, got, http.StatusOK)
}
}
if got := hitReplica(replicaB); got != http.StatusOK {
t.Fatalf("replica B request = %d, want %d", got, http.StatusOK)
}
The fourth request is over budget on either replica, and after the window's TTL elapses the budget comes back fresh. That's the property I actually cared about, and it's the one the fake can't prove, so it earns a real Redis.
What this does not claim
Fixed-window has a known sharp edge: bursts at the boundary. The window resets on a hard tick, so a tenant can send its full budget in the last moment of one window and its full budget again in the first moment of the next, and briefly do double its rate. A sliding window or a token bucket smooths that out. This limiter doesn't, and it isn't pretending to. It's a per-tenant fixed-window counter that stays correct across replicas at one round trip, and fails open on purpose. If you need boundary-burst protection you'd swap the Counter implementation for a sliding-window one, which is exactly the kind of change the interface was built to make painless. The zero-dependency in-memory counter is still the default for single-replica setups; Redis is the backend you reach for once you're running more than one.
Top comments (0)