A rate limiter can work perfectly in development and still fail in production.
You test it locally, send too many requests, and receive a 429 Too Many Requests response. Everything looks fine.
Then you deploy your Node.js application with two instances behind a load balancer.
Suddenly, users can send more requests than your limit allows.
The problem isn't necessarily your rate-limiting algorithm. It's where you're storing the request counts.
The Problem With In-Memory Rate Limiting
Consider this Express middleware:
const requests = new Map();
function rateLimiter(req, res, next) {
const key = req.ip;
const now = Date.now();
const windowMs = 60_000;
const limit = 10;
let record = requests.get(key);
if (!record || now - record.start >= windowMs) {
record = {
count: 0,
start: now,
};
}
record.count++;
requests.set(key, record);
if (record.count > limit) {
return res.status(429).json({
message: "Too many requests",
});
}
next();
}
This simplified fixed-window limiter allows up to 10 requests per IP address per minute in a single process.
But what happens when your application runs on two servers?
Imagine the load balancer distributes requests like this:
- Server A receives 10 requests.
- Server B receives another 10 requests.
- Each server maintains its own
Map.
Both servers see only 10 requests. Neither rejects the requests.
Your intended limit was 10 requests per minute, but the user managed to send 20.
With more instances, the discrepancy can become even larger.
The key issue is that each process has its own memory.
An in-memory counter is not shared across Node.js processes, containers, or separate application instances.
Why This Matters in Production
This becomes especially important when rate limits protect:
- Login and password-reset endpoints
- Public APIs with usage quotas
- Expensive database queries
- AI or third-party API calls that cost money
- Endpoints vulnerable to automated abuse
A rate limiter that works only within one process may provide weaker protection than you expect.
There is another problem: when a process restarts, its counters disappear. Scaling, restarting, or replacing instances can therefore change the effective limit.
A Better Approach: Use Redis
Redis gives multiple application instances access to shared state.
Instead of keeping request counts in a local Map, each instance updates a shared counter.
A simplified Redis fixed-window implementation might look like this:
async function rateLimiter(req, res, next) {
try {
const key = `rate-limit:${req.ip}`;
const limit = 10;
const windowSeconds = 60;
const count = await redis.incr(key);
if (count === 1) {
await redis.expire(key, windowSeconds);
}
if (count > limit) {
return res.status(429).json({
message: "Too many requests",
});
}
next();
} catch (error) {
next(error);
}
}
Here, redis represents an already configured Redis client.
Now both servers use the same counter. If Server A processes six requests and Server B processes five, the shared count reaches eleven, so the eleventh request is rejected.
However, this example illustrates the idea rather than providing a complete production-ready implementation.
There is still a subtle problem
The counter increment and expiration are separate commands.
If the process fails between INCR and EXPIRE, the key might remain without an expiration. That could cause requests to be rejected indefinitely.
For production use, execute the counter update and expiration logic atomically, for example with a Redis Lua script or an appropriate atomic Redis operation.
You should also decide what happens when Redis becomes unavailable. Failing open allows requests through but weakens protection; failing closed preserves the limit but can make an endpoint unavailable. The right choice depends on the endpoint and its risk.
Redis Doesn't Automatically Make Every Rate Limiter Correct
Using shared storage solves the per-instance counter problem, but other design decisions still matter.
1. Choose the right algorithm
A fixed-window limiter is easy to implement, but it can allow bursts around window boundaries.
For example, a client might send 10 requests just before a window resets and another 10 immediately afterward.
Depending on your requirements, a sliding-window counter, sliding log, or token bucket may provide better behavior.
2. Identify clients correctly
Using req.ip requires careful proxy configuration.
If Express trusts forwarded headers from untrusted clients, an attacker might spoof their apparent IP address. Configure trust proxy according to your actual infrastructure rather than enabling it blindly.
For authenticated APIs, you may want to rate-limit by user ID, API key, IP address, or a combination of these.
3. Set limits by endpoint
A public read endpoint and a password-reset endpoint do not have the same risk profile.
You might allow more requests for inexpensive reads while applying stricter limits to authentication or resource-intensive operations.
4. Consider an existing library
For many Express applications, using a maintained rate-limiting library with a Redis-compatible store is safer than implementing every detail yourself.
A library can handle important concerns such as response headers, window behavior, storage integration, and configuration. You still need to verify that its algorithm and failure behavior match your requirements.
How to Test Your Rate Limiter
Don't test only one request at a time against your local development server.
A more useful test plan includes:
- Send more requests than the configured limit and verify that excess requests receive
429. - Send requests through multiple application instances and confirm that the shared limit is enforced.
- Verify that the counter resets or expires as intended.
- Test concurrent requests, not just sequential requests.
- Simulate Redis failures and confirm the intended fail-open or fail-closed behavior.
- Test forwarded IP headers using your actual proxy configuration.
For production systems, also monitor rejected requests and storage errors. A rate limiter that silently fails can be difficult to diagnose.
The Real Lesson
Rate limiting is not just about counting requests. It is about maintaining consistent state across the infrastructure that processes those requests.
An in-memory counter can be perfectly reasonable for a single-process prototype. But once an application scales horizontally, local state is no longer enough for a globally enforced limit.
Before deploying a rate limiter, ask yourself one question: does every server agree on how many requests this client has made?
If the answer is no, your limit might not be the limit you think it is.
Have you implemented rate limiting in a production application? Did you use Redis, a gateway, or another approach?
Top comments (2)
Redis sharing state fixes the split-brain, but watch the increment itself: read-then-write from two instances can both pass under the limit. Use an atomic INCR with an EXPIRE set only on the first increment (or a Lua script), so the window and the counter live or die together. Two more production cases: fixed windows let a client fire 2x the limit at a window boundary, so sliding window logs or a token bucket are worth the extra Redis work on login endpoints. And decide what Redis being down means - fail open and the limiter vanishes during an outage, fail closed and your login page goes down with the cache. For most apps, fail open plus an alert on the limiter being unreachable is the survivable choice.
The shared-counter point is the one that matters, and the failure I would add is on the key, not the store: keying by IP works until a whole office or a mobile carrier's CGNAT shares one address, and then one heavy user eats the quota for everyone behind it. I learned to key authenticated routes by user ID and leave IP-keying for anonymous endpoints only, with per-endpoint limits, because a login route and a public read route never deserved the same budget. The test that caught it for me was not the single-request check but firing concurrent bursts through two instances and watching whether every server agreed on the count. Do you key your authenticated routes by user, or does everything still hang off the IP?