🔥 The Story That Made Every Backend Dev's Stomach Drop
You probably saw it: a developer shipped a React component with a useEffect that had a missing dependency array (or a state update that retriggered itself), and it quietly hammered their API with over 100,000 requests before anyone noticed. No malicious actor, no botnet — just a bracket in the wrong place and a hook that fired on every render.
The internet had a good laugh, but every backend dev reading that thread had the same intrusive thought: "my API would've just... died."
That's the uncomfortable truth. A self-inflicted traffic spike from a buggy client is functionally indistinguishable from a DDoS if your server has no defenses. The fix isn't "tell frontend devs to be careful" — it's "assume they won't be, and build accordingly."
This post walks through the three layers I now consider non-negotiable for any Node/Express API: token-bucket rate limiting, circuit breakers, and defensive defaults. I'll also talk about a real (much smaller, thankfully) spike that hit my side project, minimalist-feedback-api, and what actually saved it.
🪣 Why Token Bucket Beats Fixed Windows
Most people's first rate limiter is a fixed window: "100 requests per minute per IP." It's easy to reason about and easy to implement badly. The problem is the boundary. If your window resets at :00, a client can send 100 requests at 11:59:59 and another 100 at 12:00:01 — 200 requests in two seconds, technically "within limits."
Token bucket fixes this by modeling capacity as a continuously refilling resource instead of a hard reset:
js
// tokenBucket.js
class TokenBucket {
constructor({ capacity, refillRatePerSec }) {
this.capacity = capacity;
this.tokens = capacity;
this.refillRate = refillRatePerSec;
this.lastRefill = Date.now();
}
_refill() {
const now = Date.now();
const elapsedSec = (now - this.lastRefill) / 1000;
const refillAmount = elapsedSec * this.refillRate;
this.tokens = Math.min(this.capacity, this.tokens + refillAmount);
this.lastRefill = now;
}
tryConsume(cost = 1) {
this._refill();
if (this.tokens >= cost) {
this.tokens -= cost;
return true;
}
return false;
}
}
module.exports = TokenBucket;
Then the Express middleware, keyed per client (IP, API key, whatever identifies the caller):
js
// rateLimitMiddleware.js
const TokenBucket = require('./tokenBucket');
const buckets = new Map();
function getBucket(key) {
if (!buckets.has(key)) {
buckets.set(key, new TokenBucket({ capacity: 20, refillRatePerSec: 2 }));
}
return buckets.get(key);
}
function rateLimit(req, res, next) {
const key = req.ip; // swap for API key if you have auth
const bucket = getBucket(key);
if (bucket.tryConsume(1)) {
return next();
}
res.status(429).set('Retry-After', '1').json({
error: 'Too many requests. Slow down.',
});
}
module.exports = rateLimit;
The key insight: capacity 20 with a refill rate of 2/sec means a client gets a burst allowance (handles legitimate rapid-fire usage like a form autosave) but can't sustain more than 2 requests/sec indefinitely. That's exactly the shape of a runaway useEffect loop — it doesn't send 100 requests once, it sends them in a tight, sustained burst. Token bucket catches that pattern where a naive fixed window might not, depending on where the boundaries land.
For anything beyond a single process, don't keep buckets in memory — use Redis (via something like rate-limiter-flexible) so limits survive restarts and work across horizontally scaled instances. In-memory Map is fine for a single-instance side project; it's a liability the moment you run two replicas behind a load balancer, because each instance tracks its own bucket and your effective limit doubles per replica.
🧯 Circuit Breakers: The Second Line of Defense
Rate limiting protects your API from too many incoming requests. Circuit breakers protect your API (and its downstream dependencies) from cascading failure once something's already struggling — usually a database, a third-party API, or an internal service call that's gone slow or unresponsive.
Here's the pattern with opossum, a solid circuit breaker library for Node:
js
const CircuitBreaker = require('opossum');
const db = require('./db');
async function fetchFeedback(id) {
return db.query('SELECT * FROM feedback WHERE id = $1', [id]);
}
const breakerOptions = {
timeout: 3000, // fail fast after 3s
errorThresholdPercentage: 50, // trip if 50% of requests fail
resetTimeout: 10000, // try again after 10s
};
const breaker = new CircuitBreaker(fetchFeedback, breakerOptions);
breaker.fallback(() => ({ error: 'Feedback service temporarily unavailable' }));
app.get('/feedback/:id', async (req, res) => {
const result = await breaker.fire(req.params.id);
res.json(result);
});
Without this, a slow database under load doesn't just cause slow responses — it causes request pileup. Every incoming request holds a connection open waiting on a query that's never coming back fast enough, you exhaust your connection pool, and now healthy requests fail too. The circuit breaker trips, starts returning fast fallbacks immediately, and gives the database room to recover instead of getting buried under retries.
Rate limiting stops the flood at the door. Circuit breakers stop one struggling dependency from taking the whole system down with it. You want both — they solve different failure modes.
🛡️ Defensive Defaults I Now Bake Into Every Express API
Beyond the two big patterns above, there's a checklist of small things that cost nothing to add and save you on a bad day:
js
const express = require('express');
const helmet = require('helmet');
const compression = require('compression');
const app = express();
// Cap body size — don't let a malformed client send you a 500MB payload
app.use(express.json({ limit: '100kb' }));
// Basic security headers
app.use(helmet());
// Compress responses to reduce bandwidth under load
app.use(compression());
// Global request timeout so nothing hangs forever
app.use((req, res, next) => {
res.setTimeout(10000, () => {
res.status(503).json({ error: 'Request timed out' });
});
next();
});
// Always have a catch-all error handler, even if it feels redundant
app.use((err, req, res, next) => {
console.error(err);
res.status(500).json({ error: 'Something went wrong' });
});
None of this is exciting. That's the point — defensive defaults are boring on purpose. The bracket-typo story went viral precisely because the API had no boring safety net, and 100,000 requests met zero resistance.
📈 The Day minimalist-feedback-api Got Hit
minimalist-feedback-api is a small feedback-collection service I built as a learning project — nothing fancy, just an endpoint for apps to POST feedback and a dashboard to read it. It's not built to handle enterprise traffic, but I treated it like production because that's where you actually learn this stuff.
A few months in, one integrator's frontend had a retry loop with no backoff — every failed request immediately retried, and a brief blip in their own network turned into a sustained burst against my /feedback endpoint. It wasn't 100,000 requests, but it was enough (a few thousand in under a minute) to be a real stress test.
What actually saved it wasn't anything clever — it was the boring stuff: the token bucket limiter returned 429s immediately instead of letting requests queue up, the body size cap meant even the retries were cheap to reject, and the circuit breaker around my database call meant the brief connection pressure never turned into a full outage. The service degraded gracefully (some legitimate requests got 429'd too) instead of falling over entirely. That's the tradeoff you're signing up for: rate limiting means occasionally rejecting a request that would've been fine, in exchange for never going fully down.
🤔 What Would Your API Do?
Honestly ask yourself: if a client-side bug sent your busiest endpoint 100,000 requests in five minutes right now, what would happen? Would it 429 gracefully, or would your database connection pool just... give up?
If you're not sure, that uncertainty is the signal to add a rate limiter today — even a basic one. It's a couple hours of work that turns a viral "oops" story into a boring non-event. What's your go-to rate limiting setup, and have you ever had a spike (accidental or not) actually test it for real? I'd love to hear the war stories in the comments.
Top comments (0)