DEV Community

The BookMaster
The BookMaster

Posted on

Weighted Token-Bucket Rate Limiting for Node.js & Bun

#node #typescript #webdev

Weighted Token-Bucket Rate Limiting for Node.js & Bun

Rate limiting is one of those things every API needs and almost every implementation gets subtly wrong. The classic approaches have real blind spots:

  • Fixed-window counters ("max 100 requests/minute") are simple but bursty — a client can fire 100 requests at 11:59:59 and another 100 at 12:00:00.
  • Request-counting middlewares like express-rate-limit count requests, not weight. A cheap status check costs the same as a heavy report query, and giving power users a higher cap means forking your code.

The token bucket solves both. Tokens refill continuously at a configurable rate, and each request costs a configurable weight — so a few heavy requests can share a budget with many light ones. And when you store the bucket in Redis with an atomic Lua script, the same limit holds across every instance of a horizontally scaled service, with no read-modify-write race.

The core idea 

request ──► key (IP / user / route)
            token bucket (capacity, refillRate)
            tokens = min(capacity, tokens + elapsed_time × rate)
            if tokens >= cost ──► allow, tokens -= cost
            else ─────────────► 429 + Retry-After
Enter fullscreen mode Exit fullscreen mode

Two backends:

  • MemoryBackend — in-process, zero dependencies, perfect for a single instance.
  • RedisBackend — shared across instances, updated by one atomic Lua script so distributed limits stay correct.

A minimal example 

import { RateLimiter } from "rate-limiter-middleware";

const limiter = new RateLimiter({
  capacity: 100,      // burst allowance
  refillRate: 10,      // tokens per second
});

// Framework-agnostic handle() — works with fetch, Hono, bare http, etc.
const decision = await limiter.handle(req, res, {
  keyGenerator: (req) => req.headers["x-api-key"] ?? req.socket.remoteAddress,
});

if (!decision.allowed) {
  res.statusCode = 429;
  res.setHeader("Retry-After", decision.retryAfter);
  res.end("Too Many Requests");
}
Enter fullscreen mode Exit fullscreen mode

Or drop it in as Express-style middleware:

import { rateLimitMiddleware } from "rate-limiter-middleware";
app.use("/api", rateLimitMiddleware({ capacity: 100, refillRate: 10 }));
Enter fullscreen mode Exit fullscreen mode

Per-path weights

Heavy endpoints get their own budget without forking code:

new RateLimiter({
  routes: {
    "/api/search": { capacity: 30, refillRate: 5 },
    "/api/export": { capacity: 5,  refillRate: 1 },
  },
});
Enter fullscreen mode Exit fullscreen mode

Going distributed 

import { RedisBackend } from "rate-limiter-middleware/redis";

const limiter = new RateLimiter({
  backend: new RedisBackend({ url: process.env.REDIS_URL }),
  capacity: 100,
  refillRate: 10,
});
Enter fullscreen mode Exit fullscreen mode

Now every instance of your service shares one bucket. If Redis goes down, the limiter fails open — it degrades gracefully instead of blocking all traffic.

Why this matters

Lazy refill means no timers — an idle instance costs nothing. Every decision emits an event (key, allowed, remaining, retry-after) so you can wire it into your metrics. Weighted costs let you price heavy endpoints fairly. And the atomic Lua script is the difference between "rate limiter" and "rate limiter that's actually correct under load."

The library is rate-limiter-middleware — MIT-licensed, strictly typed, one runtime dependency (ioredis, only when you use the Redis backend). If you've been fighting bursty fixed windows or counting requests instead of weight, it's worth a look.

Top comments (0)