Designing a rate limiter: the low-level design interview, explained

Designing a rate limiter: the low-level design interview, explained

"Design a rate limiter" is a staple of the low-level design interview because it hides a surprising amount of design behind a one-line requirement: allow at most N requests per client per window. The naive version is a counter and an if; the interview version is a clean, extensible class model where the counting algorithm can change without rewriting the limiter, where two requests arriving in the same millisecond cannot both slip through on the last token, and where the same design still works when you have fifty servers instead of one. What the round tests is object-oriented design — how you turn that one-line rule into classes, where you put the algorithm, and how you defend the choice when the interviewer swaps the requirement.

This guide walks the rate limiter the way a strong candidate works it in an LLD / OOD interview: clarify the requirements, identify the core entities, sketch the classes and the key methods, and then defend the handful of decisions the interviewer always pushes on — which algorithm backs the limiter (token bucket, leaky bucket, fixed and sliding window), how those algorithms stay swappable behind one interface, what happens when two requests race for the last token, and how the whole thing behaves across many servers. In a low-level design round you write these classes out in an editor, not as a finished program but as a design you can explain and extend out loud, one method at a time.

Why the rate limiter comes up in low-level design interviews

The rate limiter earns its place because it is small enough to finish in the time given and rich enough to separate candidates. There is one obvious entry point (an allow / deny decision), one obvious variation (several counting algorithms, each with a different trade-off), one obvious extensibility axis (new algorithms, new rules per client or per endpoint), and one obvious failure (a concurrency race on the shared counter). An interviewer can watch you choose an algorithm, justify it, structure the classes so a second algorithm is a drop-in, and then handle the race — all in half an hour.

The interviewer rarely lets a working counter stand on its own. "You've got a fixed-window counter allowing 100 requests a minute. A client sends 100 requests in the last half-second of one minute and 100 more in the first half-second of the next. Walk me through what your limiter does — and whether that's really '100 per minute'." That question — the boundary burst that lets 200 requests through in one second — is the one a naive counter can never survive, and the reason the algorithm choice is the heart of this problem.

The rounds on Zynter are modelled on how the loop runs, not scraped from any candidate's recall. What matters for prep is the shape: you name an algorithm, and the next sentence attacks its weakest point — a burst at the window boundary, memory per client, a dropped request, or two threads racing the same token.

What the interviewer is really testing

A low-level design answer is read on the same four dimensions as any design round. Problem solving — did you pick an algorithm that fits the requirement (smooth versus bursty) and keep the design extensible, or hard-code one counter? Communication — can you explain in one sentence why a sliding window fixes the boundary burst that a fixed window allows? Technical execution — do you make the token check atomic so the last-token race cannot double-allow, or did you design only the single-threaded happy path? Time management — did you get a clean interface down fast and spend your time on the algorithm and the race, not on configuration plumbing? Reading this guide teaches the design; it does not teach you to defend it while someone swaps "smooth" for "allow bursts" mid-sentence.

Step 1: Clarify the requirements

Before a class, ask what "rate limit" means here — the answer changes the algorithm.

Functional requirements to confirm:

• Limit a client (identified by user ID, IP, or API key) to N requests per time window (for example 100 per minute).
• The limiter returns a fast allow or deny decision per request; denied requests are rejected (HTTP 429), not queued — unless the requirement is specifically to smooth traffic.
• Support multiple rules — different limits per endpoint, per client tier, or per API key.
• Decide whether short bursts are acceptable (token bucket) or traffic must be smoothed to a steady rate (leaky bucket).

Non-functional / constraints to surface (this is where the senior signal is):

• Extensibility — the counting algorithm should be swappable without touching the calling code, because the "right" algorithm depends on the requirement.
• Concurrency — many threads check the same client's counter at once, so the check-and-update must be atomic.
• Low latency — the limiter sits in front of every request, so it must be O(1) and in-memory on the hot path.
• Distribution — will one server enforce the limit, or must the limit be shared across a fleet? Agree the scope out loud; a single-node design and a distributed one differ sharply.

State your assumptions explicitly: "I'll design an in-memory, single-node limiter with a swappable algorithm first, keyed by client ID, returning allow/deny; then extend it to a shared store for the multi-server case — tell me if distributed enforcement is in scope from the start."

Step 2: Identify the core entities

The nouns give you the model:

• RateLimiter — the interface every algorithm implements: boolean allowRequest(String clientId). This is the seam the whole design turns on.
• TokenBucketRateLimiter, LeakyBucketRateLimiter, FixedWindowRateLimiter, SlidingWindowRateLimiter — concrete strategies, one per algorithm.
• Rule / RateLimitRule — the configuration: the limit, the window length, and what it applies to (an endpoint or client tier).
• ClientState / Bucket — the per-client counting state (tokens left and last-refill time, or a window's request timestamps), held in a map keyed by client ID.
• RateLimiterFactory — builds the right limiter from a Rule, so the calling code never does new TokenBucketRateLimiter() directly.
• RateLimiterRegistry — a (often singleton) holder that maps a client or endpoint to its limiter instance and its per-client state.

Name the relationships out loud: a RateLimiter has a Rule and owns a map of clientId → ClientState; the concrete limiters implement the RateLimiter interface (is-a); the RateLimiterFactory creates a RateLimiter from a Rule. That has-a / is-a vocabulary is what the interviewer is listening for.

Step 3: Sketch the classes

Lead with the interface, because the whole design hangs off it, then one concrete algorithm.

interface RateLimiter {
    boolean allowRequest(String clientId);   // true = allow, false = reject (429)
}

class Rule {
    int maxRequests;      // e.g. 100
    long windowMillis;    // e.g. 60_000
}

// Token bucket: tokens refill at a steady rate; a request costs one token.
// Allows short bursts up to the bucket size, then throttles to the refill rate.
class TokenBucketRateLimiter implements RateLimiter {
    private final Rule rule;
    private final Map<String, Bucket> buckets = new ConcurrentHashMap<>();

    public boolean allowRequest(String clientId) {
        Bucket b = buckets.computeIfAbsent(clientId, k -> new Bucket(rule));
        synchronized (b) {                 // one client's bucket is shared state
            b.refill();                    // add tokens for elapsed time
            if (b.tokens >= 1) {
                b.tokens -= 1;             // atomic check-and-consume
                return true;
            }
            return false;
        }
    }
}

class Bucket {
    double tokens;
    long lastRefill;
    // refill() adds (now - lastRefill) * refillRate tokens, capped at capacity
}

class RateLimiterFactory {
    static RateLimiter create(String algorithm, Rule rule) {
        return switch (algorithm) {
            case "token_bucket"   -> new TokenBucketRateLimiter(rule);
            case "sliding_window" -> new SlidingWindowRateLimiter(rule);
            // ... leaky bucket, fixed window
            default -> throw new IllegalArgumentException(algorithm);
        };
    }
}

Narrate the choices as you write: allowRequest is the single seam, so any algorithm is a drop-in; the per-client Bucket is guarded by a lock (or an atomic operation) because it is shared mutable state; the Factory keeps new TokenBucketRateLimiter() out of the caller so adding an algorithm never touches calling code.

Step 4: The design decisions interviewers probe

The classes are the easy half. The follow-ups live in the algorithm trade-offs and the race.

• Which algorithm? This is the core of the problem — know the four and their trade-offs:
  • Token bucket — tokens refill at a fixed rate; each request spends one; the bucket size allows short bursts. The common default: simple, O(1), memory-cheap, and burst-friendly.
  • Leaky bucket — requests enter a fixed-size queue that drains at a constant rate; it smooths bursty input into a steady output. Choose it when the downstream needs a steady rate, not when you want to reject fast.
  • Fixed window counter — a counter per fixed window (per minute). Simple and cheap, but it allows the boundary burst: up to 2× the limit across the seam between two windows.
  • Sliding window log — store a timestamp per request and count those inside the trailing window. Exact, no boundary burst, but memory grows with request volume.
  • Sliding window counter — a weighted blend of the current and previous fixed windows; fixes most of the boundary burst at O(1) memory. The usual "best balance" answer.
• Keep it extensible (Strategy). The reason RateLimiter is an interface: the counting algorithm is exactly the thing that varies, so it belongs behind a Strategy. Swapping token bucket for sliding window is a new class and a factory line, not an edit to the service.
• Concurrency — the race for the last token. Two threads call allowRequest for the same client at the same instant; both read "1 token left," both decrement, and you have allowed one request too many. The fix to name: make the check-and-consume atomic — a lock on that client's bucket, or an atomic/compare-and-set on the token count — so exactly one thread wins. State the insight: the per-client counter is shared mutable state, so read-check-write must be a single atomic step.
• Across many servers. A per-process counter lets a client exceed the global limit by hitting N servers. The answer: move the state to a shared store (Redis) and make the increment atomic there — an atomic INCR with an expiry for fixed window, or a small Lua script / sorted set for sliding window — trading a network hop for a correct global limit. Note the trade-off (latency, the store as a dependency) rather than hand-waving it.
• Edge cases — the first request for a new client (lazily create its bucket), clock skew across servers, and what to return when denied (429 plus a Retry-After).

Low-Level Design
LLD · Rate Limiter
Design the rate limiter and defend the boundary burst out loud

Practise on Zynter →

Design patterns that fit the rate limiter

Naming the patterns that fit — and why — is a strong signal in an OOD round.

• Strategy — the counting algorithm. A RateLimiter interface with token-bucket, leaky-bucket and sliding-window implementations is the textbook Strategy, and the single most important pattern in this problem.
• Factory — algorithm creation. A RateLimiterFactory turns a config string or Rule into the right limiter, keeping construction out of the business logic and centralising the "which class" decision.
• Singleton — the registry. The RateLimiterRegistry that holds limiters and per-client state is usually a single shared instance; note it must be thread-safe and injectable for tests.
• Decorator (optional) — layering limits. A per-second limit wrapping a per-day limit can be modelled as decorators over the same interface, each delegating then applying its own rule.

Say it crisply: Strategy for the algorithm is the one that pays off most; Factory keeps creation tidy; the registry is a thread-safe Singleton; and I'd reach for Decorator only if I need to stack multiple limits. Naming the trade-off, not just the pattern, is what reads as experience.

Low-Level Design
LLD · Core skills
Apply Strategy, Factory and Singleton out loud

Practise on Zynter →

How to talk about a low-level design in an interview

The probes from every section rhyme. Master these five and you can defend any LLD, not just the rate limiter.

• "What are the core classes here?" → A RateLimiter interface with allowRequest, concrete algorithm strategies, a Rule, per-client Bucket state, and a Factory — stated with the relationships before I write a line. (communication)
• "Which algorithm, and why?" → Token bucket for burst-friendly O(1) limiting; sliding window counter when I must avoid the boundary burst at low memory; leaky bucket when I need to smooth, not reject. (problem solving)
• "A fixed window lets a client double the limit at the boundary — fix it." → Move to a sliding window (log for exact, counter for O(1)) so the trailing window, not a resetting bucket, decides. (technical execution)
• "Two requests race for the last token — what happens?" → The counter is shared mutable state, so the check-and-consume must be atomic — a lock on that client's bucket or a compare-and-set — so exactly one is allowed. (technical execution)
• "Now there are fifty servers — does your limit still hold?" → Not with per-process counters; move the state to a shared store with atomic increments (Redis INCR + expiry, or a sorted set for sliding window), trading a hop for a correct global limit. (problem solving)

Each maps to a debrief dimension. Reading them teaches the answers; it does not teach you to deliver them while someone says "now make it distributed" — that only comes from saying them out loud.

How to prepare: a short plan

You do not need to memorise one reference solution. You need the OOD process automatic, so any "design X" prompt becomes requirements → entities → classes → decisions.

• Day 1: Nail the interface and the four algorithms — token bucket, leaky bucket, fixed window, sliding window — until you can draw each one and state its one-sentence trade-off, especially the boundary burst, from memory.
• Day 2: Drill the decisions — Strategy for the algorithm, the atomic last-token race, and the shared-store move for distribution. These are where the follow-ups live.
• Day 3: Do it out loud, timed, with someone swapping the requirement (smooth instead of burst, then make it distributed), then move to a second LLD (parking lot, elevator) to prove the process transfers.

This explainer feeds the broader low-level design interview track — the rate limiter is a compact OOD problem whose Strategy-and-concurrency shape recurs across the rest.

Frequently asked questions

What is a rate limiter low-level design interview testing?

It is testing object-oriented design, not just whether you know a counting trick. The interviewer watches how you turn the one-line rule "N requests per client per window" into a clean class model: a single interface for the allow/deny decision, the counting algorithm placed behind that interface so it can be swapped, a per-client state holder, and configuration as its own object. On top of the model, they probe the two things that make the problem real — choosing an algorithm whose trade-offs fit the requirement, and making the counter update atomic so concurrent requests cannot both consume the last token. A strong answer gets a clean interface down quickly and spends its time on the algorithm choice and the race.

What are the main rate limiting algorithms?

There are four common ones. Token bucket refills tokens at a steady rate and spends one per request, allowing short bursts up to the bucket size — simple, O(1), and the usual default. Leaky bucket queues requests and drains them at a constant rate, smoothing bursty input into steady output. Fixed window counter counts requests per fixed interval; it is cheap but allows a boundary burst of up to twice the limit across the seam between two windows. Sliding window log stores a timestamp per request for an exact trailing-window count at the cost of memory, and sliding window counter blends the current and previous windows to fix most of the boundary burst at O(1) memory. The interview-worthy move is matching the algorithm to whether the requirement wants bursts allowed, traffic smoothed, or exact fairness.

How do you make a rate limiter thread-safe?

The danger is a race on a client's counter: two threads call allowRequest at the same instant, both read that one token remains, both decrement, and one request too many is allowed. You make it thread-safe by ensuring the read-check-write is a single atomic step — hold a lock on that specific client's bucket while checking and consuming, or use an atomic compare-and-set on the token count, so exactly one thread can take the last token and the other is rejected. Keeping the critical section per-client (rather than one global lock) preserves throughput. In a distributed setting the same principle applies at the shared store, using an atomic increment or a small script so the check-and-update is atomic across servers.

How does a rate limiter work across multiple servers?

A per-process in-memory counter breaks as soon as there is more than one server, because a client can send N requests to each server and blow past the global limit. The standard fix is to move the counting state out of the process into a shared store such as Redis, and to make the increment atomic there — for a fixed window, an atomic INCR on a key that carries an expiry equal to the window; for a sliding window, a sorted set of timestamps or a short Lua script that does the check-and-update in one round trip. You trade a network hop of latency for a correct global limit, and you should name that trade-off, along with the store now being a dependency, rather than treating distribution as free.

Token bucket vs sliding window — which should I use?

They solve slightly different problems. Token bucket is about allowing controlled bursts: it is O(1), memory-cheap, and lets a client spend accumulated tokens in a short spike, which is ideal for user-facing APIs where occasional bursts are fine. Sliding window is about fairness and avoiding the fixed-window boundary burst: the sliding window counter gives you an accurate "last 60 seconds" limit at O(1) memory, and the sliding window log gives an exact count at higher memory cost. If the requirement is "allow some burstiness but cap the sustained rate," reach for token bucket; if it is "strictly no more than N in any rolling window," reach for a sliding window. Saying why you picked one over the other is what the round rewards.

How do I practise a low-level design interview out loud?

Reading a rate limiter solution is not the same as defending your algorithm choice while an interviewer swaps the requirement or pushes on the last-token race. On Zynter, the LLD track runs live voice rounds modelled on how real low-level design loops run — you write your classes and methods in the editor and a voice interviewer follows up one question at a time ("fixed window lets a client double the limit at the boundary — fix it", "two requests race for the last token — what happens?", "now make it work across fifty servers"), then gives you a debrief on problem solving, communication, technical execution and time management. The score is a practice signal, not a hiring decision.

Practise it on Zynter

You can name token bucket and sliding window. The round is won on defending the boundary burst and the last-token race out loud.

Pick the Rate Limiter low-level design round, write your classes in the editor, and defend them out loud to a voice interviewer that keeps asking "which algorithm, and why?" and "two requests just raced for the last token — what does your code do?" — then read the debrief. Zynter.ai is on-demand mock interviews modelled on real company loops, with 150+ rounds across 20 companies including the low-level design track (Sep 2026).

Practise the Rate Limiter LLD round →
Browse all interview tracks