Rate Limiting in Go With golang.org/x/time/rate

• Book: The Complete Guide to Go Programming
• Also by me: Hexagonal Architecture in Go — the companion book in the Thinking in Go series
• My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
• Me: xgabriel.com | GitHub

---

You wired up a rate limiter before shipping the new API. You picked
golang.org/x/time/rate
because it's the closest thing Go has to a blessed answer. You wrote
rate.NewLimiter(100, 100), felt good, and
moved on. Then a client opened 100 connections at once, drained the
whole bucket in a single millisecond, and your "100 requests per
second" limiter waved all of them through before the first token ever
refilled.

That is not a bug in the package. It's the burst parameter doing
exactly what you told it to. x/time/rate is a token bucket, and the
second argument is the bucket size, not a per-second cap. Most of the
confusion with this package traces back to that one number. So: the
limiter, the three ways to ask it for a token, per-key limiters, and how
to pick a burst you won't regret.

The token bucket, in one paragraph

A token bucket holds up to b tokens. Tokens refill at a steady rate
r per second. Every request takes one token. If a token is there, the
request goes through. If the bucket is empty, the request waits, gets
rejected, or reserves a future slot, depending on which method you call.

You build one like this:

import "golang.org/x/time/rate"

// 100 tokens/sec, bucket holds up to 20.
lim := rate.NewLimiter(100, 20)

The first argument is a rate.Limit, which is a float64 of tokens
per second. The second is the burst, an int. If you'd rather think in
intervals than in per-second floats, rate.Every converts for you:

// One token every 200ms == 5 tokens/sec.
lim := rate.NewLimiter(rate.Every(200*time.Millisecond), 1)

rate.Every(0) returns rate.Inf, an unlimited limiter. Handy for a
config flag that turns limiting off without branching.

Allow: the non-blocking check

Allow is the one you reach for in an HTTP handler. It takes a token if
one is available and returns immediately. No waiting.

func handler(w http.ResponseWriter, r *http.Request) {
    if !lim.Allow() {
        http.Error(w, "slow down", http.StatusTooManyRequests)
        return
    }
    serve(w, r)
}

Allow() is shorthand for AllowN(time.Now(), 1). Use AllowN when a
request costs more than one token, say a batch endpoint where each item
is a token:

if !lim.AllowN(time.Now(), len(batch)) {
    http.Error(w, "batch too big right now", 429)
    return
}

Allow never blocks and never puts the caller into debt. If the bucket
is short even one token, it returns false and takes nothing. That makes
it the right tool for user-facing traffic you want to reject fast.

Wait: block until a token frees up

Wait is for the opposite case: a background worker or an outbound
client where you'd rather pace yourself than drop work. It blocks until
a token is available or the context is done.

func callUpstream(ctx context.Context, lim *rate.Limiter) error {
    if err := lim.Wait(ctx); err != nil {
        return err // ctx cancelled or deadline hit
    }
    return doRequest(ctx)
}

Two things people miss about Wait. First, it respects the context, so
a cancelled or timed-out ctx makes it return an error instead of
blocking forever. Always pass a real context, not context.Background(),
in anything that can be shut down.

Second, Wait(ctx) fails immediately if n exceeds the burst and the
rate isn't Inf. Asking for more tokens than the bucket can ever hold
can never be satisfied, so it errors right away rather than blocking on
a refill that will never come:

lim := rate.NewLimiter(10, 5)
// WaitN for 6 tokens can never succeed: burst is 5.
err := lim.WaitN(ctx, 6) // returns an error immediately

That's a common footgun in worker pools where the batch size drifts
above the burst you configured.

Reserve: get a token and a delay you control

Reserve sits between the other two. It always gives you a reservation
and tells you how long to wait before acting on it. You decide what to
do with that delay.

r := lim.Reserve()
if !r.OK() {
    // Can't be satisfied within the limiter's limits at all.
    return errors.New("cannot proceed")
}
delay := r.Delay()
if delay > maxWait {
    r.Cancel() // give the token back
    return errors.New("too congested")
}
time.Sleep(delay)
doWork()

The r.Cancel() call is the part worth internalizing. A reservation
takes tokens from the bucket the moment you call Reserve. If you
decide not to use it, Cancel returns those tokens so the next caller
isn't punished for your abandoned request. Skip the Cancel and you've
quietly thrown tokens away.

Reserve shines when you want to enforce your own timeout policy, or feed
the delay into a Retry-After header instead of sleeping:

r := lim.Reserve()
if d := r.Delay(); d > 0 {
    w.Header().Set("Retry-After",
        strconv.Itoa(int(d.Seconds())+1))
    http.Error(w, "rate limited", 429)
    r.Cancel()
    return
}

The mental split: Allow throws away over-limit requests, Wait
sleeps on them for you, Reserve hands you the delay and lets you
choose.

Per-key limiters: one bucket per client

A single limiter caps your whole service. Usually you want a cap per
user or per IP so one noisy client can't starve everyone else. The
package doesn't do this for you, so you keep a map of limiters guarded
by a mutex.

type entry struct {
    lim  *rate.Limiter
    seen time.Time
}

type keyedLimiter struct {
    mu      sync.Mutex
    entries map[string]*entry
    r       rate.Limit
    b       int
}

func newKeyed(r rate.Limit, b int) *keyedLimiter {
    return &keyedLimiter{
        entries: make(map[string]*entry),
        r:       r,
        b:       b,
    }
}

func (k *keyedLimiter) get(key string) *rate.Limiter {
    k.mu.Lock()
    defer k.mu.Unlock()
    e, ok := k.entries[key]
    if !ok {
        e = &entry{lim: rate.NewLimiter(k.r, k.b)}
        k.entries[key] = e
    }
    e.seen = time.Now()
    return e.lim
}

Then the handler pulls the right bucket by key:

func (k *keyedLimiter) allow(key string) bool {
    return k.get(key).Allow()
}

The catch nobody warns you about: this map grows forever. Every new IP
adds an entry that never leaves. For a public endpoint that's a slow
memory leak. That's why get stamps a last-seen time on every hit —
now a background sweep can drop the idle keys:

func (k *keyedLimiter) cleanup(maxIdle time.Duration) {
    k.mu.Lock()
    defer k.mu.Unlock()
    now := time.Now()
    for key, e := range k.entries {
        if now.Sub(e.seen) > maxIdle {
            delete(k.entries, key)
        }
    }
}

Run cleanup from a background goroutine on a time.Ticker. A dropped
key just means the client gets a fresh full bucket next time, which is
fine.

The burst value, set right

Back to the number that started this. Here is how to reason about it.

r controls the sustained rate. b controls how big a spike you'll
absorb before you start throttling. They answer different questions.

• NewLimiter(100, 1) — 100/sec sustained, but only one request may arrive at the same instant. Real bursty traffic gets throttled hard even though the average is fine. Too tight for most APIs.
• NewLimiter(100, 100) — 100/sec sustained, but a client can fire 100 in a single millisecond and drain the bucket. That's the opening story. The average looks right, the instantaneous load is a spike.
• NewLimiter(100, 20) — 100/sec sustained, absorbs a burst of 20, then paces the rest. This is usually what you actually want.

Burst is not "requests per second." It's "how many can clump together
before I care." Pick it from the spike your downstream can survive, not
from your per-second target. A good starting point is a small multiple
of what one legitimate client sends in a tight loop, then tune with real
traffic.

And two edge values to keep in mind: burst 0 with a finite rate
rejects everything (AllowN(now, 1) needs at least one token of
capacity), while rate.Inf ignores burst entirely and always allows.
If a limiter is silently blocking every request, check for a zero burst
first.

You can also retune at runtime. SetLimit and SetBurst adjust a live
limiter without rebuilding it, which is how you wire limits to a config
reload:

lim.SetLimit(rate.Limit(cfg.PerSecond))
lim.SetBurst(cfg.Burst)

The short version

Allow rejects fast, Wait paces and blocks, Reserve hands you the
delay (so Cancel the ones you skip). Keep per-key limiters in a
guarded map, sweep the idle keys, and size the burst to the spike you'll
tolerate rather than your per-second rate.

Token buckets are a small idea with a lot of runtime behavior hiding
under them, which is the sort of thing The Complete Guide to Go
Programming digs into when it covers timers, the scheduler, and how
x/time/rate actually meters tokens. And once a limiter shows up in
your service, Hexagonal Architecture in Go is where I'd point you for
keeping it at the edge — an adapter concern, not something smeared
through your domain logic.