19 Requests Through a 10-Request Limit

I shipped a rate limiter recently and told the team the endpoint was now capped at 10 requests per 10 seconds. Later I watched a burst test land 19 requests in about a second and a half. All 200s. Nothing was broken — AddFixedWindowLimiter was doing exactly what it says on the tin. I'd just never read the tin carefully. A fixed window doesn't promise "at most 10 in any 10 seconds". It promises "at most 10 per window", and windows have edges.

If a client times its bursts to straddle one of those edges, it collects almost two windows' worth of requests at once. That's the kind of claim that's easy to nod at and easy to misjudge, so I built the smallest repro that could settle it.

Two policies, same numbers

A minimal API on .NET 10 with two policies that look interchangeable in a code review:

builder.Services.AddRateLimiter(options =>
{
    options.RejectionStatusCode = StatusCodes.Status429TooManyRequests;

    options.AddFixedWindowLimiter("fixed", o =>
    {
        o.PermitLimit = 10;
        o.Window = TimeSpan.FromSeconds(10);
        o.QueueLimit = 0;                 // reject instantly, no queueing
    });

    options.AddSlidingWindowLimiter("sliding", o =>
    {
        o.PermitLimit = 10;
        o.Window = TimeSpan.FromSeconds(10);
        o.SegmentsPerWindow = 5;          // 5 x 2-second segments
        o.QueueLimit = 0;
    });
});

(The middleware has been in the box since .NET 7; RequireRateLimiting("fixed") pins a policy to an endpoint.)

The fixed limiter keeps one counter and zeroes it every ten seconds. The sliding limiter keeps five little counters, one per 2-second segment, and "the window" is always the last five segments. That bookkeeping difference is the entire story.

The choreography

The app attacks itself with HttpClient. Four moves:

// 1. burn the current window down to zero
// 2. poll every 200ms until a request succeeds -> that's a replenish boundary
// 3. go quiet, come back 1s BEFORE the next boundary, burst 9
// 4. step just past the boundary, burst 10 more

Move 2 matters because window boundaries follow the limiter's internal clock, which starts when the first request lazily creates it — not yours. Rather than guessing, exhaust the window once and wait for permits to come back. The flip from 429 to 200 is a boundary announcing itself, and the next one is exactly one window later. Worth noticing: a real abuser gets the same information for free. It's just status codes.

Every 200 gets timestamped, and at the end the app reports the densest 10-second span it actually got away with. Conditions: Release build, localhost, small Linux container, wall clock. Not a lab, but this experiment isn't about milliseconds. It's about counting to 19.

What actually happened

--- attacking /fixed ---
  burn phase: 10 admitted, then 429s — window is empty
  permits came back at t=10.4s — boundary located
  pre-boundary burst  at t=19.4s: 9/9 admitted
  post-boundary burst at t=20.7s: 10/10 admitted
  => /fixed: densest 10s span held 19 admitted requests (limit says 10);
     those 19 arrived within 1.31s

Nine requests an instant before the reset, ten more an instant after. The limiter never blinked, because at the moment of reset it forgets everything. 19 requests in 1.31 seconds on a policy I would've sworn meant 10 per 10 seconds — a 1.9x hole in the promise, available at every boundary, forever.

Same choreography against the sliding window:

--- attacking /sliding ---
  pre-boundary burst  at t=19.4s: 9/9 admitted
  post-boundary burst at t=20.7s: 1/10 admitted
  => /sliding: densest 10s span held 10 admitted requests (limit says 10);
     those 10 arrived within 0.01s

The pre-boundary nine sail through, and then the second burst gets exactly one request in, because the nine from 1.3 seconds ago still count against the window. There's no moment of amnesia. The densest span it ever allowed was 10. The number on the tin.

That "10 within 0.01s" line deserves a beat too: any windowed limiter will serve its entire budget instantly to a cold window. If instantaneous burst is what actually hurts you, you want a token bucket with a small bucket, which is a different post.

Do you actually care?

Sometimes no. If the limit is a coarse backstop protecting capacity — no client gets more than a thousand requests a minute against this scraper-magnet endpoint — a brief 2x at a seam is noise, and fixed windows are the cheapest thing you can deploy: one counter per partition, nothing to segment. I'd leave those alone.

Where I've changed my default is limits that are promises to people. Per-tenant fairness, paid quota tiers, anything a client could deliberately game. A "10 per 10 seconds" that admits 19 is the kind of discovery that ends up in an awkward support thread. Sliding windows cost a little more memory per partition, since the segment counters exist for every tracked client, but with five segments it's small and the promise becomes true. My rule now: fixed for backstops, sliding for promises.

One caveat before you rip out every fixed window you own: everything above is a single process. The moment you scale out, every in-memory limiter — fixed or sliding — goes approximate anyway, because each node counts alone. At that point the boundary seam is the least of your inaccuracies, and you're shopping for a distributed limiter regardless.

Full runnable sample: https://github.com/ssukhpinder/dev-to-code-samples/tree/main/012-rate-limit-window-burst — it takes about 45 seconds to run because it has to sit through real windows.

Is your fixed window a backstop or a promise? If it's a promise, I'd go check what your densest span really is — and I'd like to hear what you find in the comments.

— still stress-testing promises nobody asked me to verify

Comments

[Ivan Rossouw]

One extension I’d add is to separate fairness over time from instantaneous pressure. A sliding window can preserve the "10 in any 10 seconds" promise and still admit all ten together, so if the scarce resource is a database connection pool or CPU-heavy render, I’d chain the per-tenant sliding limiter with a small concurrency limiter.

In a scaled-out service, I’d enforce the entitlement at a shared gateway or distributed counter but keep concurrency local to each node. Then load-test two invariants separately: the densest admitted time span and peak in-flight work. Emit Retry-After when the rejected lease supplies it, and count rejections by policy or tier rather than a raw tenant identifier. The diagnostic question becomes: are we protecting a customer quota, a node resource, or both?