DEV Community

Cover image for The 429 looked right. The limiter was wrong
agustin18
agustin18

Posted on AI-assisted

The 429 looked right. The limiter was wrong

#python #fastapi #redis #debugging

A debugging story about layered limits in a FastAPI SSE demo.

I built a small browser demo for fastapi-stream-lease to show how a Redis lease stops one user from holding several long-lived streams at the same time. Two tabs, one session cookie, a synthetic 20-second SSE stream. The first tab streams. The second tab gets 429. Everything looked correct.

It wasn't.

The 429 was real, but it came from the wrong limiter. The second tab was rejected by a demo-specific per-IP guard before it reached the per-session Redis lease I wanted to show. The tests passed. The UI looked fine. The only thing wrong was the story the demo was telling.

This is how I found it, what I changed, and why I now test the layer that rejects a request, not just the status code.

What the demo does

The demo is local-only. It runs with:

docker compose up --build demo
Enter fullscreen mode Exit fullscreen mode

It starts Redis and a small FastAPI app at http://localhost:8080. The app has two panels. Both share one signed HttpOnly session cookie. Each panel opens a synthetic SSE stream that yields tokens for 20 seconds.

The library provides per-session and global stream leases. The demo adds two extra layers:

  • an in-memory request rate limiter
  • a Redis-backed per-IP guard

The request path is:

rate limit -> per-IP guard -> per-session lease -> SSE
Enter fullscreen mode Exit fullscreen mode

The per-session lease is the library. The other two are demo guardrails. That distinction matters.

The first limiter wins

Layered limits are useful, but they are also easy to misread. If the per-IP guard rejects first, the per-session lease never runs. The client still gets 429. A test that checks only status_code == 429 passes. The demo looks correct. But the feature you wanted to demonstrate was never exercised.

This is a small detail, but it changes completely the story. I was more focused in the 429 than in the layer that produced it.

The UI did not help. It said the second tab was rejected, which was true. It did not say which layer rejected it. I looked at the status code, saw 429, and moved on. That was the mistake.

How I found it

The response body had a code. The second tab returned:

{
  "code": "demo_ip_limit",
  "detail": "Too many active demo streams from this network."
}
Enter fullscreen mode Exit fullscreen mode

The expected one was:

{
  "code": "stream_connection_limit",
  "reason": "user_limit",
  "detail": "User stream limit reached"
}
Enter fullscreen mode Exit fullscreen mode

That was the clue. The default DEMO_MAX_PER_IP was 1, so the second tab from the same browser hit the IP guard before the session lease. The test config used max_per_ip=2, so the test took a different path and passed.

That is the part I missed: the test was not using the same config as the app.

The test gap

The test looked reasonable. It created a demo app, opened a stream, waited for the lease, and opened a second stream. It asserted 429 and reason == "user_limit".

The problem was in the config helper:

def make_config(**overrides):
    values = {
        ...
        "max_per_ip": 2,
        ...
    }
Enter fullscreen mode Exit fullscreen mode

The app default was 1. The test created a world where the bug could not happen.

I removed the override so the test uses the real default. Then I added a second test that starts two sessions, fills the per-IP guard, and asserts the third one gets demo_ip_limit. The first test now asserts the layer:

assert rejected.json()["code"] == "stream_connection_limit"
assert rejected.json()["reason"] == "user_limit"
Enter fullscreen mode Exit fullscreen mode

That is the test that would have caught the original problem.

The fix

I changed the demo default to DEMO_MAX_PER_IP=2, so two tabs from the same network can reach the per-session lease. The per-IP guard still exists. A third session from the same network hits it.

I also added max_per_ip and rate_limit_per_minute to /demo/stats, and the UI now shows the request path and the layer that rejected the stream. A blocked panel says per-session lease or per-IP guard, not just 429.

The library did not change. This was a demo bug.

The library call is still the same:

return await manager.stream(user_id, token_stream())
Enter fullscreen mode Exit fullscreen mode

It takes any async token generator, acquires the lease before the stream starts, renews it while the response is open, and releases it on completion, error, or client disconnect.

What changed in v0.6.4

  • The per-IP guard default is now 2.
  • /demo/stats includes max_per_ip and rate_limit_per_minute.
  • The UI shows per-session and per-network limits, a guided "Try this" flow, and the request path.
  • The demo suite has 46 tests and 100% branch coverage for demo/app.py.
  • The library suite has 467 tests.
  • No runtime library changes.

If you are already on 0.6.3, this is a demo-only update.

What I would do differently

Three things:

  1. Return a machine-readable reason, not only 429. A status code tells you that something rejected the request. It does not tell you what.
  2. Test with the same config users get. A test-only override can hide the default path.
  3. Test each layer separately. If you have a rate limit, a per-IP guard, and a per-session lease, each one needs its own assertion.

For streaming endpoints, I would also test the disconnect path. Close the client in the middle of the stream and check that the lease is released. That is where orphaned generations and socket leaks hide.

A 429 is not a test. The layer is.

Try it 

git clone https://github.com/agustin18/fastapi-stream-lease.git
cd fastapi-stream-lease
docker compose up --build demo
Enter fullscreen mode Exit fullscreen mode

Open http://localhost:8080. Start Tab A, then Tab B. Tab B gets 429 from the per-session lease. A third session from the same network hits the per-IP guard. No hosted service or API keys are required.

If you run streaming endpoints in production, what layer do you test first?

Top comments (0)