Everyone who writes async Python has been told not to block the event loop. What I had never seen measured is how much damage one blocking call actually does, and the part of FastAPI's behaviour that decides whether a given blocking call does any damage at all.
The short version is that the same requests.get was completely harmless in a plain def endpoint and froze the entire server in an async def one. An unrelated endpoint that normally answers in 1.6 milliseconds took 34 seconds. The usual fixes have edges too: one of them stops working at exactly forty concurrent requests, and another barely helps at all.
The setup
Everything runs locally on Python 3.14.4, FastAPI 0.143.0, Starlette 1.7.0, uvicorn 0.54.0 and anyio 4.15.1, with uvicorn's default single worker and its default asyncio event loop.
A separate upstream service stands in for a slow third-party API: every request to it takes exactly one second. The app under test has a handful of endpoints that call it in different ways, plus a trivial one I use as a canary:
@app.get("/fast")
async def fast():
return {"ok": 1}
@app.get("/sync_def")
def sync_def():
return requests.get(UP).json() # blocking client, plain def
@app.get("/async_blocking")
async def async_blocking():
return requests.get(UP).json() # same blocking client, async def
@app.get("/async_proper")
async def async_proper():
return (await client.get(UP)).json() # httpx.AsyncClient
The load generator runs twenty concurrent clients against one of the slow endpoints, and while they run it measures how long the trivial /fast endpoint takes to answer, probing it every fifty milliseconds. Each case runs for fifteen seconds, three times.
One keyword, a 20,000-fold slowdown
| while 20 clients hit |
/fast median |
slow requests served in 15 s |
|---|---|---|
| nothing (baseline) | 1.6 ms | |
async def + httpx
|
1.6 ms | 320 |
plain def + requests
|
1.6 ms | 320 |
async def + requests
|
34,130 ms | 35 |
Three of those rows are indistinguishable. The fourth is a different server. In fifteen seconds of probing, /fast answered exactly once, and that one answer took 34 seconds to arrive. The slow endpoint itself managed 35 requests in the time the others managed 320, which is one per second: the server had become strictly sequential.
The difference between the second and third rows is the whole story. requests.get blocks the thread it runs on until the upstream answers. In an async def endpoint, that thread is the one running the event loop, so for that second nothing else on the server can make progress, not the other nineteen slow requests and not the trivial one either. Twenty clients each holding the loop for a second turns into a queue.
A plain def endpoint is different because FastAPI never runs it on the event loop. It sends it to a thread pool, and the loop carries on serving everything else while the thread waits. The identical blocking call is harmless there.
Which means the common refactor of adding async to an existing endpoint, without also changing every library it calls, makes it worse. The code reads as more modern and behaves like a server with one thread.
The thread pool has forty threads
Plain def endpoints being safe raised an obvious next question: safe up to what point? FastAPI's thread pool comes from anyio, and anyio's default limit is easy to check:
anyio default thread pool size (total_tokens): 40
So I held the slow def endpoint at different concurrency levels and measured a trivial def endpoint, one that does nothing except return a constant:
slow def requests in flight |
trivial def endpoint, median |
|---|---|
| 20 | 1.9 ms |
| 39 | 1.9 ms |
| 40 | 890 ms |
| 60 | 953 ms |
| 100 | 1,955 ms |
There is a cliff at exactly forty. Below it, nothing is affected. At forty, every thread is busy waiting on the upstream, so the trivial endpoint has to wait for one to come free, which takes up to a second. At a hundred it waits for two rounds, and the slow requests themselves go from one second to 2.1.
Run exactly at the boundary, the result was split. Two runs out of three showed the 890 ms median, and one showed 11.9 ms with a 914 ms worst case, depending on whether a thread happened to free up just as the probe arrived. Forty is where the behaviour changes, not a precise threshold you can sit on.
async def endpoints don't use the pool at all. With a hundred slow def requests in flight, /fast still answered in 1.6 ms.
If you have a lot of slow synchronous endpoints, the limit is adjustable at startup:
@asynccontextmanager
async def lifespan(app):
anyio.to_thread.current_default_thread_limiter().total_tokens = 200
yield
With 200, a hundred concurrent slow requests left the trivial endpoint at 1.8 ms, and 150 simultaneous one-second requests all finished in 1.28 seconds. Forty threads would have needed four rounds of a second each to get through the same 150, by arithmetic rather than measurement. Threads are not free, so this trades memory for headroom rather than solving anything, but it is a one-line change that removes a cliff.
The blocking call that doesn't look like one
Network calls are the obvious culprit. CPU work is the one nobody suspects. One endpoint serialised a 300,000-row list to JSON, which took about 117 ms, the kind of thing that looks fine in a profile of a single request:
| clients hitting the JSON endpoint |
/fast median |
/fast p99 |
|---|---|---|
| 1 | 63–71 ms | up to 108 ms |
| 4 | 427–435 ms | 891–932 ms |
A single client serialising JSON made an unrelated endpoint forty times slower. Four of them made it 270 times slower. No network call involved, nothing that would show up in a review as obviously wrong.
Moving it to a thread barely helped
The standard advice for CPU work in async code is to push it off the loop with asyncio.to_thread. I tried that, alongside a process pool, with four clients each time:
where json.dumps runs |
/fast median, three runs |
JSON requests served |
|---|---|---|
inline in async def
|
393, 396, 397 ms | 120 |
asyncio.to_thread |
292, 303, 315 ms | 112 |
| process pool | 1.6, 1.6, 1.6 ms | 365 |
The thread helped by about a quarter and served slightly fewer requests than doing the work inline. The reason is the global interpreter lock. This build of Python 3.14 still has one, and serialising Python objects needs it, so the event loop thread spends most of its time waiting for the lock to be handed back, which happens on a five-millisecond switch interval. Moving the work to another thread moved where it ran without moving who it competed with.
A process pool has its own interpreter and its own lock. /fast went straight back to 1.6 ms, and because four processes could now serialise in parallel, the endpoint served three times as many requests. The first call into the pool took half a second while the worker processes started, so I warmed it before measuring.
Python's free-threaded build should change the thread result, since it removes the lock that caused it. I did not test that, so I would not assume the table above holds there.
Finding the line that's doing it
asyncio has a debug mode that warns about slow callbacks. With PYTHONASYNCIODEBUG=1 it caught both stalls:
Executing <Task finished name='Task-11' coro=<RequestResponseCycle.run_asgi() done, ...> took 1.003 seconds
Executing <Task finished name='Task-13' coro=<RequestResponseCycle.run_asgi() done, ...> took 0.117 seconds
It tells you that something blocked and for how long. It names uvicorn's request handler as the culprit, because that is the task that was running, not your endpoint and certainly not the line. On an application with fifty endpoints, that warning tells you there is a problem without telling you where.
So I wrote a small watchdog. A coroutine on the event loop updates a timestamp every fifty milliseconds, and a separate thread watches that timestamp. If it stops moving for more than a quarter of a second, the loop is blocked, and the thread grabs the main thread's stack at that moment, while the blocking call is still on it:
def install(threshold=0.25, interval=0.05):
loop, main_id, state = asyncio.get_running_loop(), threading.get_ident(), {"tick": time.monotonic()}
async def heartbeat():
while True:
state["tick"] = time.monotonic()
await asyncio.sleep(interval)
def watch():
reported = None
while True:
time.sleep(interval)
stalled = time.monotonic() - state["tick"]
if stalled > threshold and reported != state["tick"]:
reported = state["tick"]
stack = traceback.extract_stack(sys._current_frames()[main_id])
mine = [f for f in stack if "site-packages" not in f.filename and "/lib/python" not in f.filename]
f = (mine or stack)[-1]
print(f"[watchdog] event loop blocked {stalled*1000:.0f} ms at "
f"{f.filename.split('/')[-1]}:{f.lineno} in {f.name}(): {f.line}", file=sys.stderr)
loop.create_task(heartbeat())
threading.Thread(target=watch, daemon=True).start()
Call it from your app's lifespan and it points straight at the problem:
[watchdog] event loop blocked 289 ms at app.py:35 in async_blocking(): return requests.get(UP, timeout=30).json()
The 289 ms is when it noticed, not how long the stall lasted, since it reports as soon as the threshold is crossed in order to catch the call in progress. The 117 ms JSON stall was below the quarter-second threshold and went unreported, which is the right behaviour but a reminder to pick the threshold for the latency you actually care about.
What I got wrong
Twice during this experiment I killed the shell I was working in. I stopped the app between runs with pkill -f "uvicorn app:app", and the pattern matched the shell's own command line, which contained the same text. I had already learned to write the pattern as [u]vicorn so that it cannot match itself, and it still happened, because the same script restarted the server a few lines later and that line contained the plain string. The bracket trick protects the pattern, not the rest of the script. Stopping and starting now happen in separate commands.
A smaller one: my first attempt to print anyio's thread limit was a one-liner with its logic folded into a conditional expression, and it printed an empty line. I rewrote it as three readable lines before trusting the number forty, since every conclusion in that section rests on it.
What to take from it
In FastAPI, check what a blocking library call is sitting inside before you worry about it. In a plain def endpoint it borrows a thread and the rest of the server carries on. In an async def endpoint it holds the whole server until it finishes. If you add async to an endpoint, every call inside it has to become async too, or you have made it worse.
Know the number forty. If your plain def endpoints can be slow, forty of them in flight at once will make even trivial def endpoints queue, and raising the anyio limit is a one-line change.
Treat CPU work as blocking, because it is. Serialising a large response blocked the loop as surely as a network call did, asyncio.to_thread barely helped under the GIL, and a process pool fixed it completely.
And when something is blocking and you cannot see what, asyncio's debug mode will confirm the stall but not its location. A watchdog that samples the stack while the loop is stuck will give you the file and the line.
Top comments (1)
The forty-thread cliff makes me wonder what counts against it. FastAPI's docs say plain
defdependencies also run in the external threadpool, so a def endpoint with a couple of def dependencies might take several tokens over its lifetime, even if only one at a time. Did you measure endpoints with sync dependencies, or only the endpoint body? If dependencies count, the cliff could show up well below forty concurrent requests on real apps, and it would be worth a row in that table.