One blocking call freezes every request: FastAPI's def vs async def

There's a FastAPI outage that starts with someone doing the right thing. The service has a pile of plain def endpoints written by whoever came before, somebody reads that FastAPI is an async framework and feels the codebase should act like one, and a cleanup PR turns them all into async def. Nothing breaks. Review and tests pass, latency in staging doesn't move, and the change ships as the kind of hygiene work everyone approves without much thought.

Then traffic climbs, and the incident that follows has a strange shape: p99 spikes on every endpoint at once. Not the heavy ones, all of them, including /health, which returns a constant and has no business being slow. CPU is low. The database is fine. Rolling back the hygiene PR fixes it, which is the most confusing possible outcome, because all that PR did was add the keyword that's supposed to make Python web services fast.

Two dispatch paths

FastAPI (really Starlette underneath) runs your endpoint in one of two places depending on how you declared it. An async def endpoint runs on the event loop, in the single thread where the entire async world lives. A plain def endpoint gets sent to a worker threadpool, capped at 40 threads by default, and awaited from the loop as if it were async. The framework documents this, but the docs read like a performance nuance, and it's actually a correctness contract: by writing async def, you're promising the scheduler that nothing between your awaits will block.

The event loop is cooperative scheduling with no preemption. One thread runs every coroutine in the process by hopping between them at await points, and that works only as long as everyone yields promptly. A coroutine that calls requests.get, or time.sleep, or a synchronous database driver, or a heavyweight ORM query doesn't yield, it holds the one thread hostage until the call returns. For those 800 milliseconds, nothing else in the process runs. No other request advances, no timer fires, and your /health endpoint, which needs a microsecond of loop time to respond, waits in line behind a network call to some third-party API, which is exactly the all-endpoints-at-once fingerprint from the incident.

The perverse gradient

Put the same blocking call in three different endpoint styles and rank the outcomes:

Endpoint	Inside it	Under load
def	requests.get(...)	occupies 1 of 40 pool threads, loop unaffected
async def	await client.get(...) (httpx)	loop suspends the coroutine, scales cleanly
async def	requests.get(...)	the whole process stalls for the duration of every call

The naive version, sync endpoint with a sync library, is fine. Not maximally efficient, but fine, because the threadpool absorbs the blocking and the loop never feels it, and 40 threads of cushion covers a lot of moderate traffic. The fully async version is what the framework wants from you. The hybrid, async syntax around a blocking call, is strictly the worst of the three, worse than the legacy code it replaced, because the async def keyword is precisely the thing that moved execution off the protected threadpool and onto the thread that can't afford to block. Adding async without changing what's inside isn't a lateral move, it's a downgrade wearing a modernization costume.

Node engineers have met the mirror image of this story, a hidden threadpool (libuv's) that quietly starves while the event loop looks healthy. Python inverts it: here the threadpool is the safety net, and the event loop is where you drown.

The fingerprint

You can recognize loop blocking from dashboards alone, because the correlation structure is wrong for any other cause. Every route degrades together regardless of what it does, the degradation tracks traffic on some specific route (the one with the blocking call), CPU stays low through all of it, and the floor of the latency distribution rises rather than the tail alone stretching. A slow dependency hurts the endpoints that use it; a blocked loop hurts everyone, trivial endpoints most visibly, since a constant-returning route with a 900ms p99 has no other possible explanation.

To confirm, catch the process in the act with py-spy:

py-spy dump --pid <uvicorn worker pid>

If the loop is blocked, the main thread's stack says so explicitly, something like socket.recv under urllib3 under requests.get under your endpoint function, sitting where an idle epoll wait should be. Asyncio will also confess if you ask it to: run with PYTHONASYNCIODEBUG=1 and the loop logs every callback that held it longer than 100ms ("Executing took 0.874 seconds"), which turns an invisible stall into a named function in your logs.

The fixes

The clean fix is to make the promise true end to end: async endpoints call async libraries. httpx instead of requests, asyncpg or an async SQLAlchemy engine instead of the sync session, motor rather than pymongo, and asyncio.sleep where time.sleep was. This is the version of the codebase the hygiene PR thought it was creating.

The honest fix, when the blocking dependency isn't going anywhere, is to stop claiming async: declare the endpoint as plain def and let the framework threadpool it, which is what it was silently doing for you before the cleanup. There's no shame in this. A def endpoint calling the sync SQLAlchemy session is a correct program, and in a service that's mostly CRUD against one database it performs perfectly well.

The surgical fix, for one blocking call inside an otherwise async endpoint, is to push just that call to the pool yourself:

from starlette.concurrency import run_in_threadpool

result = await run_in_threadpool(legacy_client.fetch, order_id)

Same escape hatch the framework uses, applied at call granularity. Two caveats travel with it. The pool is finite and shared, those 40 threads serve every def endpoint and every run_in_threadpool in the process, so a service that leans hard on it needs the cap raised deliberately rather than discovered as a ceiling. And CPU-bound work (bcrypt is the classic, the same villain as in Node) doesn't get better by moving to a thread, since the GIL keeps Python bytecode serialized; hashing belongs in a process pool or a worker service, not on either the loop or the threadpool.

async is a contract, not a speedup

The keyword doesn't make anything faster. It relocates your code onto a thread where blocking is forbidden, in exchange for concurrency that only materializes if the promise holds, and Python has no way to check the promise for you. The runtime can't tell a blocking call from a fast one at compile time, the type system doesn't track it, and the ecosystem is thirty years deep in excellent synchronous libraries that will accept an invitation into your event loop without a word of complaint. In JavaScript this contract is hard to violate because the platform never handed you blocking I/O in the first place, and that's the instinct transfer that hurts: engineers arriving from Node assume async code is safe by construction, when in Python it's safe by discipline.

So the rule for FastAPI specifically: async def only when everything inside is genuinely async, plain def whenever you're not sure, because the framework protects sync code and trusts async code, and you want to be on the protected side of that line until you've earned the trusted one.