The "Silent Death" Scenario
You deploy your FastAPI application. For the first few hours, it feels blazing fast. Your P99 latency is sitting comfortably at 20ms. You high-five your team and go to sleep.
The next morning, your alerts are screaming.
Latency: Spiking to 3,000ms+ randomly.
Health Checks: Failing intermittently.
The Weird Part: You check your dashboards, and CPU usage is at 5%. Memory is flat. The database is responding instantly.
If your server isn't out of CPU and isn't out of memory, why is it crawling?
The answer lies in The Event Loop. When CPU is low but latency is high, it usually means your application isn't working, it is waiting.
This guide focuses on real-world scenarios where FastAPI apps slow down over time despite low CPU usage, a pattern that only appears under sustained load.
TL;DR: If your FastAPI app has low CPU but high latency, your event loop is blocked. The fix is not scaling CPUs, it’s fixing lifecycle, async misuse, and resource ownership.
Understanding the Single-Threaded Model
To fix the problem, you must first understand the machine.
FastAPI (running on Uvicorn) relies on Python's asyncio event loop. Think of the Event Loop as a single waiter in a busy restaurant.
Async: When a customer orders food (an API request), the waiter sends the order to the kitchen (Database/External API) and immediately goes to serve the next table. He doesn't wait in the kitchen. This is why FastAPI can handle thousands of requests with one thread.
Sync (Blocking): If the waiter decides to cook the food himself (Heavy Calculation) or stand in the kitchen staring at the chef (Synchronous I/O), he ignores all other tables.
If that waiter stops for just 1 second to chop an onion, every other customer waits 1 second longer. In a high-throughput system, these delays stack up exponentially.
**Mental Model**
Think of the event loop as a single-lane highway:
- Async = cars exit quickly, highway stays free
- Blocking = one broken car stops everything behind it
Killer #1: The "Sync" Blocker (and the def Trap)
This is the most common mistake developers make when migrating from Flask or Django.
In FastAPI, the moment you define an endpoint with async def, you are promising FastAPI: "I will not block the main thread." FastAPI trusts you and runs that code directly on the main event loop.
If you put anything blocking inside an async def, you freeze the entire server.
The Mistake: Synchronous I/O in Async Path
Python
import time
import requests # <--- The silent killer
@app.get("/external-data")
async def get_data():
# BLOCKING!
# This freezes the entire server. No other request can enter.
response = requests.get("https://example.com/huge-data")
time.sleep(1) # Simulating processing
return response.json()
If 50 users hit this endpoint simultaneously, the 50th user doesn't wait 1 second. They wait 50 seconds (1s x 50). The CPU usage remains low because the CPU isn't computing - it's just idle, waiting for the network response.
The Nuance: def vs async def
FastAPI has a safety valve. If you define your endpoint with just def (no async), FastAPI assumes it might block and runs it in a separate thread pool (Starlette's run_in_threadpool).
Python
@app.get("/safe-sync")
def safe_sync():
# Safe! FastAPI runs this in a thread pool.
time.sleep(1)
return {"status": "ok"}
However, you lose the benefits of async concurrency. The best solution is to keep async def but offload the blocking work.
💡
Checkout that you are using your wokers wisely, so it doesn’t slow down your requests.
The Fix: asyncio.to_thread
If you must use a blocking library (like requests or pandas) or perform a heavy calculation (like image processing), run it in a non-blocking way.
Python
import asyncio
import time
@app.get("/heavy-async")
async def heavy_async():
# ✅ Runs in a separate thread, leaving the Event Loop free
await asyncio.to_thread(time.sleep, 1)
return {"status": "done"}
Red Flags That Your Event Loop Is Blocked
If you see any of these, suspect blocking immediately:
Health checks timing out while business endpoints are slow
Latency spikes without CPU spikes
Restarting pods “fixes” performance temporarily
Throughput collapses before memory or CPU exhaustion
Killer #2: The Hidden CPU Crushers (Password Hashing)
Sometimes, the blocker isn't I/O; it's pure CPU work.
A classic example is Password Hashing (bcrypt or argon2). Hashing a password is designed to be slow and CPU-intensive to thwart hackers.
The Mistake:
Python
from passlib.context import CryptContext
pwd_context = CryptContext(schemes=["bcrypt"], deprecated="auto")
@app.post("/login")
async def login(user: UserLogin):
# BLOCKING!
# Verifying a hash takes ~300ms of pure CPU time.
if not pwd_context.verify(user.password, user.hashed_password):
raise HTTPException(...)
return {"token": "..."}
If 10 users try to login at once, your event loop locks up for 3 seconds (300ms * 10). Your health check endpoint (which should take 1ms) will time out, causing Kubernetes to kill your pod.
Password hashing belongs to the request lifecycle, but not to the event loop.
The Fix: Even though this is CPU-bound, not I/O-bound, you must still move it off the main thread.
Python
@app.post("/login")
async def login(user: UserLogin):
# Offload CPU work to a thread
is_valid = await asyncio.to_thread(
pwd_context.verify, user.password, user.hashed_password
)
if not is_valid:
raise HTTPException(...)
Killer #3: The httpx Client Leak
If your API acts as a gateway or aggregator calling other services, you are likely using httpx or aiohttp. A common mistake is treating the async client like a standard variable.
HTTP clients should be treated like database engines: created once, owned by the application lifecycle, and cleaned up deterministically.
The Mistake: Creating a Client per Request
Python
@app.get("/proxy")
async def proxy_request():
# Creates a new Connection Pool + SSL Context every time!
async with httpx.AsyncClient() as client:
resp = await client.get("https://example.com")
return resp.json()
Why this kills performance:
TCP Handshake: Every request performs a 3-way handshake.
SSL Handshake: Expensive crypto negotiation happens every time.
File Descriptors: You will eventually run out of sockets (
OSError: [Errno 24] Too many open files).
The Fix: The Lifespan Pattern Create one client at startup and reuse it. The connection pool stays open, making subsequent requests lightning fast.
Python
from contextlib import asynccontextmanager
@asynccontextmanager
async def lifespan(app: FastAPI):
# Initialize one client for the whole app
app.state.http_client = httpx.AsyncClient()
yield
# Cleanup on shutdown
await app.state.http_client.aclose()
app = FastAPI(lifespan=lifespan)
@app.get("/proxy")
async def proxy_request(request: Request):
client = request.state.http_client
resp = await client.get("https://example.com")
return resp.json()
Killer #4: The "Zombie" Background Task
FastAPI provides BackgroundTasks, which are great. But often, developers want "fire and forget" logic using standard AsyncIO.
The Mistake:
Python
@app.post("/process")
async def process():
# DANGER!
# The Python Garbage Collector might destroy this task
# immediately because no variable references it.
asyncio.create_task(long_running_job())
return {"status": "started"}
Not only can this task fail silently, but if it hangs, you have no way to cancel it or monitor it.
The Fix: The Set-Reference Pattern You must maintain a "Hard Reference" to your running tasks.
Python
# A global set to hold references
active_tasks = set()
async def safe_process():
task = asyncio.create_task(long_running_job())
# 1. Add to set (keep it alive)
active_tasks.add(task)
# 2. Remove from set when done (prevent memory leak)
task.add_done_callback(active_tasks.discard)
Why Adding More CPU or Pods Won’t Save You
Auto-scaling hides event loop bugs, it doesn’t fix them.
When the loop is blocked:
Each new pod reproduces the same problem
Load balancers distribute latency, not relief
Costs go up while performance stays bad
Until the event loop is unblocked, scaling is cosmetic.
How to Debug: Finding the Invisible Wall
You can't see these issues with standard access logs because the logs only show total time. They don't show waiting time.
You need a sampling profiler. The best tool for Python is py-spy. It reads memory directly, so it has almost zero overhead and works on production servers.
1. Install py-spy:
Bash
pip install py-spy
2. Peek at the running process: Find your uvicorn Process ID (PID) and run:
Bash
# --rate 100: samples 100 times per second
# --subprocesses: checks worker processes too
sudo py-spy top --pid <PID> --rate 100 --subprocesses
3. Interpret the Output:
If you see
time.sleep,select, orrequestsat the top of the list: You are blocking the loop.If you see
_run_until_completedominating without specific function calls: You might be starving the loop with too many tasks.
Conclusion
Low CPU + High Latency is almost always a symptom of a Blocked Event Loop.
FastAPI apps don’t decay randomly.
They slow down because something is blocking the event loop.
Once you understand resource ownership, what runs on the loop, what must not, and what belongs in the application lifespan performance issues stop being mysterious.
If you’re also seeing database pool exhaustion or connection timeouts, read my guide on Fixing QueuePool Overflow Errors in FastAPI, where lifecycle bugs surface again in a different form.
To keep your FastAPI app responsive:
Audit your
async defendpoints: Ensure norequests,time.sleep, or heavy math exists there.Use
asyncio.to_thread: For anything blocking or CPU-intensive (like password hashing).Reuse HTTP Clients: Treat
httpx.AsyncClientlike a database connection - create it once, use it everywhere.
If you are struggling with database connection pool limits (another common cause of latency), read my guide on Fixing QueuePool Overflow errors.
Top comments (0)