DEV Community

Cover image for What the FastAPI Docs Don't Tell You About Production
Madhav Bhasin
Madhav Bhasin

Posted on • Originally published at blog.madhav.dev

What the FastAPI Docs Don't Tell You About Production

#fastapi #backend #systemdesign #gcp

The service had been live for weeks. Locally, staging, early production — fast, clean, no issues. Then a new client onboarded, traffic doubled overnight, and P99 latency climbed from under 100ms to over 4 seconds.

No 5xx spike. No memory pressure. No CPU ceiling. Just slow.

Two things were wrong. Both came directly from patterns copied out of the FastAPI docs.

Problem 1: A New DB Connection on Every Request

The FastAPI quickstart doesn't show you how to manage shared resources. So most services end up doing this:

# ❌ What most people write first
@app.get("/orders/{order_id}")
async def get_order(order_id: str):
    conn = await asyncpg.connect(DATABASE_URL)  # new connection every request
    order = await conn.fetchrow("SELECT * FROM orders WHERE id = $1", order_id)
    await conn.close()
    return order
Enter fullscreen mode Exit fullscreen mode

Fine locally. Under 50 concurrent requests on Cloud Run hitting Cloud SQL, this means 50 simultaneous TCP handshakes + authentication attempts. PostgreSQL has a finite connection limit. Requests queue waiting for a slot. Latency compounds.

The fix is a connection pool initialised once at startup via FastAPI's lifespan hook — not per-request, not as a module-level global:

# ✅ The right way
from contextlib import asynccontextmanager
from fastapi import FastAPI
import asyncpg

@asynccontextmanager
async def lifespan(app: FastAPI):
    app.state.db = await asyncpg.create_pool(DATABASE_URL, min_size=2, max_size=10)
    yield
    await app.state.db.close()

app = FastAPI(lifespan=lifespan)

@app.get("/orders/{order_id}")
async def get_order(order_id: str, request: Request):
    async with request.app.state.db.acquire() as conn:
        return await conn.fetchrow("SELECT * FROM orders WHERE id = $1", order_id)
Enter fullscreen mode Exit fullscreen mode

Same pattern applies to Redis clients, HTTP sessions, and any SDK that's expensive to initialise. If it's shared, it belongs in lifespan.

Problem 2: Stack Traces Leaking to Clients

While diagnosing the latency issue, we found something else in Cloud Logging. Clients were receiving this:

{
  "detail": "500: Internal Server Error\nTraceback (most recent call last):\n  File \"/app/routers/orders.py\", line 34...\nasyncpg.exceptions.TooManyConnectionsError: ..."
}
Enter fullscreen mode Exit fullscreen mode

Internal file paths. Dependency names. Exception types. Leaking silently for weeks.

FastAPI's default behaviour on an unhandled exception exposes more than you want in production. The fix is a single catch-all exception handler:

# ✅ Global exception handler
import logging
from fastapi import Request
from fastapi.responses import JSONResponse

logger = logging.getLogger(__name__)

@app.exception_handler(Exception)
async def unhandled_exception_handler(request: Request, exc: Exception):
    logger.exception(
        "Unhandled exception",
        extra={"path": request.url.path, "method": request.method}
    )
    return JSONResponse(
        status_code=500,
        content={"error": "internal_server_error", "message": "Something went wrong."}
    )
Enter fullscreen mode Exit fullscreen mode

Full traceback goes to Cloud Logging — where only your team sees it. Clients get a safe, consistent error shape. One file, set it once, never think about it again.

Three More Things the Docs Skip

1. Structured logging

Default uvicorn logs are human-readable text. Cloud Logging expects JSON. Replace the default logger before your first deploy:

import logging, json, sys

class JSONFormatter(logging.Formatter):
    def format(self, record):
        return json.dumps({
            "severity": record.levelname,
            "message": record.getMessage(),
            "logger": record.name,
        })

handler = logging.StreamHandler(sys.stdout)
handler.setFormatter(JSONFormatter())
logging.root.handlers = [handler]
logging.root.setLevel(logging.INFO)
Enter fullscreen mode Exit fullscreen mode

2. Uvicorn config for Cloud Run

Don't copy --reload from the quickstart into your Dockerfile. For Cloud Run:

CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8080", "--workers", "1", "--loop", "uvloop", "--timeout-keep-alive", "30"]
Enter fullscreen mode Exit fullscreen mode

Single worker per instance (Cloud Run scales horizontally, not vertically), uvloop for async performance, no --reload.

3. A health check that actually checks something

@app.get("/health")
async def health(request: Request):
    try:
        async with request.app.state.db.acquire() as conn:
            await conn.fetchval("SELECT 1")
        return {"status": "ok"}
    except Exception:
        return JSONResponse(status_code=503, content={"status": "degraded"})
Enter fullscreen mode Exit fullscreen mode

A health endpoint that just returns {"status": "ok"} is a lie. Cloud Run routes traffic based on this endpoint — a lying health check sends requests to a broken instance.

The framework didn't fail. The configuration did.

Next: S01E02 — FastAPI + Pydantic in Production: Contracts, Validation, and Versioning.

Top comments (0)