QueuePool Limit Reached - When FastAPI Runs Out of Database Connections

Hello developers,

Today I would like to talk about an error message with a real talent for sending you in the wrong direction:

sqlalchemy.exc.TimeoutError: QueuePool limit of size 5 overflow 10 reached,
connection timed out, timeout 30.00 (Background on this error at: https://sqlalche.me/e/20/3o7r)

If your FastAPI app talks to a database through SQLAlchemy, chances are high you will meet it sooner or later. Not on your machine, of course. It shows up in production, after a few quiet hours, under load. And the reflex is always the same: the pool is too small, so let us make it bigger. pool_size=20, a bit more max_overflow on top, deploy, done.

It holds. For a while. Then the error is back - only now you burn a lot more database connections on the way to it.

Three posts helped me to put the pieces together, each one from a different angle:

1. Karani Geoffrey on why pool_size is not the fix,
2. Ayush Kaushik on connection leaks in FastAPI dependencies,
3. David Muraya on connection pooling for serverless FastAPI.

This post combines them - and breaks things on purpose, so we can watch the pool run dry instead of just believing it.

Read the error message literally

Every number in that message means something, and all three are SQLAlchemy's defaults:

In the message	Setting	Meaning
size 5	pool_size	connections the pool keeps open permanently
overflow 10	max_overflow	extra connections it may open temporarily on top
timeout 30.00	pool_timeout	seconds a request waits for a free connection, then fails

So the message does not say "you need more connections". It says: all 15 connections were busy for 30 seconds straight, and nobody gave one back. Karani puts it into one sentence:

QueuePool limit reached is a statement about time, not quantity.
-- Karani Geoffrey

Thus the real question is not how many connections you have, but who is holding them, and for how long.

Breaking it on purpose

A pool with 15 connections and 30 seconds of patience can hide a problem for a very long time. So for all experiments below I shrank it to the bare minimum:

engine = create_engine(
    "sqlite:///demo.db",
    pool_size=2,     # two connections ...
    max_overflow=0,  # ... and not a single one more
    pool_timeout=1,  # give up after one second instead of 30
)

Two connections, one second of patience: every bug that holds on to a connection shows up immediately. Everything ran on FastAPI 0.142 and SQLAlchemy 2.1.

And you do not have to take my word for it: all experiments are on GitHub at andi1984/fastapi-queuepool-demo, one script per culprit. They run against SQLite, so there is not even a database to set up. Clone it and break things yourself!

Culprit #1: the leaky dependency

Ayush calls it the most common cause in FastAPI apps, and it looks so innocent:

# 🚨 Do not do this
def get_db():
    db = SessionLocal()
    return db

FastAPI happily injects that session, your endpoint uses it, the response goes out - and nobody ever calls db.close(). The connection stays checked out. With the garbage collector switched off for a moment, to make the leak deterministic, the tiny pool is empty two requests later:

/leaky 200 | Pool size: 2  Connections in pool: 0 Current Overflow: -1 Current Checked out connections: 1
/leaky 200 | Pool size: 2  Connections in pool: 0 Current Overflow: 0 Current Checked out connections: 2
/leaky sqlalchemy.exc.TimeoutError: QueuePool limit of size 2 overflow 0 reached, connection timed out, timeout 1.00

Now the interesting part. The abandoned session gets cleaned up eventually by Python's garbage collector, which hands the connection back to the pool. Eventually. When I fired 40 requests at the leaky endpoint with the garbage collector doing its normal thing, 27 of them failed - and the 13 that passed came in little bursts, whenever the collector happened to run. Both rounds are in leaky_dependency.py if you want to watch it yourself.

That is exactly what makes this bug so nasty: it is flaky. It works, then it does not, then it works again. Response times creep up, the number of open database connections only ever goes up, CPU looks perfectly fine. And a restart "fixes" it - for a few hours.

The fix is a dependency with yield, so FastAPI knows when to clean up:

def get_db():
    db = SessionLocal()
    try:
        yield db
    finally:
        db.close()  # runs after the request, even if the endpoint raised

Or shorter, as the session is a context manager anyway:

def get_db():
    with SessionLocal() as db:
        yield db

Same tiny pool, as many requests as you like: Current Checked out connections: 0 after every single one.

Culprit #2: holding the connection while waiting for someone else

This is Karani's main point, and it is the one that bites async apps. Imagine an endpoint that loads an order, charges the card at a payment provider and stores the result:

@app.post("/orders/{order_id}/pay")
async def pay(order_id: int, session: Annotated[AsyncSession, Depends(get_session)]):
    order = await session.get(Order, order_id)
    result = await charge_card(order)  # ⏳ slow HTTP call - connection still checked out
    order.status = "paid" if result.ok else "failed"
    await session.commit()

Nothing leaks here. Every connection makes it back to the pool. But for the whole duration of charge_card(), a database connection sits there checked out and doing nothing. Your database is bored, your pool is starving.

In my demo (held_across_await.py) charge_card() is a two-second asyncio.sleep(). Three concurrent requests against the two-connection pool:

/held: ['200', '200', 'TimeoutError: QueuePool limit of size 2 overflow 0 reached, ...']

The fix: give the connection back before you wait for anything that is not your database.

@app.post("/orders/{order_id}/pay")
async def pay(order_id: int):
    async with SessionLocal() as session:
        order = await session.get(Order, order_id)

    result = await charge_card(order)  # no connection held while we wait

    async with SessionLocal() as session:
        order = await session.get(Order, order_id)
        order.status = "paid" if result.ok else "failed"
        await session.commit()

Same pool of two connections, now ten concurrent requests:

/released: ['200', '200', '200', '200', '200', '200', '200', '200', '200', '200']  (2.0s)

Ten requests, two connections, all of them done within the two seconds the "payment provider" needs. The pool did not change at all - only how long each request holds on to a connection.

Yes, that is two short checkouts instead of one long one. That is the whole point. Opening a session is cheap, it borrows an already open connection from the pool.

A small disagreement

Interestingly, Ayush lists "a session per query" as an anti-pattern and recommends exactly one session per request. IMHO both are right, just at different ends. One session per request is a great default. But the thing to scope is the database work, not the HTTP request. As soon as a request waits for something else - another service, a file upload, an LLM call - the session should not wait with it.

Let us be honest about the trade-off though: two sessions mean two transactions. The order can change between them, so load it again in the second session instead of trusting what you read before the gap.

Culprit #3: a sync engine in an async world

If your endpoints are plain def functions, FastAPI runs them in a threadpool - 40 threads by default. Every request then needs a thread and a connection, and those two limits start fighting each other. Tune one and the bottleneck moves to the other.

FastAPI itself carries a little scar from exactly this fight. The cleanup code of sync yield dependencies runs outside the normal thread limit, and the comment explains why:

blocking __exit__ from running waiting on a free thread can create race conditions/deadlocks if the context manager itself has its own internal pool (e.g. a database connection pool)
-- fastapi/concurrency.py

If you are async in the web layer, be async in the database layer too:

from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine

engine = create_async_engine("postgresql+asyncpg://user:[e-mail]/app")
SessionLocal = async_sessionmaker(engine, expire_on_commit=False)


async def get_session():
    async with SessionLocal() as session:
        yield session

One gotcha on SQLAlchemy 2.1: install sqlalchemy[asyncio], not just sqlalchemy. The plain package does not pull in greenlet anymore, and the async engine refuses to import without it.

Culprit #4: the dependency lives longer than you think

This one is not in any of the three posts, but it fits right in. By default, a dependency with yield cleans up after the response has been sent. For a small JSON response that is a blink. For a StreamingResponse it is the whole stream:

@app.get("/export")
async def export(session: Annotated[AsyncSession, Depends(get_session)]):
    orders = (await session.scalars(select(Order))).all()
    return StreamingResponse(to_csv(orders))  # connection held until the last byte

A slow client downloading a big CSV keeps a database connection busy without touching the database at all. Since FastAPI 0.121.0 you can let a dependency end together with the endpoint function instead of the response:

session: Annotated[AsyncSession, Depends(get_session, scope="function")]

Measured from inside the stream (streaming_scope.py):

scope="request"  -> chunk 0 | checked out: 1 || chunk 1 | checked out: 1
scope="function" -> chunk 0 | checked out: 0 || chunk 1 | checked out: 0

The catch: with scope="function" the session is closed before the first byte goes out. Load everything you need upfront, and do not lazy-load relationships inside the generator.

Only now: the pool size

Once connections come back quickly, the pool size finally becomes what it should have been all along: a capacity decision. And for that you need to do some math, because every worker process has its own pool. A not at all exotic setup:

  4 Uvicorn workers per instance
× 20 connections per worker (pool_size=10 + max_overflow=10)
× 3 instances (pods, containers, ...)
= 240 connections in the worst case

PostgreSQL ships with max_connections = 100, and a few of those are reserved for superusers. So at peak, this setup will not see a QueuePool error - it will see Postgres itself saying no:

FATAL: sorry, too many clients already

Bumping the pool size as a first reaction does not fix anything. It only moves the wall closer to your database.

Serverless: let someone else do the pooling

David looks at the problem from the other side. On platforms like Cloud Run, instances come and go, and serverless databases like Neon even scale down to zero when idle. Long-lived pooled connections and that world do not get along well. He describes two options:

1. Keep the QueuePool and add pool_pre_ping=True. SQLAlchemy checks every connection before handing it out and silently replaces dead ones. It costs one extra round trip per checkout, and you are still bound to the database's direct connection limit.
2. Use an external pooler and switch SQLAlchemy's pooling off. Neon, Supabase & Co. offer a pooled endpoint (usually PgBouncer), and you point your app at that one:

from sqlalchemy.pool import NullPool

engine = create_async_engine(
    "postgresql+asyncpg://user:[e-mail]/app",  # the *pooled* endpoint
    poolclass=NullPool,  # no pool in the app - the pooler does the job
)

Two pools stacked on top of each other just compete, so NullPool opens a connection when you need one and closes it right after. Against PgBouncer that is cheap, because the expensive part - Postgres starting a new backend process - does not happen.

No free lunch, as always:

• In transaction mode, every transaction may land on a different server connection. Session-level things like SET search_path do not survive, so run your migrations (e.g. Alembic) over the direct connection string.
• With asyncpg behind PgBouncer, prepared statement names can collide. The SQLAlchemy docs have a section on exactly that.

Your debugging toolbox

When the error shows up, these three things tell you more than any stack trace.

1. Ask the pool. engine.pool.status() gives you a one-line snapshot:

Pool size: 2  Connections in pool: 1 Current Overflow: -1 Current Checked out connections: 0

Log it under load. In a healthy app, "checked out" goes up during bursts and drops back between them. If it climbs and stays up, something is holding on.

2. Log long checkouts. SQLAlchemy's pool events tell you whenever a connection was gone for suspiciously long:

import logging
import time

from sqlalchemy import event

log = logging.getLogger("db.pool")


# for a sync engine, listen on `engine` directly
@event.listens_for(engine.sync_engine, "checkout")
def remember_checkout(dbapi_connection, connection_record, connection_proxy):
    connection_record.info["checked_out_at"] = time.monotonic()


@event.listens_for(engine.sync_engine, "checkin")
def warn_on_long_hold(dbapi_connection, connection_record):
    started = connection_record.info.pop("checked_out_at", None)
    if started is not None and (held := time.monotonic() - started) > 1.0:
        log.warning("connection was checked out for %.2fs", held)

Against the "payment provider" endpoint from culprit #2 (pool_events.py), it immediately points at the problem:

WARNING db.pool: connection was checked out for 2.01s

3. Ask Postgres. A leaked session that ran a single SELECT keeps its transaction open - Postgres calls that idle in transaction:

SELECT pid, application_name, now() - state_change AS stuck_for, left(query, 60) AS last_query
FROM pg_stat_activity
WHERE state = 'idle in transaction'
ORDER BY stuck_for DESC;

last_query usually points you straight to the code path. As a safety net, Postgres can kill such sessions for you via idle_in_transaction_session_timeout - but that is a seatbelt, not a fix.

And one thing not to do: retrying on a pool timeout. As Karani points out, the retry just joins the same queue and makes the contention worse.

TL;DR

1. Sessions come from a yield dependency (or an async with), never from a return.
2. Never hold a session across an await that is not your database.
3. Streaming responses? Use Depends(..., scope="function") and load your data upfront.
4. Async web layer, async database layer. create_async_engine instead of a sync engine in a threadpool.
5. Do the math: workers × instances × (pool_size + max_overflow) must fit into max_connections.
6. Serverless? External pooler plus NullPool.
7. A bigger pool comes last, never first.

All scripts from this post live in andi1984/fastapi-queuepool-demo under the MIT license. Feel free to fork it - and if you find yet another creative way to drain a pool, send a pull request!

A big thank you to Karani, Ayush and David for their posts - go read them, there is more in there than fits into this one.

May your pools never run dry - pun intended.

Cheers,

Andi 🏊