Connection Leak vs. Pool Exhaustion: How to Tell the Difference

Every connection is checked out, requests are queuing, and the pool is throwing timeouts. The next decision depends on one question: is something leaking connections, or is the pool being asked for more than it can give? Both produce the same error. They need opposite fixes.

Short version: pool exhaustion is a state (every pooled connection is checked out, new callers wait). A connection leak is one cause of it (code checks out a connection and never returns it). Watch in-use connections against traffic. A leak ratchets up and stays up. Load-driven exhaustion falls when traffic falls.

• Leak: in-use climbs with uptime, ignores quiet periods, resets on restart.
• Load or slow work: in-use tracks request rate and query time, and recovers.
• Confirm at the database: leaked connections usually sit idle while the pool counts them as busy.
• Fix a leak in code (release on every path), not with a bigger pool.

The mechanism is the same for HikariCP on the JVM, pg.Pool in Node.js, SQLAlchemy's QueuePool in Python, and Go's database/sql. Where the tools differ, there's a per-stack table below. For a JVM-only walkthrough of HikariCP's exception, leak detection and thread dumps, see HikariCP "Connection Is Not Available, Request Timed Out".

Leak vs. exhaustion: the definitions

Pool exhaustion is the condition where every connection a pool may hold is checked out, so the next caller waits and eventually times out. It's a symptom.

A connection leak is a specific bug: a code path checks out a connection and never returns it, so usable capacity shrinks by one each time that path runs. A leak always ends in exhaustion if the process lives long enough. Exhaustion does not imply a leak.

The other causes of exhaustion return every connection eventually, just not fast enough:

• Load: request rate × normal hold time exceeds pool size at peak.
• Slow queries: a missing index, a lock wait, or a bigger table stretches each hold.
• Long holds in application code: a transaction that spans an HTTP call, a queue publish, or slow processing.
• Nested acquisition: a thread holding one connection waits for a second, which can deadlock the pool at modest concurrency.
• Amplified demand: client retries add requests exactly when each request is slowest.

Search Stack Overflow for pool exhaustion and most threads end with "increase the pool size" or "you have a leak", usually without evidence for either. The useful question is "leak, or one of the causes that recover on their own?" A leak needs a code change on a specific path. The others need faster work, shorter holds, less demand, or (last) more connections.

Property	Connection leak	Exhaustion without a leak
Every connection eventually returned?	No. Stays checked out until the process exits	Yes, after the query, transaction, or remote call finishes
In-use when traffic drops	Stays at its new high	Falls back to a low baseline
Correlates with	Uptime, and the rate of one code path (often an error path)	Request rate, query latency, dependency latency
Effect of a restart	Fixes it completely, until the leak catches up	Little or none if the load or slowness persists
Effect of a bigger pool	Delays the outage	Helps only if the pool is truly undersized

What they look like over a day

At the moment of the timeout both look identical: in-use equals max, idle is zero, callers are waiting. The difference only shows up as a time series. Put in-use connections and request rate on the same chart over at least one daily cycle.

CONNECTION LEAK (pool max = 10)
in-use
 10 |              ┌─────────────┐
    |         ┌────┘   traffic   │      steps up, never steps down,
  5 |    ┌────┘      falls, but  │      even while traffic falls
    |┌───┘           in-use stays│  ┌───
  0 +────────────────────────────┴──┘───   ← restart resets it
     00:00   06:00   12:00   18:00   24:00

EXHAUSTION FROM LOAD (no leak, pool max = 10)
in-use
 10 |              ┌───┐   ← pool full at peak, callers wait
    |           ┌──┘   └──┐
  5 |        ┌──┘         └──┐
    |    ┌───┘               └───┐
  0 +────┘                       └──────   recovers as traffic falls
     00:00   06:00   12:00   18:00   24:00

Three details carry most of the diagnosis:

• Does in-use come down in the quiet hours? If traffic drops 80% overnight and in-use doesn't move, those connections aren't doing work for anyone.
• What does a restart do? Instant drop to baseline followed by a slow climb is the classic leak signature. If the pool is full again within minutes of a restart, look at load or slow work first.
• Where are the steps? Leaks step up when the leaking path runs. If steps line up with a batch job, a retry burst, or error spikes on one endpoint, you have a lead on which path it is.

Autoscaling and deploys hide leaks. If instances get replaced often, a slow leak may never reach the pool limit. It shows up when deploys pause over a holiday, or when a failing dependency makes the leaking path run far more often. Chart in-use per instance against instance uptime, not only the fleet total, or the leak hides inside the average.

Exhaustion in Java, Node.js, Python, and Go

Every pool has a max size, a way to wait for a free connection, and some way to report how many are checked out. The defaults differ in ways that change what the incident looks like. Two of the four common stacks don't fail fast by default.

Stack	Pool size (default)	Wait limit (default)
JDBC · HikariCP	maximumPoolSize (10)	connectionTimeout (30 s)
Node.js · pg.Pool	max (10)	connectionTimeoutMillis (0 = wait forever)
Python · SQLAlchemy QueuePool	pool_size (5) + max_overflow (10)	pool_timeout (30 s)
Go · database/sql	SetMaxOpenConns (0 = unlimited)	None, bounded only by the caller's context

Stack	What exhaustion looks like	Where to read in-use
JDBC · HikariCP	SQLTransientConnectionException: "Connection is not available, request timed out"	Pool stats in the message, hikaricp.connections.active, leakDetectionThreshold for borrower traces
Node.js · pg.Pool	Requests hang. With a timeout set: "timeout exceeded when trying to connect"	pool.totalCount, pool.idleCount, pool.waitingCount
Python · SQLAlchemy	TimeoutError: "QueuePool limit of size 5 overflow 10 reached, connection timed out"	engine.pool.status(), checkedout(), checkout/checkin pool events
Go · database/sql	Goroutines block, then context deadline exceeded. If unlimited, the database refuses new connections instead	db.Stats(): InUse, Idle, WaitCount, WaitDuration

Defaults as documented by HikariCP, node-postgres, SQLAlchemy, and Go. Frameworks often override them, so read the effective values from your running config.

node-postgres waits forever by default

With connectionTimeoutMillis at 0, a caller that can't get a client waits with no limit. An exhausted pool doesn't throw. Requests just stop completing, which usually surfaces as a load balancer or gateway timeout far from the real cause. Set a timeout so the failure is visible, and export the counters the pool already has:

const pool = new Pool({
  max: 10,
  connectionTimeoutMillis: 5_000,   // default 0 means wait forever
  idleTimeoutMillis: 10_000,
});

setInterval(() => {
  metrics.gauge('db.pool.total', pool.totalCount);
  metrics.gauge('db.pool.idle', pool.idleCount);
  metrics.gauge('db.pool.waiting', pool.waitingCount);
  metrics.gauge('db.pool.in_use', pool.totalCount - pool.idleCount);
}, 10_000).unref();

Go's database/sql is unlimited by default

With SetMaxOpenConns at 0, a Go service never exhausts its own pool. A leak opens new connections until the database refuses them (PostgreSQL's "sorry, too many clients already", MySQL's "Too many connections"). That error can hit every other service sharing the database, so one service's leak looks like a platform-wide outage. Set a limit and log the stats:

db.SetMaxOpenConns(20)             // default 0 = unlimited
db.SetMaxIdleConns(20)             // default 2
db.SetConnMaxLifetime(30 * time.Minute)

go func() {
    t := time.NewTicker(15 * time.Second)
    defer t.Stop()
    for range t.C {
        s := db.Stats()
        log.Printf("db pool open=%d inuse=%d idle=%d waits=%d waited=%s",
            s.OpenConnections, s.InUse, s.Idle, s.WaitCount, s.WaitDuration)
    }
}()

On any stack the four numbers that matter are max, in use, idle, and waiting (or cumulative wait count and time). If you add one metric after reading this, make it in-use as a time series.

Confirm a leak from the database side

The pool knows which connections are checked out. The database knows what each of them is actually doing. Comparing the two is the fastest confirmation, and it works on any stack.

SELECT application_name, state, count(*) AS sessions,
       max(now() - state_change) AS longest_in_state
FROM   pg_stat_activity
WHERE  datname = current_database()
  AND  backend_type = 'client backend'
GROUP BY 1, 2
ORDER BY 3 DESC;

Column meanings are in the PostgreSQL cumulative statistics docs. Set application_name in each service's connection string so you can filter to one service's pool, then line the result up against the pool's counters at the same moment.

Example with a pool of 10 (illustrative numbers):

View	What it reports
Pool	10 in use, 0 idle
Database	2 active, 1 idle in transaction, 7 idle

The 2 active sessions are running queries right now. The 1 idle-in-transaction session has an open transaction and no query, so check its age. The 7 idle sessions the pool counts as busy were borrowed and never given back. Those are your leak candidates.

Pool says	Database says	Most likely cause
All in use	Most sessions idle, state_change minutes or hours old	Leak with autocommit: connections borrowed and abandoned
All in use	Many idle in transaction, old xact_start	Leak mid-transaction, or a long hold around non-database work. Age tells them apart
All in use	Most sessions active, long query_start age	Slow queries or a lock chain. Check wait_event_type
All in use	Most sessions active, queries short	Load: throughput × hold time exceeds the pool
Below max, callers waiting	Fewer sessions than expected, connection errors logged	Creation failure, e.g. the database's connection limit reached by other clients

MySQL shows the same picture in SHOW FULL PROCESSLIST (Sleep with a large Time) and information_schema.innodb_trx.

A database timeout protects the database, not the pool. PostgreSQL's idle_in_transaction_session_timeout ends sessions that sit in an open transaction too long and releases their locks. Good guardrail. It does not return the connection to your pool: the pool still counts it as checked out until application code releases it, and the next use of that connection fails. Damage control, not a leak fix.

Why leaks show up during a different incident

Most real leaks aren't on the happy path. The happy path runs in every test and every request, so a missing release there gets found within hours. Leaks survive on error and early-return paths: a row fails to parse, a downstream call throws, validation fails, a request is cancelled. So a leak drains the pool at the rate its path runs.

Time to drain = pool size ÷ leak rate (illustrative numbers):

Situation	Error path rate	Pool of 20 drains in
Normal day	~2 per hour	~10 hours (daily deploys restart it first, so nobody notices)
Dependency failing	~5 per second	~4 seconds

This is why "the pool ran out during the payment provider outage" is often a misdiagnosis. The outage was real. The exhaustion was a second, pre-existing bug the outage exposed. The tell is what happens after the dependency recovers: exhaustion from load or slowness clears within seconds, while a leak leaves the pool full until someone restarts the service.

The same logic covers leaks that appear "right after a deploy". The deploy may not have added the leak. It may have added a new failure mode, stricter validation, or a new timeout that sends more traffic down an old error branch. When you're hunting for the commit behind an incident, include changes that increase the rate of existing error paths, not only changes to data access code.

Code patterns that leak connections

Same across languages: acquisition without a guaranteed release, plus resources that pin a connection (result sets, cursors, streams, transactions) not closed on every path.

BROKEN
acquire() ──► query + map rows ──► release() ──► return
                    │
                    └── throws ──► error propagates, release() skipped
                                   (connection stays checked out forever)

FIXED
acquire() ──► try: query + map ──► finally: release() ──► exit
                    │                     ▲
                    └── throws ───────────┘
              (success and error both pass through finally)

The leak is a missing edge in the control flow. Structured release (try/finally, try-with-resources, with, defer) adds it for every path at once.

Node.js: client.release() skipped when a query throws

Leaks when a query throws:

async function transfer(from, to, amount) {
  const client = await pool.connect();
  await client.query('BEGIN');
  await client.query(DEBIT_SQL, [from, amount]);   // may throw
  await client.query(CREDIT_SQL, [to, amount]);
  await client.query('COMMIT');
  client.release();                                  // never reached on error
}

Released on every path:

async function transfer(from, to, amount) {
  const client = await pool.connect();
  let broken;
  try {
    await client.query('BEGIN');
    await client.query(DEBIT_SQL, [from, amount]);
    await client.query(CREDIT_SQL, [to, amount]);
    await client.query('COMMIT');
  } catch (err) {
    await client.query('ROLLBACK').catch((e) => { broken = e; });
    throw err;
  } finally {
    client.release(broken);   // always runs; truthy arg destroys it
  }
}

For single statements, pool.query() checks out and releases a client for you, so it can't leak. The risk lives in code that calls pool.connect() directly, usually for transactions.

Go: rows or tx left open on an early return

Leaks on a Scan error:

rows, err := db.QueryContext(ctx, listItemsSQL, orderID)
if err != nil {
    return nil, err
}
for rows.Next() {
    var it Item
    if err := rows.Scan(&it.ID, &it.SKU, &it.Qty); err != nil {
        return nil, err   // rows still open: its connection stays in use
    }
    items = append(items, it)
}
return items, rows.Err()

Closed on every path:

rows, err := db.QueryContext(ctx, listItemsSQL, orderID)
if err != nil {
    return nil, err
}
defer rows.Close()   // safe to call even after Next has returned false

tx, err := db.BeginTx(ctx, nil)
if err != nil {
    return err
}
defer tx.Rollback()  // no-op after a successful Commit

When rows.Next() returns false the rows close automatically, which is why the happy path in the broken example doesn't leak. Only the early return does, and only when a Scan fails. Exactly the kind of path tests rarely hit.

Python: a Session not closed on an exception

# Broken: close() is skipped if anything between here and there raises
def load_invoice(invoice_id):
    session = SessionLocal()
    invoice = session.get(Invoice, invoice_id)
    render_pdf(invoice)                 # raises: session keeps its connection
    session.close()

# Fixed: the context manager closes the Session on every path
def load_invoice(invoice_id):
    with SessionLocal() as session:
        invoice = session.get(Invoice, invoice_id)
        render_pdf(invoice)             # connection returned when the block exits

In CPython, reference counting sometimes returns that connection when the abandoned object is garbage collected, and SQLAlchemy may log a warning when it cleans up a connection that was never checked in. That makes Python leaks intermittent rather than absent. Don't rely on it. FastAPI dependencies with yield, or scoped_session.remove() at request end, exist to make release structural.

Other patterns that pin a connection

• Streams and cursors returned to the caller: a repository method that returns a lazy stream or server-side cursor keeps its connection until the caller closes it.
• Transactions left open by an early return between BEGIN and COMMIT, which also hold locks.
• Connections stored in fields or passed to another thread, where ownership of the release is unclear.
• Background jobs that open a connection outside the request lifecycle, so request-scoped cleanup never runs.

Track the borrower when your pool won't

HikariCP has leakDetectionThreshold. node-postgres has no built-in equivalent, but you can add one in a few lines by recording where each client was checked out:

const held = new Map();

async function connect() {
  const client = await pool.connect();
  held.set(client, { at: Date.now(), stack: new Error('checked out here').stack });
  const release = client.release;
  client.release = (err) => { held.delete(client); return release(err); };
  return client;
}

setInterval(() => {
  const now = Date.now();
  for (const { at, stack } of held.values()) {
    if (now - at > 30_000) console.warn(`db client held ${now - at} ms`, stack);
  }
}, 10_000).unref();

A warning means "held a long time", not "leaked". A warning for a client that's later released is a long hold. A warning that repeats from the same stack and never clears is a leak. In SQLAlchemy, the pool's checkout and checkin events give you the same hook.

Step by step: leak or exhaustion?

1. Read the error and the pool counters. Record max, in use, idle, and waiting at the moment of failure, plus any chained cause. If total is below max while callers wait, the pool is failing to create connections, which is a separate problem.
2. Chart in-use against request rate over a full day, per instance, against instance uptime. A ratchet that ignores quiet periods and resets on restart points to a leak. A curve that follows traffic points to load or slow work.
3. Compare the pool's view with the database's. Many connections checked out in the pool but idle or long idle in transaction at the database confirms a leak. Mostly active sessions point to load or slow queries.
4. Turn on borrower tracking. Use the pool's leak detection, or a wrapper that records checkout stack and age. Keep the warnings that never clear; they name the leaking path.
5. Find every acquisition path without structured release. Search for manual checkouts, transactions, returned cursors, and streams. Check error and early-return branches, not just the happy path.
6. Fix, then verify the shape. After the fix ships, in-use should return to baseline in quiet periods and after the next dependency failure. If it still ratchets, there's a second path.

Fixes, matched to the cause

Cause	Fix	What not to do
Connection leak	Release structurally on every path (try/finally, try-with-resources, with, defer). Close returned cursors and streams. Prefer single-call APIs like pool.query()	Raise the pool size or restart on a schedule. Both hide the leak until the error rate rises
Leak mid-transaction	Roll back on every non-commit path. Add idle_in_transaction_session_timeout as a lock guardrail	Rely on the database timeout as the fix; the pool still thinks the connection is out
Slow queries	Fix the query plan or index; add a statement timeout so one query can't hold a connection indefinitely	Raise the pool wait timeout. Users wait longer for the same failure
Long holds around remote calls	Move I/O out of the transaction and add client timeouts to the remote call	Add connections to match the slowest dependency
Load or amplified demand	Reduce demand (cache, batch, bound retries), shed load, then size the pool from throughput × hold time within the database's limit	Size each instance's pool without multiplying by instance count
Undefined limits	Set a pool max and a wait timeout explicitly (Go SetMaxOpenConns, node-postgres connectionTimeoutMillis)	Leave defaults that wait forever or open unlimited connections

Treat any change to pool size, pool timeout, or data access code as a change to a shared production resource. Several services and background jobs usually share the database's connection budget, so a local "fix" can move the exhaustion somewhere else.

Key takeaways

• Pool exhaustion is a symptom. A leak is one cause; load, slow queries, long holds, and nested acquisition are others.
• Chart in-use against request rate, per instance. A leak ratchets and ignores quiet periods. Load-driven exhaustion recovers.
• Connections checked out in the pool but idle at the database are leak candidates.
• Most leaks live on error and early-return paths, so they drain the pool at the error rate and surface during other incidents.
• node-postgres waits forever and Go opens unlimited connections by default. Set explicit limits so exhaustion is visible.
• Fix leaks with structured release on every path. A bigger pool only delays a leak.

---

Runtime metrics tell you a pool is leaking, but a path only shows up in a leak trace after it has run in production. Tomosu checks the repository and each pull request for that kind of risk before merge: acquisition paths where an error branch can skip the release, holds that stretch across remote calls, and which endpoints share the pool. A PR that adds a pool.connect() without a finally shows up in its Production Reliability Index at review time instead of during the next dependency outage. Scan a repository with Tomosu.

Originally published on tomosu.ai, part of the Production Debugging series.