Two weeks ago I told you about the index that turned a 4-second query into 12 milliseconds. Last week, about the bug that hid behind fifty perfectly fast queries: the N+1 problem. This week’s bug didn’t look like either of those.
The database dashboard was calm. CPU usage was low. Every individual query, when I checked, ran in a few milliseconds. And the app was still hanging — requests sitting there for seconds before anything happened, some timing out completely.
I kept looking at the database, because that’s where the last two bugs had lived. This one wasn’t there at all. It was sitting one layer up, in the thing that connects your app to the database in the first place.
The wrong mental model
Most of us assume that if the database itself isn’t struggling — low CPU, fast queries, nothing ugly in the slow query log — then the app talking to it should be fine too. That assumption is exactly what sends you looking in the wrong place.
Your app doesn’t open a fresh connection to the database every time it needs one. Opening a raw connection is expensive — a TCP handshake, authentication, session setup — often tens of milliseconds of pure overhead before a single query even runs. So instead, your app keeps a small, fixed set of connections open and ready, and hands them out to requests as needed. That set has a name: a connection pool. And a pool, by definition, has a limit.
**
What a connection pool actually does
**
Think of it as a small number of taxis waiting outside a building, instead of every person who needs a ride calling their own car from across town. When your app starts up, it opens a handful of connections to the database up front and keeps them alive. When a request needs to run a query, it “borrows” one of those already-open connections, uses it, and hands it back the moment it’s done — ready for the next request to borrow.
This is faster for almost every request, because nobody’s waiting on a fresh TCP handshake. It only becomes a problem when more requests want a connection at the same moment than connections are sitting in the pool.
What happens when the pool is exhausted
Say your pool has 10 connections, and 50 requests hit your app in the same second, each one needing to talk to the database. What happens to request number 11?
Take five seconds and guess before you read on. A lot of people assume the pool scales up automatically, or that request 11 fails right away. Neither is true. Request 11 waits — quietly, in a queue — for one of those 10 connections to be returned by whichever request finishes first. If it waits too long, it doesn’t fail with a database error at all. It fails with something like “connection is not available, request timed out” — an error about the pool, not the database.
A connection pool is a waiting line with a fixed number of open counters. As long as the number of requests needing a connection at any given moment stays at or below the pool size, everyone gets served instantly, and the whole thing is invisible. That’s exactly why this bug, like the last one, doesn’t show up in development — locally, you’re one person clicking around, rarely more than 2 or 3 concurrent requests, so a pool of 10 looks infinite. It only stops looking infinite once real traffic shows up.
The N+1 connection
This is worth calling out directly, because it ties straight back to last week: a request stuck in an N+1 loop, running 50 queries instead of 1, holds its connection roughly 50 times longer than a request that only needs one query. Every extra millisecond a connection stays checked out is a millisecond it’s not available for the next request in line.
Fixing N+1 doesn’t just cut your query count. It also frees up connections faster, which means your existing pool can serve more concurrent traffic without you touching the pool size at all. These two bugs compound each other more than most people realize — a slow, chatty endpoint doesn’t just look bad in its own logs; it steals capacity from every other endpoint sharing that same pool.
Reproduce pool exhaustion with Spring Boot
If you’re on Spring Boot, HikariCP is your default pool, and you can deliberately break this to feel it firsthand. Set this absurdly low in application.properties:
spring.datasource.hikari.maximum-pool-size=2
Then fire several requests at an endpoint that hits the database, at the same time — something like:
ab -n 20 -c 10 http://localhost:8080/your-endpoint
That’s Apache Bench: 20 total requests, 10 of them fired concurrently. Watch response times spike, and if you push the concurrency higher, watch requests start failing with a pool timeout instead of a database error. That’s the exact failure mode, produced on purpose, in a few seconds — a much faster way to build intuition for this than reading about it.
How HikariCP behaves
HikariCP doesn’t fail loudly the moment the pool runs out. It queues the waiting request and gives it a fixed amount of time — controlled by connection-timeout30 seconds by default — to get a connection before giving up. Only after that timeout expires does it throw, usually something like SQLTransientConnectionException: Connection is not available, request timed out after 30000ms.
That 30-second default is worth a moment of thought. It means a pool exhaustion problem doesn’t show up as an instant, obvious failure. It shows up as requests that hang for up to 30 seconds before eventually erroring out — which, from a user’s perspective, is often worse than a fast, clean failure would have been. This is also why HikariCP exposes live metrics for active, idle, and pending connections — pending specifically counts requests currently stuck waiting in that queue, and it's the single most useful number for catching this before users do.
Why “just increase the pool” is dangerous
The instinctive fix is “make the pool bigger.” That works for a while, and then quietly creates a new problem: your database also has a hard limit on total concurrent connections, and it’s usually smaller than people expect. Every connection your database holds open costs it memory and scheduling overhead, whether or not that connection is doing any work right now — so a database that’s comfortable with 100 connections can genuinely slow down if you push it to 500, even if none of those 500 are running expensive queries.
There’s a well-known starting formula from HikariCP’s own documentation: roughly ((core_count * 2) + effective_spindle_count). For a lot of standard setups that lands somewhere around 10, not hundreds — which surprises almost everyone the first time they see it, because it feels far too small for a busy app. The real fix usually isn't "bigger pool." It's "correctly-sized pool, plus fixing whatever's holding connections too long" — which, again, often means checking for N+1 queries and slow, unindexed lookups before you ever touch the pool size setting.
Pool size × application instances
Here’s the multiplication that catches people off guard. Pool size isn’t a global setting — it’s per instance of your app. If you run 10 instances behind a load balancer, and each one has a pool of 50, your database isn’t seeing 50 connections. It’s seeing up to 500, all at once, from a single service.
This is the most common way teams accidentally overwhelm their database while scaling out to “improve performance.” Adding more app instances to handle more traffic feels like it should only help — but if each new instance brings its own full-size pool, you’re multiplying connection pressure on the database at the same time you’re adding capacity everywhere else. The math to do before scaling out is simple: pool size per instance × number of instances should stay comfortably under your database's actual connection limit, with room left for migrations, admin tools, and anything else that connects directly.
How to detect pool exhaustion
Watch pool metrics, not just database metrics. If you’re using Spring Boot Actuator with Micrometer, HikariCP exposes hikaricp.connections.active, hikaricp.connections.idle, and hikaricp.connections.pending. A pending count that's regularly above zero under normal traffic is your pool telling you it's undersized, well before anything times out.
Load test with realistic concurrency, not realistic query speed. A single slow request rarely reveals pool exhaustion. Ten or twenty requests arriving at the same moment does — because that’s the actual shape of the problem.
Alert on connection timeout exceptions specifically, separate from general 500 errors. A spike in SQLTransientConnectionException is a distinct signal from a spike in application errors, and it points straight at the pool instead of sending you hunting through unrelated code.
How to size the pool
Start from your database’s real connection limit, work backwards, and treat every number as deliberate rather than copy-pasted from a tutorial:
spring.datasource.hikari.maximum-pool-size=10
spring.datasource.hikari.minimum-idle=10
spring.datasource.hikari.connection-timeout=30000
maximum-pool-size should come from the HikariCP formula above as a starting point, then get adjusted based on actual load testing, not guesswork. minimum-idle equal to the max keeps a fixed set of connections always warm, which HikariCP's own docs recommend for predictable performance. connection-timeout Controls how long a request will wait in the queue before failing — lower it if you'd rather fail fast and show an error than have users stare at a spinner for 30 seconds.
Remember while you do this: whatever number you land on here gets multiplied by every instance of your app that’s running.
Top comments (0)