DEV Community: Satyaki Saha

Just Add More Database Connections Is a System Design Trap

Satyaki Saha — Sat, 03 Oct 2026 04:18:47 +0000

During system design interviews or production firefighting sessions, a common piece of logic always resurfaces:

"If our Hibernate threads are starving for database connections, why don't we just set the connection pool size to 1000? More connections means more parallel queries, which means better database CPU usage... right?"

Wrong. It sounds intuitive, but in production, this naive scaling approach will actively slow down your system—and under the right circumstances, trigger a total database outage.

Here is what actually happens when you bloat your connection pool, and the production-grade solutions you should implement instead.

1. Latency Gets Worse, Not Better (The Myth of Infinite Parallelism)

Imagine a database server running on a machine with 8 CPU cores. If you give your application instances hundreds of active connections, all trying to execute heavy queries simultaneously, the database doesn't magically work faster. It thrashes.

Instead of doing actual processing work, the operating system spends its precious CPU execution cycles doing this:

Context Switching: Rapidly swapping execution context between hundreds of competing database threads.
Lock Contention: Waiting on row-level or table-level database locks.
Cache Thrashing: Ruining CPU L1/L2 cache locality as threads fight for resources.

The Downstream Effect

When context switching takes over, each individual query takes longer to complete. Because queries take longer, connections are held open longer. As connections stay occupied, new incoming requests pile up in the application queue. You haven't increased throughput; you have simply turned a high-performance database into a severe traffic jam.

2. How Kubernetes HPA Turns a Slowdown Into an Outage

Things get dangerous when you mix giant connection pools with cloud native auto-scaling infrastructure like a Kubernetes Horizontal Pod Autoscaler (HPA).

Imagine this textbook disaster scenario:

Your database starts slowing down due to CPU thrashing.
Because queries take longer, application response times spike.
The Kubernetes HPA sees the high latency (or CPU utilization) on your app pods and triggers a scale-out event to save the day—scaling your microservice from 2 pods to 8 pods.
If each pod has a direct connection pool size of 100, your deployment suddenly attempts to open 800 client connections simultaneously (8 pods × 100 connections).

If your Postgres max_connections configuration limit is capped at 200, the autoscaler that was supposed to save your application just delivered the killing blow:

FATAL: sorry, too many clients already

Your system shifts instantly from a minor traffic slowdown into a hard production outage.

💡 Production-Grade Solutions

How do you design a system that handles high traffic volumes without starving your microservices or crashing your core engine?

A. Introduce a Dedicated Connection Pooler

For databases like PostgreSQL, you should almost never connect highly scalable microservice replicas directly to the core database instance. Implement a proxy tool like PgBouncer or AWS RDS Proxy.

A connection pooler acts as a multiplexing buffer. Your 8 Kubernetes pods can safely open 100 client connections each to the proxy (800 total), while the proxy efficiently multiplexes those requests down into a tight, high-performance pool of just 20 to 30 actual physical connections directly routing onto the underlying Postgres engine.

B. Size the Pool to the DB Hardware, Not the Pod Count

When calculating your absolute baseline connection pool limits, rely on hardware realities over arbitrary scaling numbers. The standard industry benchmark formula established by HikariCP is:

$$\text{Connections} = (\text{Core Count} \times 2) + \text{Effective Spindles}$$

Note: Effective spindles represent the underlying disk I/O capabilities (which varies based on mechanical hard disks vs. solid-state drives).

This formula tells you the sweet spot where the database server can process tasks simultaneously before disk I/O blocking transitions into destructive CPU context thrashing. Take this overall hardware budget and divide it safely across your maximum expected replica pod count.

C. Implement CQRS (Read/Write Splitting)

If your application genuinely demands 1,000 parallel connections to handle aggressive read traffic (e.g., massive reporting dashboards or heavy user profile fetches), do not route them to your primary transactional write instance.

Separate your connection pools entirely. Route all heavy read operations to a distributed cluster of Read Replicas, keeping your primary database connection pool lean, fast, and strictly dedicated to handling state mutations and writes.

The Ultimate Takeaway

A bigger connection pool is not a faster database.

A tightly constrained pool, lightning-fast query execution, and a well-managed application-side queue will outperform a bloated, thrashing connection pool every single time.

Keep your pools small, keep your queries indexed, and use proxy poolers to protect your data layer.

We Broke Prod With Cache Misses So You Don't Have To

Satyaki Saha — Sat, 26 Sep 2026 12:45:40 +0000

Today's blog is mostly about the common ways cache misses have happened in our system, and how we resolved them.

Cache misses — and the system crashes they cause — are something you will face very often when you implement caching in a system. Until you face them, there's no way to know how the system can collapse without them.

1. Key simply doesn't exist yet (cold miss)

First-ever request for that data — nothing wrong here, just an empty cache.

Strategy: Cache-aside (lazy loading) — read from the DB on a miss, populate the cache, then return. This is the default pattern for a reason: it only caches what's actually requested. The upside here is that the request is usually new and not yet a common one, so you can afford not to overthink it.

2. Key exists but fails to deserialize

The bytes are there, but a schema change, version drift, or a bad write left the value unreadable. This one's sneaky — it looks like a miss, but it's actually a read failure being swallowed.

Strategy: Version your cache keys alongside your data schema (e.g., user:v2:123) so old, incompatible values naturally stop being looked up instead of throwing on read. Also log deserialization failures separately from true misses — conflating them hides real bugs. We faced this issue with a failing Jackson deserialization, which made the system go completely haywire.

3. TTL expiry — and the stampede risk on high-value keys

A key ages out normally. That's fine for low-traffic keys. It's dangerous for hot keys: if thousands of concurrent requests hit the same expired key at once, they all miss simultaneously and hammer the DB together.

Strategy:

Mutex/lock on rebuild — the first request past the miss recomputes the value; others wait or get a stale value.
Logical expiration — never let the TTL actually delete the key; instead, store an internal expires_at. On read, if it's expired, serve the stale value immediately and refresh it asynchronously in the background. [This is something we currently implement.]
Jittered TTLs so hot keys don't all expire in lockstep with each other.

4. Request for a resource that doesn't exist (cache penetration)

Someone queries an ID that isn't in the DB either — so nothing ever gets cached, and every repeat of that query bypasses the cache and hits the DB. This can be accidental, or an active exploit (bots probing random IDs).

Strategy:

Cache the negative result too, with a short TTL, so repeated misses for the same nonexistent key stop reaching the DB. This is negative caching, and we currently have this implemented.
Bloom filter in front of the cache/DB to cheaply reject obviously invalid keys before they ever reach the backend.

5. Entire cache cluster (server + replicas) went down

This one is severe. Thanks to the holiday-season timing, we were able to resolve it without much of a crisis — but every single request across the system ends up hitting the DB at once. This is a full avalanche, not a localized miss. In our system, we didn't have read sentinels set up properly.

Strategy:

High availability for the cache layer itself (Sentinel/Cluster with automatic failover), so one node dying isn't a total outage.
Circuit breaker / rate limiting in front of the DB as a last line of defense, so a cache outage degrades gracefully instead of taking the DB down too.
A local in-process L1 cache in front of Redis can cushion the blow even if L2 is fully down.

6. Network timeout to the cache

The cache is healthy and has the key, but the client times out or the connection drops before it gets the response. Depending on your fail-open/fail-closed policy, this often silently falls through to the DB, even though the data technically wasn't missing.

Silent Saves: How JPA Persists Your Changes Without You Asking

Satyaki Saha — Tue, 22 Sep 2026 18:28:19 +0000

One of our junior devs wrote an API that was supposed to be “read-only” and act as an internal data funnel — pull some records, transform them, pass them on. nothing that should ever come close to the database. But it corrupted the production data badly.

Then I started to dig into the code, and saw that he had @Transactional on the function. He was retrieving entities internally via EntityManager.find() (or a JPQL query, same thing) and mutating some fields on them in the middle of processing, just as a way to store intermediate state while composing the response. No save, no merge, no repository.update() anywhere in sight, which is exactly what made him feel it was safe.

But because the entities were managed and the method ran inside an active transaction, every one of those "intermediate state" field mutations got picked up by Hibernate's dirty checking at commit time. What he thought was a harmless, read-only data funnel was silently issuing UPDATE statements against production data — overwriting real records with transient values that were only ever meant to exist in memory for the duration of the request.

The fix was intuitive at that point: mark the transaction readOnly = true (@Transactional(readOnly = true)), which tells Hibernate to skip dirty checking and flushing entirely for that context. But 2 things I learnt were Hibernate engineering is a masterclass in abstraction, and learning hibernate is go deep or go home case.