Amitesh0512

Posted on Oct 3 Originally published at amiteshsurwar.com

Handling Connection Pool Exhaustion in ASP.NET Core APIs

#aspnetcore #performancetuning #observability #scalability

Quick Answer

handling connection pool exhaustion: Tuning MaxPoolSize, setting short ConnectionTimeouts, and using Polly retry/circuit‑breaker logic prevents ASP.NET Core connection pool exhaustion.

Connection Pool Bottleneck Impact

In a micro‑service that serves 5 kRPS against Azure SQL, the ADO.NET connection pool is the invisible bottleneck that turns a well‑designed API into a cascading failure. Every request that reaches the DbContext layer pulls a socket from a shared bucket. If that bucket is empty, OpenAsync blocks until the ConnectionTimeout expires, turning a quick 20 ms query into a 15‑second timeout. The result is a flood of 503s, thread‑pool exhaustion, and downstream services being hit with spurious timeouts.

Real‑world Example

During a 24‑hour flash sale, an e‑commerce platform’s order service ran 2 kRPS. The service’s connection string had MaxPoolSize=100 and no Connect Timeout override. A single long‑running inventory check that scanned 10 M rows held a connection for 45 s, never returning the slot. Within minutes the pool was exhausted; every new request had to wait for the timeout, the ASP.NET Core thread pool filled, and the service began returning 503s. The failure propagated to the cart service, causing a 5% spike in cart abandonment.

When this fails in production

All requests stall on OpenAsync and the HTTP server’s request queue builds up.
Thread‑pool starvation leads to ThreadPoolExhaustedException and application crash.
Background workers that never dispose their DbContext silently consume pool slots until the process restarts.
Database logs show a spike in “timeout expired” errors that look like network issues but are actually pool exhaustion.

Common mistakes engineers make

Assuming the default MaxPoolSize=100 is sufficient for all workloads.
Using async void in background tasks, which bypasses DI lifetime scopes and leaks connections.
Disabling CommandTimeout and relying on the database to kill long queries, which still holds the pool slot until the DB cancels.
Not instrumenting pool metrics, so exhaustion is only visible after users report timeouts.
Implementing a “retry forever” loop without backoff, causing a thundering herd when the pool recovers.

Better approach based on experience

Size the pool per request profile: MaxPoolSize = (PeakRPS * AvgQueryMs) / 1000 * 2. For 2 kRPS and 30 ms average, that’s 120 connections per pod.
Always set Connect Timeout to 5 s or lower; a longer timeout just amplifies the failure surface.
Wrap every data‑access call in a Polly policy that combines timeout, exponential backoff with jitter, and circuit breaker.
Use await using for DbContext and SqlConnection to guarantee disposal even in fire‑and‑forget scenarios.
Expose pool metrics (active, idle, wait time) via OpenTelemetry and alert when wait time > 50 ms.
When scaling horizontally, keep max_connections on the database in sync with the sum of MaxPoolSize across all pods.

Trade‑offs

High MaxPoolSize vs. DB resource limits: A larger pool reduces contention but increases the load on the database. If max_connections is too low, the DB will start rejecting connections, turning the problem into a different failure mode.
Short ConnectionTimeout vs. transient network glitches: A 5 s timeout fails fast but may mark a transient network hiccup as a failure. Adding a circuit breaker with a short break duration mitigates this.
Polly retry vs. request latency: Retries add latency but increase overall success rate. Use a capped retry count (e.g., 3) and backoff that keeps total retry time < 1 s for most use cases.
Per‑request cancellation vs. global timeout: Passing the HTTP request CancellationToken down to the data layer ensures that a client‑side timeout releases the connection immediately, but it also means that a long transaction will be aborted mid‑way, potentially leaving the DB in an inconsistent state if not wrapped in a transaction.
Connection lifetime vs. connection health checks: Setting LoadBalanceTimeout to 300 s forces the pool to recycle connections, which is good for servers that recycle sockets. However, if the DB performs its own health checks, you might double‑check and waste cycles.

Connection Pool Configuration Matrix

Use the following matrix to decide how to configure and manage your connection pool:

Scenario	Recommended `MaxPoolSize`	Recommended `Connect Timeout`	Retry Strategy	Observability
High‑frequency read‑only service (5 kRPS, 20 ms avg)	120 (per pod)	5 s	Exponential backoff, 3 attempts, 200 ms base	Pool wait time, active connections
Long‑running reporting service (10 RPS, 2 s avg)	20 (per pod)	10 s	No retry; log and alert on timeouts	Connection lifetime, query duration
Background worker with fire‑and‑forget patterns	10 (per worker)	5 s	Single retry with jitter, no circuit breaker	Leak detection, disposal logs
Shared DB across multiple services	Sum of all services’ pools ≤ DB `max_connections`	5 s	Service‑level circuit breaker, shared policy	Cross‑service wait time, DB connection pool stats

When scaling, remember that the pool is per process. In a containerized environment, each replica gets its own pool. Therefore, the total number of connections can explode if you’re not careful. Keep max_connections on the database in mind and use Connection Resiliency features of EF Core or SqlClient to handle transient faults.

Blocking Calls, ThreadPool Exhaustion, Timeouts

Connection acquisition is O(1) when slots are free, but becomes blocking when the pool is full.
Each blocked OpenAsync consumes a thread from the ThreadPool. With high RPS, this quickly saturates the pool, leading to ThreadPoolExhaustedException.
Using async/await liberates threads, but if you wrap the call in a synchronous retry loop, you risk blocking threads again.
Setting CommandTimeout lower than ConnectionTimeout ensures that a runaway query returns the connection sooner.
Instrument pool metrics at least once per second. A spike in wait time > 50 ms is a red flag.

Scaling Notes

When you scale out horizontally, recalculate MaxPoolSize per pod: MaxPoolSize = (PeakRPS / ReplicaCount) * AvgQueryMs / 1000 * 2.
Use Connection Resiliency only if you can guarantee idempotency or safe retry semantics; otherwise, rely on idempotent API design.
For Azure SQL, monitor the max_connections metric; if you hit the limit, consider elastic pools or sharding.
Leverage SqlConnectionStringBuilder to programmatically adjust pool size during deployment based on telemetry.
When running on Kubernetes, use HorizontalPodAutoscaler with metrics from the pool (e.g., average wait time) to trigger scaling before exhaustion occurs.

How do I calculate the optimal MaxPoolSize for a high‑RPS service?

Use the formula MaxPoolSize = (PeakRPS * AvgQueryMs) / 1000 * 2. For 2 kRPS and 30 ms avg, that’s 120 connections per pod.

What effect does ConnectionTimeout have on pool exhaustion?

A long timeout lets blocked OpenAsync calls hold threads for many seconds, amplifying the failure surface. Set it to 5 s or lower to fail fast.

How can Polly help mitigate connection pool exhaustion?

Wrap each data‑access call in a Polly policy that applies timeout, exponential backoff with jitter, and a circuit breaker to avoid thundering herds.

Why is async void dangerous for background workers?

async void bypasses DI lifetimes and doesn’t surface exceptions, leading to DbContext leaks that silently consume pool slots until the process restarts.

What metrics should I expose for pool observability?

Track active, idle, and wait‑time metrics via OpenTelemetry, and alert when wait time exceeds ~50 ms to catch early exhaustion.

What to Ship

Update your connection string to include explicit pooling parameters: Min Pool Size=10; Max Pool Size=200; Connection Timeout=30; Connection Lifetime=300.
Wrap all database operations in a Polly retry policy with exponential back‑off (e.g., retry 3 times with 2s, 4s, 8s delays) to gracefully handle transient pool exhaustion.
Replace any synchronous ADO.NET calls with the async counterparts (e.g., ExecuteReaderAsync) to keep ThreadPool threads free and reduce blocking.
Configure a CommandTimeout of 30 seconds on all commands to avoid hanging queries that could tie up pooled connections.
Add a health‑check endpoint that exposes current pool usage metrics (e.g., GetConnectionPoolStats()) so you can proactively detect saturation.
Deploy the updated settings and verify that the maximum pool size is never exceeded during peak load by monitoring perfmon counters or Application Insights.

DEV Community