<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: KBY Technologies</title>
    <description>The latest articles on DEV Community by KBY Technologies (@kbytech).</description>
    <link>https://dev.to/kbytech</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4032449%2F0e1275cf-3fb9-444e-9f1e-551c6a4c6d05.jpg</url>
      <title>DEV Community: KBY Technologies</title>
      <link>https://dev.to/kbytech</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/kbytech"/>
    <language>en</language>
    <item>
      <title>Why Your Postgres Is Idle But Your API Is Timing Out (PgBouncer Pool Exhaustion, Explained)</title>
      <dc:creator>KBY Technologies</dc:creator>
      <pubDate>Thu, 16 Jul 2026 16:10:30 +0000</pubDate>
      <link>https://dev.to/kbytech/why-your-postgres-is-idle-but-your-api-is-timing-out-pgbouncer-pool-exhaustion-explained-48am</link>
      <guid>https://dev.to/kbytech/why-your-postgres-is-idle-but-your-api-is-timing-out-pgbouncer-pool-exhaustion-explained-48am</guid>
      <description>&lt;p&gt;Ever had an incident where latency on a trivial &lt;code&gt;SELECT&lt;/code&gt; spikes from a few milliseconds to several seconds, CPU on the Postgres box is basically flat, and there's no slow query or missing index anywhere in sight?&lt;/p&gt;

&lt;p&gt;That's not a database problem. That's &lt;strong&gt;connection pool exhaustion&lt;/strong&gt; at the PgBouncer layer — and it's one of the most common (and most misdiagnosed) production incidents in Postgres-backed systems.&lt;/p&gt;

&lt;p&gt;I just wrote a deep dive on this: how it happens, how to diagnose it live, and how to size your pool correctly instead of guessing. Here's the short version.&lt;/p&gt;

&lt;h2&gt;
  
  
  The core issue
&lt;/h2&gt;

&lt;p&gt;PgBouncer sits between potentially thousands of client connections and a small, fixed number of physical Postgres backends. That works fine — until something makes transactions hold connections longer than expected: a slow downstream API call, a lock wait, a runaway transaction. When that happens, the queue of waiting clients grows unbounded, and every request behind it pays the tax, even the fast ones.&lt;/p&gt;

&lt;h2&gt;
  
  
  Pooling mode matters more than people think
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Session pooling&lt;/strong&gt; — 1:1 mapping, safest for compatibility, but a pool of 100 connections supports at most 100 concurrent clients. Full stop.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Transaction pooling&lt;/strong&gt; — what most production setups run. Connections are freed the instant a transaction commits, so you can multiplex hundreds of clients onto a handful of backends. The catch: anything session-scoped (&lt;code&gt;SET search_path&lt;/code&gt;, prepared statements, temp tables) becomes unreliable.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Statement pooling&lt;/strong&gt; — most aggressive, breaks multi-statement transactions, mostly used for read-only analytics.
Transaction mode is where exhaustion gets sneaky, because the pool &lt;em&gt;looks&lt;/em&gt; elastic. Teams assume Postgres's &lt;code&gt;max_connections&lt;/code&gt; is the real ceiling — it isn't. The real ceiling is how many transactions your application can hold open simultaneously.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Diagnosing it live
&lt;/h2&gt;

&lt;p&gt;Two commands tell you almost everything:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;psql &lt;span class="nt"&gt;-h&lt;/span&gt; 127.0.0.1 &lt;span class="nt"&gt;-p&lt;/span&gt; 6432 &lt;span class="nt"&gt;-U&lt;/span&gt; pgbadmin pgbouncer &lt;span class="nt"&gt;-c&lt;/span&gt; &lt;span class="s2"&gt;"SHOW POOLS;"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If &lt;code&gt;cl_waiting&lt;/code&gt; is climbing while &lt;code&gt;sv_active&lt;/code&gt; is pinned at &lt;code&gt;pool_size&lt;/code&gt;, that's the signature: every server slot is occupied and clients are queuing.&lt;/p&gt;

&lt;p&gt;Then cross-reference with Postgres directly to find out &lt;em&gt;what's&lt;/em&gt; holding those slots hostage — usually a transaction left open across a network call (a classic ORM anti-pattern), or a lock wait cascading from a migration.&lt;/p&gt;

&lt;h2&gt;
  
  
  The usual culprit: idle-in-transaction leakage
&lt;/h2&gt;

&lt;p&gt;If your app opens a transaction, does a read, then awaits an external HTTP call before committing, that connection is unusable by anyone else for the entire duration of that call. Under load, this single pattern causes more exhaustion incidents than raw traffic volume ever does.&lt;/p&gt;

&lt;h2&gt;
  
  
  Sizing the pool properly
&lt;/h2&gt;

&lt;p&gt;Little's Law gives a good starting point:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;required_connections = arrival_rate × average_transaction_duration
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;But size for the tail, not the average — one slow query can distort your numbers badly. And resist the urge to just crank &lt;code&gt;default_pool_size&lt;/code&gt; up; more Postgres backends means more memory overhead and more lock contention, not a free win.&lt;/p&gt;




&lt;p&gt;I go into a lot more depth in the full article — reference PgBouncer config, the exact diagnostic queries, failover edge cases, and the security trade-offs of exposing the admin console. Worth a read if you've ever been paged for a "database is slow" incident that turned out not to be the database at all.&lt;/p&gt;

&lt;p&gt;👉 &lt;a href="https://www.kbytechnologies.com/systems-engineering/diagnosing-pgbouncer-pool-exhaustion-under-load" rel="noopener noreferrer"&gt;Full article: Diagnosing PgBouncer Pool Exhaustion Under Load&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Curious if others have run into the prepared-statement-caching gotcha under transaction pooling — that one bit me hard the first time I hit it. Drop your war stories below.&lt;/p&gt;

</description>
      <category>postgres</category>
      <category>devops</category>
      <category>database</category>
      <category>backend</category>
    </item>
  </channel>
</rss>
