DEV Community

Libme
Libme

Posted on

Postgres Says "Too Many Clients Already": Diagnose It Before You Add a Pooler

FATAL: sorry, too many clients already almost never means your database is out of capacity. It means something in your stack opened more connections than max_connections allows — usually a per-process pool multiplied by more processes than you remembered running, or connections parked in idle in transaction. Adding PgBouncer fixes the symptom, but if you install it without finding the multiplier first, you get the same failure a month later with an extra moving part in the path.

Here is the order I work through it: count the connections and their states, find the multiplier, shrink the app-side pool, and only then decide whether a pooler is actually warranted.

What does "sorry, too many clients already" actually mean?

Postgres allocates a backend process per connection, and max_connections is a hard ceiling set at server start. When clients exceed it, the connection attempt is rejected outright. You'll see one of two messages:

FATAL:  sorry, too many clients already
FATAL:  remaining connection slots are reserved for non-replication superuser connections
Enter fullscreen mode Exit fullscreen mode

The second one is worth knowing separately: it means you've hit max_connections minus superuser_reserved_connections, so ordinary users are locked out while the reserved slots keep a superuser login available for exactly this situation. That's the escape hatch that lets you connect as a superuser and look around while the app is failing.

Both are refusals at the door, not signs of load. CPU and IO can be near idle while this happens.

Takeaway: this error is a counting problem, not a capacity problem — treat it as arithmetic until the arithmetic proves otherwise.

How do I find out where the connections are going?

Connect (as superuser if you're locked out) and group pg_stat_activity by the two things that matter: who opened it, and what it's doing.

SELECT
  usename,
  application_name,
  state,
  count(*),
  max(now() - state_change) AS longest_in_state
FROM pg_stat_activity
WHERE backend_type = 'client backend'
GROUP BY 1, 2, 3
ORDER BY count(*) DESC;
Enter fullscreen mode Exit fullscreen mode

Three patterns show up over and over:

  • A wall of idle connections with a single application_name. That's a pool that's sized correctly per process but replicated across more processes than you think.
  • idle in transaction, with longest_in_state in minutes. A code path opened a transaction, did something slow or fallible outside the database (an HTTP call, usually), and never committed. Those connections hold locks and are unusable by anyone else.
  • A scattering of active connections all above your intended pool size. Something bypasses the pool — a migration runner, a cron job, an admin tool, a metrics exporter.

The idle in transaction case has a server-side guard worth setting regardless of how you resolve the rest:

ALTER SYSTEM SET idle_in_transaction_session_timeout = '60s';
SELECT pg_reload_conf();
Enter fullscreen mode Exit fullscreen mode

That kills sessions that hold a transaction open past a minute. It converts a silent connection leak into a loud, attributable application error, which is strictly better.

Takeaway: if idle in transaction shows up at all, fix that before touching pool sizes — a pooler will happily leak those too.

Why is my app pool bigger than I think?

The multiplier is where the number actually comes from. Your config says pool size 10; your database sees 240. The count is:

pool_size x processes_per_instance x instances (x 2 if you run a separate replica/read pool)
Enter fullscreen mode Exit fullscreen mode

A Gunicorn or Puma deployment with 8 workers, a pool of 10, across 3 pods is 240 connections before anything unusual happens. Add a background worker deployment with the same defaults and you're past a default max_connections of 100 several times over. Serverless is the extreme version: every warm function instance holds its own connection, and concurrency spikes create instances faster than any pool config can constrain.

Most application pools also have a burst setting beyond the nominal size — SQLAlchemy's max_overflow, HikariCP's maximumPoolSize versus minimum idle — so multiply the ceiling, not the steady state.

And the pool is usually too big to begin with. The sizing guidance that has held up best for me is the one HikariCP has documented for years: connections should be a small multiple of your core count, not a function of your request concurrency. Beyond that, the extra connections just queue inside Postgres instead of inside your app.

Takeaway: multiply your pool ceiling by every process and every replica before you conclude the database is the problem.

When is a connection pooler actually worth adding?

Shrinking pools solves it for a fixed set of long-lived processes. A pooler earns its place when the client count is genuinely unbounded — serverless, autoscaled workers, many small services sharing one database — or when you need thousands of client connections multiplexed onto a few dozen server ones.

Option Runs where Best fit Main drawback
Smaller app pool Nowhere new Fixed process count, single service Doesn't survive autoscaling or serverless
PgBouncer Sidecar, VM, or container Anything self-managed; the default choice Single-threaded per process; you manage HA yourself
Supavisor Managed on Supabase Postgres on Supabase, incl. serverless clients Tied to that platform
pgcat Sidecar or standalone Pooling plus load balancing across replicas Smaller ecosystem; fewer people to ask when it misbehaves
RDS Proxy Managed by AWS Lambda against RDS/Aurora, IAM auth Per-connection pricing; pinning silently kills the benefit

If you're self-hosting and want the option with the longest operational track record, PgBouncer is the one that multiplexes thousands of clients onto a small server pool with a config file you can read in one sitting. If your Postgres is already on Supabase, Supavisor is the pooler that's designed for that platform's connection strings and handles serverless client churn without extra infrastructure. If you're running Lambda against RDS or Aurora and want AWS to own the uptime, RDS Proxy is the one that integrates with IAM authentication and Secrets Manager instead of asking you to distribute database passwords.

A minimal PgBouncer config for the common case:

[databases]
appdb = host=10.0.1.20 port=5432 dbname=appdb

[pgbouncer]
listen_addr = 0.0.0.0
listen_port = 6432
auth_type = scram-sha-256
auth_file = /etc/pgbouncer/userlist.txt
pool_mode = transaction
default_pool_size = 20
max_client_conn = 1000
server_idle_timeout = 600
Enter fullscreen mode Exit fullscreen mode

default_pool_size is how many real Postgres connections this pool will ever hold; max_client_conn is how many clients may queue against it. Once it's running, SHOW POOLS; on the admin console tells you whether the size is right — cl_waiting and maxwait above zero mean clients are queuing for a server connection, which is your signal to raise default_pool_size or find the slow queries holding connections.

Takeaway: a pooler is for unbounded client counts; for a fixed number of processes, the cheaper fix is to stop over-provisioning the pool.

What breaks when you switch to transaction pooling?

This is the part that surprises people, because the migration looks like a hostname and port change. In pool_mode = transaction, a server connection is handed back to the pool at the end of every transaction, so any state that lives on the session rather than the transaction is gone or, worse, leaks to whoever gets that connection next. That rules out LISTEN/NOTIFY, session-level advisory locks, SET outside a transaction, WITH HOLD cursors, and temp tables.

The one that actually bites is prepared statements. Drivers that transparently prepare and cache statements — JDBC, asyncpg, psycopg3 — will send a named statement on one server connection and try to execute it on another:

ERROR:  prepared statement "S_1" does not exist
ERROR:  prepared statement "S_1" already exists
Enter fullscreen mode Exit fullscreen mode

Modern PgBouncer (1.21 and later) can track named prepared statements in transaction mode, but you have to opt in with max_prepared_statements; if it's left at the disabled default, you get the errors above no matter how new the binary is. Check the default for the exact version you're running before assuming it's on.

The client-side fixes, if you'd rather not rely on the pooler:

# asyncpg: disable the statement cache
conn = await asyncpg.connect(dsn, statement_cache_size=0)

# psycopg3: never use the extended protocol's named prepares
conn = psycopg.connect(dsn, prepare_threshold=None)
Enter fullscreen mode Exit fullscreen mode
# SQLAlchemy in serverless or behind a transaction-mode pooler:
# don't pool twice — let the pooler own it
from sqlalchemy import create_engine
from sqlalchemy.pool import NullPool

engine = create_engine(DSN, poolclass=NullPool)
Enter fullscreen mode Exit fullscreen mode

For JDBC, the equivalent is prepareThreshold=0 in the connection URL.

Keep one session-mode port (or a separate connection string on 5432) for migrations and admin tools. Schema migrations, CREATE INDEX CONCURRENTLY, and anything using advisory locks belong on a direct connection, not the transaction pool.

Takeaway: transaction pooling is a contract change, not a hostname change — audit for session state and prepared statements before you cut over.

FAQ

Why do I get "too many clients already" when my Postgres CPU is idle?
Because the limit is max_connections, a fixed count set at server start, not a resource threshold. Postgres refuses the connection at the door regardless of how little work the existing backends are doing.

Should I just raise max_connections instead of adding a pooler?
Only modestly, and only if you have the memory for it — each connection is a backend process with its own overhead, and thousands of mostly-idle backends degrade throughput even when they're doing nothing. Raising it from 100 to a few hundred on a well-provisioned server is reasonable; treating it as the answer to serverless connection churn is not.

Can I use PgBouncer transaction mode with an ORM?
Yes, provided you disable the driver's prepared-statement cache (or enable max_prepared_statements in PgBouncer), stop the ORM from maintaining its own connection pool on top, and route migrations to a direct session-mode connection.

Bottom line

Start by counting: group pg_stat_activity by state and application, and set idle_in_transaction_session_timeout so leaks announce themselves. If your process count is fixed, shrink the per-process pool to a small multiple of your cores and the error usually disappears without new infrastructure. Add a pooler when client count is genuinely unbounded — PgBouncer for self-managed setups, Supavisor if you're on Supabase, RDS Proxy if you want AWS to own it. Whichever you choose, treat the move to transaction mode as an application change: audit session state, disable driver-side prepared statements, and keep a direct connection reserved for migrations.

Related reading

Top comments (1)

Collapse
 
mads_hansen_27b33ebfee4c9 profile image
Mads Hansen

Excellent point about doing the arithmetic before adding infrastructure. One multiplier I would add explicitly is rollout overlap: during a Kubernetes rolling deploy, old and new pods coexist, and preStop/termination grace periods can make the peak connection count much higher than the steady-state replica count. Autoscaler overshoot and background workers amplify the same window.

I like treating max_connections - reserved_slots as a budget allocated by workload, then alerting on both budget consumption and acquisition latency. Give each service and job a distinct application_name; otherwise the first incident turns into guesswork. A small pool with a bounded acquisition timeout is also a useful backpressure point: fail a request quickly or shed optional work instead of letting an unbounded queue consume all request deadlines.

A practical rollout test is to scale to the maximum expected overlap, restart every pool at once, and verify the database keeps its operational reserve. Steady-state arithmetic alone misses exactly the moment most connection incidents occur.