DEV Community: Runsite Team

S3 Egress Fees: Why Downloading Your Own Data Costs So Much

Runsite Team — Wed, 19 Aug 2026 15:47:41 +0000

Cross-posted from the Runsite blog.

You put a few hundred gigabytes of images on object storage, glance at the pricing page, and the numbers look friendly: storage is a couple of dollars a month, basically a rounding error.

Then the first real invoice arrives and it's a hundred and something. Nothing about how much you're storing changed.

The line that blew up isn't storage at all. It's egress — the charge for data leaving the bucket — and it's the part of the bill nobody shops on.

Why the storage bill blows up after the first invoice

The pricing page wasn't lying to you. Object storage genuinely is cheap to sit on. On AWS S3, standard storage runs about $0.023 per GB per month at the time of writing, so a hundred gigabytes of assets costs you around two dollars and change to keep. That's the number you compare when you're choosing where to put your files.

The number you don't compare is egress: the fee for moving data out of the provider's network. It doesn't show up when you upload, and it doesn't show up while the files just sit there. It shows up every time someone downloads something — roughly $0.09 per GB to the internet once you're past a small free allowance (about the first 100 GB a month on AWS).

Individually those are tiny fractions of a cent. The trouble is you're not billed once. You're billed per download, and a popular file gets downloaded a lot.

Where egress hides

Egress is data transfer out: every byte that leaves the provider's network. The reason it surprises people is that it isn't a single line you can point at. It's a multiplier that quietly attaches itself to things you'd never think of as "downloading":

Serving assets to users. Every image, video, PDF, or download your app hands to a visitor is egress. One 4 MB hero image on a page that gets a million views a month is four terabytes of transfer out, from a single file.
CDN origin pulls. Putting a CDN in front of your bucket helps, but it isn't free. Every cache miss means the CDN fetches the file from your bucket, and that origin pull is billed as egress. A cold cache, a low hit rate, or short cache lifetimes all quietly raise the number.
Cross-region and cross-AZ traffic. Replicating a bucket to a second region, or reading it from a service in a different availability zone, counts as data transfer too. That's egress you're paying to move data around inside the same provider.
Moving backups and exports out. Pulling a database dump back down to restore it, shipping logs to an external tool, or handing a data export to a customer all leave the network.
Retries and failed transfers. A download that drops at 90% and restarts still billed you for that 90%. At scale, retries are a line item of their own.

None of these are exotic. They're the normal behavior of a normal app. That's the point: egress isn't a penalty for doing something wrong. It's a charge on the thing storage is for, which is handing files to people.

How to calculate your real bill

You can estimate egress before a provider does it for you. The shape of it is two separate sums:

storage cost = gigabytes stored     x  storage rate per GB
egress cost  = gigabytes served out x  egress rate per GB

monthly bill = storage cost + egress cost

The storage side is easy — it's just how much you keep. The egress side is the one that runs away, because it isn't tied to how much you store, it's tied to how much you serve.

Take a small media site that keeps 100 GB of images and video and serves 2 TB of it over a month. A busy blog with a bit of video is well into that range:

Line	Volume	Rate	Cost
Storage	100 GB	$0.023/GB	$2.30
Egress	1,900 GB (2 TB served, first 100 GB free)	$0.09/GB	$171.00
Total			$173.30

The storage you actually shopped on is $2.30. The bill is $173.30.

Everything in the gap is egress, and it scales with traffic, not with how much you store. Double your visitors next month and the storage line doesn't move while the egress line does. You sized your storage, but the meter runs on your audience.

Check the rates before you trust the math. Provider egress prices change, vary by destination and region, and drop in tiers as volume climbs. The figures above are round numbers for AWS S3 to the public internet at the time of writing. Use them to understand the shape of the bill, then plug in your own provider's current numbers.

Expensive egress is also vendor lock-in

There's a second cost that doesn't show up until you try to leave. Your data lives in the bucket, and the only way to move it somewhere else is to download it — which means egress.

Say you've accumulated a terabyte and want to migrate. At around $0.09 per GB that's roughly $90 in transfer just to copy it out, before you've stored a single byte anywhere new. And a terabyte is a small dataset. For a serious archive the exit fee runs into real money.

Egress isn't only an operating cost, then. It's an exit tax, and a provider that charges a lot for it has quietly made itself expensive to leave.

Regulators have landed on the same conclusion: the EU Data Act phases out cloud switching charges precisely because exit fees keep customers stuck.

Zero-egress object storage

Flip the pricing around and the picture calms down. When data transfer out is free, the egress line goes to zero and your bill collapses back to the one thing you can predict: the gigabytes you store.

It isn't a fringe idea. Cloudflare R2, Backblaze B2, Wasabi, and Runsite all drop or heavily discount egress. They don't do it on identical terms — some cap requests, some price minimum retention, some bill differently for classes of storage — so read what each one bills instead of transfer before you pick.

Run the same media site through flat storage pricing at roughly €0.025/GB: the 100 GB of assets costs about €2.50 a month, and that's the whole bill, whether the site serves 2 TB or 20 TB. The line that was 99% of the invoice isn't discounted — it's simply gone.

The migration worry goes with it. These are all S3-compatible, so the AWS SDK, boto3, or any S3 client works unchanged: point the endpoint at the new host, swap the keys, and nothing in your code moves.

When egress actually matters — and when it doesn't

Worth being honest here, because egress isn't a crisis for everyone. It's probably your biggest bill line if you're doing one of these:

Media and video. Large files served repeatedly is exactly the shape that runs the meter hot.
Public assets at scale. A busy site's static files, a widely used app's downloads, user-generated content with real traffic.
Frequent restores or data movement. Backups you pull back often, datasets you shuttle between environments, exports you hand to customers.

And if that's not you, don't let the horror stories push you into over-engineering. A low-traffic internal tool, a private bucket that's written far more than it's read, or an archive that mostly sits untouched will barely register any egress at all. On that kind of workload the difference between metered and free transfer is a few cents.

The reason to care is the shape of your traffic, not the fact that egress exists.

The short version

Storage is the cheap, predictable part. Egress is the part that scales with how much the world uses your files — and it doubles as an exit fee when you try to leave.

Work out your own two lines before the invoice does it for you.

Disclosure: I work on Runsite, which is one of the zero-egress providers listed above. The AWS figures are its published list pricing, not my estimate — check the current rates before you budget against them.

PostgreSQL "FATAL: sorry, too many clients already" — What It Means and How to Fix It

Runsite Team — Tue, 11 Aug 2026 13:31:50 +0000

Cross-posted from the Runsite blog.

A deploy goes out, traffic picks up, and your app starts throwing 500s. You open the logs and there it is, over and over: FATAL: sorry, too many clients already. Nothing changed in your queries, the database is barely breaking a sweat on CPU, and yet new requests can't get in.

The error reads like the database is overloaded. It almost never is. What's actually full is the connection pool, and once you see why, the fix is straightforward and permanent.

What the error actually means

PostgreSQL accepts a fixed number of simultaneous client connections, set by the max_connections parameter (100 by default). When connection number 101 tries to open while the first 100 are still held, the server doesn't queue it or slow it down. It refuses outright.

You'll sometimes see its sibling, remaining connection slots are reserved for non-replication superuser connections, which is the same wall hit a few slots earlier — Postgres keeps a handful in reserve so an admin can still log in to fix the mess.

The key thing: this is a count of open connections, not a measure of load. An app running ten trivial SELECT 1 queries can exhaust the limit while one running a single heavy report stays well under it. The error is about how many doors are open, not how much work is going through them.

Why Postgres has a connection ceiling at all

It helps to know why the limit exists, because it explains why "just raise it" is a trap.

PostgreSQL uses a process-per-connection model: every connection forks a dedicated backend process on the server, and each one reserves memory for its work area, query plans, and buffers — idle or not. A hundred connections doing nothing still cost real RAM.

Push max_connections to 500 on a small instance and you can starve the database of the memory it needs to actually run queries, trading a clean refusal for thrashing and OOM kills. The ceiling isn't arbitrary; it's the database protecting itself.

Find out what's eating your connections

Before changing anything, look at who's actually connected. Postgres exposes this in the pg_stat_activity view. Group it by state to get the shape of the problem in one query:

-- How many connections, and what are they doing?
SELECT state, count(*)
FROM pg_stat_activity
GROUP BY state
ORDER BY count(*) DESC;

The state column is the tell, and each value points at a different fix:

active — real concurrent work in flight. The answer is pooling, so fewer connections do the same amount of work.
idle — your app opened the connection and walked away without closing it. The answer is fixing how it manages connections.
idle in transaction — the worst of the three: a connection that ran a query, never committed or rolled back, and is now holding both a slot and its locks hostage. A few of these can wedge a whole app.

The usual culprit: connections multiply with instances

If each app instance opens its own pool of, say, 20 connections, then four instances plus a couple of background workers and a cron job quietly add up to well past 100 — even though no single piece looks unreasonable.

Connection math is per-fleet, not per-process. Count every instance, worker, and scheduled job that talks to the database.

The real fix: pool your connections

Opening a fresh connection per request is the root cause behind most of these incidents. Connections are expensive to create and strictly limited, so the fix is to open a small set once and reuse them. There are two layers to do it at, and busy apps want both.

1. Pool inside your app

Every serious database driver ships a pool. Use it, and cap it deliberately instead of leaving it on a generous default. The cap is per process, so multiply it by how many instances you run and keep the total comfortably under max_connections:

import { Pool } from 'pg';

// One pool per process, reused across requests.
const pool = new Pool({
  connectionString: process.env.DATABASE_URL,
  max: 10, // ceiling PER instance: 10 x instances must stay under max_connections
});

// Borrow and return; never open a connection per request.
const { rows } = await pool.query('SELECT * FROM users WHERE id = $1', [id]);

This alone resolves a surprising share of "too many clients" fires. The trap to avoid is creating the pool inside a request handler, which spins up a brand-new pool on every call and reproduces the exact problem you're trying to kill. Create it once, at startup.

2. Put a pooler in front of the database

App-side pools have a hard limit: they can't coordinate across instances. Each one only knows its own connections, so a fleet that scales horizontally will still blow the ceiling no matter how careful each process is.

The fix is a single pooler that sits between your apps and Postgres, multiplexing thousands of client connections down onto a small set of real database ones. PgBouncer is the standard tool, and in transaction pooling mode it hands a real connection to a client only for the duration of a transaction, then returns it to the shared set — so a handful of backend connections can serve a large fleet.

A common way to expose this is two ports on the same database: one for a direct session, one that routes through the pooler.

# Direct connection (port 5432)
DATABASE_URL=postgresql://user:pass@db.example.com:5432/mydb

# Pooled through PgBouncer (port 6432)
DATABASE_URL=postgresql://user:pass@db.example.com:6432/mydb?pgbouncer=true

One caveat worth knowing up front: transaction pooling doesn't play nicely with session-level features like prepared statements or SET that expect to live for a whole session. Most ORMs have a setting for this — check yours before you switch the port in production rather than improvising mid-incident.

Rule of thumb: pooled endpoint for app traffic, direct endpoint for migrations and admin tasks.

Hunt down leaked and idle connections

If pg_stat_activity showed a wall of idle or idle in transaction rows, pooling alone won't save you, because something is opening connections and never letting go. The common causes are short: a code path that opens a connection and returns before closing it, a transaction that errors out without a rollback, or a worker that holds a connection open while it waits on something slow.

To buy breathing room in the moment, you can terminate the connections that have been sitting idle the longest:

-- Free connections idle for more than 10 minutes
SELECT pg_terminate_backend(pid)
FROM pg_stat_activity
WHERE state = 'idle'
  AND state_change < now() - interval '10 minutes'
  AND pid <> pg_backend_pid();

That's a fire extinguisher, not a fix; the leak will refill the slots if you don't find it. The durable version is to let the database reap them automatically by setting idle_in_transaction_session_timeout, so a forgotten transaction can't hold a slot indefinitely.

Where this error usually comes from

The error text is identical everywhere, but the thing filling the slots depends on what you're running. Four setups account for most cases, and each has a different tell.

Docker and Docker Compose

A Postgres container ships with the stock max_connections of 100, and Compose makes it easy to outgrow that without noticing. docker compose up --scale api=4 quadruples your pools while the database config stays where it was.

Restarting an app container makes it worse before it makes it better: the old backends aren't closed cleanly, so Postgres keeps them until TCP keepalive expires, and the fresh container opens a full pool on top of them. If the count drops on its own a few minutes after a restart, that's what you were looking at.

Set the pool size per container from the fleet total, not per service, and raise the container's max_connections deliberately rather than letting the default decide.

Django

Django's CONN_MAX_AGE is the usual cause. At the default of 0 every request closes its connection — wasteful but safe. Set it to a non-zero value (or None) without a pooler in front and every Gunicorn worker holds a connection open for its lifetime.

The arithmetic is workers × instances: 16 workers across three instances is 48 permanent connections before Celery or a management command touches the database.

Either keep CONN_MAX_AGE low and let a pooler do the reuse, or keep persistent connections and size the worker count against max_connections on purpose. On Django 4.2+, turn on conn_health_checks so a recycled connection that died server-side doesn't surface as an error in the request.

DBeaver, pgAdmin, and other GUI clients

A single connection in a database IDE is rarely a single connection to Postgres. DBeaver opens a separate session for metadata, and typically another per SQL editor tab, and holds all of them for as long as the app is open.

Three developers with a handful of tabs each can quietly occupy a fifth of a 100-connection limit while doing nothing at all. If pg_stat_activity shows idle rows whose application_name is a client tool, that's your answer.

A local Postgres on Ubuntu or Debian

The package default is the same 100, set in /etc/postgresql/<version>/main/postgresql.conf. Changing it needs a full sudo systemctl restart postgresql, not a reload, because max_connections is allocated at startup.

On a dev machine the culprit is usually simpler than a config value: psql sessions left in old terminal tabs, plus every project's dev server holding its own pool against the same instance.

When raising max_connections is (and isn't) the answer

Sometimes the limit really is too low for legitimate, well-pooled traffic, and the right move is genuinely a bigger instance. The order matters, though:

Pool first.
Find any leaks second.
Only then raise the ceiling — and raise the instance's memory along with it, not just the number.

A higher limit on the same small instance buys instability, not headroom.

The short version

"Too many clients already" is a connection-count problem wearing the costume of an overload.

Check pg_stat_activity to see whether you're looking at real concurrency, idle leftovers, or stuck transactions.
Pool connections inside each app, and put PgBouncer in front of the fleet so thousands of clients ride on a handful of real connections.
Reach for a bigger max_connections last, and when you do, give it the memory to match.

Do that and the error stops being a recurring incident and becomes a line you never see again.

Related, if you want to go deeper: connection pooling with PgBouncer and self-hosted vs managed PostgreSQL.

Disclosure: I work on Runsite, a deployment platform whose managed Postgres ships with PgBouncer on port 6432 — which is why that setup is the one I reach for by reflex. Everything above works the same on any Postgres you run yourself.