DEV Community

Cover image for Keepalive Is On by Default, 100 Requests Opened 200 Connections
Mustafa ERBAY
Mustafa ERBAY

Posted on Originally published at mustafaerbay.com.tr

Keepalive Is On by Default, 100 Requests Opened 200 Connections

#nginx #linux #network #performance

I ran a two-line measurement on my own server. First I counted the TIME_WAIT sockets
pointing at port :3040, then I sent 100 requests to one of this blog's endpoints,
then I counted again:

N=20  requests -> TIME_WAIT(:3040) delta=44
N=50  requests -> TIME_WAIT(:3040) before=10  after=112 delta=102
N=100 requests -> TIME_WAIT(:3040) before=112 after=312 delta=200
Enter fullscreen mode Exit fullscreen mode

A hundred requests, two hundred fresh sockets. And yet the nginx on that server is
1.30.0 — one of the versions where upstream keepalive is supposed to be on by
default. The pool should have been open, connections should have been reused, these
numbers should not exist.

They do. The reason is that I had misread exactly what the change that landed in
nginx on 24 March 2026 covers — and what it does not.

And I did not keep that misreading to myself; I published it. In
my TIME_WAIT article
back in July I listed the three 1.29.7 changes correctly and then added that the
1.30.0 on my server "has this behaviour". The version was right, the sentence was
incomplete: that behaviour arrives for an upstream block, not for a server. What
follows is the record of me refuting my own sentence.

What changed on 24 March

The nginx 1.29.7 changelog sums it up in three lines:

*) Feature: the "local" parameter of the "keepalive" directive in the
   "upstream" block.
*) Change: now the "keepalive" directive in the "upstream" block is
   enabled by default.
*) Change: now ngx_http_proxy_module supports keepalive by default; the
   default value for "proxy_http_version" is "1.1"; the "Connection"
   proxy header is not sent by default anymore.
Enter fullscreen mode Exit fullscreen mode

For years there was a three-line recipe copied into every nginx config:
keepalive 32 in the upstream block, proxy_http_version 1.1 and
proxy_set_header Connection "" in the location. Without all three, the upstream
pool did nothing. 1.29.7 made that recipe the default: the documented default for
keepalive is now keepalive 32 local, and for proxy_http_version it is 1.1.

So the thing older articles tell you to "always add" is already there on new versions.
Written from memory, the recipe can now produce a different result than you expect —
I will come back to that.

How I count whether the pool works

I did not want to publish claims without measuring them first, so a small lab went up:
two nginx containers, one proxy and one backend. The trick is in the backend's log format:

log_format up '$connection|$http_connection|$server_protocol|$uri';
access_log /dev/stdout up;
Enter fullscreen mode Exit fullscreen mode

$connection is nginx's connection serial number. Every request arriving over the
same TCP connection carries the same number. So the count of unique $connection
values in the backend's log is the number of TCP connections the proxy opened.
I count requests from the same log, which lets me divide one figure by the other.

Two free extras ride along: $http_connection is the Connection header the backend
received, and $server_protocol is the HTTP version the request arrived with.
Together, those three explain in a single line why the pool did or did not open.

The driver side is one curl call with a braced range — 200 requests:

curl -s -o /dev/null "http://ka-px/req?i=[1-200]"
Enter fullscreen mode Exit fullscreen mode

The measurement: two versions, ten configurations

Each row is 200 requests. The number is the count of TCP connections the backend saw
for those 200 requests. I ran v1, v2 and v4 three times each; all three runs
produced identical figures, so I quote a single number. The remaining rows are
single runs — I did not repeat those three times.

# Configuration nginx 1.31.8 nginx 1.28.3
v1 proxy_pass http://up:8080 — no upstream block 200 200
v2 upstream block present, no keepalive written 1 200
v3 Classic recipe: keepalive 32 + 1.1 + Connection "" 1 1
v4 upstream + keepalive_requests 10 20 —
v5 upstream + proxy_http_version 1.0 200 —
v6 upstream + keepalive 0 200 —
v7 proxy_pass straight to an IP, no upstream block 200 —
v8 upstream + keepalive 32 local (default written by hand) 1 —
v9 Two locations sharing one upstream, no directive 2 200
v10 Two locations, keepalive 32 written by hand 1 —

The headers the backend saw changed with the version too:

protocol Connection header
1.31.8, default HTTP/1.1 not sent
1.28.3, default HTTP/1.0 close
1.31.8, explicit proxy_http_version 1.0 HTTP/1.0 not sent

That second row shows which rule from RFC 9112 section 9.3 kicks in: a recipient
decides whether a connection is persistent from the version and the Connection
header of the most recent message. If close is present, the connection will not
persist; if the protocol is HTTP/1.0 without keep-alive, it will not persist either.
1.28.3 did both at once. 1.31.8 stopped doing both.

The default is not for everyone — only for those who wrote an upstream block

v1 and v7 are the ones that stung. On nginx 1.31.8, writing
proxy_pass http://up:8080 with no upstream block gives 200 connections for
200 requests. Writing an IP instead of a name also gives 200. The protocol is
HTTP/1.1 and no Connection header is sent — so half the version change is
clearly in effect — but there is no pool.

I did not want to guess at this, so I read the source. In
ngx_http_upstream_keepalive_module.c, the defaults are applied in
ngx_http_upstream_keepalive_init_main_conf(), and the first thing it does is this:

/* skip implicit upstreams */
if (uscfp[i]->srv_conf == NULL) {
    continue;
}
Enter fullscreen mode Exit fullscreen mode

The default is applied while looping over the upstream list in the configuration, and
implicit upstreams are skipped. When you write proxy_pass http://host:port,
nginx builds an upstream for you behind the scenes; that upstream has no srv_conf,
because no upstream { } block was ever written for it. The loop skips it,
max_cached stays unset, and the pool never opens.

Further down the same loop there is one more gate, and then the defaults:

if (kcf->max_cached == 0) {
    continue;
}

ngx_conf_init_msec_value(kcf->time, 3600000);
ngx_conf_init_msec_value(kcf->timeout, 60000);
ngx_conf_init_uint_value(kcf->requests, 1000);

if (kcf->max_cached == NGX_CONF_UNSET_UINT) {
    kcf->local = 1;
    kcf->max_cached = 32;
}
Enter fullscreen mode Exit fullscreen mode

That middle continue explains row v6: write keepalive 0 and max_cached
becomes zero, the loop leaves there too, and no pool is built. So one function
holds three separate gates — implicit upstreams, keepalive 0, and an unset
max_cached.

One hour, 60 seconds, 1000 requests, 32 connections: all four are here. But note
they are not under the same condition. The timeouts and the 1000 are assigned
unconditionally; only local = 1 and max_cached = 32 sit inside the if.
Writing keepalive by hand does not change the 1000 or the 60 seconds — it only
takes the 32 and the local away from you.

I downloaded the file at three release tags:
release-1.29.7 and release-1.31.8 are byte-identical (16,146 bytes), while
release-1.28.3 is 14,923 bytes and contains neither that if nor the implicit
upstream skip.

Reduced to one sentence: on nginx after 1.29.7 upstream keepalive is free, but only
if you wrote an upstream { } block. Not writing a block is not the same as
accepting the default; it is writing in a place the default never looks.

Your own keepalive 32 is not the same thing as the default

v9 and v10 are a subtler trap. Two separate locations use the same upstream,
and I split the 200 requests between them:

  • With nothing written: 2 connections.
  • With keepalive 32 written: 1 connection.

That looks backwards at first. Does writing the setting by hand make things better?
No — it makes them different. Go back to the code above: the local = 1 assignment
only happens if max_cached was unset. So the default is keepalive 32 local, while
what you wrote by hand is keepalive 32 — without local. The local parameter
(new in 1.29.7) separates the pool per location; hence two locations, two pools,
two connections. Without local, every location pointing at that upstream shares
a single pool.

There is no "correct" option here, only a trade-off. A shared pool opens fewer
connections. Separate pools stop a connection established with one location's
settings — a different proxy_set_header, a different timeout, a different
certificate — from being handed to another location. nginx picked the second,
cautious option as the default. I think that is the right call: a feature that
arrives for free should not produce surprises, and that matters more than saving
one socket.

The practical consequence: if the keepalive 32 line is still sitting in your old
config, your behaviour differs from the default once you move to 1.29.7. Deleting
that line is not a no-op.

The pool is not infinite: where 1000 comes from

The run with keepalive_requests 10 gave exactly 20 connections for 200 requests.
The division explains itself: each connection carries 10 requests and closes.
Because the default is 1000, you will not notice that recycling in daily life — but
that 1000 exists for a reason, and the nginx docs state it plainly: closing
connections periodically is necessary to free per-connection memory allocations, and
setting the number too high is not recommended because of memory use.

The same page holds one more line, and to my mind it is the most misread sentence in
it: the keepalive directive does not limit the total number of connections an
nginx worker can open to the upstream. 32 is the number of idle connections kept
in the cache. If 300 concurrent requests arrive at peak, 300 connections are opened;
when the work is done at most 32 are kept and the rest are closed. "I set keepalive
32, so at most 32 connections reach my backend" is simply not true.

Worth noting too: the default for keepalive_requests was 100 before 1.19.10. If
you find 100 in an older article, the article is not wrong — just old.

Where the documentation contradicts itself

One thing stopped me mid-measurement: on 1.31.8 the backend never sees a Connection
header at all. Yet nginx's proxy_set_header page, as I read it today, still lists
the defaults like this:

proxy_set_header Host $proxy_host;
proxy_set_header Connection close;
Enter fullscreen mode Exit fullscreen mode

What is more, a few screens up the same page, under proxy_http_version, it says
"Since 1.29.7, version 1.1 is used by default". So the page knows about 1.29.7; the
proxy_set_header box does not. The page contradicts itself. The changelog says the
Connection proxy header is no longer sent by default; the default box says close.
Both cannot be true at once, so I went to a third witness — the source.
The default header table in ngx_http_proxy_module.c reads like this on 1.28.3:

{ ngx_string("Connection"), ngx_string("close") },
Enter fullscreen mode Exit fullscreen mode

and like this on 1.29.7:

{ ngx_string("Connection"), ngx_string("") },
Enter fullscreen mode Exit fullscreen mode

In nginx an empty value means "do not pass this header". In the same file the default
for proxy_http_version has moved from NGX_HTTP_VERSION_10 to
NGX_HTTP_VERSION_11. So the source and the changelog agree with each other, my
measurement is on their side, and the only dissenting witness is the default box on
a documentation page.

Lining up five tags shows the second line of that same box has gone stale too: the
Host value is $proxy_host on 1.28.3, $proxy_internal_host on 1.29.7 and 1.30.0,
and empty on 1.31.8. The $proxy_host the documentation prints exists in no version
today. One caveat while you are here: do not file Proxy-Connection under 1.29.7 —
it was added to that table in 1.31.0 and is not part of the 1.29.7 entry. My point here is not that nginx's docs are bad — you will
rarely find a server documented this well. The issue is that in a release where
defaults move, not every corner of the documentation moves at the same speed.
Collecting three witnesses is still the cheapest route.

So what about my blog

Back to the start. My server runs nginx 1.30.0, so the change is in there. But when
I looked at my configuration I found this: six separate locations, every one of
them with proxy_pass http://127.0.0.1:3040, and not a single upstream block
anywhere. Exactly the situation in v1.

Rather than infer from the version number, I put my server's exact version in the
lab. On nginx 1.30.0, 200 requests opened 200 connections without an upstream
block and 1 with one. So the change is present on my server; what is missing is a
configuration that reaches it.

A small sourcing note: do not look for the stable 1.30 branch in the main CHANGES
file — it holds no 1.30 entries at all. The stable branch keeps its own changelog in
CHANGES-1.30.

I also worked out why the figures came to double, because the thing listening on
:3040 is not the Node process:

LISTEN 0 65535 127.0.0.1:3040 0.0.0.0:* users:(("docker-proxy",pid=1091705,fd=8))
Enter fullscreen mode Exit fullscreen mode

The blog runs in a container, and the published port puts docker-proxy in the
middle. Two separate rows pile up in the TIME_WAIT list: one for
127.0.0.1:random -> 127.0.0.1:3040 (nginx to docker-proxy) and one for
172.31.0.1:random -> 172.31.0.2:3040 (docker-proxy to the container). So an
unpooled request costs me not one socket but two, and that is where 100 requests
becoming exactly 200 comes from. The extra +2 and +4 on the 50- and 20-request runs
I did not isolate; most likely it is real traffic arriving at the site, but I did not
measure that. So base the "two sockets per request" ratio on the 100-request run,
not on the short ones.

Diagram

The obvious question: does adding an upstream block help the second hop too, or
only collapse the first? Rather than guess, I rebuilt that same architecture in the
lab — ran the backend container with a published port so docker-proxy sat in the
middle, sent 100 requests, and counted the connections the container itself saw:

Proxy configuration Connections the container saw
proxy_pass http://host:18080 (no block) 100
upstream block (no directive written) 1

I ran it twice, with the same result twice. Pooling the first hop collapses the second
as well, which makes sense: docker-proxy pairs an inbound connection with the
connection it opens to the backend for that connection's lifetime — one inbound
connection means one outbound.

Here is where I owe you honesty: I have not applied this fix to the live server
yet. My measurement holds in the lab and on a copy of the blog's architecture;
I did not want to change a live configuration just to write an article. Six
locations on the blog point at the same backend, which drags local behaviour into
the picture too, and changing this config is a maintenance-window job in its own
right. If you are a step ahead of me as you read this, congratulations — I have put
it in the queue.

Writing the fix, and what it hits

The fix for everything described so far is three lines. Before:

location / {
    proxy_pass http://127.0.0.1:3040;
}
Enter fullscreen mode Exit fullscreen mode

After:

upstream blog_backend {
    server 127.0.0.1:3040;
}

location / {
    proxy_pass http://blog_backend;
}
Enter fullscreen mode Exit fullscreen mode

That is all of it — you do not need to write keepalive, because on a post-1.29.7
version the default takes over once the block exists. But know three things before
making that change.

First, name resolution. If your backend is a name rather than 127.0.0.1 —
app, web, a Docker service name — moving that name into an upstream block makes
nginx resolve it only at startup. When the container is recreated with a new IP,
nginx keeps talking to the old one; that is exactly the failure I wrote up on this
blog as
nginx's DNS trap.
With an IP you are fine; with a name, look into resolver and a variable
proxy_pass.

Second, the other side's timeout. A pool is a two-party agreement. If your
backend closes idle connections sooner than nginx expects, nginx will pull a
closing connection out of the pool, write a request onto it, and you will see
occasional 502s. The nginx-side default is 60 seconds; find out your backend's and
keep it shorter. This is the first failure anyone who has just enabled a pool meets.

Third, the local multiplication. Because the default pool is kept per
location, the idle-connection ceiling is number of locations × 32, and that is
per worker. For my six locations that means 192 idle connections per worker,
multiplied again by worker_processes. Written by hand as keepalive 32 — that is,
without local — they all share one pool and the ceiling stays at 32. That number is
also why I have not rushed the fix on the blog: whether to point six locations at one
shared pool or accept the default and raise the ceiling is not a decision to make
while writing an article.

There is a reverse direction too: installations that upgrade to 1.29.7 and already
have an upstream block get the pool switched on by itself. If that is not what
you want, the way out is keepalive 0, row v6.

Checklist

The order I followed, if you want to measure the same thing on your own proxy:

  1. Learn your version. nginx -v. Older than 1.29.7 and the classic three-line recipe is still mandatory; newer and it is already there — but keep reading the condition.
  2. Check whether an upstream block exists. grep -n "upstream \|proxy_pass" conf. If it goes straight to proxy_pass http://host:port, there is no pool whatever your version.
  3. Count, do not guess. If the backend is nginx, log $connection and count unique values. Otherwise look at the delta of ss -tanH state time-wait | awk '{print $4}' | grep -c ":<port>$" on the proxy side — but first see who is listening on that port with ss -tlnp, because if a proxy sits in between, the numbers multiply.
  4. Account for caching. My first attempt produced zero difference across 50 requests, because they came back from proxy_cache and an HTTPS redirect and never reached the backend at all. Pick a path that genuinely traverses the proxy.
  5. Choose keepalive_requests deliberately. The default is 1000. Lowering it improves memory behaviour but increases connection churn; raising it costs memory.
  6. Know that your old keepalive 32 line is no longer local. Its behaviour differs from the default; add local if that is not what you wanted.

Conclusion

What surprised me most while writing this is that the measurement did not lead me
somewhere wrong — it led me to my own configuration. The change nginx made is a good
one: turning a decade-old copy-paste recipe into the default protects the person who
never heard the recipe most of all. But whether a default actually reaches you is a
finer question than the default itself. My configuration was living on the assumption
"I wrote no setting, so I must be getting the default", and because of a
srv_conf == NULL check it never entered the default's field of view at all.

Put more generally: when a release note says "now the default", the unanswered
question is always default for whom. Learning that from a measurement rather than
from documentation — and confirming it with a single continue in the source — was
the best twenty minutes of my week. Ask your own server the same question; the answer
is probably one grep -c away.

There is another end of connection reuse — why closed connections wait 60 seconds
in TIME_WAIT, and which setting does not touch that — over in
the TIME_WAIT article;
if you are wondering where the sockets in this piece pile up, it is there. The story
of another nginx default misleading me is in
the config that sent 100 bytes of a 5000-byte file.

Official Sources

Top comments (2)

Collapse
 
akhourianmolkumar profile image
Akhouri Anmol Kumar •

bro you boosted my motivation, confidence, aura, flex, all thing by this :)
bro I really thought that I know you're busy person and I thought you will ignore the mail of invitation or you would say just like discord that you will join later but you joined thanks bro really so much thanks
and now I can proudly declare, MUSTAFA ERBAY is in my companyyyyyyyyyyyyy
AURA 📈📈📈🔥🔥

bro THANKS multiplied by googol times

Collapse
 
merbayerp profile image
Mustafa ERBAY •

Brooo! 😂❤️ You’re giving me way too much credit!

I’m just a simple guy hanging out in a quiet little corner of the team. 😄 Nothing special, no big titles, no big expectations.

Whenever I find some free time, I’ll do my best to contribute, share what I know, and lend a hand wherever I can.

The real credit belongs to you and your passion for building something meaningful. I’m just happy to be around and support the journey in my own small way.

Keep that amazing energy, brother! And please, don’t spend all your AURA on me! 😂🔥❤️