How Nginx Handles Thousands of Connections With One Thread

You've copied an nginx config from Stack Overflow. It works. You have no idea why.

Maybe you added a location block that broke something else. Maybe you spent an hour debugging a 502 that turned out to be a missing trailing slash on proxy_pass. Maybe you're just stacking directives you found in blog posts, hoping nothing conflicts.

That's fine. Everyone starts there. But the moment you need to debug a production routing issue at midnight, "it works, don't touch it" stops being a strategy. So let's actually understand what this thing does under the hood.

---

🧠 The process model

When you start nginx, you get one master process and several worker processes. That's it. No thread pools per request, no spawning child processes for each connection.

The master runs as root. It reads your config, binds to ports 80 and 443, and spawns workers. Then it basically sits there managing the lifecycle: starting new workers, gracefully shutting down old ones during config reloads, and restarting them if they crash.

Workers do all the real work. Every connection, every request, every byte of response goes through a worker. By default, worker_processes is set to 1, but basically everyone sets it to auto, which spawns one worker per CPU core.

# /etc/nginx/nginx.conf (top-level context)
worker_processes auto;
worker_rlimit_nofile 65535;

events {
    worker_connections 4096;
    multi_accept on;
}

Each worker handles connections independently. They don't share memory or coordinate on who takes which request. The OS kernel distributes incoming connections across the listening workers.

And here's where it gets interesting.

⚡ Why the event loop wins

Apache's traditional model (prefork) spawns one process per request. Its worker MPM uses one thread per request. Either way, you're paying for an OS-level context switch and a chunk of memory for every single connection sitting there waiting for data.

Nginx does something different. Each worker runs a single-threaded event loop. One thread, thousands of connections.

The worker registers all its sockets with the kernel using epoll (Linux) or kqueue (macOS/BSD). Then it waits. When any socket has data ready, the kernel tells the worker which ones. The worker processes those events, fires off responses, and goes back to waiting. Never blocks. Never sits idle consuming resources for a connection that isn't doing anything right now.

This is why nginx can handle 10,000+ concurrent connections on hardware where Apache would fall over. It was literally built to solve the C10K problem back in 2002.

But there's a catch. If anything in the event loop blocks, like reading a large file from a slow disk, the entire worker stalls. Every other connection on that worker just waits. That's why nginx added thread pools (aio threads;) to offload blocking disk I/O to a pool of 32 threads by default. The event loop stays responsive.

How config blocks map to requests

The config is hierarchical: http wraps server wraps location. Each level inherits from its parent and can override.

http {
    # applies to all virtual hosts
    gzip on;

    server {
        listen 80;
        server_name api.myapp.com;
        # this server block handles requests to api.myapp.com

        location /users {
            proxy_pass http://127.0.0.1:3000;
        }
        location /static {
            root /var/www;
        }
    }

    server {
        listen 80;
        server_name admin.myapp.com;
        # different hostname, different routing
    }
}

When a request arrives, nginx picks the right server block by matching the Host header against server_name. Priority: exact match first, then longest leading wildcard (*.example.com), then longest trailing wildcard (mail.*), then first matching regex. No match? Falls back to default_server.

Then it finds the right location block. This is where people get tripped up.

🎯 Location matching precedence

Location matching isn't first-match-wins. It has a specific priority order that ignores the sequence in your config file:

1. = /exact - Exact match. Stops immediately if matched.
2. ^~ /prefix - Longest prefix match with the "stop searching" modifier. Skips regex.
3. ~ /regex or ~* /regex - Regular expressions (case-sensitive and case-insensitive). First regex match in config order wins.
4. /prefix - Longest prefix match without the modifier. Used only if no regex matched.

So a request to /api/users/123 checks all prefix locations, finds the longest match, then checks regexes. If a regex matches, it wins over the prefix. Unless that prefix had ^~, which blocks regex from overriding it.

Confusing? Yeah. In my experience, most routing bugs come from people assuming locations are evaluated top-to-bottom. They're not.

The six jobs nginx actually does

Nginx wears a lot of hats. But they all boil down to six things:

Static file serving with root or alias, try_files for SPA fallbacks, and sendfile on for zero-copy delivery straight from kernel space. Fast.

Reverse proxying with proxy_pass. Takes a client request, forwards it to your app server, and returns the response. Your app never sees the internet directly. I've written more about what reverse proxies do and how they differ from forward proxies <!-- not published yet, author can decide whether to include -->.

Load balancing via upstream blocks. Round-robin by default, or least_conn, ip_hash, and weighted distribution. If you want the full breakdown of how these algorithms compare, I covered that separately.

TLS termination. Nginx handles the SSL/TLS handshake, decrypts traffic, and forwards plain HTTP internally. Your app servers don't need certificates or crypto overhead.

Caching with proxy_cache_path. Stores upstream responses on disk and serves them directly for repeat requests without hitting your backend.

Rate limiting using limit_req_zone and limit_req. Throttle by IP, by endpoint, whatever you need.

upstream backend {
    least_conn;
    server 10.0.0.1:8080 weight=3;
    server 10.0.0.2:8080;
    server 10.0.0.3:8080 backup;
}

server {
    listen 443 ssl;
    server_name app.example.com;
    ssl_certificate /etc/ssl/cert.pem;
    ssl_certificate_key /etc/ssl/key.pem;

    location / {
        proxy_pass http://backend;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

That's TLS termination, load balancing, and reverse proxying in 15 lines.

🛠️ Gotchas that bite everyone

These are the ones I see constantly. Bookmark this section.

The trailing slash on proxy_pass. This single character changes everything. proxy_pass http://backend; (no trailing slash) passes the full original URI. proxy_pass http://backend/; (with trailing slash) strips the matched location prefix. So if your location is /api/ and the request is /api/users, the first forwards /api/users, the second forwards /users. Mismatching this causes mystery 404s.

Missing proxy_set_header Host. Without it, your upstream sees the internal hostname (like 127.0.0.1:3000) instead of the original Host header. And without X-Forwarded-For, your app has no idea what the client's actual IP address is. Every proxy config needs these headers.

413 Request Entity Too Large. The default client_max_body_size is 1m. One megabyte. Any file upload over that gets rejected with a 413 before your app even sees it. Set it explicitly: client_max_body_size 50m; or 0 to disable the limit entirely.

502 versus 504. A 502 means nginx connected to your upstream but got garbage back (or the connection was refused). Your app probably crashed. A 504 means nginx waited for a response and gave up after proxy_read_timeout (default 60s). Your app is alive but slow. Different problems, different fixes.

Never skip nginx -t before reload. Always run nginx -t to test config syntax. Then nginx -s reload for a graceful reload where old workers finish their current requests. A full restart drops every active connection. I've seen teams push broken configs to production because they reloaded without testing first. Don't be that team.

---

📌 Takeaways

• Nginx's event-driven workers handle thousands of connections each without spawning threads per request
• Config hierarchy is http > server > location, with inheritance flowing down
• Location matching has a fixed priority order that ignores config file sequence
• Most production nginx issues come from trailing slashes, missing proxy headers, and the 1MB body size default
• Always nginx -t before you reload. Always.

---

If you're running nginx as a load balancer, the post on load balancing algorithms goes deeper on the strategies behind upstream blocks.

---

More from me

More posts on this kind of thing at arnavsharma.dev.