Boosting Microservices Performance with NGINX Keep-Alive and HTTP/2

Benefits You’ll Gain 🚀

By the end of this post, you’ll understand:

• What keep-alive connections are
• How NGINX implements keep-alive for clients and upstream servers
• How NGINX manages upstream keep-alive connections under the hood
• How to configure and monitor keep-alive in production
• How HTTP/2 multiplexing improves microservices performance

---

1. What is Keep-Alive?

A keep-alive connection is a TCP connection that stays open to handle multiple HTTP requests instead of creating a new connection for each request.

Without keep-alive:

Open connection → Send request → Receive response → Close connection
Open connection → Send next request → Receive response → Close connection

With keep-alive:

Open connection → Send multiple requests → Receive responses → Close connection only when done

Why it matters:

• Reduces CPU/memory overhead from repeated TCP/TLS handshakes
• Lowers latency
• Increases throughput under heavy traffic

Server/Client	Keep-Alive Support
NGINX	Yes, highly configurable
Apache HTTPD	Yes, via KeepAlive On
IIS	Enabled by default
Caddy / Lighttpd / HAProxy / Envoy	Yes
Browsers/Clients	Chrome, Firefox, Safari, Postman, cURL

---

2. Why NGINX Stands Out 🌟

NGINX is ideal for high-performance web apps because:

• Event-driven architecture → handles tens of thousands of connections efficiently.
• Upstream keep-alive pools → persistent connections to backend servers.
• Efficient load balancing → reused connections reduce backend load.
• Fine-grained configuration → control over timeouts, requests, and connection pools.

Other servers support keep-alive, but NGINX excels under high traffic and proxy scenarios.

---

3. NGINX Keep-Alive Implementation

3.1 Client ↔ NGINX (Browser Connections)

Browsers send: Connection: keep-alive. NGINX responds similarly, allowing multiple requests on one TCP connection.

Config example:

http {
    keepalive_timeout 65;      # Keep client connection open for 65 seconds
    keepalive_requests 1000;   # Max requests per connection on the client side
}

3.2 NGINX ↔ Upstream Server (Backend Connections)

This manages the connections between NGINX and your microservices.

upstream backend {
    server backend1.example.com;
    server backend2.example.com;
    keepalive 32;  # Number of idle connections to keep open *per worker process*
}

server {
    location /api/ {
        proxy_http_version 1.1;
        # Crucial for upstream keep-alive: removes the Connection header
        proxy_set_header Connection ""; 
        proxy_pass http://backend;
    }
}

---

4. How Upstream Keep-Alive Works 🛠️

• Connection Pooling: Idle connections maintained for reuse (keepalive 32).
• Request Assignment: NGINX picks an idle connection from the pool or opens a new one.
• Idle Connection Management: Controlled by proxy_keepalive_timeout (not shown, but good practice).
• Load Balancing Integration: Requests use pooled connections per backend.
• Resource Efficiency: Reduces CPU, memory, and TCP handshakes on the backend.

Visual Example (Simplified):

Browser
   |
   | Keep-Alive TCP connection (Client ↔ NGINX)
   |
NGINX Worker
   |---------------------------------------|
   | Idle Upstream Pool (keepalive 32)     |
   | connection 1 (idle) -> backend1       --> backend server 1
   | connection 2 (in use) -> backend2     --> backend server 2
   | connection 3 (idle) -> backend1       --> backend server 1
   |---------------------------------------|

---

5. Real-World Example

A React SPA triggers 20 API calls for 1,000 concurrent users:

• Without keep-alive: 20,000 TCP connections opened/closed. High latency and resource usage.
• With keep-alive: NGINX reuses a small pool of connections. Low latency and low backend CPU usage.

---

6. Multiple Subdomains and TCP Connections

For microservices like:

• api.example.com → Main API
• user.example.com → User management
• cart.example.com → Cart

Key points:

• Browsers open separate TCP connections per subdomain (due to security and standard behavior).
• Connections are reused per subdomain if still alive.

Sequence Example:

1. Call 1 → user.example.com (New TCP #1 established)
2. Call 2 → cart.example.com (New TCP #2 established)
3. Call 3 → user.example.com (NGINX/Browser Reuse TCP #1)

Only 2 connections created instead of 3, but the subdomain boundary is still respected.

---

7. HTTP/2 Multiplexing ⚡️

HTTP/2 allows multiple requests/responses over one single TCP connection (multiplexing).

• Works best with wildcard or SAN certificates covering multiple subdomains.
• Reduces head-of-line blocking and repeated handshakes.
• Requests/responses are interleaved, improving latency and throughput.

Enable HTTP/2 in NGINX:

server {
    listen 443 ssl http2;
    server_name api.example.com;

    ssl_certificate /etc/ssl/certs/example.com.crt;
    ssl_certificate_key /etc/ssl/private/example.com.key;

    location / {
        proxy_pass http://api_backend;
        proxy_http_version 1.1;
        proxy_set_header Connection ""; # Crucial for upstream keep-alive
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

Flow Example (HTTP/2):

Browser single TCP connection (HTTPS + HTTP/2)
|
|-- Request: api.example.com/data (Stream 1)
|-- Request: user.example.com/profile (Stream 2)
|-- Request: cart.example.com/items (Stream 3)
|
NGINX routes requests to the correct upstream
|
Responses return over same TCP connection, interleaved

---

8. Best Practices ✅

Component	Setting	Recommended Value	Monitoring Command
Client Keep-Alive	keepalive_timeout	30–75s
	keepalive_requests	500–1000
Upstream Keep-Alive	keepalive	20–64 idle connections (per worker)	`ss -tan
Protocol		Use HTTP/1.1 or HTTP/2

• Use HTTP/2 + wildcard/SAN certificates for multi-subdomain multiplexing.
• Avoid keeping idle connections indefinitely to prevent resource exhaustion.

---

9. Practical Benefits

• Reduced Latency: Fewer TCP/TLS handshakes.
• Lower Backend Load: Connection reuse minimizes resource consumption.
• Better Scalability: Backend servers handle more requests efficiently.
• Simplified Microservice Routing: NGINX manages subdomain traffic seamlessly.

---

Conclusion

Leveraging NGINX keep-alive connections and HTTP/2 multiplexing is a powerful way to optimize microservices performance. Proper connection management reduces latency, conserves resources, and ensures your applications scale efficiently under heavy traffic.
{% raw %}