Nginx Reverse Proxy & Load Balancing: SSL Termination, Caching, and Tuning

Nginx sits at the edge of the internet, handling billions of requests daily. It is legendary for its asynchronous event-driven architecture that can handle 50,000 concurrent connections on minimal hardware.

Yet, many developers deploy Nginx with stock configuration files, missing out on massive throughput gains, security hardening, and caching advantages.

In this guide, we walk through a battle-tested, production-ready Nginx configuration for high-traffic API workloads.

1. High-Performance Core Tuning (nginx.conf)

Tuning begins in the global context:

# Auto-detect CPU cores to spawn matching worker processes
worker_processes auto;

# Maximum number of open files per worker
worker_rlimit_nofile 65535;

events {
    # Efficient epoll event model on Linux
    use epoll;

    # Max connections per worker process
    worker_connections 8192;

    # Accept multiple connections in one notification
    multi_accept on;
}

http {
    # Optimize zero-copy kernel disk streaming
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;

    # Keepalive timeouts
    keepalive_timeout 65;
    client_body_timeout 15;
    send_timeout 15;

    # Gzip Compression
    gzip on;
    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 5;
    gzip_types text/plain text/css application/json application/javascript text/xml application/xml;

    include /etc/nginx/conf.d/*.conf;
}

2. Upstream Load Balancing with Keepalive

One of the most common Nginx misconfigurations is creating a new TCP connection to the upstream application on every single incoming HTTP request.

By defining keepalive inside the upstream block, Nginx maintains a persistent pool of TCP connections to your backend:

upstream backend_api {
    # Least connection algorithm routes to least busy server
    least_conn;

    server 127.0.0.1:8001 max_fails=3 fail_timeout=10s;
    server 127.0.0.1:8002 max_fails=3 fail_timeout=10s;

    # Keep up to 64 idle connections open to each backend!
    keepalive 64;
}

server {
    listen 443 ssl http2;
    server_name api.example.com;

    ssl_certificate /etc/letsencrypt/live/api.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/api.example.com/privkey.pem;

    location / {
        proxy_pass http://backend_api;

        # REQUIRED for upstream keepalive:
        proxy_http_version 1.1;
        proxy_set_header Connection "";

        # Standard forward headers
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

3. Micro-Caching for Dynamic Endpoints

What if an endpoint receives 500 requests per second for data that changes only every 10 seconds (e.g. trending products, leaderboard, or weather)?

With Nginx Micro-Caching, you can cache dynamic API responses for just 2 seconds. This drops backend database queries by 99% during traffic spikes while keeping data virtually real-time!

# Declare cache zone in http block
proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=api_cache:10m max_size=1g inactive=60m;

location /api/v1/trending {
    proxy_pass http://backend_api;
    proxy_cache api_cache;

    # Micro-cache 200 responses for 2 seconds
    proxy_cache_valid 200 2s;

    # Allow serving stale response if backend is temporarily erroring
    proxy_cache_use_stale error timeout updating http_500 http_502;

    # Tell client cache hit status
    add_header X-Cache-Status $upstream_cache_status;
}

4. Edge Rate Limiting and Security Headers

Stop scrapers and abusive bots directly at Nginx before they consume your application threads:

# Limit zone: 10MB memory can track ~160,000 unique IP addresses
limit_req_zone $binary_remote_addr zone=api_limit:10m rate=30r/s;

server {
    # Apply rate limiting with burst buffer
    location /api/ {
        limit_req zone=api_limit burst=20 nodelay;
        proxy_pass http://backend_api;
    }

    # Harden security headers (A+ grade on SecurityHeaders.com)
    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header X-XSS-Protection "1; mode=block" always;
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
}

5. Buffer Tuning: Preventing Slow Disk Spilling

By default, if an upstream response exceeds small in-memory buffers, Nginx buffers the response to temporary files on disk (/var/lib/nginx/tmp/proxy). Disk I/O during heavy loads ruins API response times.

Tune client and proxy buffers in memory:

client_body_buffer_size 128k;
client_max_body_size 10m;
proxy_buffer_size 16k;
proxy_buffers 8 64k;
proxy_busy_buffers_size 128k;

With these 5 optimizations in place, Nginx acts as a rock-solid, ultra-fast gateway protecting your downstream services from memory exhaustion and traffic surges.