DEV Community

Cover image for Zero-Trust Reverse Proxy Hardening in Nginx: Rate Limiting, Request Smuggling Guards & mTLS
Muhammad Ali Mughal
Muhammad Ali Mughal

Posted on

Zero-Trust Reverse Proxy Hardening in Nginx: Rate Limiting, Request Smuggling Guards & mTLS

#security #nginx #webdev #devops

The most dangerous Nginx config I review is not the broken one; it is the one that works. It proxies to Laravel, gets an A on a TLS scanner, and quietly trusts everything: any X-Forwarded-For header a client sends, any certificate a public CA signed, any request rate the login endpoint receives, and any body framing the upstream is willing to guess at. Nothing in the error log hints at a problem until somebody else finds one.

I wrote this with help from AI tools and checked every directive and claim myself against the official Nginx documentation and changelog. I build PixelValut's developer and network tools, and I have run Nginx in front of PHP, Node.js and .NET services for nine years. This final part of the series is the hardening pass I apply to a reverse proxy before I call it zero-trust: identify the real client, limit what it can do, refuse ambiguous requests, and make internal hops prove who they are.

📚 Series Roadmap — Browser-Side Security & Zero-Trust Forensics

How do you harden an Nginx reverse proxy for zero-trust?

Direct answer: Trust client IP headers only from your own load balancer ranges, then rate-limit per IP and per API key with limit_req, using stricter zones for login. Run a current Nginx release, keep request buffering on and timeouts short so ambiguous bodies are rejected, and require mTLS with a private CA for partner and service-to-service traffic.

Scope note: run the test commands below only against servers you own or are authorized to test.

Step 1: Know who the client really is

Every IP-based control depends on $remote_addr. Behind a load balancer or CDN, that is the balancer's address, so every user shares one rate-limit bucket. The usual fix, trusting X-Forwarded-For, is also the usual hole: if you trust it from anywhere, any client can claim any IP.

# http {} — trust forwarding headers ONLY from your own proxy tier
set_real_ip_from 10.20.0.0/16;      # load balancer subnet
real_ip_header   X-Forwarded-For;
real_ip_recursive on;               # skip trusted hops, take the first untrusted one
Enter fullscreen mode Exit fullscreen mode

At the edge, overwrite rather than append what you forward, so the upstream never sees a client-supplied value:

proxy_set_header X-Forwarded-For   $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Real-IP         $remote_addr;
Enter fullscreen mode Exit fullscreen mode

And refuse traffic for hostnames you do not serve, instead of letting the first server block answer it (the dangling-record problem from Part 5 is worse when your default server serves real content):

server {
    listen 443 ssl default_server;
    ssl_reject_handshake on;        # no certificate needed, handshake is refused
}
Enter fullscreen mode Exit fullscreen mode

A row of steel sluice gates on a concrete dam with only two gates open releasing controlled streams of water, representing rate limiting at a reverse proxy, AI-generated
AI-generated image.

Step 2: Rate limiting that matches how clients behave

limit_req implements a leaky bucket per key. A request whose key is an empty string is not counted, which lets you select a zone with a map:

# http {}
map $http_x_api_key $key_bucket   { ""  "";                 default $http_x_api_key; }
map $http_x_api_key $anon_bucket  { ""  $binary_remote_addr; default ""; }

limit_req_zone $binary_remote_addr zone=ip_ceiling:10m rate=30r/s;
limit_req_zone $anon_bucket        zone=anon:10m       rate=5r/s;
limit_req_zone $key_bucket         zone=per_key:20m    rate=50r/s;
limit_req_zone $binary_remote_addr zone=login:10m      rate=5r/m;
limit_conn_zone $binary_remote_addr zone=conn_ip:10m;

limit_req_status  429;
limit_conn_status 429;
limit_req_log_level warn;
Enter fullscreen mode Exit fullscreen mode 
# server {}
location /api/ {
    limit_req  zone=ip_ceiling burst=60 nodelay;
    limit_req  zone=anon       burst=10 nodelay;
    limit_req  zone=per_key    burst=100 delay=50;
    limit_conn conn_ip 20;
    proxy_pass http://app_upstream;
}

location = /login {
    limit_req zone=login burst=5 nodelay;
    proxy_pass http://app_upstream;
}
Enter fullscreen mode Exit fullscreen mode

Why three zones on /api/: an anonymous client gets a tight bucket, a keyed client gets a bigger one, and the IP ceiling always applies, because an attacker can send random X-Api-Key values to escape the anonymous zone. Nginx checks keys only for counting, not validity, so the ceiling is what stops that trick. Every listed zone is checked: the request is rejected if any zone is exceeded, and delayed by the longest delay any zone asks for.

From the docs: one megabyte of zone holds about 16,000 states for 64-byte entries such as $binary_remote_addr; when a zone fills, Nginx evicts old entries and, if it still cannot create a state, returns the error status. API keys are longer than IP addresses, so give that zone more room. Before enforcing, ship with limit_req_dry_run on; for a few days and read the log lines it produces; real traffic is burstier than anyone expects, especially webhook senders retrying in batches (see Part 2).

The Nginx config generator gives me a clean baseline server block to start from; the zones above are what I add on top.

Step 3: Request smuggling guards

Request smuggling happens when two HTTP processors disagree about where one request ends, usually because of conflicting Content-Length and Transfer-Encoding headers. The front proxy forwards one request; the backend sees two, and the second rides someone else's connection.

Modern Nginx closes many of the classic gaps itself. From the official changelog:

  • 1.21.1: always returns an error if both Content-Length and Transfer-Encoding are present, and for spaces or control characters in the request line, header names or Host.
  • 1.21.2: rejects HTTP/1.0 requests with Transfer-Encoding.
  • 1.29.4: a single LF as a line terminator in a chunked body is an error.
  • 1.31.0: rejects HTTP/2 and HTTP/3 requests carrying Connection, Transfer-Encoding, Upgrade and similar headers, which closes off a class of downgrade tricks.
  • 1.31.7 (October 2026): fixes CVE-2026-95101, where with HTTP/3 part of an unknown frame could be treated as request body, bypassing client_max_body_size or injecting data into the proxied request.

Rule one: run a supported release and check nginx -v on every box, including the forgotten one. Then keep the helpful defaults and tighten the rest:

# http {}
server_tokens off;
ignore_invalid_headers on;          # default; never turn off at the edge
underscores_in_headers off;         # default
client_header_timeout 10s;
client_body_timeout   10s;
send_timeout          10s;
keepalive_timeout     15s;
client_max_body_size  2m;           # raise per location for uploads
large_client_header_buffers 4 8k;

# location {} proxying to the app
proxy_request_buffering on;         # default: read the whole body first
proxy_http_version 1.1;             # default since 1.29.7; explicit for older builds
proxy_set_header Connection "";
Enter fullscreen mode Exit fullscreen mode

proxy_request_buffering on matters: Nginx reads the full client body before talking to the upstream, so the backend receives a body with a definite length instead of whatever framing the client chose. Turn it off only for specific streaming endpoints, and then make sure the upstream parses HTTP strictly.

The remaining risk is usually a different hop. If a CDN or a second proxy sits in front of Nginx, the disagreement can happen between them. Test the chain you actually run:

printf 'POST /api/ping HTTP/1.1\r\nHost: api.example.com\r\nContent-Length: 4\r\nTransfer-Encoding: chunked\r\n\r\n0\r\n\r\n' \
  | openssl s_client -quiet -connect api.example.com:443 -servername api.example.com 2>/dev/null \
  | head -n1
# Expected from Nginx directly: HTTP/1.1 400 Bad Request
Enter fullscreen mode Exit fullscreen mode

If that returns anything other than a 400 through your CDN, find out which hop accepted it.

Step 4: mTLS for partners and internal hops

TLS proves the server to the client. Mutual TLS also makes the client prove itself with a certificate, which is the zero-trust answer for partner APIs, admin panels and service-to-service calls.

map $ssl_client_s_dn $partner_ok {
    default                         0;
    "CN=acme-billing,O=Acme Ltd"    1;
}

server {
    listen 8443 ssl;                     # dedicated listener for mTLS
    http2 on;
    server_name partners.example.com;

    ssl_certificate     /etc/nginx/pki/partners.example.com.fullchain.pem;
    ssl_certificate_key /etc/nginx/pki/partners.example.com.key;
    ssl_protocols TLSv1.2 TLSv1.3;

    ssl_client_certificate /etc/nginx/pki/partner-clients-ca.pem;  # PRIVATE CA only
    ssl_verify_client on;
    ssl_verify_depth  2;
    ssl_crl /etc/nginx/pki/partner-clients-ca.crl;

    location / {
        if ($partner_ok = 0) { return 403; }

        proxy_set_header X-Client-Verify      $ssl_client_verify;
        proxy_set_header X-Client-DN          $ssl_client_s_dn;
        proxy_set_header X-Client-Fingerprint $ssl_client_fingerprint;

        proxy_pass https://partners_api;
        proxy_ssl_certificate         /etc/nginx/pki/edge-to-api.pem;
        proxy_ssl_certificate_key     /etc/nginx/pki/edge-to-api.key;
        proxy_ssl_trusted_certificate /etc/nginx/pki/internal-ca.pem;
        proxy_ssl_verify       on;
        proxy_ssl_verify_depth 2;
        proxy_ssl_server_name  on;
        proxy_ssl_name         partners-api.internal;
    }
}
Enter fullscreen mode Exit fullscreen mode

The details that decide whether this is real security:

  • Trust a private CA, never a public bundle. If ssl_client_certificate points at the system CA store, anyone with a publicly issued certificate passes verification.
  • Verification is not authorization. A valid certificate proves it came from your CA; the map on $ssl_client_s_dn (or the fingerprint) decides which client may use this host.
  • Overwrite identity headers. proxy_set_header replaces any X-Client-* value the client sent; the upstream must accept those headers only from Nginx's network.
  • Verify the upstream too. proxy_ssl_verify is off by default, so without it the edge happily talks TLS to any server answering on that address.
  • Isolate mTLS virtual servers. CVE-2025-23419, fixed in 1.27.4, let a TLS 1.3 session from one virtual server be resumed on another sharing the listener, bypassing client certificate checks. A dedicated port or IP plus a current release removes that class of problem.

Test both paths:

curl -sS https://partners.example.com:8443/health                     # expect 400, no cert
curl -sS --cert acme.pem --key acme.key https://partners.example.com:8443/health   # expect 200
Enter fullscreen mode Exit fullscreen mode

For the public listener, the SSL Checker confirms the chain, SAN list and expiry, and the HTTP Status Checker shows which status code and redirects a hostname really returns.

Control comparison

Control Key directives Threat addressed Failure mode if misconfigured How I verify
Real client IP set_real_ip_from, real_ip_header Shared buckets, spoofed IPs Trusting X-Forwarded-For from anyone Send a fake header from outside; log shows your real IP
Rate limiting limit_req_zone, limit_req, limit_conn Credential stuffing, scraping, cost abuse Keying on a client-controlled value with no IP ceiling Parallel curl loop; count 429s
Smuggling guards Current release, proxy_request_buffering, timeouts Desync, request hijacking Old Nginx, buffering off, lax hop in front CL+TE probe returns 400 end-to-end
mTLS to clients ssl_verify_client, ssl_client_certificate Unauthenticated partner or admin access Public CA bundle, no DN or fingerprint allowlist Request without cert fails; wrong cert gets 403
mTLS to upstream proxy_ssl_verify, proxy_ssl_certificate Rogue internal service, MITM proxy_ssl_verify left off Point upstream at a self-signed server; request fails
Default server ssl_reject_handshake Host header abuse, stray hostnames Real app as implicit default openssl s_client -servername random.example.com fails

Diagnostic checklist

  1. Is set_real_ip_from limited to your own proxy ranges?
  2. Does the edge overwrite X-Forwarded-For and identity headers?
  3. Does a default server reject unknown hostnames?
  4. Do rate limits include an IP ceiling that client-supplied keys cannot escape?
  5. Is the login endpoint in its own strict zone?
  6. Did you run limit_req_dry_run before enforcing?
  7. Is every Nginx instance on a supported, patched release?
  8. Is request buffering on except for named streaming locations?
  9. Does a CL+TE probe return 400 through the full chain?
  10. Do mTLS hosts trust only a private CA, allowlist clients and sit on a dedicated listener?
  11. Is proxy_ssl_verify on set for every TLS upstream?

FAQ

Should rate limiting live in Nginx or the application?

Both. Nginx is cheap and stops floods before PHP or Node.js wakes up. The application knows the user, plan and tenant, so business quotas belong there.

Does limit_req work across several Nginx servers?

No. Zones live in each instance's shared memory. Behind N proxies, a client effectively gets up to N times the configured rate unless the balancer keeps them sticky. For a global limit, use a shared store at the application or gateway layer.

Does HTTP/2 end request smuggling?

Not by itself. Downgrading HTTP/2 to HTTP/1.1 toward the backend has its own history of issues, which is why recent Nginx releases reject connection-specific headers in HTTP/2 and HTTP/3. Keep the upstream strict and the proxy current.

How do I rotate client certificates without downtime?

Put both the old and new CA certificates in the ssl_client_certificate file, issue new client certificates, reload Nginx, then remove the old CA after clients move. Revoke individual certificates through the CRL.

Is mTLS a replacement for API keys or OAuth?

No. It authenticates the machine connection. User-level authorization, scopes and audience checks from Part 3 still apply on top.

Series wrap-up

That closes Browser-Side Security & Zero-Trust Forensics. Over six parts we decoded obfuscated payloads, verified webhook signatures against replay, audited OIDC tokens and PKCE, found catastrophic regex backtracking, hunted dangling DNS records, and finished at the proxy that every request passes through. The pattern was the same each time: never trust input because of where it seems to come from, verify it explicitly, and test the configuration you actually run.

Follow & Connect

Top comments (2)

Collapse
 
mohsen_jr profile image
Mohsen jr •

One thing worth emphasizing is how easy it is to get rate limiting wrong behind a CDN or load balancer. If every visitor shares the same proxy IP, legitimate users can end up blocking each other. Testing the real client IP chain before enforcing limits is just as important as choosing the rate itself.

Collapse
 
muhammadalimughal profile image
Muhammad Ali Mughal •

Great point, Mohsen, and it's the mistake I see most often. If nginx sits behind a CDN or load balancer, $binary_remote_addr is the proxy's IP, so one noisy client can throttle everyone behind it. The fix is the realip module: set_real_ip_from for only your CDN/LB ranges, real_ip_header set to the header your provider uses (e.g. CF-Connecting-IP or X-Forwarded-For), and real_ip_recursive on. Never trust that header from arbitrary sources, or clients can spoof their way past the limit. Before enforcing, I log $remote_addr next to $realip_remote_addr for a day and run with limit_req_dry_run on to confirm the keys look like real clients. Thanks for adding this!