DEV Community

Oleksandr Kuryzhev
Oleksandr Kuryzhev

Posted on Originally published at kuryzhev.cloud

Nginx Rate Limiting Checklist for Public APIs

#devops

Originally published on kuryzhev.cloud

A public API goes live behind Nginx. A week later, one client starts retrying a failing call in a tight loop while a scraper walks every ID. Nginx rate limiting was added "later" as a single limit_req line, and the upstream still falls over because the line keys on the wrong address or never applies to the location that matters. This checklist covers the settings that decide whether limiting protects the API or only looks like it does.

Why this checklist matters for Nginx rate limiting

Rate limiting in Nginx is a few directives, so it is easy to believe it is finished once the config reloads. The failure modes are quiet. A zone keyed on the wrong variable throttles a whole CDN instead of one client, and an empty key disables limiting for that request. A limit set at the server level can vanish when a location adds its own limit_req.

Nginx implements limiting with a leaky-bucket model. limit_req controls request rate, and limit_conn controls concurrent connections. Both are cheap and run before the request reaches your application. Neither knows who your users are, whether a token is valid, or how expensive an endpoint is. Treat them as a first layer, not a complete abuse strategy.

The list below runs roughly from foundational settings to refinements. Behavior described here follows the official limit_req module documentation. Some options depend on the Nginx version: delay= requires 1.15.7, and limit_req_dry_run requires 1.17.1. Verify them against the docs for your build. More operations-focused write-ups are on the kuryzhev.cloud home page.

The checklist

  1. Decide the identity first. Per-IP is the default fallback. Per-API-key or per-account is better for authenticated traffic. Many clients share one NAT address, so IP-only limits punish offices and mobile carriers.
  2. Restore the real client IP. Behind a load balancer or CDN, $remote_addr is the proxy. Configure set_real_ip_from and real_ip_header from the realip module, and trust only addresses you control.
  3. Use $binary_remote_addr as the key. It is more compact than $remote_addr, which matters for zone sizing. According to the documentation, a stored state takes 64 bytes on 32-bit platforms and 128 bytes on 64-bit platforms. One megabyte therefore holds about 16,000 or about 8,000 states. Longer string keys, such as API keys, can use more.
  4. Size the zone for peak distinct clients, not average. When the zone is full, Nginx removes the least recently used state. If it still cannot create a new state, it rejects the request. Plan for attacks that rotate source addresses.
  5. Set rates per endpoint class. Login, token issuance, search, and bulk export deserve separate zones with different limits from cheap reads.
  6. Choose burst deliberately. burst sets a queue size. Without nodelay or delay=, queued requests are delayed to match the rate, which ties up client connections.
  7. Return 429, not 503. Set limit_req_status 429 and limit_conn_status 429. The default is 503, which monitoring and client retry logic read as a server fault.
  8. Send Retry-After. Well-behaved clients back off when told how long. Use a named error location so the header and a JSON body are consistent.
  9. Add limit_conn for expensive streams. Downloads, long polls, and uploads cost connections, not request count.
  10. Tighten timeouts and sizes. Review client_header_timeout, client_body_timeout, send_timeout, keepalive_timeout, and client_max_body_size so slow clients cannot hold sockets indefinitely.
  11. Protect the upstream. Cap proxy_read_timeout and limit proxy_next_upstream_tries, so a rejected or slow call is not retried into a storm.
  12. Test in dry-run first. limit_req_dry_run on (1.17.1 and later) logs what would be rejected without rejecting it.

This is the http-level setup. It keys on the API key when present and falls back to the IP. It also adds a strict per-IP zone for authentication endpoints and exempts trusted internal ranges from every zone.

# http { } context
# Internal monitors and health checks are exempt from all zones (empty key).
geo $is_trusted {
    default      0;
    10.0.0.0/8   1;
}

# Fall back to client IP when no API key header is sent.
map $http_x_api_key $api_client_raw {
    default $http_x_api_key;
    ""      $binary_remote_addr;
}
map $is_trusted $api_client {
    0 $api_client_raw;
    1 "";
}
map $is_trusted $ip_key {
    0 $binary_remote_addr;
    1 "";
}

limit_req_zone  $api_client zone=api_key:20m rate=20r/s;
limit_req_zone  $ip_key     zone=api_ip:20m  rate=50r/s;
limit_req_zone  $ip_key     zone=auth_ip:10m rate=10r/m;  # per minute for fractional rates
limit_conn_zone $ip_key     zone=conn_ip:10m;

limit_req_status  429;
limit_conn_status 429;
limit_req_log_level warn;

upstream api_backend {
    server 127.0.0.1:8080;   # placeholder backend
}

And the server block that applies the zones and answers rejections with a useful body:

server {
    listen 443 ssl;
    server_name api.example.com;
    ssl_certificate     /etc/nginx/tls/example.crt;   # placeholder paths
    ssl_certificate_key /etc/nginx/tls/example.key;

    error_page 429 = @rate_limited;

    location /v1/ {
        limit_req  zone=api_key burst=40 nodelay;  # absorb short spikes, no queue delay
        limit_req  zone=api_ip  burst=100 nodelay; # exceeding either zone rejects the request
        limit_conn conn_ip 20;
        proxy_pass http://api_backend;
    }

    location = /v1/auth/token {
        limit_req zone=auth_ip burst=5 nodelay;
        proxy_pass http://api_backend;
    }

    location @rate_limited {
        default_type application/json;
        add_header Retry-After 5 always;           # 'always' is needed on error responses
        return 429 '{"error":"rate_limited","retry_after":5}';
    }
}

Commonly missed items

Watch out for empty keys. The documentation states that requests with an empty key value are not accounted. That is useful for allow-lists, as above. It becomes a hole when a variable you expected to be populated is not. A key built from a header that clients can omit, or from a variable that is only set after authentication, silently disables the limit.

Watch out for unauthenticated key headers. In the example, a client can send a fresh random X-Api-Key on every request and get a fresh bucket each time. That is why the per-IP zone sits alongside it. Only key on identity that Nginx has actually validated, for example through auth_request or a signed token check. Otherwise, keep the IP zone as the backstop.

Inheritance surprises. limit_req directives are inherited from the parent level only if the current level defines none of its own. Adding one limit_req in a location can drop the server-level limits for that location. Run nginx -T and read the effective config per location.

Per-instance state. Zones live in shared memory on one host. With several Nginx nodes behind a balancer, each node enforces its own limit. A single client can therefore get up to roughly the limit times the node count, depending on how the balancer spreads its requests. If you need a true global quota, that belongs in an API gateway or a shared store.

IPv6. Attackers with a /64 can rotate through enormous numbers of addresses, and per-address limits then do little. Aggregating the key by prefix needs a regex map on the address, so check how your traffic looks before relying on IP keys.

Retries from your own clients. SDKs that retry on 429 without jitter, or without honoring Retry-After, turn a limit into a synchronized wave. Document the header in your API reference.

Automation ideas

Keep limits in version control and treat them as code. Generate the limit_req_zone lines from a small YAML file of endpoint classes and rates, and render them with your configuration tool. Run nginx -t in CI before any deploy. A failing syntax check is cheap to catch there and expensive on a production reload.

Roll out new limits in stages. Enable limit_req_dry_run on where your version supports it (1.17.1 and later). Watch the log for what would have been rejected, then switch dry-run off.

Add a post-deploy smoke test that confirms the limit actually fires. A parallel burst against a non-production endpoint should return some 429 responses that carry a Retry-After header. Run it from an address outside the trusted range, or the exemption will hide the limit.

#!/usr/bin/env bash
# Fire a parallel burst; require 429s that carry Retry-After.
# %header{} in -w needs curl 7.84.0 or later.
URL="https://staging-api.example.com/v1/ping"
results=$(seq 1 200 | xargs -P 20 -I{} \
  curl -s -o /dev/null -w '%{http_code} %header{retry-after}\n' "$URL")
echo "$results" | sort | uniq -c
echo "$results" | grep -q '^429' || { echo "FAIL: no 429 observed"; exit 1; }
echo "$results" | grep -Eq '^429 [0-9]+' || { echo "FAIL: 429 without Retry-After"; exit 1; }

For monitoring, use a JSON log_format that includes $status, $limit_req_status (1.17.6 and later), the client key, and the upstream response time. Export the 429 rate per endpoint to your metrics system and alert on sudden changes in either direction. A spike means abuse or a misbehaving client. A drop to zero after a deploy may mean the limit was lost to inheritance.

Finally, review the numbers periodically. Limits chosen at launch rarely match real usage, and the useful signal is which legitimate clients touch them. Adjust zones from the logs, not from guesses.

Related

Top comments (0)