Rate limiters are supposed to shield downstream services from traffic spikes. But what happens to the gateway itself when hit with tens of thousands of unauthorized requests per second?
In this benchmark study, I evaluate an API Rate Limiter Gateway implemented across three distinct proxy engines: Nginx, Python (aiohttp), and Rust (Axum/Tokio). All three proxies enforce a Leaky Bucket algorithm variant to shield downstream Flask applications served by Gunicorn.
The complete code, benchmark scripts, and Docker configurations are available on the GitHub repo.
1. System Architecture Overview
The testing setup simulates a production cloud deployment using Docker Compose:
-
Traffic Generator:
wrkload-testing tool executing multi-threaded concurrent connections. - Proxy Gateway Layer: Alternates between Nginx, Python, and Rust implementations.
- Downstream Services: Multi-worker Python Flask applications managed by Gunicorn.
2. Performance Comparison
Load generation was running for 30 seconds using wrk with 2 threads and 100 persistent concurrent connections (wrk -t2 -c100 -d30s --latency). Resource utilization was sampled directly via the Docker API (docker stats).
The benchmarks evaluate performance across three traffic profiles:
- Scenario 1 (Baseline / No Limit): All incoming requests are allowed through to the downstream Flask apps.
- Scenario 2 (High Rate-Limit): A moderate rate-limit threshold where the gateway rejects the majority of incoming traffic.
- Scenario 3 (Strict Rate-Limit): A highly restrictive rate-limit threshold triggering immediate rejection for nearly ~100% of incoming flood traffic.
| Scenario / Engine | Avg Latency | p99 Latency | Throughput (RPS) | Peak CPU % | Peak RAM | Non-2xx Ratio |
|---|---|---|---|---|---|---|
| S1: Baseline (No Limit) | ||||||
| Nginx | 108.49ms | 126.28ms | 918.69 | 322.27% | 20.25MiB | 0.26% |
| Python | 128.88ms | 170.39ms | 773.15 | 105.58% | 38.34MiB | 0.00% |
| Rust | 64.29ms | 85.24ms | 1,547.96 | 882.31% | 15.14MiB | 0.00% |
| S2: High Rate-Limit | ||||||
| Nginx | 1.77ms | 4.50ms | 1,220.85 | 35.17% | 12.17MiB | 97.85% |
| Python | 13.92ms | 21.29ms | 7,300.03 | 108.50% | 28.40MiB | 99.92% |
| Rust | 2.72ms | 3.05ms | 37,187.73 | 226.38% | 10.60MiB | 99.99% |
| S3: Strict Rate-Limit | ||||||
| Nginx | 3.30ms | 9.48ms | 1,267.92 | 26.96% | 10.86MiB | 99.89% |
| Python | 13.21ms | 20.01ms | 7,568.54 | 103.06% | 25.98MiB | 100.00% |
| Rust | 2.75ms | 3.18ms | 36,538.96 | 228.07% | 9.82MiB | 100.00% |
3. Architectural Insights & Bottlenecks
Multi-Core Efficiency vs. The Thread Barrier
Rust leverages a multi-threaded, work-stealing execution framework (tokio), allowing it to dynamically distribute network polling across all allocated host CPU threads (spiking to 882.31% CPU in Scenario 1 and 228.07% in Scenario 3). Conversely, Python relies on a single-threaded asynchronous loop constrained by the Global Interpreter Lock (GIL), capping compute scaling to roughly 105% CPU (one physical core) regardless of incoming pressure.
Rate-Limit Short-Circuiting
When active enforcement triggers in Scenarios 2 and 3, average processing latencies across all engines drop to low single digits. This occurs because the proxies reject unauthorized traffic in-memory via an immediate HTTP 429 response. This gateway short-circuiting bypasses downstream payload processing, Gunicorn socket allocation, and Flask app execution entirely, preventing resource saturation on the underlying services.
Throttled Throughput Analysis
The throughput variance in Scenarios 2 and 3 (~36,500+ RPS for Rust, ~7,500 RPS for Python, and ~1,200 RPS for Nginx) is determined by specific runtime boundaries under load.
- Python's throughput is capped by single-core execution due to the GIL.
-
Nginx's performance drop stems from connection slot exhaustion (
worker_connectionssaturation) or socket allocation overhead under intense traffic spikes. - Rust circumvents these limitations by utilizing a native, multi-threaded async runtime to evaluate in-memory limits and drop rejected sockets instantly across multiple CPU cores without garbage collection or worker slot starvation.
4. Final Thoughts
This benchmark isn't a verdict on which proxy engine is "best". It highlights how different runtimes behave under active rate enforcement.
Nginx remains one of the most reliable, deterministic reverse proxies in the industry for static assets, routing, and standard load balancing. However, when serving as an in-memory rate-limiting shield during extreme connection floods, its out-of-the-box worker pool limits can become a bottleneck unless fine-tuned for high-concurrency socket dropping.
Top comments (0)