Choosing the right HTTP client in Python seems straightforward until your script moves from fetching 10 URLs locally to sustaining thousands of concurrent API calls in production.
While requests remains the gold standard for simple synchronous scripts, modern async workloads usually come down to httpx versus aiohttp. To see how they actually behave with proper connection pooling and DNS caching enabled, I ran a head-to-head benchmark across 250 concurrent I/O-bound requests.
Benchmark Summary (250 Concurrent Requests)
| Client | Mode | Avg Total Time | Peak Memory (RSS) | HTTP/2 Support | Best Use Case |
|---|---|---|---|---|---|
requests |
Sync (Session) |
14.82s |
~34 MB |
No | Sequential CLI scripts & simple jobs |
httpx |
Async (AsyncClient) |
1.94s |
~58 MB |
Yes (Native) | Modern FastAPI services & HTTP/2 APIs |
aiohttp |
Async (TCPConnector) |
1.31s |
~46 MB |
No | High-throughput scraping & raw speed |
Where Developers Lose Performance in aiohttp and httpx
Most benchmarks compare default clients out of the box, which hides two major bottlenecks:
-
Missing DNS Cache TTL (
aiohttp): By default,aiohttp.TCPConnectorcaches DNS lookups for only 10 seconds. Increasingttl_dns_cache=300eliminates repetitive resolver overhead during sustained crawls. -
Unbounded Connection Pools (
httpx): Leaving default pool limits untouched during sudden task bursts causesPoolTimeoutexceptions. Explicitly configuringhttpx.Limitskeeps socket reuse predictable:limits = httpx.Limits(max_keepalive_connections=20, max_connections=50)
async with httpx.AsyncClient(limits=limits, http2=True) as client:
response = await client.get("https://api.example.com/v1/status")
Which One Should You Pick?
-
Pick
aiohttpwhen raw throughput, lower memory footprint, and fine-grained TCP socket control matter most (such as high-volume crawlers and worker queues). -
Pick
httpxwhen you want arequests-compatible API, native HTTP/2 multiplexing, or seamless sync/async code sharing inside FastAPI apps.
Run the Benchmark Yourself (Free Standalone Script)
I uploaded the clean, reproducible http_benchmark.py test harness (with connection-pooling configs pre-wired) for direct download—no signup or redirects:
👉 Download http_benchmark.py Directly (OpenCode Mirror)
Which HTTP client are you running in production right now—httpx, aiohttp, or curl_cffi for TLS fingerprinting? Drop your setup in the comments!
Top comments (0)