DEV Community

Alex Spinov
Alex Spinov

Posted on • Edited on

requests vs httpx vs aiohttp — I Benchmarked All 3 (Results Surprised Me)

#python #webdev #performance #tutorial

Correction (June 2026). Sam Bull, a maintainer of aiohttp, emailed me about this post and was right: the original benchmark was unfair. In the aiohttp test I read the full response body (await resp.read()), but in the other tests I didn't — so aiohttp was doing strictly more work and still came out ahead. That's not apples-to-apples. I've rewritten the benchmark so every client reads the full body, re-ran it under controlled conditions, and the conclusions changed. Thank you, Sam. The corrected version is below.

I test HTTP libraries for a living

I run a lot of web scrapers. Each one makes thousands of HTTP requests per run, so the choice of client matters. I benchmarked the three most popular Python HTTP libraries head-to-head — and then a maintainer corrected my method, so here's the honest version.

The contenders

Library Async HTTP/2 Connection pooling
requests No No Yes (Session)
httpx Yes Yes (opt-in) Yes
aiohttp Yes No Yes

What was wrong, and the fix

Two things made the first version misleading:

  1. Inconsistent body reads. Only the aiohttp test read the response body. httpx already reads the body by default on await client.get(), so the right fix is to make every client read the full body explicitly — not to remove the read from aiohttp (that would just bias it the other way).
  2. Unreliable numbers. A single run against a public endpoint (httpbin.org) mixes network noise into the result. To compare the clients, I now hit a local keep-alive HTTP/1.1 server with a fixed 20 ms latency per request, warm every client equally, run each 5 times, and report the median. Every request asserts 200 + full body length, so "the body was read" is proven, not assumed. All clients use HTTP/1.1 here — httpx's HTTP/2 is not in play.

The async tests now read each body concurrently inside its own task, and both async clients are capped to the same connection pool, so httpx and aiohttp compete on equal footing.

# Full runnable benchmark: https://github.com/spinov001-art/python-http-benchmark
# Each client: request + read FULL body. Sync = sequential on one keep-alive
# connection; async = concurrent, same pool size for httpx and aiohttp.

async def _aiohttp_one(session, url):
    async with session.get(url) as r:
        body = await r.read()                 # read body (concurrent within the task)
        assert r.status == 200 and len(body) == EXPECTED

async def _httpx_one(client, url):
    r = await client.get(url)
    assert r.status_code == 200 and len(r.content) == EXPECTED   # httpx reads body by default

async def bench_aiohttp(url, n, pool):
    conn = aiohttp.TCPConnector(limit=pool)
    async with aiohttp.ClientSession(connector=conn) as s:
        await asyncio.gather(*[_aiohttp_one(s, url) for _ in range(n)])

async def bench_httpx_async(url, n, pool):
    limits = httpx.Limits(max_connections=pool)
    async with httpx.AsyncClient(limits=limits) as c:
        await asyncio.gather(*[_httpx_one(c, url) for _ in range(n)])
Enter fullscreen mode Exit fullscreen mode

Corrected results

200 GETs per run, 5 runs, median; local HTTP/1.1 server at 20 ms/request; pool = 50; every client reads the full body.

Library Mode Time (median)
urllib sync 4.87s
requests sync 4.92s
httpx sync 4.95s
httpx async 0.74s
aiohttp async 0.12s

What actually changed

  • The sync clients are basically tied. urllib, requests, and httpx land within ~1% of each other. My original claim that "sync httpx is slower than requests" was noise — at any real latency, the network dominates and the client overhead disappears.
  • Async wins because of concurrency, not because the client is magically faster per request. It hides the 20 ms latency by running many requests at once. That's the whole story behind the big numbers.
  • Under a fair, bounded pool, aiohttp is clearly faster than httpx async here (≈0.12s vs ≈0.74s, ~6×, with low run-to-run spread). This is consistent with aiohttp's lighter per-request overhead under pooled concurrency. It replaces my earlier, unsupported "aiohttp is ~12% faster" — that gap was within the noise of a single run.

One practical tip from Sam I'll pass on: install aiohttp[speedups] (C-accelerated parsing) for production.

Honest recommendation 

Simple scripts, a few requests   -> requests (everyone knows it, fine here)
One client for sync + async, or you need HTTP/2 -> httpx (clean unified API)
Max scraper throughput on HTTP/1.1 -> aiohttp (fastest here; add [speedups])
Enter fullscreen mode Exit fullscreen mode

Sam also pushed back on recommending httpx for scraping: its main differentiator is HTTP/2, which most scraping targets don't benefit from. He's right — for raw throughput on HTTP/1.1 targets, aiohttp is the pick. I still like httpx when I want one API for both sync and async, but I shouldn't have sold it as the scraping default on speed grounds.

Numbers are from a controlled local benchmark (HTTP/1.1, 20 ms latency, body read by every client); absolute ratios shift with pool size, latency, and body size — run the repo against your own workload. Thanks again to Sam Bull for the correction.

This article was written with AI assistance and corrected after expert feedback. The benchmark was run before publishing; the numbers above are from a real run.

Top comments (0)