The 3-Hour Async Bug: Why My 1000 Concurrent Requests Ran One by One

Last week, my boss dropped a requirement on my desk: scrape hundreds of thousands of product listings from competitor sites. Without a second thought, I reached for requests and fired away. Ten minutes later, I had scraped a grand total of 200 items — sequential requests, each waiting for the last, while the CPU sat there twiddling its thumbs. I knew exactly what to do: bring in asyncio, unleash coroutine concurrency, and wrap things up in no time.

So I rewrote everything with async def, sprinkled in await, hit run — the timing didn't budge. Requests still ran one after another. I stared at the screen, read the code three times. It had async, it had await — how on earth was it still serial? This bug ate three hours of my life. The culprit turned out to be a single line of reasoning: I called a synchronous library inside a coroutine.

Here's the full story: the pitfall, the debugging process, and the production-grade fix.

---

The Event Loop, Coroutines, and What Concurrency Actually Means

Python's asyncio is not "multithreading." There's only one thread, and the core is the Event Loop. Think of it as an over-caffeinated waiter: a table places an order (kicks off an I/O request), and instead of standing there frozen until the kitchen finishes, the waiter immediately moves on to serve the next table. When the dish is ready, the waiter comes back and delivers it.

In code:

• async def defines a coroutine — a function that can pause and resume.
• await tells the event loop: "I need to wait for I/O here. Go do something else, and come back to me when the result is ready."
• asyncio.gather() fires off multiple coroutines concurrently. The total time is roughly the duration of the slowest one, not the sum of all.

Sounds great, right? But there's a critical assumption: whatever you're waiting on must support async. If you await a synchronous blocking call, you freeze the entire event loop — no other coroutine can slip in.

That's exactly the mistake I made.

The Wrong Way: Why async/await Still Runs Serially

Here's the kind of code almost every asyncio beginner writes (yes, I once wrote this and felt proud):

import asyncio
import time
import requests

async def fetch(url):
    # Note: requests.get is synchronous and blocking!
    resp = requests.get(url, timeout=10)
    return resp.status_code

async def main():
    urls = [f"https://httpbin.org/delay/1?i={i}" for i in range(10)]
    start = time.perf_counter()
    # "Concurrently" launch coroutines
    results = await asyncio.gather(*[fetch(url) for url in urls])
    print(f"Elapsed: {time.perf_counter() - start:.2f}s, Results: {results}")

asyncio.run(main())

You expect 10 requests to fly out simultaneously, with a total time of roughly 1 second. In reality, it takes 10+ seconds — identical to serial execution. Why?

Because requests.get() is a synchronous blocking call. When the event loop hands control to the first coroutine, that coroutine calls requests.get(), and the entire thread is blocked inside that get call until the network responds. During that time, the event loop never regains control. Every other coroutine is stuck in the queue. Concurrency is dead on arrival.

asyncio concurrency is cooperative, not preemptive. If you don't voluntarily await an async object and yield control, no one can save you.

The Right Way: Async All the Way Down

To make concurrency actually work, you need to swap the entire call chain with proper async libraries. HTTP requests? aiohttp. File I/O? aiofiles. Database? asyncpg / aiomysql. Not a single second of synchronous blocking allowed.

Here's production-ready code with concurrency limiting, timeout control, and error handling:

import asyncio
import time
import aiohttp
from asyncio import Semaphore

# Cap concurrency to avoid overwhelming the target server or exhausting connection pools
MAX_CONCURRENT = 20
semaphore = Semaphore(MAX_CONCURRENT)

async def fetch(session: aiohttp.ClientSession, url: str):
    async with semaphore:  # Coroutines exceeding the limit wait here
        try:
            async with session.get(url, timeout=aiohttp.ClientTimeout(total=5)) as resp:
                data = await resp.text()
                return len(data)  # Return content length for demo purposes
        except Exception as e:
            print(f"Request failed {url}: {e}")
            return None

async def main():
    urls = [f"https://httpbin.org/delay/1?i={i}" for i in range(50)]
    start = time.perf_counter()

    # Reusable connection pool to reduce handshake overhead
    connector = aiohttp.TCPConnector(limit=MAX_CONCURRENT)
    async with aiohttp.ClientSession(connector=connector) as session:
        tasks = [fetch(session, url) for url in urls]
        results = await asyncio.gather(*tasks)

    print(f"Concurrency: {MAX_CONCURRENT}, Total requests: {len(urls)}, "
          f"Elapsed: {time.perf_counter() - start:.2f}s, "
          f"Successful: {sum(1 for r in results if r is not None)}")

asyncio.run(main())

Runtime: 50 requests, each with a 1-second server-side delay, total elapsed time roughly 2.5–3 seconds (50/20 batches overlapped). Compared to 50 seconds serial, that's a 15–20x speedup. This is what asyncio is supposed to look like.

Key points:

• aiohttp.ClientSession manages an internal connection pool, cutting TCP handshake overhead.
• Semaphore ensures you never exceed MAX_CONCURRENT in-flight requests, protecting both your system and the target.
• asyncio.gather() with a return_exceptions option (or manual try/except as shown) prevents one failure from crashing the entire batch.

How to Avoid This Pitfall from the Start

A hard-earned rule of thumb: "Async functions should be atomic to the async world." If you call a sync library inside an async def, you've broken the chain. Even a tiny synchronous file read or a logging call can slow everything down when it happens at scale.

A quick self-check: if you see import requests (or time.sleep, open without aiofiles) in the same file as asyncio.run, alarm bells should ring. Don't ignore them. I did for three hours. Don't be me.