One Thread, 10,000 Users: Async and Event Loops Explained

Originally published on Medium.

This server has one thread. Ten thousand people are using it right now, and none of them are waiting.

Give each of those users their own thread instead, and the same server runs out of memory and falls over.

So how does one thread beat ten thousand? This article walks through the event loop, async/await, and the one harmless-looking line of code that can freeze every user at once. If you'd rather watch, here's the 8-minute animated version:

---

1. The secret is waiting

Take one web request. It reads the request, asks the database for some rows, maybe calls another API, and sends a response.

Now time it. The CPU actually works for about two milliseconds. The other ninety-eight milliseconds are spent waiting: for the database, for the network, for the disk.

A 100 ms request spends only 2 ms on the CPU and the rest waiting on the database, network and disk. Your server isn't busy, it's bored

So your server isn't busy. It's bored. Most of every request is just waiting. And in the classic design, a waiting thread does nothing at all while still holding on to everything it owns.

2. Why one thread per user runs out

The classic answer is one thread per connection. A user connects, you give them a thread, and that thread waits for them. It's simple, and it works for a few hundred users.

But every thread has its own stack, often around a megabyte reserved. Ten thousand threads means gigabytes of memory, just to wait. And the operating system has to keep switching between all of them, so more and more time goes into switching instead of real work.

Thread per connection: a grid of 5,322 threads using 5.3 GB of RAM just to wait

In 1999, Dan Kegel gave this wall a name: the C10K problem. How do you handle ten thousand connections on one machine? The answer didn't come from adding more threads.

3. The waiter

Picture a restaurant that hires one waiter per table. The waiter takes your order, walks to the kitchen, and stands there until the food is ready. A hundred tables means a hundred waiters, mostly standing around.

Now picture a restaurant with one waiter. They take an order, hand it to the kitchen, and go straight to the next table. When the kitchen rings the bell, they pick up the food and deliver it. One waiter, a whole room of tables, and nobody waits on the waiter.

Restaurant analogy: one waiter per table means 100 waiters mostly standing around, versus one waiter serving every table

That waiter has a name in software: the event loop.

4. The event loop

An event loop is one thread running one simple cycle, over and over:

1. Check what's ready: a new connection, a database reply, a timer that just went off.
2. Run the code for each ready thing, quickly.
3. Hand off the waits. Whenever that code needs to wait, it doesn't. The wait goes to the operating system, and the loop moves on.

Then it goes back to step one.

The event loop serving 10,000 connections: check what's ready, run the code quickly, hand off the waits to the OS

Because the thread never stops to wait, it can juggle thousands of connections. It only spends time on the small slice that's real work. But notice the rule hiding in there:

Every step has to be quick.

Remember that rule. It's the key to the line that freezes everyone.

5. The operating system does the watching

How does the loop know when something is ready? The operating system tells it.

Sockets can be put in non-blocking mode. Instead of sleeping until data arrives, a read returns straight away and says: nothing yet. Then the loop hands the operating system its whole list of sockets and says: wake me up when any of these are ready.

The kernel watches every socket and wakes the event loop only when some of them are ready

On Linux that's epoll. On macOS and BSD it's kqueue. On Windows, I/O completion ports. Older calls like select scanned every socket on every call. epoll just hands back the ready ones, so ten thousand idle connections cost almost nothing.

6. From callbacks to async/await

In the early days, you gave the loop a function to call later: a callback. Read the file, and when you're done, call this. Chain a few of those and the code drifts to the right, deeper and deeper. People called it callback hell.

Promises flattened it out. A promise is a placeholder for a value that isn't ready yet. Then async and await made it look like normal code again:

const user = await getUser(id);
const orders = await getOrders(user);
return render(user, orders);

Blocking code and async code look almost the same, but they are not: only the await version gives the loop back

It reads top to bottom, exactly like blocking code. But it isn't blocking code, and the difference is the whole trick.

7. What await really does

When your code hits await, your function doesn't sit there waiting. It pauses, saves its place, and hands control back to the loop. The loop serves other requests. When the database replies, the loop picks your function up exactly where it left off.

An async handler awaiting getUser and getOrders while the loop serves other requests in between

So await means: I'm going to wait, so go help someone else. This is called cooperative multitasking. Every function promises to give the loop back quickly.

Which raises a question. What happens if one function doesn't?

8. The trap: one line freezes everyone

The line that freezes everyone looks completely harmless:

const thumb = resizeImage(upload); // 40 MB

No await. Just two seconds of pure CPU work on the only thread you have. The same goes for hashing a password, parsing a huge file, a synchronous file read, a sleep, or an old blocking HTTP library called from async code.

The trap: resizeImage on a 40 MB file holds the only thread for 0.7 seconds and freezes every user

For those two seconds, the loop is stuck. Every other request sits in the queue. Health checks time out, and your load balancer decides the server is dead. One request, and ten thousand users frozen.

If you read the last article, this should sound familiar. There, one bad request crashed everyone. Here, one slow request freezes everyone.

9. The one rule

The event loop coordinates. It never does the heavy lifting.

• Send CPU-heavy work to a pool of worker threads or processes, and await the result. In Python that's await asyncio.to_thread(resize, img) (or a process pool for CPU-heavy Python). In Node it's worker_threads.
• Node already does some of this for you. File system calls, DNS lookups and some crypto run on libuv's small hidden thread pool (four threads by default), so they don't touch the loop.
• Use async versions of everything that waits: async database drivers, async HTTP clients. requests.get() or readFileSync() inside async code blocks the loop.
• If a job is huge, cut it into chunks so the loop can breathe in between.

Moving heavy work off the loop: Node's hidden libuv thread pool and a worker pool for CPU-bound tasks

10. How the giants do it

In production you almost never see just one loop. You see one per CPU core.

• Nginx was built for exactly the C10K problem. It starts one worker process per core, and each worker runs its own event loop.
• Node apps scale the same way, with one process per core using cluster mode or a process manager like PM2.
• Python web apps run several Uvicorn workers, each with its own asyncio loop.
• Redis executes commands on a single thread and still handles massive traffic, because each command is tiny and the loop never waits.
• Go hides all of it. You write plain blocking code in goroutines, and the runtime's network poller runs an event loop underneath for you.

Processes for cores, loops for waiting: four processes, one per core, each running its own event loop

The pattern is the one from the last article: processes for cores and isolation, and inside each one, a loop for the waiting.

11. The cheat sheet

Use async when your work is mostly waiting: web servers, APIs, chat apps, proxies, anything with lots of connections doing a little work each. Its risk is blocking the loop.

Use threads or processes for CPU-heavy work. Async won't speed up computation; for that you need more cores.

In production, use both: one loop per core for the waiting, and a worker pool for the heavy lifting.

Ten thousand users, one thread. It works, as long as that thread never waits, and never does the heavy lifting itself.

---

▶️ Watch the animated version: Event Loop Explained: How One Thread Handles 10,000 Users (YouTube)

Next up: most of that 98% waiting is the database. Next time: how one index turns a 10-second query into 10 milliseconds.

If this helped, follow The Byte Mechanics here for a new visual deep dive every week.

• ▶️ YouTube: youtube.com/@thebytemechanics, full animated videos
• 📸 Instagram: instagram.com/thebytemechanics, 60-second Reels
• 📘 Facebook: facebook.com/thebytemechanics, Reels and updates
• 𝕏 X: x.com/thebytemechanic, quick takes and diagrams
• ✍️ Medium: medium.com/@thebytemechanics, the written deep dives

About: The Byte Mechanics turns how software really works into short animated explainers: system design, backend and operating systems, one diagram at a time.