DEV Community: The Byte Mechanics

Stop Starting With Microservices: The Path Real Systems Actually Follow

The Byte Mechanics — Sun, 01 Nov 2026 14:56:55 +0000

Originally published on Medium.

A new project kicks off. Someone opens a blank page and says, "Let's build it with microservices." Everyone nods. It sounds modern. It sounds scalable.

But here's the surprise: most high-scale systems don't start with microservices. They grow into them, and only when the system demands it.

This article walks through that growth one stage at a time. At each stage you'll see what breaks, why it breaks, and what teams do about it. If you'd rather watch, there's a 10-minute animated version of this article:

1. Start with a monolith

Every system starts somewhere, and the first goal isn't scale. It's correctness, speed of development and clarity. That's why most successful systems begin as a monolith.

A monolith is one application that holds every feature:

One backend, the code that runs on the server.
One deployment, so you ship it as a single unit.
Usually one database, where all the data lives.

Inside, each feature is a module, a separate area of the code. Modules call each other directly, as ordinary function calls with no network in between. That keeps two things easy:

Transactions are simple. A transaction is a group of changes that succeed or fail together, and a single database handles it for you.
Debugging is easy. One process, one log and one stack trace.

This leads to the first rule of architecture:

Don't add distributed complexity, meaning spreading work across many machines, until the system forces you to.

2. When the monolith feels heavy, fix coupling first

As features and engineers pile up, the monolith starts to feel heavy:

Builds slow down.
Deployments get risky.
A small change breaks something unrelated.
Teams collide in the same repository.

Many teams jump straight to microservices here. That's a mistake, because the real problem isn't the monolith. It's coupling: parts that depend too tightly on each other's insides.

The fix is a modular monolith. It still ships as one unit, but inside it's split into clean modules:

Each module has a defined interface, the official doorway other code must use.
Each module owns its data.
No module reaches into another's internals.
Each module follows a domain boundary, a business area like orders or payments.

This step matters more than it looks. If you split tightly coupled code into services, the coupling doesn't disappear. It just moves onto the network, where it's slower, harder to debug and can fail in new ways. Clean module boundaries now become clean service boundaries later.

3. When traffic grows, split into microservices

In a monolith, scaling means scaling everything. You add more running copies of the whole app, called instances. If payments get heavy traffic, the entire system gets more instances, including all the parts that were fine.

At small scale that's fine. At large scale it's inefficient and expensive, because different parts behave differently:

Read-heavy parts mostly fetch data.
Write-heavy parts mostly save data.
CPU-heavy parts need lots of processing power.
I/O-heavy parts spend their time waiting on disks or networks.

This is usually the moment to move to microservices: small, separate applications, each owning one domain. Each one can scale, deploy and fail on its own.

That solves scaling. But there's a catch. Before, everything lived in one process. Now everything lives on the network. Every section below is a consequence of that one sentence.

4. Services need to find each other: service discovery and the API gateway

Services now run on different machines, containers or pods. (A container is a packaged, isolated box for running an app, and a pod is a common way of grouping containers, for example in Kubernetes.) These instances restart, autoscale and move around, so hardcoding addresses is impossible.

The fix is service discovery. A registry tracks every available instance, and services ask it where to send each request.

The next problem is which service the app on your phone should call. With one backend there was one API. Now there are many. You don't want login checks or traffic limits copied into every service, and you don't want the frontend knowing your internal layout.

So you add an API gateway, a single entry point that handles:

Routing each request to the right service
Authentication, checking who you are
Logging
Throttling, limiting how fast a client can call
Sometimes combining responses from several services

5. The database becomes the bottleneck

As traffic keeps climbing, the first thing to struggle is usually the database, not the services. Services scale horizontally: you just add more machines. Databases are much harder.

The symptoms are familiar:

Queries slow down.
CPU spikes.
Locks appear. A lock is a hold on data while it's being changed, which makes everyone else wait.
Write delays grow.

Fix 1: read replicas. These are read-only copies of the database. Heavy queries move there, so reads are separated from writes. That works for a while, until the primary, the main copy that accepts writes, becomes the bottleneck itself.

Fix 2: sharding. The data is split across multiple machines, usually:

By user
By tenant, meaning a customer organization
By hash, a formula that assigns each record to a bucket

6. Transactions across services: the Saga pattern

In a monolith, one transaction can safely update many tables. That's an ACID transaction: a guarantee that a group of changes happens completely or not at all, and stays correct even when others are working at the same time.

Now one request, like create an order, may touch the order, payment, inventory and notification services, each with its own database. There's no easy way to wrap all of them in one ACID transaction. Forcing strong consistency, where everyone sees the same correct data at the same moment, usually hurts performance and reliability.

So high-scale systems accept eventual consistency: data may be briefly out of sync, but it catches up. They get there with three tools:

Events, to tell other services what happened
Retries, to recover from temporary failures
Compensation, which means undoing an earlier step

The common pattern that combines them is the Saga. Each step succeeds independently. If a later step fails, earlier steps are retried or undone. For example, if stock can't be reserved, the payment is refunded and the order is cancelled.

It's more scalable, but it's also more complex. That trade-off comes up at every stage.

7. Stop waiting: event-driven design

With a few services, synchronous calls work fine. Synchronous means the caller waits for an answer, usually over HTTP. As services multiply, these call chains become slow and fragile:

One slow service delays everything.
One failure breaks the whole flow.
Traffic spikes overload the services further down the line.

So high-scale systems switch to messaging. An event is a small message saying something happened. Instead of calling a service directly, you publish the event to a queue or stream, an ordered line of messages waiting to be processed. Consumers, the services that read events, handle them independently.

What you gain:

Services become loosely coupled.
Spikes are absorbed by the queue.
Failures can be retried.
Work happens asynchronously, so no one waits.

That's why large systems lean on Kafka, a popular event-streaming platform, as well as queues and pub-sub, where one sender reaches many listeners.

8. Retries are dangerous without idempotency

In distributed systems, failures are normal. Messages get retried. Requests time out, giving up after waiting too long. A consumer can crash after doing the work but before acknowledging it, that is, before telling the queue "I'm done."

So the same request may run more than once. For payments, orders, wallet updates or inventory changes, that's dangerous. Picture a customer who pays \$20 once and gets charged \$40.

The answer is idempotency: the same request produces the same result, even if it runs multiple times. Systems achieve it with two tools:

Idempotency keys (or unique request IDs), labels attached to each request
Deduplication tables, a record of which requests have already been handled

When a request arrives, the consumer checks its key. If the key isn't in the table, it processes the request and records the key. If the key is already there, it skips the work and returns the earlier result.

At scale, idempotency isn't an optimization. It's a requirement.

9. Designing to fail safely: rate limits and backpressure

At high scale, traffic isn't steady. Requests can suddenly spike, and a system that tries to handle everything at once can crash. So it has to control the flow:

Rate limiting restricts how much traffic is allowed in.
Backpressure slows down producers, the senders, when consumers can't keep up.
Queues buffer short spikes.
Dead-letter queues capture messages that keep failing, so they can be inspected instead of blocking everything else.

Sometimes the right move is to reject a request with a polite "try again later" (HTTP 429) instead of crashing the whole service.

High-scale systems aren't designed to never fail. They're designed to fail safely.

10. Observability and the CAP trade-off

In a monolith, logs are usually enough. In a distributed system, one request can pass through many services, and you need three more tools:

Tracing follows one request's path, so you can see which service is slow.
Metrics are numbers tracked over time, so you can see where load is growing.
Alerts tell you about problems before users complain.

Together these make up observability, the ability to see inside a running system. At this scale it has to be part of the design, not an afterthought.

Then comes the biggest trade-off, known as CAP:

Consistency: everyone sees the same latest data.
Availability: the system always answers.
Partition tolerance: the system keeps working when the network splits parts apart.

Networks do split in distributed systems, so partition tolerance isn't optional. When a split happens, you have to choose: keep answering, even with slightly stale data (availability), or refuse until the data is correct (consistency). You can't have all three, and different parts of the same product choose differently:

Payments and wallets favor consistency. A wrong balance is worse than a short wait.
Feeds and search favor availability. A slightly stale feed beats an error page.
Analytics accepts eventual consistency. Numbers that catch up in a minute are fine.

The path, in five steps

Most successful systems follow the same path:

Start simple.
Add structure.
Split when needed.
Distribute when required.
Optimize when scale demands it.

Three takeaways

1. Start with a monolith. Don't add distributed complexity until the system forces you to.

2. Fix coupling first. Use a modular monolith before you split into services. Otherwise you'll just move the coupling onto the network.

3. Microservices aren't a starting point. They're a response to scale, and every stage brings new problems to solve: discovery, data, transactions, retries, traffic and visibility.

Good architecture isn't the most advanced design. It's the right design for today, plus knowing when it's time to evolve.

▶️ Watch the animated version: Stop Starting With Microservices (YouTube)

Next up: why one bad request can crash every user on a server, and how processes and threads decide the blast radius. Watch Processes vs Threads.

If this helped, follow The Byte Mechanics here for a new visual deep dive every week.

▶️ YouTube: youtube.com/@thebytemechanics, full animated videos
📸 Instagram: instagram.com/thebytemechanics, 60-second Reels
📘 Facebook: facebook.com/thebytemechanics, Reels and updates
𝕏 X: x.com/thebytemechanic, quick takes and diagrams
✍️ Medium: medium.com/@thebytemechanics, the written deep dives

About: The Byte Mechanics turns how software really works into short animated explainers: system design, backend and operating systems, one diagram at a time.

One Bad Request Crashed Everyone: Processes vs Threads

The Byte Mechanics — Sun, 01 Nov 2026 14:56:31 +0000

Originally published on Medium.

A user uploads one corrupted image. Your server crashes. And every other user on it goes down too: their uploads, their checkouts, their sessions, all gone.

One bad request shouldn't be able to do that. But on a lot of servers, it can. Whether it does comes down to one choice: processes or threads.

This article explains both, the traps that come with each, and how real systems combine them. If you'd rather watch, here's the 8-minute animated version:

1. The problem: many requests at once

Hundreds of requests hit a server at the same time: uploads, logins, checkouts. Handle them one by one, and everyone waits, while the CPU sits idle every time a request waits on the disk or the database.

So the operating system gives us two ways to work on many requests at once: processes and threads.

2. A process is a house

A process is a running program with its own private world. The operating system gives it its own memory (code, data, heap and stack), its own ID, its own open files and its own permissions.

Think of a process as a house. Everything inside belongs to that house, and the neighbours can't walk in and move your furniture.

A process is a house: its own code, heap, stack, files and PID, walled off from neighbouring processes

This isolation is enforced by hardware. Each process sees its own virtual addresses, and the CPU translates them to real memory. Process A literally cannot point at Process B's memory. So one process can corrupt its own memory, freeze or crash, and the house next door doesn't notice.

3. Threads are the people inside

A thread lives inside a process. It's one path of execution, a sequence of instructions the CPU runs. Every process starts with one thread and can create more.

If the process is the house, threads are the people living in it. Each person has their own to-do list and remembers where they are in it: that's the thread's own stack and registers. But they share everything else: the same heap, the same global variables, the same open files.

That sharing is the whole point of threads. It's also the whole problem.

4. Why threads became the default

Cheaper to create. A new process needs a whole new memory space. Copy-on-write makes that cheaper, but it's still real work. A new thread only needs its own stack.
Cheaper to switch. Switching between threads of the same process keeps the same memory map. Switching between processes changes the map, and the CPU's address cache goes cold. One switch costs microseconds, and thousands per second add up.
Easier to share data. Processes must send messages through pipes, sockets or queues, and every message is copied. One thread can simply write a variable that another thread reads.

Fast, simple, no copying. But "any thread can change anything, at any time" is a dangerous sentence.

5. The trap: race conditions

Two threads each add one to a shared counter that starts at 100. You expect 102. But count = count + 1 isn't one step. It's three: read, add, write.

Thread A reads 100. Before it writes, the CPU switches. Thread B also reads 100, adds one and writes 101. Thread A resumes, adds one to its stale 100, and also writes 101. Two increments happened. One got lost.

Race condition: two threads read a shared counter of 100, both add one and write 101, so one update is lost

That's a race condition. It only shows up under load, disappears when you add a print statement, and never reproduces on your laptop. Now imagine the counter is an account balance, or the number of concert tickets left.

6. The fix, and the next trap

The standard fix is a lock (a mutex). A thread takes the lock before touching the counter, and everyone else waits until it's released. Read, add and write now happen as one unit.

But locks bring their own trap. Thread A holds lock 1 and waits for lock 2. Thread B holds lock 2 and waits for lock 1. Neither will ever let go.

Deadlock: thread A holds lock 1 and waits for lock 2 while thread B holds lock 2 and waits for lock 1, so everything stops

That's a deadlock. Nothing crashes. Everything just stops. Shared memory is fast, but every shared thing needs rules. Get them wrong, and you get lost updates or a frozen system.

7. Back to the corrupted image

Image decoders are often written in C. A broken file can make one touch invalid memory. When a thread does that, the operating system doesn't kill just that thread. It kills the whole process, and every other thread dies with it. That's how one upload took down 500 users.

Shared memory, shared fate: one thread touches invalid memory, the OS sends a segfault, and every thread in the server process dies

If each request were handled by a separate worker process, only that worker would die. A supervisor starts a fresh one, and nobody else notices.

Separate worker processes: the bad photo.jpg crashes only worker 3 while workers 1, 2 and 4 keep running

Chrome made this choice on purpose. Each site gets its own renderer process, so a crashing page only kills its own tab. It's a security win too: a malicious site in one process can't read your banking tab's memory. The price is RAM, which is why Chrome uses so much of it. Isolation costs resources.

8. Your workload and language matter

I/O-bound work (mostly waiting on databases, networks or disks): threads are great. While one waits, another runs.
CPU-bound work (image processing, number crunching): you want real parallel execution on every core.
Python: in standard CPython, the Global Interpreter Lock lets only one thread run Python code at a time. For CPU-heavy Python, use multiple processes, one per core.
Node.js: your JavaScript runs on a single thread with an event loop, and heavy work goes to worker threads or separate processes.

9. Real systems use both

PostgreSQL starts a process per connection. A failed query stays inside its own process, but every backend also shares one block of memory for cached data. So if one backend crashes outright, the postmaster restarts all of them to be safe. Processes are also heavy, which is why people put a connection pooler in front of it.
MySQL handles each connection with a thread inside one server process. Lighter per connection, with shared memory.
Nginx runs a few worker processes, usually one per core, each with an event loop juggling thousands of connections.
A typical Python web app runs several worker processes with threads or async inside each.

The pattern used by PostgreSQL, MySQL, Nginx and Python web apps: processes for isolation and every core, threads or async for cheap concurrency

The pattern is the same everywhere: processes for isolation and to use every core, threads or async inside them for cheap concurrency.

10. How to choose

How to choose: use processes when crashes must stay contained or you need security boundaries; use threads when tasks share lots of data

Reach for processes when a crash must stay contained, when you need security boundaries, or when your language can't run threads in parallel.

Reach for threads when tasks share a lot of data, when you need many lightweight workers, and when you're willing to protect shared state with care.

In production, expect both: a few processes, with many threads or async tasks inside. With every request in threads under one roof, one corrupted image took everyone down. With separate worker processes, one worker dies, restarts, and nobody else notices.

▶️ Watch the animated version: One Bad Request Crashed Everyone: Processes vs Threads (YouTube)

Next up: how async and event loops handle thousands of connections with a single thread. Watch One Thread, 10,000 Users.

If this helped, follow The Byte Mechanics here for a new visual deep dive every week.

▶️ YouTube: youtube.com/@thebytemechanics, full animated videos
📸 Instagram: instagram.com/thebytemechanics, 60-second Reels
📘 Facebook: facebook.com/thebytemechanics, Reels and updates
𝕏 X: x.com/thebytemechanic, quick takes and diagrams
✍️ Medium: medium.com/@thebytemechanics, the written deep dives

About: The Byte Mechanics turns how software really works into short animated explainers: system design, backend and operating systems, one diagram at a time.

One Thread, 10,000 Users: Async and Event Loops Explained

The Byte Mechanics — Sun, 01 Nov 2026 14:55:41 +0000

Originally published on Medium.

This server has one thread. Ten thousand people are using it right now, and none of them are waiting.

Give each of those users their own thread instead, and the same server runs out of memory and falls over.

So how does one thread beat ten thousand? This article walks through the event loop, async/await, and the one harmless-looking line of code that can freeze every user at once. If you'd rather watch, here's the 8-minute animated version:

1. The secret is waiting

Take one web request. It reads the request, asks the database for some rows, maybe calls another API, and sends a response.

Now time it. The CPU actually works for about two milliseconds. The other ninety-eight milliseconds are spent waiting: for the database, for the network, for the disk.

A 100 ms request spends only 2 ms on the CPU and the rest waiting on the database, network and disk. Your server isn't busy, it's bored

So your server isn't busy. It's bored. Most of every request is just waiting. And in the classic design, a waiting thread does nothing at all while still holding on to everything it owns.

2. Why one thread per user runs out

The classic answer is one thread per connection. A user connects, you give them a thread, and that thread waits for them. It's simple, and it works for a few hundred users.

But every thread has its own stack, often around a megabyte reserved. Ten thousand threads means gigabytes of memory, just to wait. And the operating system has to keep switching between all of them, so more and more time goes into switching instead of real work.

Thread per connection: a grid of 5,322 threads using 5.3 GB of RAM just to wait

In 1999, Dan Kegel gave this wall a name: the C10K problem. How do you handle ten thousand connections on one machine? The answer didn't come from adding more threads.

3. The waiter

Picture a restaurant that hires one waiter per table. The waiter takes your order, walks to the kitchen, and stands there until the food is ready. A hundred tables means a hundred waiters, mostly standing around.

Now picture a restaurant with one waiter. They take an order, hand it to the kitchen, and go straight to the next table. When the kitchen rings the bell, they pick up the food and deliver it. One waiter, a whole room of tables, and nobody waits on the waiter.

Restaurant analogy: one waiter per table means 100 waiters mostly standing around, versus one waiter serving every table

That waiter has a name in software: the event loop.

4. The event loop

An event loop is one thread running one simple cycle, over and over:

Check what's ready: a new connection, a database reply, a timer that just went off.
Run the code for each ready thing, quickly.
Hand off the waits. Whenever that code needs to wait, it doesn't. The wait goes to the operating system, and the loop moves on.

Then it goes back to step one.

The event loop serving 10,000 connections: check what's ready, run the code quickly, hand off the waits to the OS

Because the thread never stops to wait, it can juggle thousands of connections. It only spends time on the small slice that's real work. But notice the rule hiding in there:

Every step has to be quick.

Remember that rule. It's the key to the line that freezes everyone.

5. The operating system does the watching

How does the loop know when something is ready? The operating system tells it.

Sockets can be put in non-blocking mode. Instead of sleeping until data arrives, a read returns straight away and says: nothing yet. Then the loop hands the operating system its whole list of sockets and says: wake me up when any of these are ready.

The kernel watches every socket and wakes the event loop only when some of them are ready

On Linux that's epoll. On macOS and BSD it's kqueue. On Windows, I/O completion ports. Older calls like select scanned every socket on every call. epoll just hands back the ready ones, so ten thousand idle connections cost almost nothing.

6. From callbacks to async/await

In the early days, you gave the loop a function to call later: a callback. Read the file, and when you're done, call this. Chain a few of those and the code drifts to the right, deeper and deeper. People called it callback hell.

Promises flattened it out. A promise is a placeholder for a value that isn't ready yet. Then async and await made it look like normal code again:

const user = await getUser(id);
const orders = await getOrders(user);
return render(user, orders);

Blocking code and async code look almost the same, but they are not: only the await version gives the loop back

It reads top to bottom, exactly like blocking code. But it isn't blocking code, and the difference is the whole trick.

7. What await really does

When your code hits await, your function doesn't sit there waiting. It pauses, saves its place, and hands control back to the loop. The loop serves other requests. When the database replies, the loop picks your function up exactly where it left off.

An async handler awaiting getUser and getOrders while the loop serves other requests in between

So await means: I'm going to wait, so go help someone else. This is called cooperative multitasking. Every function promises to give the loop back quickly.

Which raises a question. What happens if one function doesn't?

8. The trap: one line freezes everyone

The line that freezes everyone looks completely harmless:

const thumb = resizeImage(upload); // 40 MB

No await. Just two seconds of pure CPU work on the only thread you have. The same goes for hashing a password, parsing a huge file, a synchronous file read, a sleep, or an old blocking HTTP library called from async code.

The trap: resizeImage on a 40 MB file holds the only thread for 0.7 seconds and freezes every user

For those two seconds, the loop is stuck. Every other request sits in the queue. Health checks time out, and your load balancer decides the server is dead. One request, and ten thousand users frozen.

If you read the last article, this should sound familiar. There, one bad request crashed everyone. Here, one slow request freezes everyone.

9. The one rule

The event loop coordinates. It never does the heavy lifting.

Send CPU-heavy work to a pool of worker threads or processes, and await the result. In Python that's await asyncio.to_thread(resize, img) (or a process pool for CPU-heavy Python). In Node it's worker_threads.
Node already does some of this for you. File system calls, DNS lookups and some crypto run on libuv's small hidden thread pool (four threads by default), so they don't touch the loop.
Use async versions of everything that waits: async database drivers, async HTTP clients. requests.get() or readFileSync() inside async code blocks the loop.
If a job is huge, cut it into chunks so the loop can breathe in between.

Moving heavy work off the loop: Node's hidden libuv thread pool and a worker pool for CPU-bound tasks

10. How the giants do it

In production you almost never see just one loop. You see one per CPU core.

Nginx was built for exactly the C10K problem. It starts one worker process per core, and each worker runs its own event loop.
Node apps scale the same way, with one process per core using cluster mode or a process manager like PM2.
Python web apps run several Uvicorn workers, each with its own asyncio loop.
Redis executes commands on a single thread and still handles massive traffic, because each command is tiny and the loop never waits.
Go hides all of it. You write plain blocking code in goroutines, and the runtime's network poller runs an event loop underneath for you.

Processes for cores, loops for waiting: four processes, one per core, each running its own event loop

The pattern is the one from the last article: processes for cores and isolation, and inside each one, a loop for the waiting.

11. The cheat sheet

Use async when your work is mostly waiting: web servers, APIs, chat apps, proxies, anything with lots of connections doing a little work each. Its risk is blocking the loop.

Use threads or processes for CPU-heavy work. Async won't speed up computation; for that you need more cores.

In production, use both: one loop per core for the waiting, and a worker pool for the heavy lifting.

Ten thousand users, one thread. It works, as long as that thread never waits, and never does the heavy lifting itself.

▶️ Watch the animated version: Event Loop Explained: How One Thread Handles 10,000 Users (YouTube)

Next up: most of that 98% waiting is the database. Next time: how one index turns a 10-second query into 10 milliseconds.

If this helped, follow The Byte Mechanics here for a new visual deep dive every week.

▶️ YouTube: youtube.com/@thebytemechanics, full animated videos
📸 Instagram: instagram.com/thebytemechanics, 60-second Reels
📘 Facebook: facebook.com/thebytemechanics, Reels and updates
𝕏 X: x.com/thebytemechanic, quick takes and diagrams
✍️ Medium: medium.com/@thebytemechanics, the written deep dives

About: The Byte Mechanics turns how software really works into short animated explainers: system design, backend and operating systems, one diagram at a time.