This is Part 12 of my "From One User to One Million" series, where we'll build an understanding of System Design by following a simple application as it grows from a single user to millions. Instead of memorising technologies, we'll learn why they exist by solving real problems as they appear.
Look at what we've built around the application.
The infrastructure distributes traffic, data, background work, and content across dozens of machines. Load balancers route requests across application servers. Caches absorb repeated reads. Read replicas share the database load. Shards divide the data. Message queues decouple slow work from fast responses. CDN edge locations serve static content from nearby.
Every time a new bottleneck appeared, we distributed something. Traffic. Reads. Storage. Work over time. Content over geography.
But Part 11 ended by pointing at something we haven't touched yet.
The application code itself is still one thing.
While the infrastructure around it grew more distributed and sophisticated, the application running on those servers remained a single, unified codebase. One repository. One deployment process. One running process serving every feature the product offers.
For a long time, that was fine. Now it's becoming the problem.
--
Section 1: When the Application Was Small
It helps to go back to the beginning of the application's life, because the architecture that's now causing problems was originally a perfectly sensible choice.
When the application launched, it had a handful of features: user accounts, order management, payments, search, and notifications. A team of five engineers built it all in one codebase. They deployed it as one unit. When something broke, anyone on the team could find it. When a new feature needed to be added, one person could make the change and understand how it fit into the whole.
The early application:
[ One Codebase ]
- Users
- Orders
- Payments
- Search
- Notifications
One team. One deployment. One database.
This is called a monolith: a single application that contains all the business logic and runs as one deployable unit.
The monolith isn't a mistake. It's often exactly the right choice early on. There's nothing to coordinate between services, no network calls between components, no distributed systems problems to manage. A developer can run the entire application on their laptop and understand the whole thing in a few days. Changes are fast, deployments are simple, and debugging is straightforward.
For a small team building a product that's still finding its footing, a monolith is the pragmatic choice.
The problems don't come from the architecture being wrong. They come from the application growing far beyond the scale it was originally designed for.
--
Section 2: When One Codebase Becomes Too Much
The company grows. Users grow. Features multiply. The engineering team expands from five people to fifty, then to a hundred and fifty.
Now there are separate teams for each major product area: one team working on the payment system, another focused on search, another on notifications, another on the core ordering flow. Each team has its own roadmap, its own priorities, its own release schedule.
But they all share the same codebase.
This is where the friction begins, and it's worth being specific about what that friction actually looks like in practice.
A developer on the payments team makes a change to how the application processes refunds. Before that change can go to production, the entire application has to be tested. Not just the payments code. Everything. Because in a single codebase, a change in one part can have unexpected effects in another. The test suite for the entire application has to run. It takes forty minutes.
Then the deployment. To push the refund fix, the company deploys the entire application. The search features go with it. The notification system goes with it. The user account code goes with it. All of it is deployed as one unit, even though only a few lines of payment code changed.
That deployment carries risk. If something goes wrong with the refund change, it can affect search results, or notification delivery, or the order flow. Features that had nothing to do with refunds can break. The more code that ships in one deployment, the larger the blast radius of anything that goes wrong.
And as the codebase grows, something else happens: it becomes harder to understand. A developer who joins the team to work on search has to navigate a codebase that also contains the entire payment system, the complete notification logic, and everything else. None of that is relevant to their work, but it's all there, and the boundaries between different areas of the code become increasingly blurry.
The dependency problem grows with it. The search code might depend on a utility function in the payments module, not because search needs payments, but because someone took a shortcut years ago and it was never cleaned up. Now a change to payments requires verifying that search still works.
The application isn't necessarily slow. The users might be getting fast responses. By every performance metric, the system might look healthy.
But the teams are slowing down. Deployments are getting more dangerous. The codebase is getting harder to change confidently. The bottleneck is no longer in the infrastructure.
The application itself has become the bottleneck.
--
Section 3: What If the Application Didn't Have to Be One Thing?
At some point, someone on the team asks the question that reframes the whole problem.
"What if payments didn't have to live in the same codebase as search? What if they didn't have to be deployed together? What if the team working on notifications could release a change without touching anything related to orders?"
Think about what that would require.
The payment functionality would have to be separated into its own independent unit. It would have its own codebase, its own deployment process, its own team ownership. The search functionality would be another independent unit. Notifications another. Orders another. Users another.
Each unit would be responsible for one specific area of the product. And when those units needed to talk to each other, they'd communicate through a well-defined interface, the same way any two separate systems communicate.
BEFORE: One application
[ Monolith ]
Users + Orders + Payments + Search + Notifications
|
Deployed as one unit
AFTER: Separated responsibilities
[ Users ] [ Orders ] [ Payments ]
[ Service ] [ Service ] [ Service ]
[ Search ] [ Notifications ]
[ Service ] [ Service ]
Each deployed independently.
The word for this idea, when taken to a deliberate architectural pattern, is microservices.
--
Section 4: Meet Microservices
A microservice is a relatively small, independently deployable service responsible for a specific business capability.
Notice what that definition doesn't say. It doesn't specify a maximum file size or line count. It doesn't prescribe exactly how many services an application should have. Those details vary enormously from one company to another.
What the definition does say is that the service is independently deployable and responsible for a specific capability. Those two properties are the point.
Independently deployable means the payments team can ship a change to the Payment Service on Tuesday afternoon without needing to coordinate with the search team, wait for the notification team's release window, or risk breaking the order flow. The deployment is scoped to one service.
Responsible for a specific capability means the service has clear ownership. The Payment Service handles payments. It knows its own data, its own logic, its own dependencies. Other services don't reach into its internals. They request things from it through its interface, and it handles the rest.
In practice, an API gateway often sits in front of the services, acting as the single entry point for incoming requests and routing them to the appropriate service.
[ API Gateway ]
|
.-----------------+-----------------.
| | | |
[ Users [ Orders [ Payments [ Search
Service] Service] Service] Service]
|
[ Notifications
Service ]
A user request comes in through the gateway. The gateway determines which service needs to handle it and routes accordingly. The services handle their own domains.
--
Section 5: The Benefit: Independent Scaling and Deployment
The most immediate practical benefit of this separation is that things which used to be coupled can now move independently.
Consider what happens when the application runs a flash sale. Search traffic spikes dramatically as users browse and compare products. In a monolith, handling that spike means scaling the entire application: more instances of the whole thing, including the payment code, the notification logic, and everything else that isn't under any additional load.
With separate services, the Search Service can be scaled independently. More instances of search spin up to handle the spike. The Payment Service, which isn't receiving unusual traffic, stays as it is. The Notification Service doesn't change. You're not wasting resources scaling code that doesn't need it.
Flash sale traffic spike:
Monolith approach:
Scale everything x5 (payments, orders, search, notifications, users)
Most of that capacity is wasted on code that isn't under load.
Microservices approach:
Scale Search Service x5
Everything else unchanged.
Resources go exactly where the demand is.
The deployment story changes too. The payments team can release a fix on Wednesday. The search team can release an improvement on Thursday. The notification team can release a new feature on Friday. Each team operates on its own schedule. A bug in one service doesn't block another team's release. A problem discovered in the search code doesn't delay a critical payment fix.
Over time, this compounds. Teams move faster when their work is truly independent. Ownership becomes clearer. Codebases become smaller and easier to reason about. Onboarding a new engineer to the Payment Service means learning one service, not understanding the entire product.
--
Section 6: The Trade-off: Now the Network Is Part of Your Application
If microservices were purely beneficial, every application would use them from day one. They're not.
In a monolith, when the order logic needs to call the payment logic, it's a function call. It happens in memory, in the same process, in microseconds. It either works or it doesn't.
In a microservices architecture, that same interaction is a network request.
Monolith:
Order logic --> calls --> Payment function
(in memory, same process, microseconds)
Microservices:
Order Service --> network request --> Payment Service
(crosses a network, takes time, can fail)
And networks fail. That's not a design flaw or an implementation problem. It's a fundamental property of distributed systems. A service can be slow. It can be temporarily unavailable. It can process a request and send a response that never arrives. It can be overloaded and start dropping requests.
Think through what this means for a simple order placement.
A user places an order. The Order Service receives the request. It needs to charge the user, so it calls the Payment Service. The Payment Service is slow right now because of unrelated load. The Order Service waits. How long should it wait before giving up? If it gives up too quickly, it might cancel a payment that was actually being processed. If it waits too long, the user's request hangs.
The Payment Service processes the payment successfully. Before the response travels back to the Order Service, a network hiccup drops the packet. The Order Service never receives confirmation. Does it retry? If it retries, and the payment already went through, the user might be charged twice.
The Order Service calls the Inventory Service to reserve the item. The Inventory Service is down. The order is paid but the inventory isn't reserved. The state across services is now inconsistent.
None of these scenarios existed in the monolith, because none of them could exist. There was no network between the order logic and the payment logic. They lived in the same process.
Microservices don't eliminate this complexity. They introduce it, deliberately, in exchange for the independence they provide. The distributed systems problems of latency, partial failure, retries, and consistency now belong to the application layer, not just the infrastructure layer.
This is the trade-off stated plainly:
Monolith:
Simpler communication (function calls, not network calls)
Simpler deployments (one unit)
Simpler local development (run one thing)
Coupled scaling (can't scale one part independently)
Coupled deployments (one change ships everything)
Coupled teams (changes in one area risk others)
Microservices:
Independent scaling (scale what needs it)
Independent deployments (ship one service at a time)
Independent teams (own your service end to end)
Network communication (with all its failure modes)
Distributed consistency challenges
More complex observability and debugging
Neither column is inherently better. The right choice depends on the specific situation.
--
Section 7: When Should You Actually Split?
The honest answer is: later than most teams think.
Microservices are often presented as a modern best practice, something sophisticated engineering teams do. That framing leads teams to adopt them too early, before the problems they solve have actually materialized.
A team of five engineers building an early-stage product almost certainly doesn't need microservices. The overhead of coordinating deployments across multiple services, setting up inter-service communication, managing distributed failures, and operating multiple independent codebases will slow them down more than the monolith ever would. The monolith is a feature, not a liability, at that stage.
The signals that suggest a split might be worth the complexity are specific.
Different parts of the application have genuinely different scaling needs, and the cost of scaling everything together is becoming real. Different teams own different areas, and shared deployments are creating real coordination friction. A clear business boundary exists between two areas of the application, and crossing that boundary requires constant negotiation. The risk surface of a single deployment has grown large enough that changes feel genuinely dangerous.
When those things are true, and when the team is large and experienced enough to manage distributed systems complexity, separation starts to pay off.
When those things aren't yet true, a well-organized monolith, with clear internal boundaries and disciplined code ownership, is often the better choice. The goal was never to have microservices. The goal was always to be able to move fast and build reliable software. Microservices are one way to achieve that, in the right circumstances, at the right scale.
--
Conclusion
Here's the complete picture of what this series has built.
We started with one server. We progressively distributed every part of the system that became a bottleneck.
Traffic was concentrated at one server, so we added load balancers and distributed it across many.
Repeated database work was concentrated at one database, so we added caching and eliminated it.
Read traffic was concentrated at one database, so we added replicas and spread it.
Data was concentrated in one database, so we added sharding and partitioned it.
Slow background work was concentrated in the user's request, so we added queues and moved it out.
Static content was concentrated at one origin server far from users, so we added CDNs and distributed it geographically.
And now, the application logic was concentrated in one codebase and one deployment, so we split it into services that could be developed, deployed, and scaled independently.
The lesson across all twelve parts has been the same. Find what's concentrated. Understand why. Distribute it in the way that fits the problem.
But there's one question this series hasn't asked yet, and it might be the most important one.
We've spent every article asking: how do we make the system handle more?
More traffic. More data. More requests. More users.
Now look at what we've built. Multiple services. Multiple databases. Read replicas. Shards. Queues. Workers. CDN edge nodes. Cache layers.
Dozens of moving pieces, spread across many machines, connected by a network.
What happens when one of those pieces fails?
A database server loses power. A service crashes under unexpected load. A data center loses network connectivity. A disk fills up. A deployment goes wrong and takes a service offline.
We've built a system that can scale. But can it survive?
How do you build a system that keeps working even when parts of it fail?
That question is what Part 13 is about.
Top comments (0)