What if the architecture that "doesn't scale" is the one that gets you to production fastest?
I keep seeing the same pattern. A dev starts a side project, draws 8 services on a whiteboard before writing a single line of code, spends three weekends wiring up service discovery and inter-service auth, and the actual product logic is 200 lines across all of them. The project dies in a Docker Compose file.
Microservices aren't wrong. But they solve problems most early-stage projects don't have. And the cost of splitting too early is way higher than the cost of splitting too late.
🎯 Four options, not two
People talk about this like it's binary. Monolith or microservices. But there are actually four architecture patterns on the spectrum, and two of them don't get enough attention.
The monolith. One deployable unit. All modules share a process, a database, and a release cycle. Not a dirty word. Just means you ship one thing.
The modular monolith. Still one deployment, but with enforced internal boundaries. Modules have explicit public APIs and can't reach into each other's guts. Shopify runs a 2.8 million line Rails app this way. They use a tool called Packwerk to enforce dependency rules between components. One deploy, but internally decoupled. This is honestly where most teams should land.
Microservices. Independently deployable services, each owning its own data, talking over the network. The defining trait isn't size. It's that you can deploy service A without touching service B.
The distributed monolith. The failure mode nobody plans for. You split your code into 15 services, but they still have to be deployed together, tested together, and they fail together. You've got all the operational complexity of a distributed system with none of the independence benefits. The worst of both worlds. And honestly? This is what most failed microservices migrations actually produce.
What you gain when you split
I'll be fair to the microservices side. When the split is done well, you get real benefits:
| Dimension | Monolith | Microservices |
|---|---|---|
| Deployment | Ship everything together | Deploy one service independently |
| Team ownership | Coordinate releases across teams | Each team owns full lifecycle |
| Fault isolation | One bug can bring down the whole app | Failures stay contained per service |
| Scaling | Scale the entire unit | Scale only the hot path |
| Tech flexibility | One stack for everything | Each service picks its own tools |
These are real. At Amazon's scale, with thousands of teams, independent deployment isn't a nice-to-have. It's survival. But Amazon has the org structure to back it up.
🧠What gets worse (the hidden costs)
Here's what people skip over when they're excited about service boundaries.
Network calls replace function calls. An in-process method call takes nanoseconds. A network request takes milliseconds plus serialization overhead. Chain five services together and your p99 latency floor is 250ms before your code even does anything. That latency budget adds up fast, especially when you're trying to hit sub-second response times. If you want more on how traffic flows through these layers, the post on API gateways covers the routing side.
No cross-service transactions. Forget ACID. You're in saga territory now, dealing with eventual consistency and compensation logic. For a lot of apps, this complexity just isn't worth it.
CI/CD per service. N services times M environments equals an explosion of build pipelines. Each one needs monitoring, alerting, and someone who knows what to do when it breaks at 3am.
Observability costs more than the services themselves. Distributed tracing, log aggregation, correlation IDs, and service meshes. You need all of it just to answer "why did this request fail?" Something that was a stack trace in a monolith becomes a detective hunt across 6 services.
On-call surface multiplies. Each service needs an owner. With 50 services you need 50 runbooks, and you'd better hope those runbooks are up to date.
Cognitive load. Developers now need to understand network failure modes, retries, idempotency, and circuit breakers. That's a lot of accidental complexity for a team that just wants to ship features.
Conway's law is not optional
Here's a thing that doesn't get said enough. Your architecture will mirror your org structure whether you plan for it or not. Conway's Law from 1967. It's not a suggestion.
Amazon's two-pizza team model works because one team equals one service equals clear ownership. But if you're a 20-person company with 3 teams and you've drawn 50 microservices on a diagram? You don't have the people to support that. You'll end up with roughly as many real services as you have real teams, no matter what the architecture doc says.
So match your architecture to your org. Not your aspirations.
📌 Signals it's time to split
Start with a monolith. But stay alert for these signals:
- Deployments require cross-team coordination and are becoming a bottleneck. Teams are waiting on each other to ship.
- Teams keep stepping on each other's code. Merge conflicts every sprint, broken features from unrelated changes.
- One component needs to scale 100x while the rest stays flat. Your checkout service gets hammered during sales but your admin panel sits idle.
- Compliance or security requires data isolation. PCI, PII regulations that demand certain data lives in its own boundary.
- Build times are so long they destroy productivity. If your CI takes 45 minutes, people stop running it.
None of these signals mean "rewrite everything into 30 services overnight." They mean: identify one bounded context, extract it using the strangler fig pattern, and see if your life gets better. One at a time.
âš¡ The playbook (short version)
If you're going to split, do it in this order:
- Start modular. Enforce boundaries inside the monolith first. Shopify's approach. Separate your modules with clear interfaces, even if they still deploy together. This is cheap and reversible.
- Identify the bounded context. Which module has a genuinely different scaling profile, team ownership, or data isolation need?
- Strangler fig. Route traffic to the new service incrementally. Old code handles what's left. No big-bang rewrite. Martin Fowler described this pattern back in 2004, and it remains the safest way to extract.
- Own the operational cost. Before you extract, make sure you have CI/CD, monitoring, tracing, and an on-call rotation for the new service. If you don't have these, you're not ready.
If you're partitioning data across services and using event-driven communication, understanding how Kafka partitions and consumer groups work will save you some grief on the messaging side.
Real-world evidence
I'm not just making this up. The pattern "start monolith, split later" has strong backing:
Martin Fowler wrote in 2015: "Almost all the successful microservice stories have started with a monolith that got too big and was broken up." His explicit advice is don't start a new project with microservices, even if you're sure it'll be big enough to justify them eventually.
Segment had 140+ microservices (one per integration partner) and consolidated them back into a single service. Three engineers were spending most of their time keeping the system alive instead of building features. After consolidating, their shared library improvement rate jumped from 32 to 46 per year and on-call paging for load spikes disappeared entirely.
Amazon's Prime Video team moved a video quality monitoring tool from Step Functions plus Lambda back into a single ECS process and cut that tool's infrastructure cost by over 90%. But here's the thing people get wrong about this one: it was one monitoring tool, not Prime Video's entire streaming platform. Prime Video still runs hundreds of services. The 90% figure applies to this specific pipeline's cost, not Amazon's bill.
DHH has been running Basecamp on a single Rails monolith serving millions of users with a small team since 2016. His argument: microservices add complexity that only pays off at organizational scale most companies never reach. Hard to argue with the results.
The short answer
Start with a monolith. Make it modular. Split when a real signal appears, not when a conference talk makes you feel behind. The distributed monolith is the actual enemy here, and you create it by splitting before you understand your domain boundaries.
Most of us aren't building the next Prime Video. We're building apps that need to ship, iterate, and not collapse under the weight of their own infrastructure.
Where else to find me
I write about system design and architecture decisions at arnavsharma.dev.
Top comments (0)