DEV Community: Sushant Gaurav

Communication Between Services: REST, gRPC, and Message Queues

Sushant Gaurav — Tue, 11 Aug 2026 05:41:00 +0000

In the previous article, we explored how large applications gradually evolve from monoliths into collections of smaller, independently deployable services. Instead of one massive application responsible for authentication, payments, inventory, notifications, and analytics, we now have multiple services, each responsible for a single business capability.

At first glance, this seems like a clean and elegant solution. Every service owns its own logic, its own database, and can be deployed independently without affecting the rest of the system.

But splitting an application into multiple services immediately creates a new challenge.

The components that once lived inside the same process now live on different machines.

Inside a monolithic application, communication is almost effortless. If the payment module needs information about a user, it simply calls a function from the user module. The call happens inside the same process, uses the same memory space, and usually completes in a fraction of a millisecond. Developers rarely stop to think about this communication because, from their perspective, it is almost free.

Microservices change this assumption entirely.

The payment service no longer has direct access to the user service. They may be running on different servers, in different containers, or even in different regions of the world. Every interaction between them must now travel through a network.

What was once a simple function call has become a network request.

This seemingly small architectural change has enormous consequences.

Network requests introduce latency. They can fail unexpectedly. They may experience congestion, timeouts, packet loss, or temporary service outages. Unlike local function calls, network communication is inherently unreliable.

This is why experienced engineers often say that moving to microservices means moving into the world of distributed systems.

The application is no longer just executing code.

It is coordinating communication between independent systems.

Why Communication Becomes a System Design Problem

Imagine a customer placing an order on an e-commerce platform.

From the customer's perspective, the process appears straightforward. They click the "Place Order" button, wait a few seconds, and receive a confirmation message.

Behind the scenes, however, the request may travel through several independent services before the order is successfully processed.

The order service must verify the customer.

The inventory service must confirm that the requested items are available.

The payment service must process the transaction.

The notification service must send a confirmation email or SMS.

Each of these services performs its own responsibility, but together they must behave as if they were a single application.

Notice something interesting about this interaction.

The Order Service is not solving business problems alone.

It spends a significant amount of time communicating with other services.

As systems grow larger, communication itself becomes one of the biggest consumers of time and resources.

This is one of the reasons distributed systems are fundamentally different from traditional applications.

The challenge is no longer writing business logic.

The challenge is coordinating independent systems efficiently and reliably.

Not Every Conversation Between Services Is the Same

Suppose the Order Service needs to verify whether a product exists before allowing a purchase.

The answer is needed immediately.

The customer is waiting.

The Order Service cannot continue until it receives a response.

Now consider a different situation.

A payment has been completed successfully, and the customer should receive a confirmation email.

Does the customer really need to wait until the email has been delivered?

Probably not.

The order should complete successfully even if the email arrives a few seconds later.

Now imagine another scenario.

A recommendation engine wants to learn which products users purchase so that it can improve future recommendations.

Should the payment process slow down while waiting for the recommendation engine to process analytics?

Again, the answer is no.

These examples illustrate an important principle in distributed systems:

Not every interaction between services has the same urgency.

Some requests require an immediate response.

Others can happen later.

Some require a direct conversation.

Others simply need to notify another service that an event has occurred.

Understanding these differences is the key to choosing the right communication mechanism.

Synchronous vs Asynchronous Communication

Before discussing REST, gRPC, or messaging systems, we need to understand two fundamental communication models that appear throughout distributed systems.

The first is synchronous communication.

In synchronous communication, one service sends a request and waits for the response before continuing.

It is very similar to a phone call.

When you call someone, the conversation happens in real time. You ask a question and wait for the answer before continuing the discussion.

Most web applications work this way.

When your browser requests a webpage, it waits until the server responds.

Likewise, when one microservice calls another using REST or gRPC, it often waits for the response before moving forward.

This approach is simple to understand because it closely resembles traditional programming.

However, it also creates dependencies.

If Service B becomes slow, Service A also becomes slow.

If Service B becomes unavailable, Service A may fail entirely.

In other words, synchronous communication couples the availability and performance of multiple services.

Asynchronous communication takes a very different approach.

Instead of waiting for an immediate response, one service simply publishes a message and continues its own work.

Receiving services process that message whenever they are ready.

A useful analogy is email.

When you send an email, you do not wait with your computer open until the recipient replies.

You send the message and continue with your day.

The recipient responds later.

Distributed systems often behave in exactly the same way.

This model dramatically reduces coupling between services.

The Order Service no longer needs to know whether the Notification Service is currently running.

Its responsibility ends once the message has been successfully placed into the queue.

The notification can be processed seconds—or even minutes—later without affecting the customer experience.

As systems become larger, this asynchronous style of communication becomes increasingly common because it improves resilience and allows services to operate independently.

Three Ways Services Usually Communicate

Almost every distributed system relies on one of three primary communication mechanisms.

The first is REST, which has become the standard way of building web APIs over the last two decades. It is simple, human-readable, and supported by virtually every programming language and framework.

The second is gRPC, a high-performance communication framework designed for fast and efficient communication between services. Instead of exchanging human-readable JSON, gRPC uses a compact binary protocol, making it significantly faster in many internal service-to-service scenarios.

The third approach uses message queues, where services communicate indirectly by publishing events instead of sending direct requests. Rather than waiting for immediate responses, services exchange information through brokers such as queues or event streams, enabling asynchronous communication and improving system resilience.

Although all three approaches allow services to communicate, they solve different problems.

Choosing between them is rarely about which technology is better.

It is about understanding the nature of the conversation taking place between services.

Should the sender wait for a response?

Does the receiver need to respond immediately?

Can the work happen later?

Should failures affect the user?

The answers to these questions determine the most appropriate communication model.

And this is where our discussion truly begins.

REST: The Language of the Web

Long before microservices became popular, applications already needed a way to communicate over networks.

Web browsers needed to request web pages.

Mobile applications needed to fetch user profiles.

Third-party developers needed access to payment systems, maps, weather data, and social media platforms.

The internet itself needed a common language that every application, regardless of the programming language it was written in, could understand.

This is where REST entered the picture.

REST, which stands for Representational State Transfer, is not a protocol or a programming language. It is an architectural style proposed by Roy Fielding in his doctoral dissertation in the year 2000. Rather than introducing a completely new communication protocol, REST embraced technologies that were already powering the web—primarily HTTP.

This decision played a significant role in REST's widespread adoption.

Instead of asking developers to learn an entirely new ecosystem, REST leveraged concepts they were already familiar with. URLs identified resources, HTTP methods described operations, and responses were returned using formats such as JSON or XML.

Over time, REST became the de facto standard for building APIs, and today, it powers a significant portion of the modern internet.

Whether you're checking your bank balance, booking a flight, ordering food, or scrolling through social media, there's a good chance your device is communicating with backend services through REST APIs.

Thinking in Resources Instead of Functions

One of the biggest mindset shifts when learning REST is understanding that it is resource-oriented, not function-oriented.

In traditional programming, we tend to think in terms of actions.

We write functions like:

createUser()
getUser()
deleteUser()
updateUser()

REST encourages us to think differently.

Instead of focusing on actions, we focus on resources.

A user is a resource.

An order is a resource.

A product is a resource.

A payment is a resource.

Once a resource exists, HTTP methods describe what we want to do with it.

For example:

GET    /users/125
POST   /users
PUT    /users/125
DELETE /users/125

Notice how the URL represents what we are working with, while the HTTP method represents what operation we want to perform.

This separation makes APIs easier to understand because they resemble the structure of the real-world entities they represent.

How a REST Request Travels Through the System

Let's consider a simple example.

A customer opens an e-commerce application and wants to view the details of a product.

The frontend application sends an HTTP request to the Product Service.

The Product Service processes the request, retrieves the necessary information from its database, converts the result into JSON, and sends it back to the client.

The interaction looks something like this:

From the user's perspective, this entire interaction feels almost instantaneous.

Behind the scenes, however, several things have happened.

The request travelled through the internet.

The server authenticated the client.

Business logic was executed.

A database query was performed.

The response was serialised into JSON.

Finally, the data was transmitted back over the network.

Every REST request follows this general request-response lifecycle.

Why REST Became So Popular

One of REST's greatest strengths is its simplicity.

Almost every programming language today can send an HTTP request.

Browsers understand HTTP natively.

Firewalls are designed to work with HTTP traffic.

Cloud platforms, API gateways, reverse proxies, load balancers, and CDNs all understand HTTP exceptionally well.

Because of this universal support, REST APIs can be consumed by virtually anything.

A web browser.

A mobile application.

A desktop application.

Another backend service.

Even command-line tools like curl.

This universality is one of the reasons REST became the default communication mechanism for modern applications.

It reduced friction.

Developers no longer needed specialised libraries or proprietary communication protocols.

If two systems could speak HTTP, they could usually communicate with one another.

REST Is Stateless - And That Is a Good Thing

One of the defining characteristics of REST is that it is stateless.

At first, the word sounds technical, but the idea is remarkably simple.

Every request should contain all the information required to process it.

The server should not rely on memory from previous requests.

Imagine asking someone for directions.

If every time you ask, you provide your current location and destination, they can answer immediately.

They don't need to remember your previous conversations.

REST works in the same way.

Suppose a client sends the following request:

GET /orders/1254
Authorization: Bearer <token>

The request already contains everything the server needs.

It specifies:

Which resource is being requested.
Who is making the request.
Authentication credentials.
Any additional headers or parameters.

The server processes the request, sends the response, and then forgets everything about that interaction.

The next request starts from scratch.

This stateless design provides significant scalability benefits.

Since servers do not need to remember client sessions, incoming requests can be distributed across multiple application instances using a load balancer.

Any server can process any request because every request is self-contained.

This aligns perfectly with the horizontal scaling principles we discussed earlier in this series.

JSON: The Universal Language of REST

Although REST itself does not require JSON, the two have become almost inseparable.

JSON is lightweight, human-readable, and supported by virtually every modern programming language.

A typical REST response might look like this:

{
  "id": 105,
  "name": "Mechanical Keyboard",
  "price": 89.99,
  "stock": 42
}

One reason developers appreciate JSON is that it is easy to inspect.

If something goes wrong, developers can open browser developer tools, examine the response, and immediately understand what the server returned.

This readability makes debugging significantly easier than many binary communication formats.

However, this convenience comes with a trade-off.

Text-based data is generally larger than binary data.

A JSON document contains field names, punctuation, quotation marks, and whitespace, all of which increase the size of the response.

For applications serving millions of requests every minute, these additional bytes become significant.

This limitation eventually motivated the development of faster communication mechanisms such as gRPC, which we'll explore in the next part.

Where REST Truly Shines

REST is exceptionally well suited for communication between clients and servers.

When a web browser requests product information, when a mobile application fetches a user's profile, or when an external partner integrates with your platform, REST is often an excellent choice.

Its simplicity, interoperability, and widespread tooling make it ideal for public-facing APIs.

It is also highly cache-friendly.

Since REST is built on HTTP, it naturally benefits from HTTP caching, proxy servers, reverse proxies, and Content Delivery Networks (CDNs). Responses that do not change frequently can often be cached closer to users, reducing latency and improving performance.

This is one of the reasons many public APIs continue to rely on REST despite the emergence of newer communication technologies.

Its ecosystem is mature, battle-tested, and universally understood.

REST Is Not Perfect

Despite its popularity, REST is not the answer to every communication problem.

Because REST relies on HTTP and typically exchanges JSON, each request carries a certain amount of overhead.

Headers must be transmitted.

JSON must be serialised by the sender.

JSON must be parsed by the receiver.

Every interaction involves opening, processing, and completing an HTTP request-response cycle.

For occasional communication, this overhead is almost negligible.

But imagine hundreds of microservices communicating thousands of times every second.

The cumulative cost becomes noticeable.

Applications requiring extremely low latency or high throughput often begin looking for more efficient alternatives.

Another limitation is that REST is fundamentally request-driven.

One service asks another for information and waits until the response arrives.

As we discussed in the previous part, this synchronous style of communication creates dependencies between services.

If one service becomes slow, the calling service also slows down.

If one service becomes unavailable, requests may begin failing throughout the system.

These limitations do not make REST a poor choice.

They simply highlight that every communication model involves trade-offs.

And understanding those trade-offs is precisely what system design is about.

gRPC: Built for High-Performance Service-to-Service Communication

REST revolutionised the way applications communicate over the web, and even today it remains one of the most widely adopted architectural styles for building APIs. It is simple, readable, and supported by virtually every programming language and framework.

However, as organisations embraced microservices, engineers began noticing a different kind of challenge.

The majority of service-to-service communication was no longer happening between browsers and backend servers.

Instead, it was happening between backend services themselves.

A single user request could trigger communication between dozens of internal services.

An Order Service might call the Inventory Service, which in turn communicates with the Pricing Service, the Recommendation Service, the Shipping Service, and the Notification Service before the original request is completed.

Each REST call may take only a few milliseconds, but when hundreds of these calls occur during the processing of a single request—and millions more occur every day—the overhead begins to accumulate.

The issue wasn't that REST was slow.

The issue was that REST was designed primarily for interoperability and simplicity rather than maximum performance.

This is where gRPC enters the picture.

Developed by Google, gRPC is a high-performance Remote Procedure Call (RPC) framework specifically designed for efficient communication between services.

Unlike REST, which encourages thinking in terms of resources and HTTP operations, gRPC focuses on calling methods on remote services almost as if they were local functions.

This makes service-to-service communication feel much closer to traditional programming while retaining the advantages of distributed systems.

Understanding Remote Procedure Calls

Before understanding gRPC, it helps to understand what a Remote Procedure Call (RPC) actually means.

Imagine writing the following function in your application:

calculateShipping(orderId)

Inside a monolithic application, calling this function is straightforward.

The code executes within the same process, accesses local memory, and immediately returns a result.

Now imagine that the shipping logic has been extracted into its own microservice.

The function still appears to exist, but in reality it now executes on an entirely different machine.

Instead of calling local code, the application must:

Serialise the request.
Send it across the network.
Wait for the remote server to execute the operation.
Receive the response.
Deserialise the result.

From the developer's perspective, however, the interaction still resembles a normal function call.

This is the philosophy behind RPC systems.

Rather than thinking in terms of URLs and HTTP resources, developers think in terms of methods and services.

How gRPC Communicates

One of the biggest differences between REST and gRPC lies in how data is transmitted.

REST typically exchanges data using JSON.

JSON is human-readable, easy to debug, and universally supported.

However, it is also relatively verbose.

Consider a simple JSON response:

{
  "productId": 101,
  "name": "Mechanical Keyboard",
  "price": 89.99
}

Every field name is transmitted over the network.

Quotation marks, commas, braces, and whitespace all contribute to the size of the payload.

For applications serving millions of requests every second, these extra bytes matter.

gRPC takes a different approach.

Instead of JSON, it uses Protocol Buffers (Protobuf), a compact binary serialisation format.

Rather than transmitting descriptive text, Protobuf represents data in a highly efficient binary format.

The resulting payloads are significantly smaller.

Smaller payloads require less bandwidth.

Less bandwidth means faster transmission.

Faster transmission contributes to lower latency and higher throughput.

Although these improvements may appear minor for a single request, they become substantial when multiplied across billions of service calls every day.

Defining APIs Before Writing Code

One particularly elegant aspect of gRPC is that communication contracts are defined before implementation begins.

Instead of writing API endpoints directly, developers first create a Protocol Buffer definition file, commonly called a .proto file.

This file describes:

The services available.
The methods each service exposes.
The structure of every request.
The structure of every response.

Conceptually, it looks like this:

service PaymentService {
    rpc ProcessPayment(PaymentRequest)
        returns (PaymentResponse);
}

From this single definition, tools automatically generate client libraries and server code in multiple programming languages.

This process eliminates much of the repetitive work developers traditionally perform when building APIs.

More importantly, both the client and the server now share the same contract.

This significantly reduces integration errors because both sides are generated from the same specification.

HTTP/2 Makes a Difference

Another reason gRPC performs exceptionally well is that it is built on top of HTTP/2 rather than traditional HTTP/1.1.

Without diving too deeply into networking internals, HTTP/2 introduces several improvements that make communication more efficient.

Instead of opening multiple independent connections, HTTP/2 allows many requests and responses to travel simultaneously over a single connection.

This reduces connection overhead and allows multiple conversations to happen concurrently.

HTTP/2 also compresses headers and improves how data is transmitted across the network.

The result is faster communication with lower latency, especially in environments where services exchange thousands of requests every second.

For large distributed systems, these efficiencies accumulate into significant performance improvements.

Streaming: A Capability REST Doesn't Naturally Provide

Perhaps one of gRPC's most impressive features is its support for streaming.

Traditional REST communication generally follows a simple pattern:

One request.

One response.

The conversation ends.

Sometimes, however, applications need continuous communication rather than isolated requests.

Imagine a live stock market dashboard.

A multiplayer online game.

A GPS navigation system.

A live sports score application.

A monitoring dashboard displaying server metrics in real time.

In these scenarios, repeatedly sending HTTP requests every second becomes inefficient.

gRPC supports multiple communication models.

A client can send one request and receive a continuous stream of responses.

A server can continuously receive streamed requests.

Or both sides can exchange messages simultaneously.

This capability makes gRPC particularly attractive for applications involving live updates and continuous data exchange.

Where gRPC Excels

gRPC is particularly well suited for communication inside distributed systems.

When hundreds of microservices continuously exchange information, performance becomes increasingly important.

The smaller payload sizes, efficient serialisation, persistent HTTP/2 connections, and automatic code generation all contribute to a communication model optimised for speed.

This is why many organisations use gRPC internally while continuing to expose REST APIs to external clients.

External developers appreciate REST because it is simple, human-readable, and easy to integrate.

Internal services benefit from gRPC because efficiency matters more than human readability.

Many modern architectures therefore combine both approaches.

This hybrid architecture allows systems to take advantage of the strengths of each communication model.

gRPC Is Not Always the Right Choice

Despite its impressive capabilities, gRPC is not intended to replace REST entirely.

Because it uses binary Protocol Buffers, developers cannot simply open a browser and inspect responses as easily as they can with JSON.

Debugging often requires specialised tooling.

Public APIs also tend to favour REST because nearly every programming language, browser, and third-party integration platform already understands HTTP and JSON.

REST therefore remains an excellent choice for communication between external clients and backend systems.

gRPC shines when communication happens primarily between trusted internal services where performance, efficiency, and strong API contracts become more important than human readability.

As with every architectural decision we have discussed throughout this series, neither approach is universally better.

REST optimises for simplicity and interoperability.

gRPC optimises for efficiency and performance.

Understanding the requirements of the system is what determines the better choice.

Up to this point, every communication model we have discussed has shared one common characteristic.

Whether we were using REST or gRPC, one service directly contacted another service and waited for it to perform some work.

The communication was immediate.

The sender knew exactly who the receiver was.

The receiver processed the request and returned a response.

While this model works exceptionally well for many scenarios, it also creates an important dependency.

If the receiving service is unavailable, the sender cannot continue.

If the receiving service becomes slow, the sender also becomes slow.

In other words, the health of one service directly affects another.

As distributed systems become larger, these dependencies begin to accumulate.

Imagine a modern e-commerce platform during a festival sale.

A customer clicks the "Place Order" button.

At first glance, the operation seems simple.

The order is created, payment is processed, inventory is updated, a confirmation email is sent, loyalty points are added, analytics are recorded, invoices are generated, warehouse systems are notified, and recommendation engines learn from the purchase.

Although these activities are all triggered by the same user action, they do not all have the same urgency.

The customer certainly expects payment to be processed immediately.

However, the customer does not care whether analytics are updated within five milliseconds or five seconds.

Similarly, recommendation engines can learn about the purchase later without affecting the shopping experience.

This observation leads to one of the most important ideas in distributed systems:

Not every task needs to happen immediately.

Once engineers recognise this, a completely different communication model becomes possible.

Instead of directly asking another service to perform work, a service can simply announce that something has happened.

Any interested service can process that information whenever it is ready.

This is the philosophy behind message queues.

Thinking in Events Instead of Requests

Traditional APIs are request-driven.

One service asks another service to act.

Message queues are event-driven.

Instead of saying,

"Please send an email."

a service simply announces,

"An order has been created."

The service publishing the event does not know who will consume it.

It does not need to know.

Its responsibility ends after successfully publishing the message.

Other services independently decide whether that event is relevant to them.

This subtle difference fundamentally changes the architecture.

Instead of tightly coupling services together, communication becomes loosely coupled.

Imagine dropping a letter into a mailbox.

Your responsibility ends once the letter has been posted.

You do not stand beside the mailbox waiting for the postal worker.

You trust that the postal system will eventually deliver the message.

Message queues operate in much the same way.

Introducing the Message Broker

A message queue introduces an intermediary known as a message broker.

Rather than communicating directly with one another, services communicate through this broker.

The architecture now looks something like this:

Notice what has changed.

The Order Service no longer needs to know anything about the Notification Service.

It does not know whether the Notification Service is running.

It does not know whether analytics are temporarily unavailable.

It simply publishes an event.

The broker takes responsibility for delivering that message to interested consumers.

This greatly reduces dependencies between services.

A Real-World Example

Let's revisit our e-commerce platform.

A customer successfully places an order.

Inside the Order Service, the business transaction completes successfully.

Immediately afterwards, an event named OrderCreated is published to the message broker.

Notice something remarkable.

The Order Service only performs one communication.

The broker handles everything else.

As additional services are introduced in the future—perhaps fraud detection, inventory forecasting, recommendation systems, or customer rewards—the Order Service remains unchanged.

The architecture naturally supports growth.

Why Message Queues Improve Reliability

Suppose the Notification Service suddenly crashes.

What happens?

In a synchronous REST-based system, the Order Service attempts to contact the Notification Service and receives an error.

Depending on how the application is designed, this failure may delay or even fail the entire user request.

With a message queue, the situation is very different.

The Order Service publishes the event successfully.

The broker safely stores the message.

The Notification Service can process it later after recovering.

The customer still receives a successful order confirmation because the critical business operation—the purchase itself—has already completed.

This ability to temporarily decouple producers and consumers is one of the biggest reasons message queues are so widely adopted.

Services no longer have to be available at exactly the same moment.

The broker absorbs temporary failures and smooths communication between systems.

Handling Traffic Spikes

Another major advantage of message queues appears during periods of unusually high traffic.

Imagine an online retailer during Black Friday.

Millions of customers begin placing orders simultaneously.

The Order Service processes purchases as quickly as possible.

However, sending emails, generating invoices, updating analytics, and notifying warehouse systems all require additional processing time.

If every service attempted to perform all this work immediately, the entire platform could become overwhelmed.

Message queues naturally absorb these spikes.

Incoming events accumulate inside the queue.

Consumers process them at a sustainable rate.

This buffering effect prevents downstream services from becoming overloaded.

Instead of rejecting requests, the system gracefully handles temporary bursts in demand.

This is one reason message queues are often compared to waiting lines at supermarkets.

Customers continue arriving.

Some wait briefly.

Cashiers process them one at a time.

The line absorbs fluctuations in arrival rates.

Scaling Consumers Independently

Suppose the Notification Service cannot keep up with incoming messages.

Unlike synchronous communication, we do not necessarily need a faster server.

Instead, we can simply start more consumer instances.

Each consumer retrieves messages independently.

The workload becomes distributed across multiple workers.

As demand increases, additional consumers can be added.

As traffic decreases, unnecessary consumers can be removed.

This makes message queues naturally compatible with horizontal scaling, one of the concepts we explored earlier in this series.

Rather than scaling the producer, we scale the workers responsible for processing queued tasks.

Popular Message Queue Technologies

Over the years, several technologies have become industry standards for asynchronous communication.

RabbitMQ is one of the most widely used traditional message brokers. It provides reliable message delivery, flexible routing, acknowledgements, retries, and mature tooling, making it an excellent choice for many business applications.

Apache Kafka approaches the problem from a slightly different perspective. Rather than acting solely as a message queue, Kafka functions as a distributed event streaming platform capable of handling enormous volumes of events every second. Organisations use Kafka extensively for log aggregation, analytics pipelines, financial systems, IoT platforms, and real-time data processing.

Cloud providers also offer managed messaging services such as Amazon SQS, Google Cloud Pub/Sub, and Azure Service Bus, allowing teams to adopt asynchronous communication without managing broker infrastructure themselves.

Although these technologies differ in implementation, they all pursue the same goal:

Allow independent systems to communicate without requiring them to be available at exactly the same time.

The Cost of Asynchronous Communication

Message queues provide remarkable flexibility, but they also introduce new challenges.

One obvious trade-off is that communication is no longer immediate.

If an email service processes messages five seconds later, that delay is usually acceptable.

If a payment confirmation arrives five seconds later, the customer may become concerned.

Choosing asynchronous communication therefore requires understanding which operations are time-sensitive and which are not.

Another challenge is debugging.

In synchronous systems, following a request is relatively straightforward.

With asynchronous systems, events may pass through brokers, retries, dead-letter queues, and multiple consumers before finally completing.

Understanding the lifecycle of a single business operation becomes considerably more difficult.

This is why mature event-driven systems rely heavily on distributed tracing, centralised logging, and observability tools.

As systems become more asynchronous, visibility becomes just as important as functionality.

Asynchronous Communication Is About Independence

Perhaps the most important lesson about message queues is that they are not simply another communication technology.

They represent a different philosophy.

REST asks another service to perform work immediately.

gRPC performs the same idea more efficiently.

Message queues remove the requirement for immediate cooperation altogether.

Instead of saying,

"Do this now."

they simply say,

"This happened."

That small change dramatically reduces coupling, improves resilience, smooths traffic spikes, and allows systems to evolve independently.

Throughout this article, we have explored three fundamentally different ways in which services communicate inside modern distributed systems.

We began with REST, the communication style that powers much of today's internet. We then looked at gRPC, a high-performance framework designed specifically for efficient service-to-service communication. Finally, we explored message queues, which abandon direct conversations altogether in favor of asynchronous event-driven communication.

At first, these technologies may appear to compete with one another.

After all, they all allow one system to communicate with another.

So a natural question arises:

Which one should we use?

Interestingly, experienced system designers rarely ask that question.

Instead, they ask a different one.

What kind of conversation are these services trying to have?

That subtle shift in thinking often leads to the correct architectural decision.

There Is No Universal Winner

One of the biggest mistakes engineers make when learning system design is searching for the "best" technology.

Should we always use gRPC because it is faster?

Should we replace every REST API with Kafka?

Should every microservice communicate asynchronously?

The answer to all of these questions is no.

Each communication model was designed to solve a different problem.

Using the wrong one is similar to using a screwdriver to hammer a nail.

The tool is not bad.

It is simply solving a different problem.

REST optimises for simplicity.

gRPC optimises for efficiency.

Message queues optimise for independence.

Understanding those goals is far more valuable than memorising implementation details.

When REST Is the Right Choice

REST remains the most appropriate choice whenever communication involves external clients.

Browsers, mobile applications, desktop applications, and third-party developers all understand HTTP exceptionally well.

If your application exposes a public API, REST is often the safest and most practical option.

Imagine a food delivery application.

A customer's mobile app needs to:

View nearby restaurants.
Browse menus.
Place an order.
Track delivery status.
View previous orders.

Each of these actions follows a straightforward request-response pattern.

The client asks for information.

The server responds immediately.

REST fits naturally into this style of interaction.

It is also an excellent choice when readability and interoperability are more important than absolute performance.

Because REST commonly uses JSON, developers can inspect requests with a browser, Postman, or curl, making development and debugging remarkably convenient.

When gRPC Becomes a Better Choice

Now consider the communication happening inside the backend.

The Order Service may need to communicate with the Inventory Service dozens of times every second.

The Recommendation Service may continuously exchange information with machine learning systems.

A Search Service may call several ranking engines before returning results.

These services are all controlled by the same organisation.

There is no need for human-readable JSON.

There is no requirement for browser compatibility.

Performance becomes the primary concern.

This is precisely where gRPC shines.

Its binary serialisation, HTTP/2 transport, persistent connections, and strongly defined contracts make it ideal for internal communication between trusted services.

The user never interacts with gRPC directly.

Instead, it quietly enables fast communication behind the scenes.

Many organisations therefore expose REST APIs publicly while using gRPC internally.

This hybrid architecture combines the accessibility of REST with the efficiency of gRPC.

When Message Queues Are the Better Solution

Some operations simply do not require immediate responses.

Suppose a customer successfully purchases a product.

Several additional activities may need to occur afterwards.

A confirmation email must be sent.

Analytics should record the purchase.

Inventory forecasting should update demand predictions.

Warehouse systems should prepare packaging.

Recommendation engines should learn from customer behaviour.

Notice that none of these tasks should delay the checkout experience.

The customer only cares that the order has been placed successfully.

Everything else can happen afterwards.

This makes asynchronous communication a far better choice than synchronous requests.

Instead of contacting every downstream service directly, the Order Service simply publishes an OrderCreated event.

Every interested service processes that event independently.

The checkout experience remains fast.

The architecture becomes more resilient.

New services can subscribe to the event without modifying existing code.

This ability to evolve naturally is one of the greatest strengths of event-driven architectures.

Modern Systems Rarely Choose Just One

One of the most interesting observations about large technology companies is that they rarely commit to a single communication mechanism.

Instead, they combine several approaches, allowing each to solve the problems it handles best.

A typical architecture might resemble the following:

Notice how each communication model has a clearly defined responsibility.

The mobile application communicates with backend services using REST because HTTP and JSON are universally supported.

Internal services communicate with one another using gRPC because efficiency matters more than readability.

Business events are distributed through message queues because many downstream services do not require immediate responses.

Rather than competing, these technologies complement one another.

Choosing the Right Communication Style

Whenever designing communication between services, it is useful to begin by asking a series of simple questions.

The first question is whether the caller requires an immediate response.

If the answer is yes, synchronous communication such as REST or gRPC is usually appropriate.

The second question concerns performance.

If communication happens frequently between internal services and latency is critical, gRPC often provides significant advantages.

The third question is whether the work must happen immediately.

If the answer is no, asynchronous messaging usually results in a simpler and more resilient architecture.

Finally, consider ownership.

If the API will be consumed by external customers, partners, or public developers, REST remains the most approachable option because of its universal adoption and extensive tooling.

Thinking about the nature of the conversation rather than the technology itself almost always leads to better architectural decisions.

Communication Is About Trade-offs

Throughout this System Design series, one theme has appeared repeatedly.

There is rarely a perfect solution.

Monoliths trade flexibility for simplicity.

Distributed systems trade simplicity for scalability.

Horizontal scaling trades hardware upgrades for coordination.

Microservices trade organisational independence for operational complexity.

Communication mechanisms follow the same pattern.

REST is easy to understand but introduces additional overhead.

gRPC improves efficiency but sacrifices some simplicity and requires specialised tooling.

Message queues increase resilience and decoupling but introduce eventual consistency and make systems harder to observe and debug.

Good system design is therefore not about selecting the newest technology.

It is about selecting the technology whose trade-offs best match the problem you are trying to solve.

Final Thoughts

As applications evolve from monoliths into distributed systems, communication becomes one of the most important architectural concerns.

The quality of a distributed system is determined not only by how well individual services are implemented but also by how effectively those services cooperate.

REST, gRPC, and message queues each represent a different philosophy of communication.

REST emphasises simplicity and broad compatibility.

gRPC emphasises speed, efficiency, and strongly defined contracts.

Message queues emphasise independence, resilience, and asynchronous processing.

The most successful systems are rarely built around just one of these approaches.

Instead, they combine all three, allowing each communication model to solve the problems for which it was designed.

Perhaps that is the most important lesson from this article.

In system design, the goal is not to find a single technology that solves every problem.

The goal is to understand the strengths and limitations of each tool well enough to know when to use it.

And that ability to choose the right tool for the right problem is what ultimately distinguishes a good software engineer from a great system designer.

Microservices Architecture: Benefits, Challenges, and When to Use It

Sushant Gaurav — Tue, 28 Jul 2026 05:41:00 +0000

For many engineers, microservices are one of the first concepts that come to mind when discussing modern system design.

Job descriptions mention them constantly. Conference talks celebrate them. Architecture diagrams often contain dozens or even hundreds of small services communicating with one another. It is easy to come away with the impression that microservices represent the final destination of software architecture - that every successful application eventually becomes a collection of independently deployed services.

The reality is considerably more nuanced.

Microservices are not the next version of monoliths in the same way that smartphones replaced feature phones. They are not universally better, nor are they the inevitable future of every application. Instead, microservices are an architectural response to a very specific set of problems that begin to appear as systems and organisations grow.

To understand why microservices exist, we first need to understand the world before them.

For a very long time, most applications were built as monoliths. User authentication, product management, payments, inventory, notifications, analytics, reporting, and administration all lived inside a single codebase and were deployed together as a single application.

For small and medium-sized systems, this approach worked remarkably well.

A developer could clone the repository, run the application locally, and understand the request flow from beginning to end. Debugging was relatively straightforward because everything lived in one place. Deployments involved building and releasing a single artefact. Transactions across modules were simple because all the code ran within the same process and often accessed the same database.

This simplicity is one of the reasons monoliths continue to power a large percentage of software systems around the world.

However, successful systems have a tendency to grow in ways that architecture diagrams rarely anticipate.

Teams become larger.
Features become more numerous.
Release cycles become more frequent.
Different parts of the application begin evolving at different speeds.
What started as a clean and elegant codebase slowly begins to feel heavier.
A small change to one module unexpectedly affects another.
Deployment times increase.
Testing becomes slower.
The number of developers working in the same repository continues growing until coordination itself becomes a challenge.

Eventually organizations discover that the problem is no longer computational complexity.

It is organisational complexity.

A payment team wants to deploy a new feature, but must wait for the inventory team to finish testing their changes.

The recommendation engine requires additional compute resources during peak traffic periods, but scaling means scaling the entire application, including components that are barely being used.

Different teams want to adopt different technologies, databases, and deployment strategies, yet the architecture forces everyone into the same decisions.

The application begins behaving less like a product and more like a city that has grown without urban planning.

This is the environment from which microservices emerged.

Instead of treating the application as a single deployable unit, the system is divided into smaller services that each own a specific business capability.

The payment service owns payments.
The inventory service owns inventory.
The notification service owns notifications.
The recommendation service owns recommendations.

Each service becomes responsible for its own logic, data, deployment, and scaling requirements.

Conceptually, the architecture starts looking something like this:

The most important aspect of this diagram is not the number of services.

It is ownership.

Each service becomes a small system with clear responsibilities and well-defined boundaries. Teams can work independently because they no longer need to understand the entire application to make progress in their own domain.

This shift introduces one of the biggest philosophical differences between monoliths and microservices.

In a monolith, components communicate through function calls.
In microservices, communication happens over a network.

At first, this sounds insignificant.

After all, a service call is just another request.

In reality, this changes almost everything.

A function call inside a monolith takes microseconds and rarely fails.

A network request may take milliseconds, may experience congestion, may time out, may be retried, or may fail.

What used to be a simple method invocation suddenly becomes a distributed systems problem involving latency, retries, availability, and fault tolerance.

This is one of the reasons many engineers say that adopting microservices means adopting distributed systems.

The complexity does not disappear.

It simply moves.

Instead of managing complexity inside a codebase, we begin managing complexity between services.

This is neither good nor bad.

It is a trade-off.

And like every major architectural decision in system design, understanding those trade-offs is far more important than understanding the technology itself.

Perhaps the greatest misconception surrounding microservices is the belief that they are primarily a scaling strategy.

They are not.

Companies rarely move to microservices because their servers cannot handle traffic.

More often, they move because their teams cannot handle coordination.

When hundreds or thousands of engineers are working on the same product, organisational scalability becomes just as important as technical scalability.

Microservices allow teams to move independently, deploy independently, and evolve independently.

This alignment between software architecture and team structure is so common that it is often summarised by Conway's Law:

Organisations tend to design systems that mirror their communication structures.

A company with ten independent teams often ends up building ten independent services.

A company with a single tightly integrated team often builds a monolith.

Architecture and organisational design are frequently reflections of one another.

This is why the decision to adopt microservices is rarely just an engineering decision.

It is often a business decision, a team decision, and an operational decision all at the same time.

Understanding this context is essential because, without it, microservices can appear to be an obvious improvement over monoliths.

They are not.

They solve certain problems extremely well.

They also create entirely new categories of problems that monoliths never had to worry about.

Independent Deployment Changes the Development Process

One of the biggest advantages of microservices is that they allow different parts of the system to evolve independently.

Consider an e-commerce platform built as a monolith.

The payment team develops a new payment integration. At the same time, the recommendation team is experimenting with a new machine learning model, while the notification team is redesigning the email delivery pipeline.

In a monolithic architecture, all these changes eventually converge into the same application deployment.

Even though these teams are working on completely unrelated features, they still share release cycles, testing pipelines, deployment windows, and rollback procedures.

This creates coordination overhead.

A delay in one team may delay everyone else.

A bug in one component may prevent unrelated features from reaching production.

As organisations grow, these dependencies become increasingly painful.

Microservices attempt to solve this by allowing services to be deployed independently.

The payment team can deploy payment changes without waiting for the recommendation team.

The notification service can release a new version without affecting inventory management.

The recommendation service can experiment rapidly without introducing risk into unrelated parts of the application.

The result is not necessarily faster code execution.

The result is faster organisational movement.

Independent Scaling Allows Resources to Follow Demand

Another significant advantage appears when different parts of the system experience different workloads.

In most applications, traffic is rarely distributed evenly.

An online marketplace may process millions of product searches every hour while receiving far fewer payment requests.

A video streaming platform may generate enormous traffic for video delivery while user profile services remain relatively idle.

A social network may receive vastly more feed requests than account creation requests.

In a monolithic architecture, scaling one part of the application often means scaling everything.

Even if only the search functionality is under pressure, additional instances of the entire application may need to be deployed.

This increases infrastructure costs and wastes resources.

Microservices allow organisations to scale individual services according to their own requirements.

The search service may run on twenty instances while the payment service operates comfortably on three.

This granularity allows infrastructure decisions to reflect actual usage patterns rather than architectural limitations.

Technology Diversity Becomes Possible

One of the less discussed advantages of microservices is technological flexibility.

Different problems are often best solved using different tools.

A recommendation engine performing machine learning inference may benefit from Python.

A high-performance messaging system may be better suited to Go.

A payment system requiring mature transactional guarantees may prefer Java and relational databases.

An analytics pipeline may rely heavily on distributed data processing frameworks.

In a monolith, these decisions are constrained by the technology choices of the entire application.

Microservices make this separation possible because services communicate through APIs rather than internal language constructs.

The recommendation service does not need to know how the payment service is implemented.

It only needs to understand the contract exposed by its API.

This flexibility is powerful, but as we will see shortly, it can also become dangerous if left uncontrolled.

Fault Isolation Improves Resilience

Failures are inevitable in software systems.

Servers crash.

Databases become unavailable.

Network partitions occur.

Dependency time out.

The question is rarely whether failure will happen.

The question is how much of the system fails when it does.

In a tightly coupled monolith, failures can propagate rapidly.

A memory leak in one component may exhaust resources for the entire application.

A slow database query may impact unrelated functionality.

An overloaded subsystem may cause widespread degradation.

Microservices improve isolation by creating boundaries between components.

If the recommendation service experiences problems, users may temporarily lose personalised recommendations while the rest of the platform continues operating normally.

The application degrades gracefully rather than collapsing completely.

This idea is often referred to as the blast radius of failure.

Microservices aim to reduce the blast radius.

Reducing the impact of failures is one of the strongest arguments in favour of service decomposition.

The Complexity That Replaces the Monolith

At this point, microservices can sound almost ideal.

Independent deployments.

Independent scaling.

Fault isolation.

Technology flexibility.

So why doesn't every company immediately move to microservices?

Because microservices solve one type of complexity by introducing another.

Remember the observation from the previous part:

Complexity does not disappear. It moves.

In a monolith, communication happens through local function calls.

In microservices, communication happens through networks.

And networks are fundamentally unreliable.

A function call rarely fails.

A network request can fail for dozens of reasons:

Packet loss
Timeouts
Congestion
DNS failures
Load balancer failures
Service crashes
Partial outages

Suddenly, developers must think about retries, circuit breakers, backoff strategies, and timeout management.

The application is no longer just software.

It is a distributed system.

Data Becomes One of the Hardest Problems

Monoliths often enjoy a luxury that microservices lose:

A shared database.

A single transaction can update multiple tables atomically.

Consistency is relatively easy to achieve.

Microservices intentionally avoid this model because shared databases create coupling between services.

Instead, each service typically owns its own data.

This improves service independence but introduces difficult questions.

What happens when a payment succeeds but order creation fails?

How do we maintain consistency across multiple databases?

How do we roll back distributed operations?

How do services coordinate business workflows?

These problems eventually lead engineers toward concepts such as event-driven architectures, sagas, outbox patterns, and distributed transactions.

Problems that never existed inside a simple monolith suddenly become central architectural concerns.

Observability Stops Being Optional

Debugging a monolith can often be as simple as following logs from a single application instance.

Microservices remove that convenience.

A single user request may travel through:

API Gateway
Authentication Service
User Service
Payment Service
Inventory Service
Notification Service

When something goes wrong, finding the source of the problem becomes significantly harder.

This is why mature microservice architectures invest heavily in:

Distributed tracing
Centralised logging
Metrics collection
Monitoring systems

Observability evolves from a useful feature into a critical requirement.

By this point, microservices can feel both exciting and intimidating at the same time.

On one hand, they offer independent deployments, independent scaling, fault isolation, and organisational flexibility. On the other hand, they introduce distributed systems complexity, operational overhead, and entirely new categories of failure.

This naturally leads to the most important question in the entire discussion:

When should you actually use microservices?

Interestingly, the answer is not "as early as possible."

In fact, for many systems, that answer would be actively harmful.

One of the biggest misconceptions in modern software engineering is the belief that successful companies started with microservices and scaled effortlessly from day one.

That is not what happened.

Many of today's largest technology companies began with monolithic architectures.

The early versions of companies like Amazon, Netflix, and Facebook were significantly more monolithic than many engineers realise.

This was not because the engineers lacked knowledge.

It was because monoliths optimise for something that startups desperately need:

speed.

When a company is trying to validate an idea, acquire users, and find product-market fit, the biggest risk is rarely scalability.

The biggest risk is building the wrong product.

During this stage, simplicity is a competitive advantage.

A monolith allows teams to move quickly. Features can be built without worrying about service boundaries, network contracts, distributed tracing, or inter-service communication protocols.

Developers can change database schemas quickly.

Refactoring is easier.

Testing is simpler.

Deployment pipelines are smaller.

Operational costs are lower.

Most importantly, the engineering team can spend its energy solving business problems instead of infrastructure problems.

This is why one of the most common pieces of advice in software architecture is:

Start with a monolith and earn your microservices.

That statement is not a criticism of microservices.

It is recognition that complexity should arrive only when it solves a real problem.

Microservices are not free.

Every service introduces additional deployment pipelines, monitoring requirements, infrastructure costs, API contracts, and operational responsibilities.

A system with fifty microservices is not managing one application.

It is managing fifty applications.

That distinction becomes incredibly important as organisations grow.

Signs That a Team May Be Ready for Microservices

While there is no universal threshold, certain patterns appear repeatedly in organisations that successfully adopt microservices.

One common signal is organisational growth.

When dozens or hundreds of engineers are contributing to the same codebase, coordination begins to slow development. Teams start stepping on each other's changes. Releases become increasingly risky because every deployment contains modifications from many unrelated teams.

At some point, the bottleneck is no longer technical architecture.

It is communication.

Microservices can reduce this friction by allowing teams to own individual services and deploy independently.

Another signal appears when different parts of the application have drastically different scaling requirements.

Consider a streaming platform.

The video processing pipeline may require enormous compute resources while account management requires comparatively little infrastructure.

Scaling both components together becomes inefficient.

Microservices allow resources to follow actual demand.

The same pattern appears in machine learning systems, recommendation engines, search platforms, and analytics pipelines.

Some components naturally become much larger than others.

Independent scaling starts becoming valuable.

A third signal appears when release velocity becomes a problem.

If a minor update to the notification system requires coordination across multiple teams and weeks of testing for unrelated components, the deployment process itself becomes an obstacle to business growth.

Independent deployments can dramatically improve development speed in these environments.

Signs That a Team Probably Should Not Use Microservices

Interestingly, identifying situations where microservices are unnecessary is often easier.

A small engineering team working on a relatively straightforward application rarely benefits from distributed complexity.

If five developers are building an internal business application with moderate traffic, microservices often create more problems than they solve.

The team now needs:

Service discovery
Monitoring infrastructure
Centralised logging
API versioning
Deployment orchestration
Network security policies
Inter-service authentication
Distributed tracing

None of these problems existed before.

The architecture becomes more sophisticated while the business problem remains the same.

This is one of the reasons many organisations accidentally build what engineers jokingly call:

a distributed monolith.

A distributed monolith has all the operational complexity of microservices but none of their advantages.

Services are tightly coupled.

Deployments still require coordination.

Failures cascade between systems.

Scaling remains difficult.

The architecture becomes harder to understand without becoming more flexible.

This is often considered one of the most painful outcomes of premature microservice adoption.

The Hybrid Reality of Modern Systems

The discussion is often framed as a choice between monoliths and microservices.

Real systems are usually far more nuanced.

Many organisations operate somewhere in the middle.

A company may begin with a monolith and gradually extract services as scaling requirements emerge.

Certain domains may remain inside the monolith for years while others become independent services.

A recommendation engine may become a standalone service because it has unique computational requirements.

Payment processing may become isolated because it has strict security and compliance requirements.

Analytics pipelines may move into separate services because they require entirely different storage and processing technologies.

Meanwhile, user management and administration may continue living inside the monolith.

The result is neither a pure monolith nor a pure microservice architecture.

It is a pragmatic architecture shaped by business needs rather than ideology.

This gradual evolution is far more common than complete rewrites.

In fact, many experienced architects prefer incremental extraction because it allows systems to evolve naturally rather than forcing large architectural migrations.

The Most Important Lesson About Microservices

Perhaps the most valuable lesson in this entire discussion is that architecture is ultimately a tool for managing complexity.

Monoliths manage complexity by keeping everything together.

Microservices manage complexity by separating things apart.

Neither approach eliminates complexity.

They simply decide where it lives.

A small system with microservices may end up more complicated than necessary.

A massive global platform with a monolith may eventually become impossible to maintain.

Good architecture is rarely about following trends.

It is about understanding trade-offs.

The best architects are not the ones who always choose microservices.

They are the ones who know when not to.

Final Thoughts

Microservices changed the software industry because they allowed systems and organisations to grow beyond the limits of a single application and a single team.

They enabled independent deployment, independent scaling, and independent ownership.

But they achieved these benefits by embracing the realities of distributed systems: unreliable networks, partial failures, eventual consistency, and operational complexity.

That trade-off is the essence of microservices.

They are not an upgrade from monoliths.

They are a different answer to a different problem.

And understanding the problem is far more important than understanding the technology.

Data Partitioning and Sharding: How Systems Scale Data

Sushant Gaurav — Tue, 14 Jul 2026 05:41:00 +0000

When engineers first learn about scalability, the conversation usually revolves around application servers.

If traffic increases, add more servers. If requests increase, add a load balancer. If latency increases, introduce caching. For a while, this works remarkably well.

Application servers are relatively easy to scale because they are typically stateless. If one server becomes overloaded, another can be added behind the load balancer, and traffic can simply be distributed between them.

Databases are different.

Unlike application servers, databases hold state. They contain user information, transactions, product catalogues, messages, orders, and every other piece of information that gives an application meaning. They are the memory of the system.

And because databases hold state, scaling them becomes significantly harder.

Eventually, almost every growing system encounters the same problem:

The application can still handle more traffic, but the database cannot.

This is where data partitioning enters the picture.

Why Bigger Databases Are Not Always the Answer

The first response to a struggling database is usually straightforward.

Increase the machine size.
Add more CPU cores.
Add more memory.
Upgrade to faster SSDs.
Move to a more powerful server.

This approach is known as vertical scaling, and just like application servers, databases can benefit from it for a surprisingly long time.

For many startups and medium-sized applications, vertical scaling is often enough.

However, every machine eventually reaches physical and financial limits.

There is always a larger server available—until there isn't.

At some point, upgrading hardware stops providing meaningful improvements. Even worse, the entire system becomes dependent on a single machine.

No matter how powerful that machine becomes, it is still only one machine.

If it fails, the database fails. If the database fails, the application fails.

This creates both a scalability problem and an availability problem.

The question therefore, changes from:

"How do we make this database bigger?"

to:

"How do we distribute the database itself?"

Understanding Data Partitioning

Data partitioning is the process of dividing a large dataset into smaller and more manageable pieces.

Instead of storing all users, all products, and all orders on a single database server, the data is split into multiple partitions and distributed across several machines.

The idea itself is surprisingly intuitive.

Imagine a library containing one hundred million books.

Storing every book inside a single building would eventually create problems. The building would become too large, too expensive, and too difficult to maintain.

A more practical solution would be to divide the books among multiple buildings.

Each building stores only a subset of the collection. Together, the buildings form a complete library system. Individually, each building only manages a portion of the data.

Databases use the same principle.

From the perspective of the application, the database may still appear to be a single logical system.

Behind the scenes, however, the data is physically distributed across multiple machines.

This separation between logical simplicity and physical complexity is one of the defining characteristics of modern distributed systems.

What Is Sharding?

The terms partitioning and sharding are often used interchangeably, and in many practical discussions they effectively mean the same thing.

Strictly speaking, partitioning refers to the broader concept of splitting data into smaller pieces.

Sharding is a specific type of partitioning in which those pieces are distributed across different physical machines.

You can think of sharding as horizontal partitioning of data.

Instead of making a single database server larger, we make the database wider by adding more machines.

Each shard becomes responsible for storing a subset of the overall dataset.

The application no longer talks to a single database server.

Instead, it communicates with a routing layer that determines which shard owns the requested data.

This may sound like a small architectural change.

In reality, it changes everything.

Because the moment data exists in multiple locations, entirely new problems begin to emerge.

Why Replication Alone Cannot Solve This Problem

When discussing database scaling, many engineers initially assume replication can solve everything.

After all, if one database becomes overloaded, why not simply create additional copies of it?

Replication certainly helps, but it solves a very different problem.

Replication creates multiple copies of the same data.

Sharding divides the data into different pieces.

Consider an application with one hundred million users.

With replication, every replica contains all one hundred million users.

This approach improves read scalability because read requests can be distributed across multiple replicas.

It also improves availability because the system can continue operating if one replica fails.

However, the primary database still receives every write request.

Every new user registration, every order creation, and every transaction still goes to the same machine.

Eventually, that machine becomes the bottleneck.

Sharding addresses a completely different dimension of scalability.

Instead of asking:

"How do we serve more readers?"

It asks:

"How do we store and write more data?"

Why Large Internet Systems Depend on Sharding

Modern internet companies operate at scales that simply cannot be supported by a single database server.

A social media platform may store billions of posts.

An e-commerce platform may process millions of transactions every day.

A ride-sharing platform may continuously ingest location updates from millions of devices.

The volume of data alone makes single-node databases impractical.

Sharding allows these systems to continue growing incrementally.

When storage requirements increase, additional shards can be introduced.

When write throughput increases, new shards can absorb part of the workload.

Instead of upgrading to increasingly expensive hardware, capacity can be expanded horizontally.

This is one of the reasons why sharding is often described as bringing horizontal scaling to databases.

The philosophy is the same as horizontal scaling for application servers.

Rather than building larger machines, we build larger systems.

The Hidden Complexity of Sharding

At this point, sharding may sound almost too good to be true.

More storage.
More throughput.
Better scalability.

So why doesn't every application use it from day one?

Because sharding introduces complexity that simply does not exist in single-database systems.

The system must now answer difficult questions.

How does the application know which shard contains a particular user's data?
What happens when one shard becomes significantly larger than the others?
How are transactions handled when data spans multiple shards?
What happens if a shard fails?
How are backups managed?
How are joins performed across shards?
These problems are not implementation details.
They are some of the hardest challenges in distributed systems.

And solving them requires entirely new techniques and architectural patterns.

The Fundamental Question of Sharding

The moment a system decides to split its data across multiple database servers, a surprisingly simple question becomes one of the most important architectural decisions in the entire system:

How do we decide which shard stores which data?

A single database never has to think about this problem. Every query goes to the same machine. Every user record lives in the same place. Every order, product, message, and transaction can be found by asking a single database server.

Sharding changes this completely.

Now the system must determine where data belongs before it can even execute a query.

If a user with ID 125847 logs into the application, the system first needs to answer:

Which shard owns user 125847?

Only then can the actual database query begin.

This process is known as a sharding strategy, and the choice of strategy has enormous implications for scalability, performance, and operational complexity.

Range-Based Sharding

One of the earliest and most intuitive approaches is range-based sharding.

The idea is simple.

Each database server becomes responsible for a specific range of values.

For example:

Shard 1 stores users with IDs from 1 to 1 million.
Shard 2 stores users with IDs from 1 million to 2 million.
Shard 3 stores users with IDs from 2 million to 3 million.

Visually, the architecture looks something like this:

At first glance, this approach seems almost perfect.

It is easy to understand. It is easy to implement. It is easy to debug. If someone asks where user 2,345,678 lives, the answer is immediately obvious.

The problem is that real-world data is rarely distributed evenly.

Imagine a social media platform where new users are constantly signing up.

Most new users will have the latest IDs.

This means almost all write traffic ends up hitting the newest shard while older shards remain relatively idle.

One database server becomes overloaded while others sit underutilised.

This phenomenon is known as a hotspot, and it is one of the biggest challenges in distributed databases.

Understanding Hotspots

Hotspots occur when certain shards receive significantly more traffic than others.

The issue may arise from user growth patterns, geographical concentration, or uneven business activity.

Consider an e-commerce platform during a flash sale.

If products are partitioned based on category and electronics happens to be the most popular category, the shard responsible for electronics suddenly receives an overwhelming amount of traffic.

Meanwhile, other shards continue operating normally.

The irony is that the system may have dozens of database servers available, yet performance still suffers because the load distribution itself is uneven.

This is one of the reasons engineers started looking for more balanced approaches.

Hash-Based Sharding

Hash-based sharding attempts to solve this problem by distributing data more evenly across servers.

Instead of storing data according to ranges, the system applies a mathematical hash function to the shard key.

The result of the hash determines where the data will live.

Conceptually, it works like this:

hash(user_id) % number_of_shards

Suppose the system currently has four shards.

If the hash value of a user ID results in 2, that user is stored on shard two.

If the result is 3, the user belongs to shard three.

The actual values are less important than the outcome:

The distribution becomes much more uniform.

Unlike range-based sharding, newly created users are unlikely to end up on the same shard.

Writes become naturally distributed across the cluster.

This significantly reduces hotspots and improves scalability.

However, hash-based sharding introduces a new problem.

Humans lose predictability.

With range sharding, engineers immediately know where a user lives.

With hash sharding, finding the location of a record requires executing the hashing algorithm.

The routing layer becomes mandatory.

The Problem of Growth

Eventually, every successful system encounters another difficult question:

What happens when four shards are no longer enough?

The obvious answer seems simple:

Add a fifth shard.

Unfortunately, hash-based systems make this surprisingly painful.

Consider the following example:

hash(user_id) % 4

Now suppose we add a fifth shard:

hash(user_id) % 5

Suddenly, almost every user maps to a different shard than before.

Data that previously belonged to shard one may now belong to shard three.

Data from shard three may move to shard five.

A huge percentage of the database needs to be migrated.

For systems containing billions of records, this migration can become extraordinarily expensive and risky.

This challenge led to one of the most elegant ideas in distributed systems.

Consistent Hashing

Consistent hashing was developed specifically to minimise data movement when infrastructure changes.

Instead of mapping data directly to servers, both servers and data are placed on a logical ring.

Each piece of data is assigned to the next available server in the ring.

Now imagine adding a new server.

Instead of redistributing the entire dataset, only a small portion of the data moves to the new machine.

Most records remain exactly where they were.

This dramatically reduces migration costs and makes scaling much safer.

Consistent hashing is widely used in distributed systems because infrastructure growth becomes a normal operational activity rather than a major migration project.

Systems such as distributed caches, distributed databases, and object storage platforms frequently rely on this technique.

Choosing the Right Shard Key

Interestingly, one of the most important decisions in sharding is often overlooked:

choosing the shard key.

The shard key is the attribute used to determine where data lives.

Examples include:

User ID
Customer ID
Geographic region
Product category
Organisation ID

Choosing the wrong shard key can create a severe imbalance in the system.

Suppose a ride-sharing platform partitions data by city.

This may work well initially.

But if one city suddenly becomes ten times larger than every other city, that shard becomes a permanent hotspot.

Similarly, partitioning social media users by country may appear reasonable until a handful of countries dominate global traffic.

A good shard key distributes data evenly while still allowing efficient queries.

Finding that balance is often one of the hardest design decisions in large systems.

Why Cross-Shard Queries Become Expensive

In a traditional database, a query can freely join tables because all the data lives on the same machine.

Sharding changes this assumption.

Imagine a query asking for information that spans multiple shards.

The database can no longer answer the question locally.

Instead, the system must:

Send requests to multiple shards.
Wait for responses from each server.
Merge the results.
Return the final response.

A query that once required a single machine now becomes a distributed operation involving network communication and coordination.

This is why sharding often forces teams to rethink their data models and access patterns.

The Reality of Modern Systems

The truth is that most large internet companies use a combination of techniques rather than relying on a single strategy.

Sharding is combined with:

Replication for availability.
Caching for performance.
Load balancing for traffic distribution.
CDNs for global content delivery.

Modern distributed systems are rarely built from one idea alone.

They are ecosystems of complementary techniques working together.

A shard may contain replicas.

Those replicas may sit behind load balancers.

Frequently accessed data may never reach the database at all because it is served from a cache.

This layered architecture is what allows modern systems to operate at a global scale.

Final Thoughts

Sharding is often introduced as a database optimisation technique.

In reality, it is much more than that.

It represents the moment when data itself becomes distributed.

And once data becomes distributed, the system enters an entirely new world of trade-offs involving routing, coordination, consistency, and fault tolerance.

Scaling application servers is relatively straightforward.

Scaling data is where distributed systems become truly interesting.

Because at the internet scale, the challenge is no longer simply storing information.

The challenge is knowing where that information lives, how to find it quickly, and how to keep the entire system functioning while millions of users are trying to access it simultaneously.

Databases in System Design - SQL vs NoSQL (When and Why)

Sushant Gaurav — Tue, 30 Jun 2026 05:41:00 +0000

Every system, no matter how simple or complex, eventually converges to a single fundamental need: it must remember things.

Users sign up, data gets generated, transactions happen, content is created, relationships form, and all of this needs to be stored, retrieved, and updated reliably. At a small scale, this seems trivial. You pick a database, store your data, and move on.

But as systems grow, data stops being passive.

It becomes the centre of gravity around which everything else revolves.

Performance depends on it. Scalability depends on it. Consistency, availability, and even user experience are shaped by how data is stored and accessed.

And this is where one of the most important decisions in system design emerges:

Should you use a relational database (SQL), or a non-relational database (NoSQL)?

At first glance, this appears to be a technology choice. But in reality, it is a design philosophy decision; one that reflects how your system models the world.

The Structured World of SQL

Relational databases, often referred to as SQL databases, are built on a simple but powerful idea: data should be organised into structured tables with clearly defined relationships.

This model has been around for decades and is grounded in mathematical principles. Each piece of data fits into a predefined schema. Tables are connected through relationships, and queries allow you to retrieve and combine data in flexible ways.

This structure brings clarity.

When you design a relational schema, you are effectively defining the shape of your data upfront. You decide:

What entities exist
What attributes do they have
How they relate to each other

Once defined, this structure is enforced strictly. Every piece of data must conform to it.

This may feel restrictive at first, but it provides a powerful guarantee:

The data in the system is consistent, predictable, and reliable.

This is why relational databases are widely used in systems where correctness is critical. Financial systems, transactional platforms, and enterprise applications rely on the guarantees provided by SQL databases to maintain integrity.

For example, systems within companies like Oracle and Microsoft have long relied on relational databases to manage structured, high-integrity data.

The Power of Relationships

One of the defining strengths of SQL databases is their ability to model relationships between data.

Consider a simple scenario:

A user places an order
An order contains multiple items
Each item belongs to a product

In a relational database, these relationships are explicitly defined. You can join tables together and retrieve complex, interconnected data with a single query.

This ability to perform joins allows systems to answer rich questions:

What products has a user purchased?
Which items are most frequently bought together?
What is the total value of all orders in a given time period?

This expressiveness is one of the reasons SQL remains dominant in many domains.

Consistency

Relational databases are designed with strong consistency in mind.

They follow properties often referred to as ACID:

Atomicity
Consistency
Isolation
Durability

While these terms may sound theoretical, their impact is very real.

They ensure that:

Transactions either complete fully or not at all
The database remains in a valid state
Concurrent operations do not interfere in harmful ways
Data is not lost, even in the event of failure

This makes SQL databases particularly well-suited for systems where correctness cannot be compromised.

Think of banking systems. If money is deducted from one account but not credited to another due to a failure, the system becomes unreliable. SQL databases are designed to prevent such scenarios.

The Limits Begin to Appear

Despite their strengths, relational databases begin to show limitations as systems scale.

The first challenge is scalability.

SQL databases are traditionally designed to scale vertically. You increase the power of a single machine to handle more load. But as we explored earlier, vertical scaling has limits, both in terms of hardware and cost.

The second challenge is flexibility.

Because the schema is predefined, making changes to the structure of data can be complex. Adding new fields, modifying relationships, or evolving the data model often requires careful migrations.

In rapidly evolving systems, this rigidity can slow down development.

The third challenge is distribution.

Distributing relational databases across multiple nodes while maintaining strong consistency is difficult. It introduces coordination overhead, which can impact performance and availability.

These challenges do not make SQL obsolete, but they create space for a different approach.

Enter NoSQL - A Different Philosophy

NoSQL databases emerged not as a replacement for SQL, but as a response to the challenges of scale, flexibility, and distribution.

Instead of enforcing a rigid schema, NoSQL systems embrace a more flexible approach to data modelling.

Data is often stored in formats that closely resemble how it is used in applications:

Documents
Key-value pairs
Wide-column structures
Graphs

This flexibility allows developers to evolve data models quickly without being constrained by predefined schemas.

But more importantly, NoSQL databases are designed with distribution as a first principle.

From the ground up, they assume:

Data will be spread across multiple machines
Systems will operate across regions
Failures will occur

This makes them naturally suited for horizontally scaled systems.

Companies like Amazon and Netflix have leveraged NoSQL databases to handle massive amounts of distributed data with high availability.

A Shift in Trade-offs

With NoSQL, the trade-offs begin to shift.

Instead of strict consistency, many NoSQL systems embrace eventual consistency.

Instead of rigid schemas, they offer flexibility.

Instead of centralised control, they enable distributed scalability.

But these benefits come at a cost:

More responsibility on the application to manage data integrity
Reduced ability to perform complex joins
Increased complexity in ensuring consistency when needed

This is not a drawback; it is a design choice.

And understanding this shift is key to choosing the right database.

Where This Is Heading

At this point, we have two distinct worlds:

SQL: structured, consistent, relationship-driven
NoSQL: flexible, scalable, distribution-friendly

But the real question is not which one is better.

It is:

When should you use each?

This is where database selection moves from theory to system design strategy.

The Many Faces of NoSQL

When people say NoSQL, they often imagine a single alternative to relational databases. In reality, NoSQL databases vary significantly in how they store and retrieve data.

Each type exists because different applications have different needs.

Document Databases - Modelling Data as It Is Used

Document databases store data in a format similar to JSON. Instead of splitting data across multiple tables, related information is often stored together in a single document.

This structure mirrors how applications actually consume data.

Instead of performing multiple joins to assemble related information, everything is already grouped together. This reduces query complexity and improves performance for read-heavy workloads.

This model is widely used in systems where:

Data structures evolve frequently
Relationships are hierarchical
Fast reads are more important than complex joins

Key-Value Stores

Key-value databases are the simplest form of NoSQL.

Data is stored as a collection of key-value pairs. You provide a key, and the system returns the associated value.

This simplicity allows for extremely fast lookups and high scalability.

However, it comes with limitations:

No support for complex queries
No inherent relationships between data

These databases are often used for:

Caching layers
Session storage
Simple lookup services

Wide-Column Databases

Wide-column databases organise data into rows and columns, but unlike SQL, each row can have a different set of columns.

This flexibility allows systems to handle large-scale, sparse datasets efficiently.

These databases are designed for:

High write throughput
Distributed storage across many nodes
Analytical workloads at scale

Graph Databases

While SQL can model relationships, graph databases treat them as the core of the system.

Data is stored as nodes and edges, making it ideal for highly connected data.

This model excels in scenarios where relationships are complex and deeply interconnected, such as:

Social networks
Recommendation systems
Fraud detection

Where SQL Still Shines

Despite the rise of NoSQL, relational databases remain dominant in many critical systems.

This is not by accident.

SQL databases are still the best choice when:

Data relationships are complex and require joins
Transactions must be strongly consistent
Data integrity is critical
The schema is stable and well-defined

For example, financial systems and transactional platforms within companies like Microsoft continue to rely heavily on relational databases.

Because in these systems, correctness is not negotiable.

Where NoSQL Excels

NoSQL systems shine in environments where scale and flexibility are more important than strict consistency.

They are particularly effective when:

The system needs to scale horizontally
Data models change frequently
High availability is required
Workloads are distributed globally

Platforms like Netflix use NoSQL databases to handle massive volumes of user activity, content metadata, and streaming data.

In such systems, slight inconsistencies are acceptable if they enable better performance and availability.

The Hybrid Reality

At this point, it might seem like you must choose one approach over the other.

But in modern system design, that is rarely the case.

Most large-scale systems use a combination of SQL and NoSQL databases, depending on the requirements of each component.

For example, a system might:

Use SQL for payments and transactions
Use NoSQL for user activity and analytics
Use caching for performance

This approach allows systems to utilise the strengths of each model while minimising their weaknesses.

Companies like Amazon follow this strategy extensively, using different databases for different parts of their architecture.

The Real Decision Framework

Choosing between SQL and NoSQL is not about trends or popularity.

It comes down to understanding your system’s priorities.

Ask yourself:

Do I need strict consistency or can I tolerate eventual consistency?
Is my data highly structured or flexible?
Will my system scale vertically or horizontally?
Are relationships central to my queries?

These questions guide the decision more than any feature comparison ever could.

Final Thought

Databases are not just storage systems.

They define how your system thinks about data, how it evolves, and how it scales.

Choosing the right database is not about picking the most powerful tool:

It is about choosing the tool that aligns with your system’s reality.

Because in system design, the best solutions are not the most sophisticated ones.

They are the ones that fit the problem.

Caching in System Design - The Secret to High Performance

Sushant Gaurav — Tue, 16 Jun 2026 05:41:00 +0000

There is a point in every system’s growth where adding more servers stops being enough.

You scale horizontally. You introduce load balancers. You distribute traffic efficiently. And yet, something still feels off.

Requests are slower than expected. Databases are under constant pressure. Systems that should scale effortlessly begin to struggle under repeated work.

And then you realise something fundamental:

The system is doing the same work again and again.

The same queries.
The same computations.
The same responses.

Over and over.

This is not a scaling problem.

It is a redundancy problem.

And the solution to this problem is one of the most powerful ideas in system design:

Caching.

What Is Caching, Really?

At a surface level, caching is often defined as storing frequently accessed data in a faster storage layer so it can be retrieved quickly.

But this definition, while correct, does not capture its true significance.

Caching is not just a performance optimisation.

It is a shift in how systems think about work.

Instead of asking:

Can we compute this quickly?

Caching asks:

Do we need to compute this at all?

That shift from computation to reuse is what makes caching so powerful.

The Cost of Repeated Work

To understand why caching matters, we need to look at what happens without it.

Imagine a system where every user request requires:

Fetching data from a database
Performing business logic
Formatting a response

This process may take only a few milliseconds per request. But at scale, those milliseconds add up.

When thousands or millions of users request the same data, the system is forced to:

Execute identical database queries repeatedly
Perform the same computations
Generate the same responses

This creates unnecessary load on the system, especially on components like databases, which are often the most expensive and limited resources.

Over time, this repeated work becomes the bottleneck.

Caching addresses this by eliminating redundant effort.

The Core Idea - Store Once, Serve Many

At its heart, caching is simple.

When a request is processed, instead of discarding the result, the system stores it in a cache. The next time the same request arrives, the system can return the cached result instead of recomputing it.

This introduces two fundamental concepts:

Cache Hit - The data is found in the cache and returned immediately
Cache Miss - The data is not in the cache, so it must be computed and then stored

The effectiveness of a caching system is often measured by its cache hit rate.

Higher hit rate → fewer expensive operations → better performance.

Why Caching Changes Everything

Caching has a profound impact on system behaviour.

Reduced Latency

Fetching data from memory is significantly faster than querying a database or calling an external service.

Increased Throughput

By reducing the load on core systems, caching allows more requests to be handled simultaneously.

Lower System Load

Databases, APIs, and backend services experience less pressure, improving overall system stability.

Better Scalability

Systems can handle larger traffic without proportionally increasing infrastructure.

This is why caching is used extensively in large-scale systems.

Platforms like Netflix and Google rely heavily on caching at multiple layers to serve massive amounts of data efficiently.

Where Does the Cache Live?

One of the most important design decisions in caching is where to place the cache.

Because caching is not a single layer, it can exist at multiple points in the system.

Application-Level Cache

The simplest form of caching happens within the application itself.

Data is stored in memory inside the server process.

This is:

Extremely fast
Easy to implement

But it has limitations:

Not shared across servers
Lost when the server restarts

This works well for small-scale systems or single-node setups.

Distributed Cache

As systems scale horizontally, caching must also scale.

Instead of storing cache locally, systems use distributed caching systems that are shared across multiple servers.

This allows:

Consistent access to cached data
Better cache utilisation
Scalability across nodes

However, it introduces:

Network overhead
Cache synchronisation challenges

Edge Cache (CDN)

At the highest level, caching can move closer to the user.

Content Delivery Networks (CDNs) store cached data in geographically distributed locations.

When a user requests content, it is served from the nearest location rather than the origin server.

This drastically reduces latency and server load.

This is how platforms like Amazon and Netflix deliver content globally with high performance.

The Trade-off Begins

At this point, caching may seem like a perfect solution.

Faster responses.
Lower load.
Better scalability.

So why not cache everything?

Because caching introduces a new and unavoidable challenge:

Data can become stale.

When the underlying data changes, cached data may no longer be accurate.

And this leads us to one of the hardest problems in system design:

Cache invalidation.

But as systems grow, a deeper and more uncomfortable truth begins to emerge:

Caching is easy to add… but very hard to get right.

Because the moment you introduce a cache, you are no longer just optimising performance, you are managing two versions of reality:

The source of truth (database)
The cached copy (fast, but potentially outdated)

And keeping these two in sync is where the real challenge begins.

Cache Invalidation - The Hardest Problem

There’s a well-known saying in system design:

There are only two hard things in Computer Science: cache invalidation and naming things.

It sounds like a joke, but it isn’t.

Cache invalidation is the process of ensuring that cached data remains accurate when the underlying data changes.

Let’s say a product’s price changes in the database. If the old price is still stored in the cache, users may see outdated information.

So the system must decide:

When should the cache be updated?
Should it be updated immediately or later?
Should it be removed entirely?

Each choice comes with trade-offs between consistency, performance, and complexity.

Common Approaches to Cache Invalidation

There is no single correct way to handle cache invalidation. Instead, systems use different strategies depending on their requirements.

Time-Based Expiration (TTL)

One of the simplest approaches is to assign a time-to-live (TTL) to cached data.

After a fixed duration, the cache entry expires and is removed.

This approach is:

Easy to implement
Predictable
Widely used

But it has limitations.

If the TTL is too long:

Data may remain stale for too long

If the TTL is too short:

Cache effectiveness decreases (more misses)

So choosing the right TTL becomes a balancing act.

Write-Based Invalidation

Another approach is to update or invalidate the cache whenever data changes.

For example:

When a product is updated → update or delete its cache entry

This ensures better consistency, but introduces complexity:

Every write operation must handle cache updates
Failures in cache updates can lead to inconsistencies

This approach works well when accuracy is critical.

Explicit Invalidation

Sometimes, systems explicitly remove cache entries when they know data has changed.

Instead of updating the cache, they simply delete it, forcing the next request to fetch fresh data.

This is simple and safe, but may temporarily increase load due to cache misses.

Caching Strategies - When to Read and Write

Beyond invalidation, another important question is:

When should the system interact with the cache?

This leads to different caching strategies.

Cache-Aside (Lazy Loading)

This is the most commonly used strategy.

When a request arrives:

Check the cache
If data exists → return it (cache hit)
If not → fetch from database, store in cache, then return

This approach is:

Simple
Flexible
Widely adopted

But it can lead to stale data if not invalidated properly.

Write-Through Cache

In this strategy, data is written to both:

Cache
Database

simultaneously.

This ensures that the cache is always up-to-date.

However:

Writing becomes slower
More coordination is required

Write-Back (Write-Behind)

Here, data is first written to the cache, and the database is updated later.

This improves write performance but introduces risk:

If the cache fails before writing to the database, data may be lost

This strategy is used when performance is prioritised over immediate consistency.

Cache Eviction - Making Space for New Data

Caches are not infinite.

At some point, they run out of space.

So the system must decide:

Which data should be removed to make room for new data?

This is handled through eviction policies.

LRU (Least Recently Used)

Removes data that has not been accessed recently.

This works well because frequently accessed data tends to remain in the cache.

LFU (Least Frequently Used)

Removes data that is accessed the least often.

This is useful when certain data is consistently popular.

TTL-Based Eviction

Data is removed after a fixed time, regardless of usage.

Each policy reflects a different assumption about how users interact with data.

Choosing the right one depends on your access patterns.

When Caching Goes Wrong

Caching is powerful, but when misused, it can create serious problems.

Stale Data Issues

Users see outdated information, leading to inconsistencies.

Cache Stampede

When a popular cache entry expires, many requests hit the database simultaneously, overwhelming it.

Increased Complexity

Managing cache logic, invalidation, and consistency adds significant engineering overhead.

Hidden Bugs

Caching can mask underlying issues, making debugging harder.

The Deeper Insight

At this point, caching should no longer feel like a simple optimisation.

It is a trade-off system.

You trade:

Freshness for speed
Simplicity for performance
Consistency for scalability

And like everything in system design, there is no perfect choice.

Only the choice that best fits your requirements.

Final Thought

The most important thing to understand about caching is this:

Caching does not make your system faster; it makes your system do less work.

And at scale, doing less work is the only way to survive.

Because the systems that scale are not the ones that compute faster:

They are the ones that avoid unnecessary computation altogether.

Load Balancing Explained - How Systems Handle Millions of Requests

Sushant Gaurav — Tue, 02 Jun 2026 05:41:00 +0000

There is a moment in the life of every growing system when a single server quietly becomes a bottleneck.

At first, the system works exactly as expected. Users send requests, the server processes them, and responses are returned almost instantly. Everything feels smooth, predictable, and under control.

But as usage grows, something subtle begins to change.

Requests start arriving faster than they can be processed. The server becomes overloaded. Response times increase. Eventually, some requests begin to fail, not because the logic is incorrect, but because the system simply cannot keep up.

This is not a bug.

It is a limitation.

And this is the moment when adding more servers becomes necessary.

But adding more servers introduces a new challenge:

How do you decide which server should handle which request?

Because without coordination, adding more machines does not solve the problem; it just spreads the chaos.

This is where load balancing enters the picture.

What Is Load Balancing, Really?

At its core, load balancing is the process of distributing incoming requests across multiple servers so that no single machine becomes overwhelmed.

But that definition, while correct, does not capture the full picture.

Load balancing is not just about distribution.

It is about:

Maximising resource utilisation
Minimising response time
Ensuring high availability
Preventing system overload

It acts as the invisible layer that allows systems to scale horizontally without collapsing under pressure.

The Role of a Load Balancer

A load balancer sits between users and your servers.

Instead of users directly interacting with a specific machine, they send requests to the load balancer, which then decides where those requests should go.

From the user’s perspective, the system still appears as a single entity.

But behind the scenes, requests are being intelligently distributed across multiple machines.

This abstraction is powerful.

It allows you to:

Add or remove servers without affecting users
Handle traffic spikes dynamically
Improve fault tolerance

But the real complexity lies in how the load balancer makes decisions.

The Simplest Approach - Equal Distribution

The most intuitive way to distribute requests is to spread them evenly across all servers.

Each incoming request is sent to the next server in line.

This is known as round-robin distribution.

At first glance, this seems perfectly fair.

Every server gets an equal share of requests.

But real systems are rarely that simple.

Because not all requests are equal.

The Problem with Equal Distribution

In practice, different requests require different amounts of work.

Some requests might:

Query large datasets
Perform complex computations
Call multiple downstream services

While others may be simple and lightweight.

If all requests are distributed equally without considering their complexity, some servers may become overloaded while others remain underutilised.

This is where more intelligent load-balancing strategies come into play.

Smarter Distribution - Load-Aware Routing

Instead of blindly distributing requests, load balancers can make decisions based on the current state of each server.

For example:

Send requests to the server with the least number of active connections
Route traffic to the server with the lowest response time
Avoid servers that are showing signs of stress

This allows the system to adapt dynamically to changing conditions.

But this introduces a new requirement:

The load balancer must constantly monitor the health and performance of servers.

Health Checks - Knowing When a Server Is Failing

One of the most critical responsibilities of a load balancer is detecting when a server is no longer able to handle requests.

This is done through health checks.

The load balancer periodically sends requests to each server to verify:

Is the server responding?
Is it responding within an acceptable time?
Is it returning correct responses?

If a server fails these checks, it is temporarily removed from the pool.

This ensures that user requests are only sent to healthy servers, improving reliability and user experience.

Load Balancing and Availability

Load balancing plays a crucial role in achieving high availability.

Without it, even a horizontally scaled system can fail.

Imagine having multiple servers, but users are directly connected to just one of them. If that server goes down, the system becomes unavailable, even though other servers are perfectly functional.

A load balancer prevents this by acting as a single entry point, ensuring that traffic is always routed to available resources.

This is one of the reasons why large-scale systems, including those built by Netflix, rely heavily on load balancing at multiple layers.

Where This Is Heading

So far, we’ve explored:

Why load balancing is necessary
How it distributes traffic
Why simple strategies are not enough
How health checks ensure reliability

But as systems grow more complex, the role of a load balancer expands beyond simple request routing.

Because not all requests are the same.
Not all servers behave the same.
And not all failures look the same.

To handle real-world traffic at scale, load balancing itself evolves into a multi-layered system with different levels of intelligence.

Layer 4 vs Layer 7 - Two Ways to Think About Traffic

To understand modern load balancing, we need to look at the network stack; specifically, how requests move through it.

At a high level, load balancers operate at two common layers:

Layer 4 (Transport Layer)
Layer 7 (Application Layer)

These are not just technical distinctions; they define how much the load balancer understands about the request.

Layer 4 Load Balancing - Fast and Blind

A Layer 4 load balancer operates at the transport level (TCP/UDP). It does not inspect the content of the request. Instead, it makes decisions based on basic network information like:

IP address
Port number

From its perspective, a request is just a stream of packets.

This makes Layer 4 load balancing:

Extremely fast
Low overhead
Highly efficient for raw traffic distribution

But it also means it lacks context.

It cannot differentiate between:

/login vs /checkout
API calls vs static content
High-cost vs low-cost operations

It simply forwards traffic.

Layer 7 Load Balancing - Intelligent Routing

Layer 7 load balancers operate at the application level (HTTP/HTTPS). They can inspect the actual content of the request:

URL paths
Headers
Cookies
Request types

This allows for much smarter routing decisions.

For example:

Send /images requests to a caching server
Route /api/payments to a specific service
Direct mobile users to a different backend

This level of control is powerful, but it comes with trade-offs:

Higher latency than Layer 4
More computational overhead
Increased complexity

This leads to a common pattern in real systems:

Use Layer 4 for speed, Layer 7 for intelligence, often together.

Sticky Sessions - When State Gets in the Way

Earlier, we discussed how scalable systems aim to be stateless. But in practice, not all systems achieve this immediately.

Some applications rely on session state stored on individual servers.

This creates a problem.

If a user’s first request goes to Server 1, and their session data is stored there, sending their next request to Server 3 may break the experience.

To handle this, systems sometimes use sticky sessions.

Sticky sessions ensure that requests from the same user are always routed to the same server.

While this solves the immediate problem, it introduces new challenges:

Uneven load distribution
Reduced fault tolerance (if the server fails, the session is lost)
Difficulty scaling dynamically

This is why modern systems try to avoid sticky sessions and instead externalise state into shared systems like distributed caches.

Global Load Balancing - Scaling Across Regions

So far, we’ve discussed load balancing within a single region. But large-scale systems operate globally.

Users from different parts of the world expect fast response times. Sending every request to a single data centre is not practical.

This is where global load balancing comes into play.

Instead of routing traffic between servers, global load balancing routes traffic between regions.

The system decides:

Which region is closest to the user
Which region is currently healthy
Which region has capacity

This reduces latency and improves availability.

If one region goes down, traffic can be redirected to another.

This is how companies like Google and Amazon maintain global-scale systems that remain responsive even under failures.

Load Balancing Meets Caching and CDNs

As systems scale further, load balancing does not operate in isolation.

It works alongside:

Caching layers
CDNs (Content Delivery Networks)

A CDN can serve requests directly from edge locations, reducing the load on origin servers.

This introduces a powerful optimisation:

The best request is the one that never reaches your server.

By serving content closer to users, CDNs reduce latency, decrease server load, and improve scalability.

Load balancers then handle the remaining dynamic traffic efficiently.

The Deeper Insight

At this point, load balancing should no longer feel like a simple routing mechanism.

It is a control system that:

Distributes load
Detects failures
Optimises performance
Enables scalability

And more importantly, it connects multiple system design concepts:

Horizontal scaling
Availability
Latency optimization
Fault tolerance

This is why load balancing is often one of the first components introduced when systems begin to scale.

Final Thought

When systems are small, you think in terms of servers.

When systems grow, you think in terms of clusters.

But when systems reach scale, you think in terms of traffic flow.

Where requests come from, how they move, and where they are handled become the defining factors of system performance and reliability.

And at the centre of that flow is the load balancer.

Not just distributing requests, but shaping how the system behaves under pressure.

Redis Essentials: Architecture, Caching, and Setup

Sushant Gaurav — Tue, 26 May 2026 05:41:00 +0000

Redis is often a misunderstood tool in the backend developer's arsenal. While many view it simply as a "topic" to be covered in an hour, its role in modern system design is pivotal for building high-performance, scalable applications. This article explores what Redis is, why it is used, and how to set it up locally for development.

Understanding Redis: The In-Memory Powerhouse

At its core, Redis is an in-memory data store, often referred to as a "lightning-fast" hash map or key-value store. Unlike traditional databases like MongoDB or PostgreSQL that primarily store data on a hard disk (SSD or HDD), Redis keeps its state in the RAM (Random Access Memory).

Core Concept: The In-Memory Advantage

The fundamental difference between Redis and traditional databases (like MongoDB or PostgreSQL) is where they store data. While standard databases primarily use disk storage (SSDs/HDDs), Redis keeps its state in RAM (Random Access Memory).

Because RAM access is significantly faster than mechanical or electronic disk reads, Redis is often described as "lightning fast".

Architecture: The Caching Layer

In a typical application, Redis acts as an intermediary between the backend application and the primary database. This setup creates two primary scenarios:

Cache Hit: The backend finds the required data in Redis and returns it immediately to the user, bypassing the slower database.
Cache Miss: If the data isn't in Redis, the backend queries the primary database. It then stores a copy of this "hot record" in Redis for future requests before responding to the user.

This architecture dramatically reduces "read pressure" on the primary database, which should remain the "Source of Truth" for permanent records.

Key Features and Data Management

Persistence: Contrary to the myth that in-memory data is always lost on restart, Redis offers persistence features. It can load data from saved files back into memory upon a server reboot.
Key-Value Pairs: Redis stores data in simple pairs. Developers are encouraged to use human-readable, colon-separated keys (for example, user:session:123 or product:all) to avoid collisions and simplify debugging.
TTL (Time to Live): This is one of Redis's most powerful features. You can set an expiration time on a key (for example, 90 seconds). Once the time expires, Redis automatically deletes the record, ensuring the memory remains uncluttered.

Advanced Use Cases

Beyond simple data caching, Redis is used for:

Session Management: Storing user login states (Active/Inactive) across multiple distributed servers.
OTP Management: Holding temporary One-Time Passwords for a few minutes, they are valid.
Rate Limiting: Tracking IP addresses or user IDs to prevent abuse (for example, blocking a user for 10 minutes after too many failed login attempts).
Job Queues: Maintaining lists of background tasks. "Workers" (secondary backend applications) pull jobs from Redis to process time-consuming tasks like sending emails in batches.
Shared Counters: Tracking live metrics like page views or "likes" across various application instances.

Strategic Local Setup

For development, the sources recommend using Docker and Docker Compose to spin up a local environment. A standard configuration involves:

Redis Image: Using redis:7-alpine for a lightweight footprint.
Port Mapping: Binding the default Redis port 6379 to the host machine.
Persistence Command: Running the server with --appendonly yes to ensure data is written to a log.

In a Node.js environment, the ioredis library is the industry-standard package for communication. A basic connection is established by creating a new Redis client using the local URL: redis://localhost:6379. Developers can test the connection using the PING command, which should return a PONG response from the server.

When to Use Redis

Redis is not a solution for every problem. Use it if your application needs to:

Remove read pressure from the primary DB.
Manage rapidly expiring temporary data.
Handle background job queues or shared counters.

However, it is not a replacement for a primary database.

When NOT to use Redis

Redis is not a "magic bullet". It should not be used if you don't have a clear bottleneck or if your data doesn't fit the patterns described above. If you have a write-heavy application where data doesn't need to be read frequently, or if you are trying to use it as a primary database for complex relational data, Redis may not be the right solution.

Scalability in System Design - Vertical vs Horizontal Scaling

Sushant Gaurav — Tue, 19 May 2026 05:41:00 +0000

There comes a point in every system’s life where things start to break; not because the system is poorly designed, but because it is being used more than it was ever intended to handle.

At first, everything feels smooth. Requests are processed quickly, users are satisfied, and the system behaves predictably. But as usage grows, subtle changes begin to appear. Pages take longer to load. APIs respond more slowly. Databases struggle to keep up. Eventually, what once worked effortlessly starts becoming unreliable.

This is not a failure of design.

It is a signal.

The system has reached the limits of its current capacity.

And this is where scalability enters the conversation.

What Does Scalability Really Mean?

Scalability is often misunderstood as simply handling more users. But that definition is incomplete.

A system is considered scalable if it can handle increasing load without a proportional drop in performance or reliability.

Notice the nuance here.

It is not just about handling more requests - it is about doing so efficiently. A system that doubles its resources every time the load increases is not truly scalable; it is simply brute-forcing the problem.

True scalability is about growing intelligently.

And to achieve that, systems typically rely on two fundamental approaches:

Scaling up (vertical scaling)
Scaling out (horizontal scaling)

At a high level, both aim to solve the same problem: increasing capacity. But the way they approach it, and the consequences of those choices, are fundamentally different.

Vertical Scaling - Growing Taller

Vertical scaling, often referred to as scaling up, is the simpler and more intuitive approach.

Instead of changing the structure of the system, you make the existing machine more powerful.

You increase:

CPU
RAM
Disk capacity

In essence, you are upgrading the machine so it can handle more work.

From an engineering perspective, vertical scaling feels natural.

There is no need to redesign the system. The application continues to run as it always has, just on better hardware. Databases remain centralised. Communication patterns remain unchanged. There is no need to think about distribution, coordination, or synchronisation.

This simplicity is incredibly valuable - especially in the early stages of a system.

It allows teams to focus on building features rather than solving infrastructure complexity.

Why Vertical Scaling Works So Well (Initially)

In the early lifecycle of a product, vertical scaling often provides the fastest path to growth.

If your database is slowing down, you can upgrade it to a machine with more memory. If your application server is under load, you can increase its CPU capacity.

The system continues to function exactly as before, just with more headroom.

This is why many systems, including those built by companies like Instagram in their early days, start with vertically scaled architectures.

The benefits are clear:

Minimal architectural changes
Lower operational complexity
Faster implementation

For a small team trying to move quickly, this is often the most practical choice.

The Limits of Vertical Scaling

But like everything in system design, vertical scaling has limits.

The first limitation is physical.

A machine can only be upgraded to a certain extent. There is a maximum amount of CPU, memory, and storage you can add. Beyond that point, scaling up is no longer possible.

The second limitation is cost.

As machines become more powerful, their cost increases disproportionately. A machine that is twice as powerful is often significantly more than twice as expensive.

This leads to diminishing returns.

The third and perhaps most critical limitation is risk.

When your entire system depends on a single machine, that machine becomes a single point of failure.

If it goes down:

The entire system goes down

No matter how powerful the machine is, it cannot protect you from hardware failures, network issues, or unexpected crashes.

This is where the need for a different approach begins to emerge.

Horizontal Scaling - Growing Wider

Horizontal scaling, or scaling out, takes a fundamentally different approach.

Instead of making a single machine more powerful, you add more machines and distribute the workload among them.

Now, instead of relying on one powerful server, the system relies on multiple smaller servers working together.

This introduces a new concept: distribution of work.

Requests are no longer handled by a single machine. They are spread across multiple nodes, often using a load balancer that decides where each request should go.

At first, this might seem like a straightforward extension of vertical scaling. But in reality, it changes the nature of the system entirely.

Because the moment you introduce multiple machines, you introduce:

Network communication
Data synchronisation
Failure handling across nodes

In other words, you are stepping into the world of distributed systems.

A Shift in Complexity

Vertical scaling keeps complexity low but limits growth.

Horizontal scaling removes those limits but introduces a new kind of complexity.

This is not just an implementation detail; it is a fundamental shift in how systems are designed and reasoned about.

In a vertically scaled system:

There is one source of truth
Communication is local
Failures are simpler to reason about

In a horizontally scaled system:

Data may exist in multiple places
Communication happens over networks
Failures become partial and unpredictable

This is the same shift we saw earlier when moving from monolithic to distributed systems.

Because in many ways, horizontal scaling is what forces systems to become distributed.

Where This Is Heading

At this point, we’ve built the intuition:

Vertical scaling is simple, powerful, and limited
Horizontal scaling is flexible, scalable, and complex

But this is only the surface. To truly understand horizontal scaling, we need to answer a deeper question:

What actually happens to data and traffic when a system scales horizontally?

Because adding more machines is easy.

Making them work together correctly and efficiently is the real challenge.

Distributing Traffic - The Role of Load Balancing

The moment you introduce multiple servers, you need a way to decide:

Which request goes to which machine?

This is the job of a load balancer.

A load balancer sits between users and your servers, acting as a traffic controller. Instead of users directly hitting a specific server, their requests are routed through the load balancer, which distributes them across available machines.

At a surface level, this seems simple—just spread requests evenly. But in practice, it involves subtle decisions:

Should requests be distributed round-robin?
Should they go to the least loaded server?
Should user sessions stick to the same server?

These choices affect both performance and correctness.

And more importantly, they introduce a critical requirement:

Each server should be able to handle requests independently.

This leads to a key design principle in scalable systems:

Statelessness.

Stateless vs Stateful Systems

In a vertically scaled system, state is easy to manage. Since everything runs on a single machine, user sessions, data, and temporary state can be stored locally.

But in a horizontally scaled system, this approach breaks down.

If a user’s request goes to Server 1, and their next request goes to Server 3, that second server must still understand the user’s context.

This is why scalable systems aim to make application servers stateless.

Instead of storing session data locally, they store it in shared systems such as:

Databases
Distributed caches
External storage systems

This allows any server to handle any request, making load balancing effective.

But this shift pushes complexity elsewhere, into how data is stored and accessed.

Scaling Data - The Real Challenge

Handling more requests is only part of the problem.

The bigger challenge is handling more data.

In a vertically scaled system, data typically lives in a single database. As the load increases, you upgrade the database server. But just like application servers, databases have limits.

This is where horizontal scaling forces a fundamental shift in data strategy.

Two major approaches emerge:

Replication - Copying Data Across Nodes

Replication involves creating multiple copies of the same data across different machines.

This allows:

Multiple servers to read data simultaneously
Improved availability if one node fails

For example, one database node may handle writes, while multiple replicas handle read requests.

This improves throughput, but introduces consistency challenges—something we explored earlier in CAP.

Sharding - Splitting Data Across Nodes

Sharding takes a different approach.

Instead of copying data, it divides it.

Each server is responsible for a subset of the data:

User A–M on one server
User N–Z on another

This allows the system to scale almost indefinitely by adding more shards.

But it introduces new complexity:

How do you decide which shard stores which data?
What happens when data needs to move between shards?
How do you handle queries that span multiple shards?

Sharding improves scalability dramatically, but at the cost of operational and architectural complexity.

Availability - Why Horizontal Scaling Wins

One of the most powerful advantages of horizontal scaling is fault tolerance.

In a vertically scaled system, everything depends on a single machine. If it fails, the system goes down.

In a horizontally scaled system, failure becomes partial.

If one server crashes:

Other servers continue handling requests
The system degrades, but does not collapse

This is the foundation of high-availability systems.

It is also why companies like Netflix design their systems to run across multiple machines, zones, and even regions.

They assume failure will happen—and design systems that survive it.

Cost, Complexity, and Trade-offs

At this point, horizontal scaling may seem like the obvious choice.

But it comes with trade-offs that cannot be ignored.

Cost Dynamics

While horizontal scaling can start with cheaper machines, the total cost can grow quickly as you add more infrastructure, networking, and operational overhead.

Engineering Complexity

You now need to handle:

Distributed communication
Data consistency
Failures across nodes
Monitoring and observability

Debugging Challenges

A single request may pass through multiple machines. Debugging issues becomes significantly harder compared to a single-node system.

This leads to a critical insight:

Horizontal scaling solves scalability problems by introducing distributed systems complexity.

The Hybrid Reality

In practice, most systems do not rely purely on vertical or horizontal scaling.

They combine both.

A common approach is:

Scale vertically first (quick wins, low complexity)
Introduce horizontal scaling as limits are reached

For example:

Start with a powerful database server
Add read replicas as traffic grows
Eventually introduce sharding when needed

This gradual evolution allows systems to grow without unnecessary complexity early on.

The Deeper Insight

At its core, scalability is not about choosing between vertical and horizontal scaling.

It is about understanding when each approach makes sense.

Vertical scaling is about simplicity and speed.
Horizontal scaling is about resilience and long-term growth.

And the transition between them is one of the most important decisions in system design.

Because once you move toward horizontal scaling, you are no longer just scaling a system:

You are designing a distributed system.

Final Thought

Scalability is often seen as a technical challenge.

But in reality, it is a reflection of success.

Systems only need to scale when they are being used, when they are growing, when they matter.

And the way you scale them defines:

Their performance
Their reliability
Their future evolution

Because in the end, scalability is not just about handling more users:

It is about building systems that can grow without breaking.

CAP Theorem Explained Simply (And Why It Matters in Real Systems)

Sushant Gaurav — Tue, 05 May 2026 05:41:00 +0000

There is a moment in every system design journey where things stop feeling simple.

Until that point, systems seem manageable. You think in terms of databases, APIs, scaling strategies, maybe even caching layers. But then you encounter distributed systems in their true form—data spread across machines, services communicating over unreliable networks, failures happening in unpredictable ways.

And suddenly, a question emerges that is far more difficult than it first appears:

How do we ensure that all parts of a system behave correctly when they are no longer in the same place?

This is the question that gave rise to the CAP Theorem.

At first glance, CAP is often presented as a rule - something to memorize:

A distributed system can only guarantee two out of three: Consistency, Availability, and Partition Tolerance.

But this simplified statement, while technically correct, hides the deeper truth.

CAP is not just a rule.

It is a constraint imposed by reality.

To truly understand it, we need to go beyond definitions and step into the world where distributed systems actually operate.

The Reality of Distribution

In a monolithic system, everything runs within a single environment. Data is stored in one place, and operations happen in a predictable sequence. If you update a value, every part of the system immediately sees that update.

But in a distributed system, things are fundamentally different.

Data is no longer centralized. It is spread across multiple nodes—possibly across regions, continents, or even different cloud providers. These nodes communicate over a network, and that network is not perfect.

Messages can be delayed.
Packets can be lost.
Connections can break.

And when that happens, parts of the system can no longer talk to each other.

This situation is known as a network partition.

Partition Tolerance — The Unavoidable Reality

Partition tolerance refers to a system’s ability to continue functioning even when communication between nodes is disrupted.

And here’s the critical insight:

In distributed systems, partitions are not optional—they are inevitable.

You cannot design a real-world distributed system and assume that the network will always be reliable. Sooner or later, something will fail.

This means that partition tolerance is not a choice you make.

It is a condition you must accept.

Once you accept this, the CAP theorem becomes much clearer.

Because now the real question is:

When a partition happens, what do you prioritize - consistency or availability?

Consistency — One Truth Across the System

Consistency, in the context of CAP, means that all nodes see the same data at the same time.

If a user updates a piece of data, any subsequent read—no matter which node it comes from—should return that updated value.

There is a single, unified truth.

This is straightforward in a centralized system. But in a distributed system, maintaining this guarantee requires coordination between nodes.

When a write happens, all replicas must agree on the updated value before it is considered complete.

This coordination takes time. And during a network partition, it may not be possible at all.

So if you insist on strong consistency, the system must sometimes refuse to respond rather than risk returning incorrect data.

Availability — Always Responding

Availability means that every request to the system receives a response.

It does not necessarily mean the response is correct or up-to-date—only that the system does not fail to respond.

In highly available systems, the priority is to keep the system operational, even under failure conditions.

This often means allowing different nodes to respond independently, even if they do not have the latest data.

The system continues to function, but it may temporarily serve inconsistent data.

The Core Trade-off

Now we arrive at the heart of CAP.

When a network partition occurs, you are forced to make a decision:

If you choose consistency, you may have to reject requests to ensure correctness.
If you choose availability, you may return outdated or inconsistent data.

You cannot guarantee both at the same time.

This is the essence of the CAP theorem.

It is not about picking any two out of three in general conditions. It is about what happens during a partition.

And since partitions are inevitable, this trade-off is unavoidable.

Why This Matters More Than You Think

At this point, CAP might seem like an abstract concept. But in reality, it influences almost every large-scale system you interact with.

When you see slightly outdated data on a social media feed, that is a system choosing availability over strict consistency.

When a payment system refuses to process a transaction until it confirms the latest state, that is a system prioritizing consistency over availability.

These are not accidental behaviours. They are deliberate design choices shaped by CAP.

Companies like Amazon often design different parts of their systems with different priorities. For example, product catalogues may favour availability, while payment systems enforce strict consistency.

This highlights an important idea:

CAP is not applied to an entire system uniformly; it is applied at the level of individual components.

Now the natural question becomes:

What kinds of systems make which choices? And how do real-world architectures actually deal with this trade-off?

To answer that, we need to look at how CAP is commonly categorized in practice.

The Three CAP Categories (And the Truth Behind Them)

CAP is often explained using three system types:

CP (Consistency + Partition Tolerance)
AP (Availability + Partition Tolerance)
CA (Consistency + Availability)

At first glance, this looks like a clean classification. But there is a subtle—and very important—truth hidden here.

In real distributed systems, CA is not actually achievable.

Why?

Because partition tolerance is not optional.

If your system is distributed, you cannot ignore the possibility of network failures. And the moment you accept partitions as inevitable, you are always operating in the world of P (Partition Tolerance).

So in practice, the real trade-off is always:

CP vs AP

CP Systems — Choosing Consistency Over Availability

A CP system prioritizes correctness above all else.

When a partition occurs, and nodes cannot communicate reliably, the system chooses to reject or delay requests rather than risk returning inconsistent data.

This often means:

Some parts of the system become temporarily unavailable
Users may experience errors or delays
But the data remains correct and trustworthy

This approach is critical in systems where correctness is non-negotiable.

Think about financial transactions. If your bank shows two different balances depending on which server you hit, the system is fundamentally broken.

This is why systems dealing with payments, inventory management, or critical state often lean toward CP.

For example, services within Google that require strict coordination (like distributed databases with strong guarantees) are designed to favor consistency, even if it means temporarily sacrificing availability.

AP Systems — Choosing Availability Over Consistency

AP systems take the opposite approach.

When a partition occurs, they continue to serve requests no matter what, even if that means returning stale or inconsistent data.

This results in:

High availability
Faster response times during failures
Temporary inconsistencies across nodes

But here’s the key: these inconsistencies are not permanent.

AP systems rely on a concept called eventual consistency, where all nodes will converge to the same state once the network stabilizes.

This model works well for systems where perfect accuracy at every moment is not required.

For example, platforms like Facebook prioritize keeping the platform responsive. If your feed shows a slightly outdated like count for a few seconds, it does not break the user experience.

The system favours availability and responsiveness over strict consistency.

Why CA Systems Don’t Really Exist

It is tempting to think that some systems can achieve both consistency and availability.

And technically, in systems that are not distributed, this is true.

A single-node database can provide both:

Immediate consistency
Always-available responses (as long as the node is up)

But the moment you distribute the system across multiple nodes, the network becomes a factor.

And once the network becomes a factor, partitions become inevitable.

So any system that claims to be CA is either:

Not truly distributed
Or quietly sacrificing partition tolerance (which is unrealistic in real-world systems)

Real Systems Don’t Pick One Side Completely

Here’s where things get even more interesting.

Real-world systems rarely choose to be purely CP or purely AP.

Instead, they mix and match based on the needs of different components.

For example, in a large system like Amazon:

The shopping cart might be AP
(you can still add items even if some nodes are out of sync)
The payment system is CP
(transactions must be accurate and consistent)
The product catalog might lean toward AP
(slight delays in updates are acceptable)

This layered approach allows systems to optimize for different trade-offs depending on the context.

This is a critical mindset shift:

CAP is not a system-wide decision. It is a per-component design choice.

How Modern Systems Soften the Trade-off

While CAP defines a hard constraint, modern systems use clever techniques to reduce the pain of the trade-off.

They cannot eliminate it—but they can make it less noticeable.

Eventual Consistency with Conflict Resolution

AP systems often allow temporary inconsistencies but resolve them later using strategies like:

Last write wins
Version vectors
Conflict-free data structures

Retries and Idempotency

Systems retry failed requests intelligently, ensuring that operations can be safely repeated without corrupting data.

Graceful Degradation

Instead of failing completely, systems reduce functionality under stress:

Showing cached data
Disabling non-critical features

Geo-Partitioning

Data is partitioned geographically so that most operations happen locally, reducing the impact of global partitions.

Common Misconceptions About CAP

Even experienced engineers sometimes misunderstand CAP. Let’s clear up a few common myths.

You can choose any two at any time

No — the trade-off only matters during a partition.

AP systems don’t care about consistency

They do — they just relax when consistency is achieved.

CP systems are always better

Not necessarily — they can lead to poor user experience during failures.

CAP is outdated

Not at all — it is still one of the most fundamental constraints in distributed systems.

The Real Lesson of CAP

CAP is not about memorizing three letters.

It is about understanding this deeper truth:

In distributed systems, failure forces you to make trade-offs.

And those trade-offs are not bugs.

They are design decisions.

The best system designers are not the ones who avoid trade-offs—they are the ones who choose them wisely, based on the needs of the system.

Final Thought

If you truly understand CAP, you start seeing systems differently.

You begin to ask better questions:

What happens when this service cannot reach another service?
Is it better to fail or return stale data?
Where can we tolerate inconsistency, and where can we not?

And those questions lead to better designs.

Because at scale, systems are not defined by how they behave when everything works—

They are defined by how they behave when things break.

System Design Fundamentals - Latency, Throughput, Availability, Consistency, Redundancy, Replication, and Congestion Explained

Sushant Gaurav — Tue, 21 Apr 2026 05:41:00 +0000

When people first step into system design, they often expect to learn about architectures—microservices, databases, load balancers, and scaling strategies. But very quickly, something becomes clear.

The real language of system design is not architecture diagrams.

It is a set of fundamental forces—concepts that quietly govern how systems behave under load, failure, and growth. These forces exist whether you acknowledge them or not, and every architectural decision is ultimately an attempt to balance them.

Among these, a few stand out as foundational: latency, throughput, availability, consistency, redundancy, replication, and congestion.

At first, they may seem like isolated technical terms. But in reality, they are deeply interconnected. Changing one often impacts the others. Optimizing for one can degrade another. And understanding their relationships is what separates a surface-level understanding from true system design thinking.

This is where we begin.

Latency — The Cost of Time

Every interaction with a system has a cost, and that cost is measured in time.

Latency is the amount of time it takes for a request to travel through a system and produce a response. It begins the moment a user initiates an action—clicking a button, loading a page, sending a message—and ends when the system responds.

At a small scale, latency often feels negligible. A request goes from the user to the server, gets processed, and returns almost instantly. But as systems grow, latency becomes one of the most critical challenges.

Because in reality, a single request is rarely simple.

It might pass through multiple layers:

A load balancer
An API gateway
Authentication services
Business logic
Databases
External APIs

Each step adds a small delay. Individually, these delays may seem insignificant. But together, they accumulate.

This is why latency is not just about speed—it is about distance, complexity, and coordination.

In a monolithic system, latency is often lower because components communicate internally. In distributed systems, latency increases because communication happens over networks, where delays are unavoidable.

This leads to an important realization:

You don’t eliminate latency—you manage it.

And managing latency becomes a central concern in system design.

Throughput - The Capacity to Handle Load

If latency is about how fast a single request is handled, throughput is about how many requests the system can handle over time.

A system with high throughput can process a large number of requests per second. This becomes critical when dealing with real-world traffic—thousands, millions, or even billions of users interacting with the system concurrently.

At first glance, it might seem like increasing throughput is simply a matter of adding more resources. But the reality is more nuanced.

Throughput is limited by bottlenecks.

A system is only as fast as its slowest component. If a database can only handle a certain number of queries per second, it does not matter how fast the application layer is—the system’s overall throughput will be constrained.

This is why scaling systems often involves identifying and removing bottlenecks, rather than just increasing capacity.

Distributed systems improve throughput by allowing different parts of the system to operate independently and in parallel. Requests can be distributed across multiple services, multiple machines, and even multiple regions.

But this introduces a subtle trade-off.

As throughput increases through distribution, latency often increases due to coordination overhead. Requests may need to travel further, wait for responses from multiple services, or handle retries in case of failures.

This creates a tension:

Systems that handle more work often take longer to respond to individual requests.

Balancing latency and throughput is one of the most fundamental challenges in system design.

Availability - The Promise of Being There

A system is only useful if it is accessible when users need it.

Availability measures the probability that a system is operational and able to respond to requests at any given time. It is often expressed as a percentage - commonly referred to as "uptime".

For example:

99% availability allows ~3.65 days of downtime per year
99.9% reduces that to ~8.7 hours
99.99% brings it down to less than an hour

At scale, even small differences in availability can have a significant impact.

For companies like Amazon or Netflix, downtime is not just a technical issue - it directly translates to revenue loss and user dissatisfaction.

But achieving high availability is not as simple as making systems "more reliable".

Failures are inevitable:

Servers crash
networks fail
databases become overloaded
software contains bugs

The goal is not to prevent failures entirely, but to design systems that continue to function despite them.

This is where concepts like redundancy and replication (which we will explore in detail next) become critical.

Distributed systems are often designed with availability as a primary goal. By spreading services across multiple nodes and regions, they reduce the likelihood that a single failure will bring down the entire system.

But again, there is a trade-off.

Improving availability often requires relaxing consistency, which brings us to one of the most important and nuanced concepts in system design.

Consistency — The Truth of Data

Consistency is about whether all parts of a system agree on the same data at the same time.

In a perfectly consistent system, every read returns the most recent write. There is a single, unified view of truth.

This is relatively straightforward in monolithic systems with a single database. But in distributed systems, maintaining strong consistency becomes significantly more challenging.

Because data is no longer stored in one place.

It may be:

replicated across multiple servers
distributed across regions
cached at different layers

When a piece of data changes, ensuring that every copy reflects that change instantly is difficult—and sometimes impossible within acceptable latency limits.

This leads to different models of consistency.

Some systems prioritize strong consistency, ensuring that all users see the same data at all times. Others adopt eventual consistency, where updates propagate over time, and temporary inconsistencies are tolerated.

For example, in a messaging system, seeing a message appear a fraction of a second later might be acceptable. But in a banking system, inconsistencies in account balances are not.

This highlights a key principle:

Consistency is not absolute—it is a design choice based on requirements.

And choosing the right level of consistency often involves trade-offs with availability and latency.

Where This Is Leading?

At this point, we’ve covered four fundamental forces:

Latency
Throughput
Availability
Consistency

And something important should already be clear.

These are not independent concepts. They are deeply intertwined, constantly influencing each other in subtle ways.

Replication — Copying for Continuity

Replication is one of the most fundamental techniques used in system design. At its core, it simply means maintaining multiple copies of the same data or service.

If one copy fails, another can take over.

At first glance, replication seems straightforward. But in practice, it introduces some of the most complex challenges in distributed systems.

Imagine a database that stores user data. Instead of keeping a single copy, the system maintains replicas across multiple servers, possibly in different geographic regions. When a user writes new data, that update must be propagated to all replicas.

But here’s the catch: this propagation is not instantaneous.

There is always a delay—sometimes milliseconds, sometimes seconds. During this window, different replicas may hold different versions of the data. This is where consistency challenges emerge.

Systems must decide:

Should reads always go to the latest updated replica (strong consistency)?
Or is it acceptable for some reads to return slightly stale data (eventual consistency)?

This decision is not purely technical—it is deeply tied to the nature of the application.

For example, platforms like Netflix can tolerate slight delays in data synchronization because user experience is not critically affected by minor inconsistencies. On the other hand, financial systems require strict guarantees, making strong consistency essential.

Replication improves availability and fault tolerance, but it complicates consistency and coordination.

And this trade-off is unavoidable.

Redundancy — Designing for Failure

While replication focuses on copying data or services, redundancy is a broader concept.

Redundancy is about having extra components in the system that can take over when something fails.

These components may or may not be identical copies.

For example:

Multiple servers running the same application
Backup databases
Secondary data centers in different regions

If one component fails, another is already in place to handle the load.

This is how large-scale systems achieve high availability.

Companies like Amazon operate across multiple regions, ensuring that even if one region experiences failure, traffic can be routed to another. From the user’s perspective, the system continues to function.

But redundancy is not free.

It introduces:

Increased infrastructure cost
Complexity in synchronization
Challenges in failover mechanisms

There is also a subtle challenge known as failover correctness.

Switching from a failed component to a backup must happen seamlessly. If not handled properly, failover itself can introduce new failures—such as duplicate requests, inconsistent data, or partial system states.

So while redundancy improves availability, it also increases the operational complexity of the system.

Congestion — When Systems Get Overwhelmed

Even well-designed systems can struggle under heavy load.

Congestion occurs when the demand on a system exceeds its capacity to handle requests. This can happen due to sudden traffic spikes, inefficient resource usage, or slow downstream services.

At first, congestion might appear as increased latency. Requests start taking longer to process. Queues begin to form. Eventually, the system may start rejecting requests or timing out.

In monolithic systems, congestion tends to affect the entire application. Since all components share the same resources, a bottleneck in one part can slow down everything else.

Distributed systems handle congestion differently—but not necessarily better by default.

Because services depend on each other, congestion in one service can propagate through the system.

Consider a scenario where a database becomes slow. The service relying on it starts waiting longer for responses. As a result, incoming requests pile up. Upstream services may start retrying requests, further increasing the load.

This creates a dangerous feedback loop.

What begins as a small slowdown can escalate into a cascading failure, where multiple parts of the system degrade simultaneously.

To handle congestion, systems implement protective mechanisms such as:

Limiting incoming requests
Dropping excess traffic
Temporarily isolating failing services

These are not optimizations - they are survival strategies.

Bringing It All Together

At this point, we’ve explored all the fundamental forces:

Latency
Throughput
Availability
Consistency
Replication
Redundancy
Congestion

Individually, each concept is understandable. But the real challenge lies in how they interact.

Improving availability through replication may weaken consistency.
Increasing throughput through distribution may increase latency.
Adding redundancy improves resilience but increases complexity.
Handling congestion may require rejecting requests, impacting availability.

This is why system design is not about finding perfect solutions.

It is about making informed trade-offs.

And those trade-offs depend entirely on the system you are building.

The Real System Design Mindset

A strong system designer does not think in terms of isolated concepts. They think in terms of constraints and priorities.

Is low latency critical for user experience?
Can the system tolerate stale data?
How important is availability compared to consistency?
What happens when traffic suddenly spikes?

The answers to these questions shape every architectural decision.

And this is why two systems solving similar problems may look completely different internally.

Because they are optimizing for different trade-offs.

Final Thought

If there is one idea that defines system design, it is this:

Everything is a trade-off.

There is no architecture that maximizes latency, throughput, availability, and consistency all at once. Every system chooses what to prioritize and what to sacrifice.

Understanding these trade-offs is what transforms system design from a collection of concepts into a way of thinking.

And once you start thinking this way, every system you encounter—no matter how complex—begins to make sense.

Monolithic vs Distributed Systems: Trade-offs, Evolution, and Real-World Design Decisions

Sushant Gaurav — Tue, 07 Apr 2026 05:41:00 +0000

There was a time when building software did not require thinking about millions of users, global traffic distribution, or handling failures across continents. Systems were smaller, teams were tighter, and the primary goal was simple: build something that works, and ship it fast.

In that world, the most natural way to build software was to keep everything together. Not because it was the "best architecture", but because it was the most intuitive one.

This is what we now call a monolithic architecture.

A monolithic system is essentially a single, unified application where all functionalities coexist within the same codebase and are deployed as one unit. The user interface, business logic, authentication, database interactions—everything resides in one place. When a user makes a request, it flows through this single application, which internally handles all responsibilities.

What makes this model powerful is not just its simplicity, but the absence of boundaries. Components do not need to communicate over networks; they simply call each other directly within the same process. There is no need for serialization, no waiting for responses over HTTP, no retries, and no concern about network failures. The system behaves like a tightly coordinated machine.

Because of this, monolithic architectures tend to be extremely efficient in their early stages. Latency is minimal since everything runs locally. Debugging is straightforward because the entire flow of execution exists within a single environment. Deployment is predictable—build once, deploy once, and the entire system is updated.

This is why, even today, many highly successful products begin their journey as monoliths. At a small to moderate scale, the overhead of distributed thinking often introduces more problems than it solves. A monolith allows teams to focus on business logic rather than infrastructure complexity.

But as systems grow, something subtle yet inevitable begins to happen.

The very strength of a monolith—its tight integration—starts becoming its biggest limitation.

Imagine a scenario where an application gains traction and user traffic increases rapidly. Perhaps a particular feature, like payments or search, begins receiving significantly more load than other parts of the system. In a monolithic architecture, there is no way to scale just that part. The entire application must be scaled as a whole, even if only one component is under stress. This leads to inefficient resource utilization and increased infrastructure costs.

At the same time, development teams begin to expand. What was once a small group of engineers has become dozens, sometimes hundreds, of contributors working on the same codebase. Coordination becomes harder. A small change in one module requires rebuilding and redeploying the entire system. Testing cycles grow longer. The risk associated with each deployment increases, because any bug—no matter how isolated—can potentially impact the entire application.

Then comes the issue of failures. In a tightly coupled system, components are deeply interconnected. If one critical module fails or behaves unexpectedly, it can cascade through the system, leading to a complete outage. The application, being a single unit, often becomes a single point of failure.

None of these problems appears suddenly. They emerge gradually, almost invisibly, as scale increases—both in terms of users and engineering complexity.

This is the point where the industry began to rethink how systems should be designed.

Instead of building one large application that does everything, what if we broke it down into smaller, independent pieces? What if each part of the system could operate on its own, scale independently, and fail without bringing everything else down?

This line of thinking gave rise to distributed systems.

A distributed system is not a single application but a collection of smaller services, each responsible for a specific functionality. These services run independently and communicate with each other over a network. Unlike a monolith, where everything exists within a single boundary, a distributed system is defined by boundaries and interactions.

In this model, each service can evolve on its own. The authentication service can be updated without touching the payment system. The order service can be scaled independently based on demand. Teams can work in parallel, owning different services without stepping on each other’s toes.

More importantly, failures become isolated. If the recommendation service crashes, the core checkout flow might still continue to function. The system does not necessarily fail completely; instead, it degrades gracefully.

This architectural shift was not driven by trends or preferences. It was driven by necessity. As companies like Amazon, Netflix, and Google began operating at a massive scale, the limitations of monolithic systems became impossible to ignore. Distributed systems allowed them to handle global traffic, improve availability, and enable rapid innovation across large teams.

However, this evolution came with a cost.

In a monolith, communication between components is instantaneous and reliable because it happens within the same process. In a distributed system, every interaction becomes a network call. This introduces latency, uncertainty, and the possibility of partial failures. A service might respond slowly, or not at all. Messages can be delayed, duplicated, or lost. Debugging becomes significantly more complex because a single user request might traverse multiple services across different machines.

Data consistency also becomes a challenge. In a monolith, a single database ensures a consistent view of data. In a distributed system, each service may have its own database, and keeping them in sync requires careful design. Concepts like eventual consistency, retries, and idempotency start becoming part of everyday engineering discussions.

In other words, distributed systems solve the problems of scale, but they introduce a new class of problems rooted in complexity.

This leads to a critical realization, one that is often overlooked:

Distributed systems are not a replacement for monoliths. They are a response to specific challenges.

In fact, many modern systems today adopt a hybrid approach. They begin as monoliths, evolve into distributed systems where necessary, and sometimes even retain monolithic components where simplicity is more valuable than scalability.

Understanding this evolution is crucial for system design. Because the real question is not:

"Which architecture is better?"

The real question is:

"Which architecture is appropriate for the problem you are solving?"

And that is where the deeper comparison begins.

To understand the difference between monolithic and distributed systems, we need to understand these forces—not as definitions, but as behaviors that emerge under load.

Latency vs Throughput - The First Trade-off

Whenever a user interacts with a system, two things matter:
how fast the system responds, and how many such requests it can handle over time.

Latency is the time taken to serve a single request. It answers the question: "How long did this request take?"

Throughput, on the other hand, is the number of requests a system can handle per unit time. It answers: "How many requests can we process per second?"

At first glance, these seem independent. But in practice, they are deeply connected.

In a monolithic system, latency is often lower for internal operations because everything runs within the same process. A function call is just a jump in memory—fast, predictable, and reliable. There is no network overhead, no serialization, no waiting for responses from other services.

This gives monoliths a natural advantage when it comes to low-latency execution.

However, as traffic increases, throughput becomes the bottleneck. Since the system is deployed as a single unit, scaling requires replicating the entire application. Even if only one part of the system is under heavy load, the whole system must scale. This creates inefficiencies and limits how effectively resources are used.

Distributed systems approach this problem differently. By breaking the system into smaller services, they allow different parts of the system to scale independently. This significantly improves throughput because multiple services can handle requests in parallel, each optimized for its own workload.

But this comes at a cost.

Every interaction between services now involves network communication. A single user request might pass through an API gateway, then an authentication service, then an order service, and finally a payment service. Each hop adds latency. Even if each service is fast individually, the cumulative delay can be significant.

This is why distributed systems often trade higher throughput for increased latency.

And this is one of the first key insights in system design:

You rarely optimize for both latency and throughput at the same time. You choose what matters more for your use case.

The Role of Caching — Fighting Latency

As systems grow, repeatedly computing the same results becomes inefficient. This is where caching comes into play.

Caching is the idea of storing frequently accessed data in a location where it can be retrieved faster.

In a monolithic system, caching is relatively straightforward. Since everything runs in a single process, data can be cached in memory and accessed instantly. The system has a unified view of data, making cache invalidation simpler (though never trivial).

In distributed systems, caching becomes both more powerful and more complicated.

Instead of a single cache, you may have:

Caches at the service level
Distributed caches shared across services
Edge caches closer to users

This introduces a new layer of decision-making: where should the cache live?

For example, caching at the edge closer to the user can drastically reduce latency. This is often done using CDNs (Content Delivery Networks), which store copies of data in geographically distributed locations.

When a user requests content, it is served from the nearest location rather than the origin server.

This is why platforms like Netflix can stream content smoothly across the globe. Instead of serving every request from a central system, they rely heavily on distributed caching and CDNs to reduce latency and improve user experience.

But caching introduces its own challenge: stale data.

In distributed systems, ensuring that all caches reflect the latest data is extremely difficult. This leads to trade-offs between consistency and performance.

Replication vs Redundancy — Designing for Failure

As systems scale, failure is no longer a possibility—it is an expectation.

The question is not if something will fail, but when.

This is where replication and redundancy come into play.

Replication refers to maintaining multiple copies of the same data or service. If one copy fails, another can take over.

Redundancy, on the other hand, is a broader concept. It involves having extra components or systems that can handle failures, even if they are not exact copies.

In a monolithic system, replication usually happens at the application level. Multiple instances of the same application are deployed behind a load balancer. If one instance goes down, others continue to serve requests.

While this improves availability, the system still behaves as a single logical unit. A bug in the code can affect all instances simultaneously.

Distributed systems take this further. Different services can be replicated independently. Databases can be replicated across regions. Entire data centres can act as backups for each other.

This allows for fine-grained fault tolerance.

For example, if a payment service fails in one region, traffic can be routed to another region without affecting other services.

This is how companies like Amazon achieve high availability at a global scale.

But again, complexity increases. Keeping replicated data consistent across regions introduces challenges like:

Data synchronization delays
Conflict resolution
Eventual consistency

Availability — Keeping the System Alive

Availability is the measure of whether a system is accessible and operational when needed.

A system that is highly available continues to function even in the presence of failures.

Monolithic systems can achieve availability through replication and load balancing. However, because they are tightly coupled, failures can propagate more easily. A critical bug or resource exhaustion can impact the entire system.

Distributed systems are designed with availability as a core principle. By isolating services, they prevent failures from spreading across the system. Even if some services are down, others can continue to operate.

This leads to the concept of graceful degradation.

For example, an e-commerce platform might still allow users to browse products even if the recommendation service is down.

This kind of resilience is much harder to achieve in a monolithic system.

Congestion — When Systems Get Overwhelmed

As traffic increases, systems can become congested. This happens when the demand exceeds the system’s capacity to process requests.

In a monolithic system, congestion often affects the entire application. Since all components share the same resources, a spike in one area can slow down everything else.

Distributed systems handle congestion more effectively by isolating workloads. If one service becomes overloaded, it does not necessarily impact others.

However, congestion in distributed systems can propagate through dependencies. If a downstream service becomes slow, upstream services may start timing out, retrying requests, and amplifying the load.

This can lead to cascading failures—a common challenge in distributed architectures.

To handle this, systems implement strategies like:

Rate limiting
Circuit breakers
Load shedding

These mechanisms help prevent the system from collapsing under pressure.

The Deeper Insight

At this point, something important should start becoming clear.

Monolithic and distributed systems are not just different ways of organizing code. They represent fundamentally different approaches to handling time, scale, and failure.

A monolithic system optimizes for:

Simplicity
Low latency
Easier reasoning

A distributed system optimizes for:

Scalability
Fault isolation
High availability

But neither is universally better.

Every improvement in distributed systems comes with a trade-off:

Better scalability → higher complexity
Better availability → harder consistency
Better fault tolerance → more operational overhead

This is why system design is always about trade-offs, not choices.

Monolithic vs Distributed Systems — Where Each One Truly Shines

By now, the discussion has moved beyond definitions and into the deeper mechanics of how systems behave under load, failure, and scale. But system design, in the real world, is rarely about understanding concepts in isolation. It is about making judgment calls under constraints.

And this is where one of the biggest misconceptions in the industry needs to be addressed.

There is a tendency—especially among engineers preparing for system design interviews—to assume that distributed systems are inherently superior. That breaking everything into microservices is the "modern" or "correct" way to build software.

But in reality, many of the most effective engineering teams deliberately choose not to distribute their systems too early.

Because sometimes, the smartest architecture is the simplest one.

Where Monolithic Systems Still Shine

A monolithic architecture excels in environments where speed of development and simplicity of reasoning matter more than extreme scalability.

Consider the early stages of a product. At this point, the biggest risk is not system failure under massive load—it is whether the product solves a real problem at all. Teams need to iterate quickly, ship features rapidly, and respond to user feedback without being slowed down by infrastructure complexity.

In such scenarios, a monolith provides a powerful advantage.

Because everything lives in one place, developers can easily understand the system. There is no need to manage inter-service communication, no distributed tracing to debug requests across multiple services, and no need to handle network-level failures. A single codebase allows for faster onboarding, quicker experimentation, and more predictable deployments.

This is one of the reasons why companies like Shopify and Basecamp have historically leaned heavily on monolithic architectures, even as they scaled to serve large user bases. Their focus has been on maintaining developer productivity and system clarity rather than prematurely introducing complexity.

Another often overlooked advantage of monoliths is strong consistency. Since all components typically share a single database, maintaining a consistent view of data is straightforward. Transactions can span multiple parts of the system without worrying about network partitions or synchronization delays.

Monoliths also tend to perform better in scenarios where low latency is critical within the system itself. Since components communicate via direct function calls, the overhead is minimal compared to network-based communication.

However, these advantages come with limits. As the system grows, the same simplicity that once accelerated development can start slowing it down. Codebases become harder to navigate, deployments become riskier, and scaling becomes inefficient.

Where Distributed Systems Become Necessary

Distributed systems begin to shine when the constraints shift from simplicity to scale, resilience, and organizational complexity.

At a large scale, it is no longer practical to treat the system as a single unit. Different parts of the application experience different levels of load. Some services require high availability, while others can tolerate occasional downtime. Some need to scale aggressively, while others remain relatively stable.

A distributed architecture allows each of these concerns to be handled independently.

This is particularly important in systems with massive user bases and global reach. Companies like Netflix and Amazon operate across multiple regions, serving millions of requests per second. For them, distributing services across data centers is not an optimization it is a necessity.

Distributed systems also enable fault isolation, which becomes critical at scale. In a monolith, a failure in one part of the system can potentially bring down the entire application. In a distributed system, failures can be contained within individual services, allowing the rest of the system to continue functioning.

Another key advantage is team scalability.

As organizations grow, having multiple teams work on the same codebase becomes increasingly difficult. Distributed systems allow teams to own specific services, define clear boundaries, and work independently. This reduces coordination overhead and enables faster development cycles.

However, these benefits come at a cost that is often underestimated.

Distributed systems introduce complexity at every level—networking, data consistency, deployment pipelines, monitoring, and debugging. Engineers must now think about retries, timeouts, circuit breakers, and eventual consistency. Observability becomes a critical requirement rather than a nice-to-have.

In many cases, teams adopt distributed architectures without fully understanding these challenges, leading to systems that are harder to manage than the monoliths they replaced.

The Hybrid Reality — Where Most Systems Actually Live

In practice, very few systems are purely monolithic or purely distributed.

Most real-world architectures exist somewhere in between.

A common pattern is to start with a monolith and gradually extract services as needed. Instead of breaking the system into dozens of microservices from the beginning, teams identify natural boundaries—areas of the system that require independent scaling or have distinct responsibilities and separate them over time.

This approach allows teams to retain the simplicity of a monolith while selectively introducing the benefits of distribution.

For example, a system might keep its core business logic in a monolith while offloading specific concerns—such as search, recommendations, or notifications to separate services. These services can then be scaled and optimized independently without complicating the entire system.

This hybrid model reflects a deeper understanding of system design:

Not everything needs to be distributed—only the parts that truly benefit from it.

The "Distributed Monolith" Trap

One of the most common pitfalls in modern system design is what is often referred to as a distributed monolith.

This happens when a system is split into multiple services, but those services remain tightly coupled. They depend heavily on each other, require coordinated deployments, and cannot function independently.

From the outside, it looks like a distributed system. But internally, it behaves like a monolith—only with added network complexity.

This is arguably the worst of both worlds.

The system inherits:

The operational complexity of distribution
The tight coupling of a monolith

Avoiding this trap requires careful design of service boundaries, clear ownership, and a strong emphasis on loose coupling.

A Subtle but Powerful Insight

At this stage, a deeper pattern begins to emerge.

Monolithic systems optimize for clarity and control.
Distributed systems optimize for scale and resilience.

But the real skill in system design lies in knowing:

When clarity matters more than scalability
When resilience justifies complexity
When to delay decisions instead of over-engineering early

This is why experienced engineers often say:

"Design for today’s requirements, but keep tomorrow in mind".

Not every system needs to handle millions of users from day one. But every system should be designed in a way that allows it to evolve when the need arises.

How do you actually decide what to build?

Because in real engineering—and especially in system design interviews—you are not rewarded for knowing definitions. You are evaluated on your ability to make the right trade-offs for a given problem.

And that decision is rarely obvious.

There Is No “Best Architecture” — Only Context

One of the most important mindset shifts in system design is understanding that architecture is not about picking the “best” option. It is about choosing the most appropriate one given your constraints.

These constraints can include:

Number of users
Expected growth
Team size
Time to market
Reliability requirements
Budget and infrastructure

A startup building its first product and a global platform serving millions of users are solving fundamentally different problems. Expecting them to use the same architecture would not just be inefficient—it would be incorrect.

This is why strong engineers do not jump to solutions. They start by asking:

What problem are we actually trying to solve?

A Practical Decision Framework

When deciding between a monolithic and distributed architecture, it helps to think in terms of pressure points rather than trends.

A monolithic system is often the right choice when the primary goal is to move fast, validate ideas, and keep complexity low. If your system does not yet face heavy traffic, if your team is small, and if your features are still evolving rapidly, introducing distributed complexity too early can slow you down significantly.

On the other hand, distributed systems become valuable when certain pressures start to emerge.

One such pressure is uneven scaling. If specific parts of your system—such as search, payments, or media processing—require significantly more resources than others, splitting them into independent services allows you to scale efficiently.

Another pressure is availability requirements. If your system must remain operational even when parts of it fail, distributing responsibilities across services enables fault isolation and graceful degradation.

A third pressure is team structure. As organizations grow, having multiple teams work on a single codebase can become a bottleneck. Distributed systems allow teams to own services independently, reducing coordination overhead.

And finally, there is geographical scale. If your users are spread across regions, distributing services closer to them reduces latency and improves performance.

The key insight here is simple:

You don’t move to distributed systems because it’s modern.
You move because your system demands it.

Common Mistakes Engineers Make

Despite understanding these principles, many engineers fall into predictable traps.

One of the most common is over-engineering too early. Influenced by large-scale architectures from companies like Netflix or Amazon, teams attempt to replicate microservices-based designs without having the scale to justify them.

The result is often a system that is harder to build, harder to debug, and slower to evolve.

Another mistake is ignoring operational complexity. Distributed systems require robust monitoring, logging, and tracing. Without these, debugging becomes extremely difficult, as a single request may traverse multiple services.

There is also a tendency to underestimate data consistency challenges. Engineers accustomed to monolithic systems often assume strong consistency guarantees, only to encounter issues when working with distributed databases and eventual consistency models.

And perhaps the most subtle mistake is failing to define clear service boundaries, leading to tightly coupled services that behave like a monolith—just spread across a network.

Thinking Like a System Designer

At this point, the discussion moves beyond architecture into mindset.

A strong system designer does not think in terms of monolith vs distributed as binary choices. Instead, they think in layers:

What are the core components of the system?
Which parts are likely to scale independently?
Where can failures occur, and how should they be handled?
What level of consistency is required?
How will the system evolve?

These questions naturally guide the architecture.

For example, a system might start as a monolith, then gradually extract services for high-load components, and eventually adopt a more distributed structure as scale increases. This evolution is not forced—it is driven by real needs.

The Evolution Mindset

One of the most powerful ideas in system design is this:

Good architectures are not designed once—they evolve.

Many of today’s large-scale systems did not start as distributed architectures. They started simple, learned from real usage, and adapted over time.

Trying to predict every future requirement upfront often leads to unnecessary complexity. Instead, systems should be designed with evolution in mind.

This means:

Writing modular code
Defining clear boundaries
Avoiding tight coupling
Keeping deployment flexible

These practices make it easier to transition from monolithic to distributed systems when the time comes.

Final Takeaways

As we bring everything together, a few key ideas stand out.

Monolithic systems are not outdated—they are foundational. They provide simplicity, speed, and clarity, making them ideal for early-stage development and many real-world applications.

Distributed systems are not inherently better—they are specialized tools designed to handle scale, failure, and complexity.

The real skill lies in understanding when to use each, and how to transition between them without introducing unnecessary complexity.

Because at the end of the day, system design is not about architecture diagrams or buzzwords.

It is about building systems that:

Solve real problems
Scale when needed
Remain reliable under pressure
And are understandable by the people who build and maintain them

Closing Thought

If there’s one idea worth carrying forward, it’s this:

Start simple. Scale smart. Evolve intentionally.

That is the essence of great system design.

From DevOps to Platform Engineering and GitOps: The Complete Guide to Modern Software Delivery

Sushant Gaurav — Tue, 24 Mar 2026 05:41:00 +0000

Modern software delivery feels almost magical.

Manual Ops
   ↓
Continuous Integration
   ↓
DevOps
   ↓
Platform Engineering
   ↓
GitOps

A developer writes code, pushes it to Git, and within minutes the application is built, tested, containerized, deployed, monitored, and running in production.

But this level of automation did not appear overnight.

Behind it lies an evolution that spans multiple stages:

Traditional software operations
DevOps
CI/CD pipelines
GitOps
Platform Engineering

If you’ve ever wondered questions like:

How were deployments handled before DevOps?
What exactly is the difference between Continuous Delivery and Continuous Deployment?
What is GitOps and why is everyone talking about it?
What is Platform Engineering and how is it different from DevOps?

Then this article will walk you through the entire journey.

We’ll go step by step and build the mental model together.

1. How Software Deployment Worked Before DevOps

Before the DevOps movement, most companies followed the Waterfall development model.

Teams were divided into strict silos:

Development team
QA team
Operations team

Each team had its own responsibilities and very little overlap.

A typical workflow looked like this:

Developers write code
↓
Code handed to QA team
↓
QA performs manual testing
↓
Operations team deploys the software

The operations team was responsible for running production servers.

Deployments were usually manual and looked something like this:

1. Operations receives build artifact (ZIP/JAR)
2. SSH into production server
3. Copy new files
4. Restart application
5. Hope nothing breaks

This approach created several problems:

Deployments were slow
Releases happened every few months
Debugging production failures was difficult
Developers and operations blamed each other when things failed

You may have heard the classic phrase:

“It works on my machine.”

That phrase existed because development environments and production environments were completely different.

2. The First Major Improvement: Continuous Integration

As systems grew larger and teams grew bigger, integration became painful.

Imagine 50 developers working on the same codebase.

If everyone works independently for weeks and merges their code at the end, conflicts become massive.

To solve this, the concept of Continuous Integration (CI) was introduced.

Continuous Integration simply means:

Developers integrate their code frequently into a shared repository, and automated tests run every time.

The workflow becomes:

Developer writes code
↓
Push code to Git
↓
CI server runs build
↓
Automated tests execute
↓
If something fails → developer fixes immediately

Instead of integrating code after weeks, integration happens multiple times per day.

CI servers automate this process.

Popular CI tools include:

Jenkins
GitHub Actions
GitLab CI

With CI, teams gained:

faster feedback
fewer integration conflicts
more stable codebases

But CI alone did not solve the entire problem.

Deployment was still largely manual.

3. Continuous Delivery vs Continuous Deployment

At this stage many engineers encounter two terms that often cause confusion:

Continuous Delivery
Continuous Deployment

Both are abbreviated as CD, which makes things even more confusing.

Let’s clarify the difference.

Continuous Delivery

Continuous Delivery means:

Every code change is automatically built, tested, and prepared for release, but deployment to production requires human approval.

The pipeline typically looks like this:

Code commit
↓
Build
↓
Automated tests
↓
Security checks
↓
Deploy to staging
↓
Manual approval
↓
Production deployment

The key idea here is:

The software is always ready to be deployed.

But the business decides when the release happens.

Continuous Deployment

Continuous Deployment takes automation one step further.

It means:

Every change that passes the pipeline is automatically deployed to production.

The pipeline becomes:

Code commit
↓
Build
↓
Tests
↓
Security checks
↓
Automatically deploy to production

There is no manual approval step.

As soon as the pipeline succeeds, the change goes live.

Companies like Netflix and Facebook deploy hundreds or thousands of times per day using this model.

The Simplest Way to Remember the Difference

Ask one question:

Does a human press the deploy button?

If yes → Continuous Delivery
If no → Continuous Deployment

4. The Rise of DevOps

Even with CI/CD pipelines, organizations faced another major issue.

Developers and operations teams still worked separately.

Developers wanted:

faster releases
frequent changes

Operations teams wanted:

stability
minimal downtime

These goals often conflicted.

The DevOps movement emerged to solve this problem.

DevOps is not just a role or a toolset.

DevOps is primarily a culture and philosophy.

It encourages:

collaboration between development and operations
automation of repetitive tasks
shared responsibility for software delivery

Instead of separate teams, organizations started creating cross-functional teams responsible for the entire lifecycle of an application.

This includes:

development
testing
deployment
monitoring
operations

With DevOps practices, the pipeline evolved into something like this:

Developer writes code
↓
Push to Git
↓
CI builds and tests application
↓
Container image created
↓
Deployment to staging
↓
Deployment to production

Automation tools made this possible.

Technologies like Docker and Kubernetes dramatically accelerated the DevOps movement by making applications easier to package and deploy.

5. The DevOps Scaling Problem

DevOps worked well for small teams.

But as organizations grew, a new challenge appeared.

Imagine a company with:

100 engineering teams
hundreds of microservices
thousands of deployments per day

Each team started implementing its own DevOps pipelines.

Team A might use:

Jenkins
Terraform
Kubernetes
Helm

Team B might use:

GitHub Actions
ArgoCD
Terraform
Kubernetes

Team C might create completely custom scripts.

Over time the organization ends up with:

dozens of CI pipelines
different deployment strategies
inconsistent security policies
duplicated infrastructure work

At this point DevOps becomes difficult to manage at scale.

Companies tried solving this by creating central DevOps teams, but that introduced another issue.

Developers now had to open tickets for infrastructure requests.

For example:

Developer: please create a new service deployment
DevOps: ticket received
DevOps: implementation scheduled
Developer: waiting...

DevOps slowly turned back into a bottleneck.

And this is exactly where Platform Engineering enters the picture.

6. The Emergence of Platform Engineering

As companies scaled their systems and teams, a recurring problem appeared.

Even though DevOps promoted collaboration and automation, every team still had to manage a large amount of infrastructure complexity.

A typical developer working in a modern cloud-native environment might need to understand:

containerization
CI/CD pipelines
Kubernetes deployments
networking
observability
security policies
secrets management
scaling configurations

This created a problem known as cognitive overload.

Developers were spending too much time managing infrastructure and too little time solving business problems.

To address this challenge, organizations began introducing internal developer platforms.

The teams responsible for building and maintaining these systems became known as Platform Engineering teams.

7. What Platform Engineering Actually Means

Platform Engineering focuses on building internal platforms that simplify software delivery for developers.

Instead of every team building its own infrastructure pipelines, a platform team builds reusable systems that application teams consume.

Think of it as building an internal cloud platform inside your company.

Developers use the platform just like they use external cloud services.

For example:

Instead of configuring infrastructure manually, a developer might simply run:

create-service payment-service

Or use a web interface to generate a new service.

Behind the scenes the platform automatically generates:

repository structure
CI pipeline
container build configuration
deployment configuration
monitoring dashboards

The developer can then focus purely on writing application logic.

8. A Practical Example: Creating a Microservice

To understand this better, let’s walk through a real example.

Suppose a developer wants to create a new payment microservice.

In many companies this process starts in an internal developer portal such as Backstage.

Backstage is an open source developer portal created by Spotify that helps manage internal services and infrastructure.

The developer opens the portal and selects:

Create New Service

Then fills out a simple form:

Service name: payment-service
Programming language: Python
Database: PostgreSQL

After clicking Create, the platform takes over.

9. What Happens Behind the Scenes

The platform automatically performs several steps.

Step 1: Create a Repository

A new repository is generated in the organization's source control system.

Example structure:

payment-service/

This repository already contains a standardized project layout.

Step 2: Generate Starter Code

The platform inserts a predefined template.

Example structure:

payment-service/
 ├── src/
 │   └── app.py
 ├── tests/
 ├── Dockerfile
 ├── deployment/
 │   └── kubernetes.yaml
 └── ci-pipeline.yml

This template follows best practices defined by the platform team.

Step 3: CI Pipeline Is Already Configured

The repository includes a prebuilt CI workflow.

For example:

Push code
↓
Run automated tests
↓
Build container image
↓
Push image to container registry

The developer does not need to configure this manually.

Step 4: Containerization

The platform provides a standard container configuration.

For example:

Dockerfile

This ensures consistent container builds across all services.

Step 5: Deployment Configuration

Deployment manifests are already included.

For example:

deployment/
 └── kubernetes.yaml

These files define how the application runs inside a Kubernetes cluster.

Step 6: Monitoring and Observability

The platform automatically connects the service to the company’s monitoring stack.

Dashboards and metrics are generated automatically.

Developers can immediately see:

request rates
error rates
latency metrics

10. What the Developer Actually Does

After the platform sets everything up, the developer workflow becomes extremely simple.

1. Generate service from template
2. Write application code
3. Push code to Git

Everything else happens automatically.

This dramatically reduces the complexity developers must deal with.

11. What Platform Engineers Actually Build

Platform engineers do not build business applications.

Instead, they build the systems that enable other engineers to build applications efficiently.

Their responsibilities include several areas.

Building Service Templates

Platform engineers create standardized templates for different service types.

Examples:

Python microservice template
NodeJS microservice template
Java microservice template

Each template includes:

repository structure
CI pipeline
container configuration
deployment configuration
security policies

Creating Developer Portals

Internal developer portals allow teams to interact with the platform.

These portals provide features such as:

service catalogs
project templates
documentation
deployment management

Backstage is one of the most widely used tools for this purpose.

Managing Infrastructure Platforms

Platform engineers maintain the underlying infrastructure.

This often includes:

Kubernetes clusters
container registries
networking infrastructure
storage systems

They ensure the platform remains stable, scalable, and secure.

Building Deployment Automation

Platform engineers design automated pipelines that handle:

build
test
containerization
deployment
monitoring integration

These pipelines are reused across many services.

Infrastructure as Code

Infrastructure is defined using code.

Platform engineers use tools that allow infrastructure to be managed programmatically.

This enables consistent environments and reproducible deployments.

12. Introducing GitOps

Another major innovation in modern infrastructure management is GitOps.

GitOps is a deployment model where Git becomes the single source of truth for system state.

Instead of directly deploying changes to infrastructure, engineers update configuration stored in Git repositories.

Automated systems then synchronize the infrastructure with the desired state defined in Git.

13. Traditional CI/CD Deployment Model

In a traditional CI/CD system, deployment works like this:

Developer pushes code
↓
CI pipeline runs
↓
Pipeline deploys application to cluster

The CI system pushes changes directly to the infrastructure.

14. GitOps Deployment Model

GitOps changes this model slightly.

Instead of the CI system pushing changes to infrastructure, a GitOps controller pulls changes from Git.

The workflow becomes:

Developer pushes code
↓
CI builds container image
↓
CI updates deployment configuration in Git
↓
GitOps controller detects change
↓
Cluster synchronizes with Git

The cluster continuously ensures its state matches the configuration stored in Git.

15. Advantages of GitOps

GitOps offers several major advantages.

Complete Audit History

Every infrastructure change is stored in Git.

This creates a full audit trail.

Easy Rollbacks

If a deployment causes issues, rolling back becomes as simple as reverting a commit.

Improved Security

CI systems no longer require direct access to production clusters.

The cluster pulls updates instead of receiving pushed deployments.

Self-Healing Systems

GitOps controllers constantly monitor the cluster state.

If someone manually changes something in the cluster, the controller automatically restores the correct configuration from Git.

16. GitOps and Platform Engineering Together

Platform Engineering and GitOps often complement each other.

The platform team builds systems that generate deployment configurations automatically.

Those configurations are stored in Git repositories.

GitOps tools then ensure the running infrastructure matches those configurations.

This architecture creates a powerful and scalable deployment system.

17. The Concept of "Golden Paths"

One of the most important ideas in modern platform engineering is the concept of Golden Paths.

A Golden Path is a recommended way of building and deploying services within an organization.

Instead of forcing developers to follow strict rules, the platform team provides well-designed templates that make the recommended approach the easiest option.

Developers can still customize their systems if needed, but most teams naturally follow the Golden Path because it simplifies development.

Golden Paths typically include:

standardized service templates
predefined CI pipelines
secure deployment configurations
built-in observability

This approach balances flexibility with consistency.

18. The Modern Cloud-Native Stack

Today’s cloud-native platforms often include technologies such as:

Kubernetes for container orchestration
GitOps tools for deployment synchronization
container registries for artifact storage
CI systems for automated testing and builds
developer portals for platform interaction

Together these systems create a powerful ecosystem that allows organizations to deploy software safely and rapidly.

Final Thoughts

The journey from traditional operations to modern cloud-native platforms represents a major transformation in software engineering.

We have moved through several phases:

Manual Operations
↓
Continuous Integration
↓
DevOps
↓
Platform Engineering
↓
GitOps

Each stage emerged to solve scaling challenges created by the previous one.

The ultimate goal has always remained the same:

enable developers to deliver reliable software faster and more safely.

As systems continue to grow in complexity, internal platforms and automated infrastructure will become even more important.

Understanding these concepts will help engineers design better systems and contribute effectively to modern cloud-native environments.