One of the biggest promises of distributed systems is that they allow applications to scale beyond the limits of a single machine. By distributing data across multiple servers, applications become capable of handling millions of users, surviving hardware failures, and serving requests from different parts of the world.
At first glance, this sounds like the perfect solution.
If one database server is no longer enough, simply add another.
If one data centre becomes overloaded, deploy another in a different region.
If a server fails, allow another replica to take over.
Everything appears straightforward.
However, the moment multiple copies of the same data begin existing on different machines, distributed systems encounter one of their most fundamental challenges.
How do we ensure that every copy of the data remains correct?
This question sounds deceptively simple.
In reality, it is one of the hardest problems in computer science.
A Single Database Never Faces This Problem
Imagine an application that stores all of its information inside a single database server.
A customer updates their delivery address.
The application writes the new address into the database.
Every future request reads that exact value.
There is no confusion.
There is only one copy of the data, so every user sees the same information.
Life is simple.
This simplicity is one of the reasons monolithic applications are relatively easy to reason about.
There is only one source of truth.
As long as that database is healthy, every request returns identical information.
Now imagine the application becomes enormously successful.
Millions of users begin accessing it every day.
The single database starts struggling under the load.
To improve availability and scalability, engineers decide to introduce database replication.
Instead of maintaining one database server, they now maintain several identical copies.
At first glance, nothing appears to have changed.
The application still stores the same data.
Users still perform the same operations.
But internally, the architecture has become dramatically more complicated.
There is no longer one copy of the data.
There are several.
And those copies must somehow remain synchronised.
When Copies Disagree
Suppose a customer changes their password.
The application successfully writes the new password to the primary database.
Immediately afterwards, the updated information begins replicating to the remaining database replicas.
But replication is not instantaneous.
It takes time.
Perhaps only a few milliseconds.
Sometimes hundreds of milliseconds.
Occasionally even longer.
Now imagine another request arrives before every replica has received the update.
One server contains the new password.
Another still contains the old one.
Which version should the application return?
Consider another example.
A customer transfers money between two bank accounts.
The transaction completes successfully.
One database replica reflects the new account balance.
Another replica has not yet received the update.
If the customer immediately refreshes their banking application, should they see the old balance or the new one?
Neither answer feels satisfactory.
Returning outdated information can confuse users.
Waiting indefinitely until every server agrees can make the system painfully slow.
This is the central dilemma of consistency in distributed systems.
Understanding Consistency
When engineers talk about consistency, they are not referring to data correctness in the traditional sense.
Instead, they are asking a very specific question:
If one client writes new data, when should every other client be able to observe that change?
Notice what this definition focuses on.
It is not asking whether the data is valid.
It is asking when different parts of the system should agree on the latest value.
That single word—when—is incredibly important.
Some systems require agreement immediately.
Others can tolerate small delays.
The acceptable answer depends entirely on the application.
Why Perfect Consistency Is Surprisingly Difficult
At first, the solution seems obvious.
Whenever data changes, simply update every server before responding to the user.
Problem solved.
Unfortunately, distributed systems operate over networks.
Networks introduce latency.
Messages can be delayed.
Servers can temporarily become unavailable.
Entire data centres can lose connectivity.
Imagine an application with database replicas in New York, London, Singapore, and Sydney.
Every update must now travel thousands of miles across the internet.
Even travelling at nearly the speed of light, information still requires time to move between continents.
Network congestion, routing changes, and temporary failures increase that delay even further.
Now consider a customer in Singapore updating their profile picture.
Should a user in London immediately see the new image?
What if the network cable connecting Europe and Asia is temporarily unavailable?
Should the application wait?
Should it reject all requests?
Or should it temporarily allow users to see slightly outdated information?
These are not theoretical questions.
Large internet companies make decisions like these every day.
The Relationship Between Replication and Consistency
In our previous article about Data Partitioning and Sharding, we discussed how systems distribute data across multiple machines to improve scalability.
In another article, we explored Replication, where multiple copies of the same data improve availability and fault tolerance.
Replication solves one problem while creating another.
The moment multiple copies of the same information exist, the system must decide how quickly those copies should become identical.
The faster replicas synchronise, the more consistent the system becomes.
The longer synchronisation takes, the greater the possibility that different users temporarily observe different versions of the same information.
Replication and consistency are therefore deeply connected.
One cannot exist without influencing the other.
A Real-World Example Everyone Has Experienced
Most people have experienced eventual consistency without realising it.
Suppose you change your profile picture on a social media platform.
Immediately afterwards, you refresh your own profile and see the new image.
A friend opens your profile from another country a few seconds later.
Surprisingly, they still see the old picture.
A short time afterwards, they refresh again.
Now the new picture appears.
Was the system broken?
Not at all.
The update simply required time to propagate across multiple servers distributed around the world.
During those few seconds, different users observed different versions of the same data.
This behaviour is completely normal in many distributed systems.
The important question is whether that temporary inconsistency is acceptable.
For a profile picture, probably yes.
For a bank balance, almost certainly not.
And that distinction leads us to the two primary consistency models used in distributed systems:
Strong Consistency and Eventual Consistency.
Although both models attempt to solve the same problem, they make very different trade-offs between correctness, availability, latency, and user experience.
Understanding Strong Consistency
Imagine you withdraw money from an ATM.
Your account balance is $2,500.
You withdraw $500.
The transaction completes successfully.
A few seconds later, you check your balance using your banking application's mobile app.
How much money should it display?
There is really only one acceptable answer.
It should display $2,000.
Seeing the old balance—even for a few seconds—would immediately reduce your trust in the banking system.
You would naturally wonder:
"Did the withdrawal actually happen?"
This is precisely the guarantee that strong consistency provides.
Once a write operation has been acknowledged as successful, every subsequent read must return that latest value, regardless of which server processes the request.
There is only one version of the truth.
Every client sees it immediately.
From the user's perspective, it feels almost as though there is only a single database, even though the data may actually be replicated across multiple machines.
How Strong Consistency Is Achieved
Achieving strong consistency sounds simple in theory.
Whenever data changes, ensure that every replica receives the update before confirming success to the client.
However, implementing this guarantee in a distributed environment is considerably more challenging.
Consider a distributed database with one primary node and three replicas.
Notice the order of events.
The client does not immediately receive a success response after writing to the primary database.
Instead, the primary waits until the replicas acknowledge the update.
Only after sufficient replicas confirm the change does the system inform the client that the operation has completed successfully.
This waiting period is what allows every future read to return the same value.
The consistency guarantee comes from coordination.
And coordination always has a cost.
Why Strong Consistency Increases Latency
One of the recurring themes throughout this System Design series has been that every architectural decision involves trade-offs.
Strong consistency is no exception.
Imagine a database replicated across three continents.
A customer in India updates their shipping address.
The primary database accepts the request.
Before confirming success, however, it must replicate that update to servers located in Europe and North America.
Although data travels incredibly fast across fibre-optic networks, it is still constrained by the laws of physics.
Information cannot travel instantaneously.
Every additional replica introduces communication delays.
Every acknowledgement requires another network round trip.
The client remains waiting until the coordination process finishes.
As a result, the response time increases.
In other words, stronger consistency often comes at the cost of higher latency.
This is one of the most important trade-offs in distributed systems.
Applications demanding immediate agreement frequently sacrifice speed to achieve correctness.
The Cost of Waiting
Now imagine that one of the replicas becomes temporarily unreachable.
Perhaps a network cable has been damaged.
Perhaps a data centre is experiencing maintenance.
Perhaps temporary congestion has increased network delays.
The primary database now faces a difficult decision.
Should it continue waiting?
Should it reject the write request?
Or should it proceed without updating every replica?
If the system chooses to wait, users experience delays.
If it chooses to reject requests, availability decreases.
If it proceeds anyway, consistency is no longer guaranteed.
Notice how this dilemma resembles the trade-offs we discussed in the article on CAP Theorem.
Maintaining immediate consistency during failures often requires sacrificing availability.
This is one reason why strong consistency is relatively expensive to maintain in globally distributed systems.
Strong Consistency in the Real World
Despite these challenges, many applications simply cannot tolerate inconsistent data.
Imagine purchasing the last available ticket for an international flight.
If two customers simultaneously purchase what appears to be the final seat because different replicas disagree, the airline now has a serious operational problem.
The same issue appears in hotel reservations.
Suppose only one room remains available.
Two booking requests arrive at different replicas before synchronisation completes.
Without strong consistency, both customers may successfully reserve the same room.
Financial systems provide another obvious example.
If your account balance differs depending on which database replica processes your request, users quickly lose confidence in the system.
Healthcare systems also depend heavily on strong consistency.
Imagine two doctors viewing different versions of a patient's allergy information because replicas have not yet synchronised.
The consequences could be severe.
Inventory management systems face similar challenges.
When only one product remains in stock, every customer must observe the same inventory count.
Otherwise, multiple customers may purchase an item that no longer exists.
In each of these scenarios, correctness is significantly more important than minimising response time.
Users are generally willing to wait an additional few hundred milliseconds if it guarantees that the information they receive is accurate.
Consensus: Getting Multiple Servers to Agree
Strong consistency introduces another fascinating challenge.
How do multiple independent servers actually agree on the latest value?
Suppose four replicas receive slightly different information because of temporary network delays.
Which version becomes the correct one?
Distributed databases solve this using consensus algorithms.
Although the underlying algorithms are mathematically sophisticated, the central idea is surprisingly intuitive.
Before accepting certain operations, multiple servers participate in a coordinated decision-making process.
Rather than allowing individual replicas to decide independently, the cluster reaches agreement collectively.
This process ensures that every server eventually shares the same understanding of the system's state.
Several well-known distributed databases rely on consensus algorithms such as Raft or Paxos to provide strong consistency guarantees.
While these algorithms are beyond the scope of this article, it is important to recognise that strong consistency is not achieved through replication alone.
It requires coordination, agreement, and carefully designed protocols capable of handling failures.
Strong Consistency Is About Trust
Perhaps the easiest way to understand strong consistency is to think about trust.
Whenever users interact with a banking application, stock trading platform, payment gateway, airline reservation system, or healthcare database, they expect the system to provide one definitive answer.
There should never be uncertainty regarding the latest account balance, available seat, completed payment, or medical record.
Strong consistency provides that confidence.
Every successful write becomes immediately visible to every subsequent read.
The system behaves as though there were only a single, perfectly synchronised source of truth.
Achieving this guarantee is technically demanding.
It increases latency.
It requires coordination.
It becomes more difficult during network failures.
Yet for many applications, those costs are entirely justified because the business value of correctness far outweighs the additional complexity.
However, not every application requires this level of precision.
Many systems prioritise responsiveness, scalability, and availability over immediate agreement.
Instead of forcing every server to synchronise instantly, they allow updates to propagate gradually throughout the system.
This alternative approach is known as Eventual Consistency, and it has become one of the defining characteristics of many modern internet-scale applications.
Understanding Eventual Consistency
Imagine updating your profile picture on a social media platform.
You upload a new image.
Within a second, you refresh your own profile and see the updated picture.
A friend living on another continent opens your profile almost immediately afterwards.
Surprisingly, they still see the old picture.
A few moments later, they refresh again.
Now the new image appears.
Nothing was broken.
The system simply required time to distribute the update across its globally distributed infrastructure.
This is the essence of eventual consistency.
Unlike strong consistency, eventual consistency does not require every replica to agree immediately.
Instead, it guarantees something slightly different.
If no new updates occur, all replicas will eventually converge to the same value.
The word eventually is doing all the work here.
The system accepts that different users may temporarily observe different versions of the same data.
However, given enough time, every replica will synchronise and reach the same final state.
Why Eventual Consistency Exists
At first, allowing inconsistent data sounds like a terrible idea.
After all, haven't we spent the entire article discussing why consistency matters?
The answer depends entirely on the type of data involved.
Consider the homepage of a video streaming platform.
Suppose a movie's thumbnail changes.
If some users continue seeing the previous thumbnail for five seconds while others see the updated version immediately, has the application failed?
Probably not.
Now imagine an online news website.
An article receives a new featured image.
Readers in Europe see the change immediately.
Readers in Asia see the previous image for another ten seconds.
Again, the user experience remains almost unaffected.
The cost of delaying every request until every server agrees would likely outweigh the temporary inconsistency.
In situations like these, eventual consistency becomes an extremely practical trade-off.
Rather than waiting for worldwide synchronisation, applications prioritise speed and availability.
Users receive responses almost instantly, while the system quietly propagates updates in the background.
How Eventual Consistency Works
Let's revisit the replicated database architecture from the previous section.
This time, however, the write process follows a different sequence.
Instead of waiting for every replica to acknowledge the update, the primary database accepts the write and immediately responds to the client.
Replication happens afterwards.
Notice the difference from strong consistency.
The user receives confirmation almost immediately.
The application does not pause while waiting for every replica to synchronise.
Instead, replication continues in the background.
This significantly improves response time.
However, it also creates a short period during which different replicas may contain different versions of the data.
That temporary disagreement is an intentional design decision.
Living With Temporary Inconsistency
To understand eventual consistency, it helps to stop thinking of inconsistency as an error.
Instead, think of it as a transition period.
Suppose an e-commerce company updates the description of a product.
Some users immediately receive the updated description.
Others continue seeing the previous version for a short time.
Eventually, every replica contains the new information.
The inconsistency existed only while the update was propagating.
The same idea appears in many cloud services.
When a DNS record changes, internet providers around the world do not update simultaneously.
Some DNS servers refresh their records immediately.
Others continue serving cached information until their cache expires.
Eventually, every DNS server reflects the new record.
The internet itself relies heavily on eventual consistency.
Eventual Consistency in Everyday Applications
One of the reasons eventual consistency is so widely adopted is that users rarely notice it.
Consider a messaging application.
You send a message to a group chat.
Some participants receive it almost instantly.
Others experience a slight delay because of network conditions.
Despite these temporary differences, everyone eventually receives the same message.
Similarly, when you like a social media post, the like count may not increase simultaneously for every user viewing that page.
One person may see 1,024 likes while another briefly sees 1,023 likes.
A few seconds later, both users observe the same value.
Streaming platforms exhibit similar behaviour.
Suppose a creator uploads a new video.
The video may appear immediately in one region while taking a little longer to become available elsewhere.
This delay is often caused by content propagation across geographically distributed servers.
Again, eventual consistency allows the platform to scale globally without forcing every server to synchronise before serving users.
These examples illustrate an important principle.
Not every piece of information requires perfect synchronisation.
For many user experiences, slight delays are practically invisible.
Why Eventual Consistency Improves Availability
Another significant advantage of eventual consistency appears during failures.
Imagine that one replica temporarily loses network connectivity.
In a strongly consistent system, writes may need to pause until coordination is restored.
An eventually consistent system often behaves differently.
The primary database continues accepting updates.
Healthy replicas continue receiving changes.
The unavailable replica simply catches up later after connectivity returns.
This ability to continue operating despite temporary failures makes eventual consistency particularly attractive for globally distributed applications.
Rather than sacrificing availability whenever communication becomes difficult, the system tolerates temporary divergence and resolves it later.
This design aligns closely with the trade-offs discussed in our article on the CAP Theorem, where distributed systems frequently prioritise availability over immediate consistency during network partitions.
Eventual Does Not Mean Random
One misconception beginners often have is that eventual consistency means replicas simply update whenever they feel like it.
That is not how distributed systems work.
Modern databases use carefully designed replication mechanisms, version tracking, timestamps, conflict resolution strategies, and synchronisation protocols to ensure replicas eventually converge toward the same state.
The delay may vary depending on network conditions, infrastructure, or workload, but the synchronisation process is deliberate and carefully managed.
Without these mechanisms, replicas could permanently diverge, creating conflicting versions of the same data.
Eventual consistency accepts temporary disagreement.
It does not accept permanent disagreement.
That distinction is crucial.
Eventual Consistency Is About Practicality
Perhaps the easiest way to understand eventual consistency is to think about priorities.
Instead of asking,
"Can every server agree immediately?"
the system asks,
"Is immediate agreement actually necessary?"
For profile pictures, product descriptions, recommendation lists, analytics dashboards, news feeds, and countless other types of information, the answer is often no.
Users generally prefer receiving information immediately—even if it is a few seconds behind—rather than waiting longer for perfect synchronisation.
This practical mindset is one of the reasons eventual consistency powers so many of today's largest distributed platforms.
It acknowledges an important reality of globally distributed systems:
Sometimes delivering information quickly is more valuable than delivering perfectly synchronised information.
A Side-by-Side Comparison
One of the easiest ways to understand these consistency models is to compare how they behave when the same event occurs.
Imagine a user updates their profile information.
Under Strong Consistency
The system waits until the update has been synchronised according to its consistency requirements before acknowledging success.
Every subsequent request immediately observes the updated information.
There is never any ambiguity regarding which version is correct.
Users receive one consistent answer regardless of which server processes the request.
Under Eventual Consistency
The system acknowledges the update almost immediately.
Replication continues in the background.
For a short period, different users may observe different versions of the data depending on which replica serves their request.
Eventually, every replica converges to the same state.
The difference is not whether synchronisation happens.
The difference is when synchronisation becomes visible to users.
Visualising the Difference
The following diagram illustrates the behaviour of both consistency models after a write operation.
Notice that both systems ultimately reach the same final state.
The distinction lies in whether the client waits for synchronisation or whether synchronisation continues after the response has already been returned.
Choosing the Right Model Depends on the Business Problem
A common mistake among beginners is assuming that every application should strive for strong consistency because it sounds safer.
In reality, insisting on strong consistency everywhere often leads to unnecessary complexity and reduced performance.
Imagine an online retail platform.
When a customer updates their shipping address, seeing the previous address for two or three seconds is unlikely to create significant problems.
However, when that same customer submits a payment, displaying an outdated account balance would be completely unacceptable.
The application therefore treats different pieces of information differently.
Profile information may tolerate eventual consistency.
Financial transactions generally require strong consistency.
The same application may therefore use multiple consistency models simultaneously.
This idea surprises many engineers when they first encounter distributed systems.
Consistency is rarely an application-wide decision.
It is usually made at the level of individual business operations.
Real-World Examples
Large technology companies rarely commit themselves exclusively to one consistency model.
Instead, they carefully choose the appropriate model for each type of data.
Consider a typical social media platform.
When you upload a new profile picture, the update propagates across content delivery networks and replicated databases around the world.
Some users may briefly continue seeing the previous image.
This temporary inconsistency has almost no business impact.
The platform therefore favours eventual consistency because it delivers excellent scalability and responsiveness.
Now consider the payment system used by that same company.
Suppose a user purchases a premium subscription.
The payment must only be processed once.
The account balance must remain accurate.
Subscription status must not fluctuate between different servers.
In this scenario, strong consistency becomes essential.
The same organisation therefore employs both models—simply for different purposes.
Can a Single Application Use Both?
Absolutely.
In fact, this is exactly how many modern distributed systems are designed.
Consider an e-commerce platform.
Notice how different business capabilities have different requirements.
The payment service cannot afford inconsistent account balances.
The inventory service must accurately determine whether products remain available.
On the other hand, recommendation systems, analytics dashboards, search indexes, and notification services can safely process information asynchronously.
Attempting to force every component to use strong consistency would increase latency and reduce scalability without providing meaningful business value.
Conversely, making payment processing eventually consistent could result in duplicate transactions or oversold inventory.
Good system design therefore involves identifying which operations genuinely require immediate correctness and which can tolerate temporary inconsistency.
Strong Consistency vs Eventual Consistency
The following table summarises the primary differences between the two models.
| Characteristic | Strong Consistency | Eventual Consistency |
|---|---|---|
| Data visibility | Latest data is always returned | Temporary stale reads are possible |
| Read behavior | Every client observes the same value | Different replicas may briefly return different values |
| Latency | Typically higher because of coordination | Usually lower because responses are returned immediately |
| Availability during failures | May decrease while replicas coordinate | Generally higher because updates continue despite temporary failures |
| Scalability | More difficult at global scale | Easier to scale across regions |
| Typical applications | Banking, payments, reservations, inventory | Social media, analytics, recommendations, DNS, caching |
Rather than viewing these differences as advantages or disadvantages, think of them as design trade-offs.
Every distributed system decides where it wants to position itself along this spectrum.
Consistency Is a Business Decision
One of the most important lessons for system designers is that consistency is not merely a technical concern.
It is ultimately a business decision.
Technology provides the available options.
Business requirements determine which option is appropriate.
A stock trading platform cannot allow customers to observe outdated portfolio values.
An airline cannot sell the same seat to multiple passengers.
A hospital cannot display different versions of a patient's medical history.
These applications willingly accept additional complexity to guarantee correctness.
Conversely, a social media platform benefits far more from responsiveness and availability than from forcing every profile update to synchronise globally before users can continue browsing.
The acceptable level of inconsistency depends entirely on the consequences of stale data.
Understanding those consequences is often more important than understanding the underlying algorithms.
Final Thoughts
Throughout this System Design series, one idea has consistently appeared across every topic we have explored.
No architecture is universally optimal.
Monolithic systems trade flexibility for simplicity.
Distributed systems trade simplicity for scalability.
Horizontal scaling trades hardware limitations for operational complexity.
Microservices trade organisational independence for increased communication overhead.
Consistency models follow the same pattern.
Strong consistency prioritises correctness, even if that means accepting additional latency and coordination.
Eventual consistency prioritises scalability, responsiveness, and availability, while accepting that replicas may temporarily disagree.
Neither philosophy is inherently better.
They simply optimise for different priorities.
As you continue studying distributed systems, you will notice that nearly every architectural decision revolves around balancing competing trade-offs rather than maximising a single metric.
The best engineers are not those who always choose the most advanced technology.
They are the ones who understand the strengths and limitations of every approach and select the one that best serves the problem they are trying to solve.
That, ultimately, is the essence of system design.
Top comments (0)