Why consistency and availability cannot coexist in distributed systems?

Problem

Have you ever pondered why consistency and availability cannot coexist in distributed systems, as the title suggests?

While it may be achievable in a single server mode, it could be considered a single point of failure, therefore making availability uncertain. In distributed systems, with multiple partitions, attaining both consistency and availability is not feasible.

CAP theorem

CAP theorem, also known as Brewer's theorem, is an essential concept in distributed computing that explains the trade-offs involved in building a distributed system. It was introduced by Eric Brewer in 2000, and it is still relevant today in the development of modern distributed systems.

The CAP theorem states that it is impossible to achieve all three of the following guarantees in a distributed system simultaneously:

Consistency: All nodes in the system see the same data at the same time.

Availability: Every request made to the system gets a response, without guarantee of the freshness of the data.

Partition tolerance: The system continues to function even when network partitions occur.

The theorem's fundamental idea is that a distributed system can provide at most two of these three guarantees at any given time. In other words, we have to make a trade-off between consistency, availability, and partition tolerance.

• Consistency means that all nodes in the distributed system see the same data at the same time. Achieving consistency is essential for many applications where data accuracy is critical. However, ensuring consistency can be challenging in a distributed system, especially when multiple nodes try to update the same data simultaneously.

C in CAP is different than the C in ACID. CAP-C means that data is consistent across all the nodes.ACID-C denotes the data correctness at the database level. However both denotes consistency.

• Availability means that every request made to the system gets a response, but it doesn't guarantee the freshness of the data. If a node cannot respond to a request, the system is considered unavailable. In some cases, it may be acceptable to serve stale data, but in others, it may not be.

• Partition tolerance refers to the ability of a distributed system to continue functioning when communication between nodes is disrupted. A network partition occurs when a group of nodes becomes disconnected from the rest of the system due to a network failure or other issues. Partition tolerance is essential for the system to continue operating in such situations.

Use-cases

The CAP theorem tells us that in a distributed system, we have to choose between consistency and availability while ensuring partition tolerance. In reality, different applications have different requirements and priorities, and there is no one-size-fits-all solution. The choice of which two guarantees to provide depends on the application's specific needs, the scale of the system, and the potential impact of network partitions.

For example, a system that handles financial transactions must prioritize consistency to ensure that all transactions are accurate and complete. On the other hand, a social media platform can prioritize availability to ensure that users can access the platform even during network disruptions, even if it means serving stale data.

Advancing solutions

1. Eventual consistency
It is nothing but a version of consistency model used in distributed systems, where all updates made to a data store will eventually be propagated to all replicas or nodes in the system, resulting in a consistent state over time. It is a weak form of consistency that allows for temporary inconsistencies or conflicts in the data until all updates are fully propagated across all nodes.

2. Consensus algorithms
A consensus algorithm is a distributed computing protocol that enables a group of nodes or processes to agree on a single value or decision, even if some of the nodes fail or behave maliciously. It is an algorithm to achieve consistent data across all nodes, in fact, they're backbone for the blockchain applications as well.

3. Hybrid architecture
For database design, with the idea of CAP theorem, the architect can make informed decisions such that, the strong consistency can be applied to critical data, and would give weak consistency to non-critical data.

This problem exists for around 2 decades, yet there are no ideal solutions to fix this completely. Let me know if there are new approaches that is able to solve this problem.

Conclusion

The CAP theorem is a critical concept in distributed systems that highlights the trade-offs involved in building such systems. While we cannot have all three guarantees of consistency, availability, and partition tolerance simultaneously, understanding the theorem's implications can help us make informed decisions when building distributed systems.