The problem
Every distributed systems interview eventually gets to CAP theorem: during a network partition, you can have consistency or availability, not both. Engineers love quoting this. It sounds rigorous, it name-drops a real theorem, and it explains an outage story well.
Here's what it doesn't explain: why your "strongly consistent" database is slower than your "eventually consistent" one on a totally healthy Tuesday afternoon, with zero partitions in sight.
CAP only makes a claim about the P — the rare window when the network is actually split. For a well-run multi-region system, that might be minutes per year. The other 99.99% of the time, you're still making a consistency trade-off, on every single request, and CAP has nothing to say about it. The theorem that does is PACELC, and almost nobody outside distributed-systems papers has heard of it.
Why it happens
PACELC, coined by Daniel Abadi in 2010, says: if Partitioned, choose Availability or Consistency (that's CAP); Else, choose Latency or Consistency.
That "Else" is the part that actually governs your day-to-day production behavior. Strong consistency isn't free even when the network is fine — it costs coordination, and coordination costs round trips.
Take a quorum-based system with N=3 replicas, W=2, R=2 (Dynamo-style, or Cassandra with QUORUM consistency). To satisfy a strongly-consistent read after a write, you need overlapping quorums, which means a write has to get acknowledged by 2 of 3 nodes before it returns, and a read has to check 2 of 3 nodes and reconcile versions before it returns. If those replicas are same-AZ, that's maybe 1-2ms of extra round-trip. If they're cross-region — which is exactly when you'd want the durability guarantee most — each of those round trips is 40-150ms depending on the region pair.
Compare that to reading from the nearest single replica with no quorum check: sub-5ms, no coordination, no waiting on the slowest of N acks. Same data. Same hardware. The only difference is how much agreement you demanded before answering, and that demand is a tax you pay on every request, forever, not just during a partition.
This is why Spanner (optimizes for the C, pays the L, uses TrueTime and atomic clocks to shrink the tax as much as physics allows), DynamoDB in eventual-read mode (optimizes for L, accepts stale reads), and Cassandra with tunable consistency (lets you dial the trade-off per query) all make genuinely different, defensible choices — they're not disagreeing about CAP, they're landing in different spots on the E-L-C axis, which is a decision you make thousands of times a day, not once during an incident.
What to do about it
Stop treating "consistency" as a single on/off setting picked once at database-selection time. Instead:
- Identify which specific reads actually need read-your-writes or linearizable guarantees (payment status, inventory decrement, leader election) versus which ones are fine stale by a few hundred milliseconds (a feed, a dashboard, a "last seen" timestamp).
- For the first group, pay the latency tax deliberately and measure it — know your P99 quorum round-trip, don't discover it in an incident.
- For the second group, actually use the weaker consistency mode your database offers. Most teams default every query to the strongest available mode "to be safe" and then wonder why a read-heavy service has a P99 three times higher than it needs.
- If you're using a system with tunable consistency (Cassandra, ScyllaDB, CockroachDB's follower reads), treat the consistency level as a per-query parameter, not a cluster-wide constant.
- When evaluating a new datastore, ask what it does on the "Else" branch specifically — vendors will happily tell you their partition behavior, but the everyday latency-consistency trade-off is usually buried three pages into the architecture docs.
Key takeaways
- CAP theorem describes rare-event behavior (during a partition); it says nothing about normal operation.
- PACELC's "Else" clause — latency vs. consistency with no partition present — governs your system's behavior essentially all the time.
- Strong consistency costs real, measurable round-trip latency because it requires coordination (quorums, consensus), not because of anything partition-related.
- Different databases making different CAP choices often aren't disagreeing about partitions at all — they're landing in different spots on the everyday latency/consistency trade-off.
- Treat consistency level as a per-query decision informed by what that specific read or write actually needs, not a single global setting.
Top comments (0)