DEV Community

Subham
Subham

Posted on

Dissecting Databases: CockroachDB Has Almost Every Postgres Feature - So Why Is It the Default Answer for Distributed SQL?

The Question Nobody Asks Out Loud

Open CockroachDB's docs and the first thing you notice is how familiar everything looks. It speaks the Postgres wire protocol. psql connects to it without complaint. It has JSONB, window functions, foreign keys, ACID transactions, standard SQL indexes. If you handed a backend engineer a CockroachDB shell without telling them, they might spend twenty minutes assuming it's Postgres with a slightly different EXPLAIN output.

And yet - ask anyone "what do I use for distributed SQL?" and CockroachDB (or something architecturally like it - Spanner, YugabyteDB, TiDB) is the answer, not "Postgres, but bigger."

That's the actual puzzle worth dissecting. It's not a features conversation. It's an architecture conversation, and the difference only shows up the moment something goes wrong - or the moment you ask "who is the writer for this row, right now, and what happens if that node dies mid-transaction."

Give Postgres Its Due First

It's tempting to write posts like this as a strawman - set Postgres up to look primitive, then knock it down. That's dishonest, and Postgres deserves better, because it can genuinely get you further than people assume:

  • Read replicas & logical replication - you can fan out read traffic across as many replicas as you want.
  • Citus and other sharding extensions - turn a single logical Postgres database into a sharded cluster, with the planner handling distributed queries.
  • Partitioning - split large tables by range or hash, keep index sizes sane, keep vacuum manageable.
  • Connection poolers like PgBouncer or PgCat - solve the connection-scaling problem that trips up most people's first encounter with "Postgres doesn't scale."

If your bottleneck is read throughput or table size, Postgres - properly operated - will take you much further than most people give it credit for. This is not the place CockroachDB wins.

Where It Actually Breaks: Failure, Not Load

The gap shows up not under load, but under failure. Two scenarios:

1. The single-writer problem.
No matter how you shard reads or partition tables, Postgres has exactly one primary accepting writes for any given piece of data. Citus shards data across nodes, but each shard still has one writer. If that node is slow, overloaded, or down, every write that touches that shard blocks - full stop.

2. Losing a node (or a datacenter).
When a Postgres primary dies, something has to promote a replica. That's not instantaneous - it takes seconds to minutes depending on your failover tooling, and if your replication was asynchronous, you can lose the last few transactions that hadn't shipped yet. You've traded correctness for speed, or speed for correctness - Postgres makes you choose.

Now stretch that across regions. Want a write to survive an entire AWS region going dark, with zero data loss and no manual promotion step? Postgres has no native answer to that question. You'd be building your own consensus layer on top of it - which is, incidentally, most of what CockroachDB is.

Dissecting the Machinery: How the Cluster Actually Holds Itself Together

This is the part most "distributed SQL" explainers wave their hands through. Let's not.

The cluster's nervous system: Gossip

Before any range or Raft group can do anything, every node needs to know two things: who else is alive, and where the data actually lives. CockroachDB solves this with a gossip protocol running continuously between nodes - not for every read/write (that would be far too slow), but for cluster-level metadata:

  • Node liveness (is this node's lease still valid, or has it expired and needs takeover)
  • Store capacity and load (used for rebalancing decisions)
  • The location of a small number of special ranges that bootstrap everything else

Every node maintains a local NodeLiveness record, refreshed via a heartbeat. If a node stops heartbeating within its liveness threshold (default ~9s), the cluster treats its leases as expired and other replicas can take over - this is what makes failover automatic rather than something an operator or a failover script triggers.

Finding data: the meta ranges

A table's data lives across potentially thousands of ranges. So how does a client's query find the right one without gossiping the location of every single range to every node? CockroachDB uses a two-level indirection, itself stored as regular ranges:

  • meta1 - points to the nodes holding meta2
  • meta2 - maps key ranges to the actual data ranges holding them

This is conceptually identical to a B-tree lookup, just distributed. Nodes cache these lookups aggressively (RangeDescriptorCache) so a hot key doesn't need to re-walk the meta ranges on every access.

Ranges, Raft, and the leaseholder - properly

Ranges. Every table's data is broken into contiguous chunks called ranges, roughly 512MB each by default (configurable). As a table grows, CockroachDB automatically splits it into more ranges, and can also split "hot" ranges based on load, not just size (load-based splitting) - a range taking disproportionate QPS gets split even if it's small.

Raft, per range. Each range is independently replicated via the Raft consensus protocol - copies spread across nodes and, ideally, across availability zones or regions via replication zone configs. Every range runs its own tiny consensus group. A write to range A and a write to range B are two entirely independent Raft decisions, happening on potentially different sets of nodes. This is the structural answer to Postgres's single-writer problem: there isn't one writer for the table, there's one leaseholder per range, and a large table has many leaseholders spread across the cluster.

The leaseholder. Within each range's Raft group, one replica holds the lease or if we want to relate a bit, it's just like a leader node - the right to serve reads and coordinate writes without a full consensus round-trip on every read. Two lease types matter here:

  • Expiration-based leases, which must be periodically renewed (used for most ranges).
  • Epoch-based leases, tied to the node's liveness record - the lease is valid as long as the node's liveness epoch hasn't changed, avoiding constant renewal overhead for the majority of ranges.

Writes still go through Raft - a write only commits once a majority of replicas in that range acknowledge it - but the leaseholder is what lets reads skip a full quorum round-trip.

Surviving a node loss. If a replica dies, its Raft group already has a majority among the remaining replicas - no manual promotion, no failover script, no window of unavailability beyond the time it takes Raft to elect a new leader for that range (typically sub-second). If an entire AZ or region goes dark, and replicas were placed across zones, the surviving replicas still hold quorum and the range keeps serving writes.

The write path: what actually happens when you INSERT

This is the part that's usually skipped, and it's where CockroachDB earns the "distributed transactions without a lock manager" claim:

  1. The transaction is assigned a provisional commit timestamp, drawn from the coordinating node's hybrid logical clock (HLC).
  2. Instead of taking locks, a write creates a write intent - a provisional MVCC value tagged with the transaction's ID, stored right where the final value would go. Any other transaction that encounters this intent knows a conflicting write is in-flight.
  3. One range is chosen to hold the transaction record, tracking whether the transaction is PENDING, COMMITTED, or ABORTED.
  4. On commit, CockroachDB uses parallel commits - instead of a two-phase commit that waits for every intent to be explicitly resolved before returning success, it fires all writes in parallel and only waits for all of them to be durably replicated via Raft, then atomically flips the transaction record to COMMITTED (technically "staged" then implicitly committed). This shaves a full round-trip off every multi-range transaction compared to classic 2PC.
  5. Intents are asynchronously resolved into final MVCC values afterward, off the critical path.

Handling clock skew: uncertainty intervals

Serializable isolation across independent physical clocks is dangerous if you assume clocks are perfectly synced - they aren't. CockroachDB doesn't assume synced clocks; it assumes bounded clock skew (via NTP, default max offset ~500ms) and handles the uncertainty explicitly. Every transaction carries an uncertainty interval - if a transaction reads a value whose timestamp falls within that window, CockroachDB can't be sure of true ordering, so it forces a restart with an adjusted timestamp rather than risk a stale or out-of-order read. This is the mechanism that lets HLC-based ordering be safe, not just fast.

Running a query across the cluster: DistSQL

A single SELECT ... JOIN ... GROUP BY might touch ranges on a dozen nodes. CockroachDB's DistSQL execution engine plans the query as a physical execution DAG, pushes filtering, projection, and even partial aggregation down to the nodes that actually hold the data, and streams intermediate results toward the node performing the final merge - rather than pulling every row back to one coordinator first. This is closer to how Spark or Presto execute a distributed query plan than to how a single-node Postgres executor works.

Keeping the cluster balanced

Node liveness and store capacity gossip feed a background replicate queue on every node, which continuously asks: is this range under-replicated, over-replicated, or sitting on an overloaded store? Ranges get rebalanced automatically - moved between stores, and leaseholders shifted to be closer to where the load actually is (follow-the-workload). None of this requires an operator to shard, resplit, or manually move data.

Postgres vs CockroachDB: The Architecture Diff, Scaling Lens

| Dimension | Postgres (+ Citus/replicas) | CockroachDB |
|---|---|---|
| Unit of replication | Whole instance (WAL streamed to replicas) | Per-range, independently (Raft group per range) |
| Write ownership | One primary per shard/instance | One leaseholder per range; thousands of ranges, thousands of leaseholders |
| Failover | Manual or tooling-driven promotion (seconds–minutes) | Automatic Raft leader re-election (sub-second) |
| Consistency on failover | Depends on sync vs async replication - can lose data | No data loss on minority node loss - majority already has it |
| Isolation default | Read Committed | Serializable, always |
| Ordering mechanism | Single-node transaction log (WAL LSN order) | Hybrid logical clocks + uncertainty intervals, cluster-wide |
| Cross-shard transactions | Hard - Citus has real limitations here | Native - any transaction can span ranges via parallel commits |
| Horizontal write scale | Requires manual sharding (Citus) | Native, automatic, via range splits |
| CAP leaning | Tends AP under async replication, CP-ish under sync | CP - unavailable on minority partition, never inconsistent |
| Clock dependency | None | Requires bounded clock skew (NTP); large skew forces transaction restarts |
| Operational model | One engineer can hold it in their head | Genuinely more moving parts: gossip, Raft, rebalancing, zone configs |
Enter fullscreen mode Exit fullscreen mode

The short version: Postgres distributes load. CockroachDB distributes ownership of the data itself. That's the actual architectural fork, and it's why bolting sharding onto Postgres never fully closes the gap - you can shard the data, but you still can't make a single shard survive its own primary dying without an external consensus mechanism, which is exactly what Raft-per-range gives CockroachDB for free.

The Trade-Off Nobody Puts on the Marketing Page

None of this is free, and a post that pretends otherwise isn't being honest with you:

  • Write latency. A write that needs Raft consensus across nodes - potentially across regions - is slower than a write that just needs to hit one machine's WAL and fsync. If your ranges span regions for survivability, you're paying speed-of-light round-trip costs on every write quorum.
  • Clock dependency. Postgres doesn't care if your server clocks drift. CockroachDB's correctness guarantees lean on bounded clock skew - badly configured NTP is a real, if rare, failure mode that has no Postgres equivalent.
  • Operational surface area. A well-tuned single Postgres instance (or a simple primary/replica pair) is something one engineer can hold in their head. A distributed cluster with range rebalancing, replication zones, and multi-region topology is a genuinely bigger operational commitment.
  • You probably don't need it yet. Most startups' actual bottleneck is engineering time, not distributed consensus. If you can survive on a well-indexed Postgres instance with a hot standby, that's usually the right call - CockroachDB is solving a problem you may not have.

So, Why Is It the Default Answer?

Not because of features - as established, Postgres has almost all of them. It's the default answer to "give me distributed SQL" because the unit of ownership itself is distributed: every range runs its own consensus group, has its own leaseholder, and survives node loss without an operator in the loop - and the whole cluster still agrees on a global transaction order without a central lock manager anywhere in the picture.

Postgres gets you 90% of the features. CockroachDB gets you the last 10% - the part that's actually hard to build, and the part that only matters the day something fails.

Top comments (0)