Most candidates say "use consistent hashing" in a system design interview and stop there.
That's not enough anymore.
I made a video breaking this down from first principles — the hash ring, virtual nodes, and a nuance most tutorials skip entirely: virtual nodes fix data balance, not traffic balance. Here's the short version, if you want the TL;DR before watching.
The problem with naive hashing
The obvious way to distribute keys across servers is hash(key) % N. Simple — until N changes.
Add or remove one server, and the modulo result flips for almost every key. In practice, adding a single server to a naively-hashed cluster can remap ~75% of your keys. That's a cache stampede waiting to happen: every remapped key is a cache miss, and your database eats the traffic your cache was supposed to absorb.
The fix: a hash ring
Instead of a line, imagine the hash space as a circle — 0 to 2³²−1, wrapping back to 0.
Both servers and keys get hashed onto this same ring. A key belongs to the first server it hits walking clockwise. That one change means:
Adding a server only remaps the small arc of keys between it and its predecessor — not the whole ring.
Removing a server only shifts its keys to the next server clockwise — everything else is untouched.
Roughly K/N keys move per change, not the whole dataset.
The catch: uneven load
With only a handful of servers, their positions on the ring are essentially random — which means they can cluster unevenly. One server can end up owning a huge arc while others sit nearly idle.
Fix: virtual nodes. Hash each physical server onto the ring 100-200 times (ServerA-1, ServerA-2, ...) instead of once. More points → the arcs even out via the law of large numbers → under 5% load variance between machines in practice.
Most candidates stop here. That's a mistake.
The part most people miss: hot keys
Virtual nodes fix structural imbalance — every server holds roughly the same number of keys. They do nothing for traffic imbalance.
If one specific key goes viral — a celebrity profile, a trending product — every read for that key still routes to the same server, no matter how many virtual nodes you have. Two common mitigations:
Read replicas — replicate the hot key, load-balance reads across copies.
Key salting — append a random suffix (key-0 through key-9) to spread it across multiple ring positions, then aggregate on read.
If you can articulate the difference between data balance and traffic balance in an interview, you're already ahead of most candidates.
Where this actually runs
Not theoretical — this is running in production systems you already use:
DynamoDB — partition key routing
Cassandra — token-ring based partitioning
Redis Cluster — hash slots
CDNs (Akamai) — routing to the nearest edge node
Riak — distributed KV partitioning
Load balancers — sticky sessions without a central session map
Full video
The video covers all of this with visual, ring-based diagrams for each step (adding/removing nodes, virtual nodes, hot keys) — worth a watch if you want the full walkthrough rather than the text summary.
Curious what other system design topics people want broken down like this — drop a comment below.
Top comments (0)