Sharding, Explained: How to Talk About It in a System Design Interview

Sharding is one of those topics that shows up in almost every system design interview once scale enters the conversation. Yet many candidates reach for it reflexively, without justifying whether it's actually needed, and without a clear framework for explaining how they'd do it. This article breaks down why sharding exists, how to choose a shard key, the main distribution strategies, the challenges sharding introduces, and how to bring all of this together in an interview setting.

Why Sharding Exists

Imagine you've just launched an app backed by a single large database — say an AWS RDS Postgres instance with around 70 terabytes of storage and the capacity to handle roughly 10,000 writes per second. In the early days, this is more than enough. Traffic grows, but the single database keeps up comfortably.

Eventually, though, growth catches up with you. Maybe you now need 20,000 writes per second, or your storage is creeping toward that 70 TB ceiling. Queries slow down, backups take forever, and it's clear you've hit a limit.

The first instinct is usually vertical scaling — move to bigger hardware. This buys you time. AWS offers machines that can handle upwards of 140 terabytes of storage and around 50,000 writes per second, and most companies will never even approach that ceiling. But if your app keeps growing — more users, global traffic, sustained load — even the biggest single machine eventually saturates its CPU, storage, and I/O.

At that point, no amount of vertical scaling solves the problem. This is when you reach for sharding.

What Sharding Actually Is

Sharding means splitting your data across multiple independent databases so that no single machine holds the entire dataset. Each shard is a standalone database with its own CPU, memory, storage, and connection pool, holding just a portion of the overall data. Need more capacity? Add another shard.

This solves the scaling problem, but it introduces a new set of questions: How do you decide how to split the data? How do you know which shard to query? What happens when one shard gets far more traffic than the others? What happens if a shard goes down and data needs to be rebalanced? Sharding trades a scaling problem for an operational complexity problem.

Choosing a Shard Key

The first decision in any sharding strategy is the shard key — the field used to determine how data is grouped and distributed. In an interview, this should be one of the first things you state explicitly, along with your reasoning.

A strong shard key has three properties:

High cardinality — many unique values, so data spreads across shards rather than clustering.
Even distribution — values that naturally spread out, so no shard ends up disproportionately large.
Query alignment — the key should match how the data is actually queried.

For example, if the dominant access pattern is "fetch all posts for a given user," sharding by user ID means each user's data lives on one shard, so most queries only need to hit a single database.

Good shard key examples:

A social media app where the core operation is loading a user's profile and posts → shard by user ID.
An e-commerce platform where the core operation is retrieving or updating a single order → shard by order ID.

Poor shard key examples:

A boolean field like "is premium user" — only two possible values, which caps you at two shards.
Creation date, in an app where most queries target recent data — this concentrates nearly all traffic on the newest shard, creating a hotspot.

How to Distribute Data Across Shards

Once you have a shard key, you need a strategy for mapping its values to specific shards.

Range-based sharding splits the key into contiguous ranges (e.g., user IDs 0–10M go to shard one, 10M–20M to shard two, and so on). It's simple and intuitive, but tends to produce uneven load — early shards may sit empty while the current range absorbs all new traffic, especially if IDs increase monotonically. It's a reasonable starting point but rarely used in production at scale.

Hash-based sharding is the industry default. You hash the shard key and mod the result by the number of shards to decide placement, which produces solid, even distribution. The catch is rebalancing: adding or removing a shard changes the modulus, which forces nearly all existing data to move. Consistent hashing solves this by placing both keys and shards on a virtual ring — you hash to a point on the ring and walk it to find the right shard, with refinements like virtual nodes to smooth out distribution further. This avoids the mass-reshuffling problem entirely. In interviews, especially at senior levels, hash-based sharding with consistent hashing is generally the expected default; at junior or mid-levels, you may need to explain the mechanics rather than simply naming it.

Directory-based sharding uses a lookup table that maps each record to a specific shard, rather than a formula. This gives you flexibility — you can move an overloaded user to their own dedicated shard just by updating the mapping. The tradeoff is an extra lookup on every request (added latency, two round-trips instead of one) and the directory itself becomes a single point of failure. It's rarely the default answer in interviews but is useful to mention as an option when flexibility matters more than raw throughput.

The practical takeaway: default to hash-based sharding with consistent hashing, and bring up the alternatives only when the scenario calls for them.

The Core Challenges Sharding Introduces

Hotspots and Load Imbalance

Even a well-chosen shard key can produce uneven load in practice — the classic "celebrity problem." If you shard by user ID and one extremely popular user's data lands on a single shard, that shard can get flooded with traffic while others sit idle. Two common fixes:

Compound shard keys, which append extra data (like a number or time bucket) to the key before hashing, spreading one entity's data across multiple shards.
Dedicated shards for outliers, where high-traffic entities are detected and routed via a directory lookup to a special, higher-capacity shard, while everyone else follows the standard hashing scheme.

Most systems never need this, but platforms with extreme traffic outliers — large social networks, for instance — often do.

Cross-Shard Operations

Some queries need data from more than one shard, requiring a fan-out to multiple databases and then aggregation of the results — far more expensive than a single-shard query. This usually happens when a query doesn't align with the shard key. Fetching one user's profile is cheap; fetching "the ten most popular posts across the entire platform" requires querying every shard and combining the results.

You can't eliminate cross-shard queries completely, but you can minimize their cost:

Cache expensive cross-shard results (in something like Redis) with an expiration, trading some staleness for speed — useful for feeds, leaderboards, and trending pages.
Denormalize data so related information lives on the same shard, reducing the need for cross-shard reads at the cost of more complex writes.

If cross-shard queries keep showing up for common use cases, that's usually a sign the shard key needs rethinking, or that caching and denormalization should be applied more aggressively.

Consistency Across Shards

On a single database, a transaction like transferring money between two accounts is atomic by default — both updates succeed, or neither does. Once those two accounts live on different shards, that guarantee breaks down, and a partial failure can leave the system in an inconsistent state.

The textbook fix is two-phase commit (2PC), where a coordinator asks every involved shard if it's ready, waits for agreement, and then tells them all to commit. In practice, 2PC is slow and fragile — if a shard or the coordinator fails mid-process, the system can get stuck in a hard-to-resolve locked state, which is why most production systems avoid it.

Better alternatives:

Avoid cross-shard transactions where possible, by keeping transactionally related data on the same shard.
Use the saga pattern, breaking a transaction into a sequence of smaller steps, each with a compensating action to undo it if a later step fails — rather than relying on an atomic rollback.

Bringing It Together in an Interview

Sharding usually comes up during the deep-dive portion of a system design interview, when you're addressing a non-functional requirement around scale. Before proposing it, do the math on storage, write throughput, and read throughput to justify whether sharding is actually necessary.

For example: 500 million users at 5 KB each is only about 2.5 TB — well within the range a single Postgres instance can handle comfortably. But 50,000 writes per second at peak would strain a single database, which is a legitimate reason to shard. High read volume from a large active user base, even with read replicas in place, can also justify distributing load.

This distinction matters. Modern single-instance hardware can go a long way — well past 100 TB of storage and tens of thousands of writes per second. Many candidates jump straight to sharding without checking whether it's warranted; showing the math for why sharding isn't necessary yet can be just as impressive as knowing how to implement it.

When sharding is justified, structure your answer around four steps:

Propose a shard key based on the dominant access pattern (e.g., shard by user ID because most queries are user-centric).
Choose a distribution strategy — typically hash-based sharding with consistent hashing, explained in more depth if you're earlier in your career.
Call out the tradeoffs, such as cross-shard queries becoming more expensive, and how you'd mitigate that with caching or precomputation.
Address future growth, such as starting with enough shards for headroom and noting that consistent hashing makes adding more shards later far less disruptive.

Delivered well, this shouldn't feel like a checklist — it should read as a natural progression from the numbers you've calculated to the design decisions those numbers demand.