Imagine, there are 4 database sharding strategies with 4 completely different outcomes. Pick the wrong one and your primary database melts on Black Friday.
Your Postgres orders table just hit 500 million rows. Queries that used to take 40ms are now crawling at 800ms. Vertical scaling is dead. You have to shard.
But before you pick a strategy, look at your actual workload.
80% of reads are customers checking their own order history from the last 30 days.
15% are analytics queries on date ranges.
Only 5% are new writes, around 400 requests per second, doubling on sale days.
So which sharding strategy do you choose?
A) Hash sharding on order_id
B) Range sharding on created_at
C) Directory based sharding
D) Consistent hashing with virtual nodes
What would you pick? Drop your answer below. 👇
All four are real production strategies. But only one fits this specific e-commerce pattern. One of these is the senior engineer trap that looks smart in tutorials but will destroy your latency in reality.
Let us break it down like we are explaining it to a friend over chai.
English Breakdown:
Option A, Hash Sharding on Order ID
This sounds perfect because it distributes data evenly. No hotspots. But here is the problem. When a customer logs in and asks for their last 30 days of orders, their data is scattered across every single shard. Your app has to query all shards and stitch results together. This is called scatter-gather and it is painfully slow for user-facing queries. This is the trap. YouTube loves it, but your users will hate it.
Option B, Range Sharding on Date
Great for analytics, terrible for sales. On Black Friday, every new order hits the current month shard while older shards sit idle. That one shard gets crushed under double traffic and crashes. The rest of your infrastructure watches helplessly. This is how you melt a primary database during peak revenue moments.
Option D, Consistent Hashing
Amazing for rebalancing when adding servers. Terrible for customer-centric queries. Same scatter-gather problem as Option A. Your customers do not care about your elegant ring topology. They care about seeing their order history instantly.
Option C, Directory Based Sharding
This is your winner. Think of it like a phone directory. You maintain a fast lookup service that maps each customer to their specific shard. When Customer X asks for their orders, you check the directory, go directly to Shard 5, and return results in milliseconds. No scattering. No gathering. Just direct access.
Yes, the directory itself can become a bottleneck. But you solve that by caching it in Redis with replication. High availability, low latency, perfect fit for 80% customer-centric reads.
The lesson here is simple. Do not pick a sharding strategy because it sounds sophisticated. Pick it because it matches your actual query patterns. Numbers tell stories. Listen to them.
If your team is debating a database migration right now, share this with them. Someone is about to make an expensive mistake.
Hinglish:
Step 1: Terms Ko Simple Bhasha Mein Samjho
Question mein likha hai: "80% reads... 15% reads... 5% writes..."
Yeh kya matlab hai?
Supermarket (Database)
Reads (Padhna/Dekhna):
Jab customer shelf par jaakar dekhta hai ki "Mere liye kya product available hai?" ya "Mera purana bill kya tha?" — Yeh Read operation hai.Writes (Likhna/Change karna):
Jab customer counter par jaakar naya saman khareedta hai aur cashier usse bill mein add karta hai — Yeh Write operation hai.RPS (Requests Per Second):
Har second kitne log server ko hit kar rahe hain.
400 RPS: Matlab har second 400 naye orders aa rahe hain. Sale ke din yeh double (800) ho jata hai.Sharding (Baantna):
Ab tumhara Supermarket (Database) itna bada ho gaya hai ki ek hi building mein saara maal rakhna mushkil ho raha hai.
Sharding ka matlab hai: Tum apne pure database ko chhote-chhote tukdon (Shards) mein baant do aur alag-alag servers par rakh do.
Real Question:
Tumhare paas 50 Crore (500M) orders hain.
Pehle jab orders kam the, toh search karna 40ms (milliseconds) leta tha.
Ab itna data hai ki search karte waqt database ko poora table scan karna padta hai, jo 800ms le raha hai. User ko lagta hai app hang ho gayi hai!
Tumhe ab decide karna hai ki in 50 Crore orders ko kaise baantein (Shard kaise karein) taaki user ko apna order history turant mile.
Workload Yaad Rakho:
80% Queries: "Customer X ke pichle 30 din ke orders dikhao." (Yeh customer-centric hai).
15% Queries: Analytics (Date range wale).
5% Writes: Naye orders.
Step 3: Options Ko Decode Karein (Why & Where)
Chaliye har option ko simple example se samajhte hain:
Option A: Hash Sharding on order_id
Kya hai? Computer ek mathematical formula (Hash Function) use karta hai order_id par. Jaise order_id = 101 shayad Shard 1 par jaye, aur order_id = 102 Shard 5 par.
Fayda: Data bilkul barabar (evenly) distribute ho jata hai. Koi ek shard overload nahi hota.
Nuksan (The Trap): Agar mujhe "Customer X" ke saare orders dhundhne hain, toh mujhe pata nahi ki uske orders kis-kis shard par bikhre hain. Mujhe saare shards ko query karna padega (Scatter-Gather problem). Yeh slow ho jayega.
Option B: Range Sharding on created_at (Date)
Kya hai? Data ko date ke hisaab se baanta hai.
Shard 1: Jan-Mar 2023 ke orders.
Shard 2: Apr-Jun 2023 ke orders.
Fayda: Agar analytics team puche "Pichle mahine kitne sale hue?", toh wo sirf ek specific shard ko query karegi. Very fast for analytics.
Nuksan (Hotspot Problem): Black Friday ya Sale ke din, naye orders sirf "Current Month" wale shard par aayenge. Baaki shards khali padenge. Current shard par load badh jayega aur wo "melt" (crash) ho sakta hai. Ise Hotspotting kehte hain.
Option C: Directory-based Sharding (Lookup Table)
Kya hai? Ek alag se "Phone Directory" (Lookup Service) hota hai.
Jab request aati hai "Customer X ke orders lao", pehle Directory se pucha jata hai: "Bhai, Customer X kis Shard par hai?"
Directory batata hai: "Wo Shard 3 par hai."
Phir direct Shard 3 se data fetch kiya jata hai.
Fayda: Yeh exactly wahi karta hai jo humein chahiye. Kyunki 80% queries Customer-Centric hain, hum directly usi shard par ja sakte hain jahan wo customer ka data hai. No need to search everywhere.
Nuksan: Directory service khud ek single point of failure ban sakti hai (agar wo down ho gayi toh koi kuch nahi dhundh payega), lekin ise cache karke manage kiya ja sakta hai.
Option D: Consistent Hashing with Virtual Nodes
Kya hai? Yeh Option A (Hashing) ka hi advanced version hai. Isme servers ko ring ki tarah arrange kiya jata hai taaki agar ek server add/remove ho, toh poora data reshuffle na karna pade.
Fayda: Scaling aur rebalancing bahut aasaan hai.
Nuksan: Yeh bhi hashing hi hai. Agar maine order_id par hash kiya, toh wahi Scatter-Gather problem aayegi. Customer ke saare orders alag-alag jagah bikhre honge.
Subscribe :)
Top comments (0)