DEV Community: Bhanu partap

Suppose it is late at night and you decide to order pizza from Zomato. While making the payment, Paytm shows an invalid balance because it is month end. From your perspective, you created the order, paid, and then it got cancelled......... read :)

Bhanu partap — Sat, 31 Oct 2026 06:55:05 +0000

Bhanu partap

Oct 31

Zomato - System Design

#zomato #systemdesign #sqs #onlineordersystemdesign

4 min read

Zomato - System Design

Bhanu partap — Sat, 31 Oct 2026 06:51:41 +0000

Suppose it is late at night and you decide to order pizza from Zomato. While making the payment, Paytm shows an invalid balance because it is month end.

From your perspective, you created the order, paid, and then it got cancelled.

But in Zomato’s backend, there is an order service connected to SQS that is still processing events.

SQS is a data structure that works like a queue. Standard SQS does not guarantee FIFO ordering. It maintains best-effort ordering and is much cheaper than FIFO queues.

So there are 5 workers processing messages ahead of yours.

Suppose the cancellation event reaches Chef1 first because standard SQS does not guarantee order. Chef1 sees the order was cancelled but finds nothing in the database. Then Chef2 receives the creation event and creates the order in the DB. Now you have a paid order that you never actually paid for. 😅

Then you ordered the same pizza again and enjoyed a movie. 📽️

Comment your favourite movie name below!

Meanwhile the finance team tells engineers to stop vibe coding and fix this mess. You have until Monday. What do you do?

A) Switch to SQS FIFO plus MessageGroupId per order_id and let AWS handle ordering per order.
B) Keep standard SQS, add a sequence number plus reorder buffer in the consumer, and hold out-of-order events until earlier ones arrive.
C) Replace the event stream with a Saga where each step waits for the previous step’s completion signal before firing the next.
D) Add an event version or timestamp and make the state machine idempotent plus order-agnostic by rejecting stale transitions and accepting any arrival order.

All four are patterns I have seen in real production systems. Only one actually fits this problem. One of them is the trap that catches senior engineers on the whiteboard.

Pick one, A, B, C, or D, and tell me why. Full breakdown in the comments.

Drop your answer 👇

Explanation

SQS Standard vs FIFO

Standard SQS is like a normal line where people can sometimes get slightly out of order. It gives best-effort ordering, meaning 99% of the time the sequence is correct but 1% of the time messages may arrive out of order. It is cheap and very fast.

FIFO SQS is a strict line. Whoever comes first gets served first. But it is slower and more expensive.

Consumers and Workers

These are background programs that pick messages from the queue and update the database. In this scenario, 5 workers are processing messages in parallel.

State Machine

This is a strict set of rules. For example, it says “I will only accept a Cancelled state if a Created state already exists.” This prevents data corruption.

Decoding the Options

Now let us fix this problem by looking at each option.

Option A: Switch to SQS FIFO plus MessageGroupId per order_id ❌ (The Senior Engineer Trap)

You replace the standard queue with a FIFO queue. MessageGroupId ensures all messages for one order_id stay in the same line.

AWS will guarantee that Created processes first, then Paid, then Cancelled.

But here is the trap. FIFO queues have much lower throughput than standard queues, around 3000 messages per second per group. If you have 2000 orders per minute with 3 messages each, the FIFO queue becomes a bottleneck. The system slows down. This is a band-aid solution, not a permanent fix.

Option B: Add a sequence number plus reorder buffer in the consumer ⚠️ (Too Complex)

You send a sequence number with every message and keep a buffer in the consumer. If Cancelled arrives first, you store it in the buffer and wait for Created to arrive.

But this is very hard to implement correctly. If the Created message never arrives due to a network issue, the Cancelled message stays stuck in the buffer forever causing memory leaks or deadlocks. Distributed systems avoid such complex buffers.

Option C: Replace the event stream with a Saga ❌ (Wrong Use Case)

Saga pattern is used when you have multiple microservices like Order Service, Payment Service, and Inventory Service and need to manage distributed transactions between them.

But here the problem is queue ordering, not distributed transactions. Saga is overkill for this situation.

Option D: Add event version or timestamp and make the state machine idempotent plus order-agnostic ✅ (The Winner)

You make your state machine smarter and more flexible.

Order-agnostic means the state machine does not care about event arrival order. If Cancelled arrives first, it accepts it and sets the order directly to Cancelled state even if Created comes later.

Idempotent means even if the same event arrives 10 times, the result stays the same.

You fixed the root cause which was a fragile state machine. Now you can use standard SQS for high throughput without worrying about out-of-order messages crashing the system.

The Correct Answer and Logic

Correct Answer: D) Add an event version or timestamp and make the state machine idempotent plus order-agnostic.

The logic is simple. There is a golden rule in distributed systems: Never trust the network. The network can delay messages or deliver them out of order at any time.

If your state machine is so strict that it rejects out-of-order messages, your system will always be fragile.

:))))))))))))))))) SUBSCRIBE :))))))))))))))))))))))))))

You went to bathroom.... and got crashed

Bhanu partap — Fri, 30 Oct 2026 05:33:08 +0000

Imagine you are in a bathroom with a 5 minute timer.

You go in but crash at minute 3. The lock is still on for 2 more minutes.

Your friend arrives at minute 3 and just stands outside waiting even though the bathroom is empty.

That wasted time is exactly what happens in software when a process dies but the Redis lock stays alive.

Now imagine three servers behind a load balancer. Every 5 minutes a cron job fires to generate a daily report and all three try to grab it. You use Redis SETNX with a 5 minute TTL. Instance A wins and thinks it owns the bathroom. But then it gets OOM killed at minute 3. The lock remains. No report runs for 2 more minutes. Then another instance grabs it. But wait, the first instance restarted and also thinks it still owns the bathroom. Now both are writing to the same output. Data corruption. 💥

This is the split brain problem. Redis cannot tell if a client is working or just frozen.

So what is the right pattern here?

A) Redis SETNX plus short TTL plus fencing token
B) Redlock across 5 independent Redis nodes
C) Database pessimistic lock using SELECT FOR UPDATE
D) Optimistic concurrency with unique key inserts

All four are real patterns engineers ship. But only one survives this exact failure mode. One of them is what Martin Kleppmann publicly tore apart 8 years ago and people still argue about it today.

Firstly, What would you pick? Drop your answer below. 👇

Let us break it down simply.

Option B, Redlock

This sounds bulletproof. Acquire locks on 5 separate Redis nodes and proceed only if majority agree. But Martin Kleppmann proved that if system clocks drift even slightly, two instances can get the lock simultaneously. It is complex, fragile, and usually over-engineering for most teams. This is the trap.

Option C, Database Pessimistic Lock

You lock the row in your jobs table until the transaction completes. If Instance B dies, the connection drops and the lock releases automatically. Safe? Yes. But heavy. Holding a database connection open for a long running report job is bad practice. Your database becomes the bottleneck instead of your compute.

Option D, Optimistic Concurrency

No locks at all. Each instance generates the full report and tries to insert it with a unique key. Database rejects duplicates. Sounds elegant until you realize both instances just wasted 10 minutes of compute generating a report that gets thrown away. For heavy jobs, this is pure waste.

Option A, Fencing Token

This is your winner. Think of it like a ticket number at a government office. Every time someone gets the lock, they receive a monotonically increasing number. Token 101, then 102, then 103.

When Instance B comes back from its GC pause with Token 101 and tries to write, the storage layer checks: “Current valid token is 103. You have 101. Your lock is stale. Rejected.”

Instance C with Token 103 writes safely. No corruption. No wasted compute. No clock dependency.

The fencing token makes the downstream resource the source of truth, not Redis. Redis just hands out tickets. The storage layer enforces order.

The lesson here is simple. Distributed locks are not about preventing two workers from starting. They are about preventing a zombie worker from writing after it should be dead. TTL alone cannot do that. You need a monotonic guarantee.

If your team has ever argued about Redlock versus SETNX at 3am, share this with them. The debate itself is the learning.

SystemDesign #DistributedSystems #Redis #BackendEngineering #shortTTL #longTTL #dailyupdates

Black Friday Sale: System Design

Bhanu partap — Thu, 29 Oct 2026 06:53:11 +0000

Imagine, there are 4 database sharding strategies with 4 completely different outcomes. Pick the wrong one and your primary database melts on Black Friday.

Your Postgres orders table just hit 500 million rows. Queries that used to take 40ms are now crawling at 800ms. Vertical scaling is dead. You have to shard.

But before you pick a strategy, look at your actual workload.

80% of reads are customers checking their own order history from the last 30 days.
15% are analytics queries on date ranges.
Only 5% are new writes, around 400 requests per second, doubling on sale days.

So which sharding strategy do you choose?

A) Hash sharding on order_id
B) Range sharding on created_at
C) Directory based sharding
D) Consistent hashing with virtual nodes

What would you pick? Drop your answer below. 👇

All four are real production strategies. But only one fits this specific e-commerce pattern. One of these is the senior engineer trap that looks smart in tutorials but will destroy your latency in reality.

Let us break it down like we are explaining it to a friend over chai.

English Breakdown:

Option A, Hash Sharding on Order ID

This sounds perfect because it distributes data evenly. No hotspots. But here is the problem. When a customer logs in and asks for their last 30 days of orders, their data is scattered across every single shard. Your app has to query all shards and stitch results together. This is called scatter-gather and it is painfully slow for user-facing queries. This is the trap. YouTube loves it, but your users will hate it.

Option B, Range Sharding on Date

Great for analytics, terrible for sales. On Black Friday, every new order hits the current month shard while older shards sit idle. That one shard gets crushed under double traffic and crashes. The rest of your infrastructure watches helplessly. This is how you melt a primary database during peak revenue moments.

Option D, Consistent Hashing

Amazing for rebalancing when adding servers. Terrible for customer-centric queries. Same scatter-gather problem as Option A. Your customers do not care about your elegant ring topology. They care about seeing their order history instantly.

Option C, Directory Based Sharding

This is your winner. Think of it like a phone directory. You maintain a fast lookup service that maps each customer to their specific shard. When Customer X asks for their orders, you check the directory, go directly to Shard 5, and return results in milliseconds. No scattering. No gathering. Just direct access.

Yes, the directory itself can become a bottleneck. But you solve that by caching it in Redis with replication. High availability, low latency, perfect fit for 80% customer-centric reads.

The lesson here is simple. Do not pick a sharding strategy because it sounds sophisticated. Pick it because it matches your actual query patterns. Numbers tell stories. Listen to them.

If your team is debating a database migration right now, share this with them. Someone is about to make an expensive mistake.

Hinglish:

Step 1: Terms Ko Simple Bhasha Mein Samjho
Question mein likha hai: "80% reads... 15% reads... 5% writes..."
Yeh kya matlab hai?

Supermarket (Database)

Reads (Padhna/Dekhna):
Jab customer shelf par jaakar dekhta hai ki "Mere liye kya product available hai?" ya "Mera purana bill kya tha?" — Yeh Read operation hai.
Writes (Likhna/Change karna):
Jab customer counter par jaakar naya saman khareedta hai aur cashier usse bill mein add karta hai — Yeh Write operation hai.
RPS (Requests Per Second):
Har second kitne log server ko hit kar rahe hain.
400 RPS: Matlab har second 400 naye orders aa rahe hain. Sale ke din yeh double (800) ho jata hai.
Sharding (Baantna):
Ab tumhara Supermarket (Database) itna bada ho gaya hai ki ek hi building mein saara maal rakhna mushkil ho raha hai.
Sharding ka matlab hai: Tum apne pure database ko chhote-chhote tukdon (Shards) mein baant do aur alag-alag servers par rakh do.

Real Question:
Tumhare paas 50 Crore (500M) orders hain.
Pehle jab orders kam the, toh search karna 40ms (milliseconds) leta tha.
Ab itna data hai ki search karte waqt database ko poora table scan karna padta hai, jo 800ms le raha hai. User ko lagta hai app hang ho gayi hai!

Tumhe ab decide karna hai ki in 50 Crore orders ko kaise baantein (Shard kaise karein) taaki user ko apna order history turant mile.
Workload Yaad Rakho:

80% Queries: "Customer X ke pichle 30 din ke orders dikhao." (Yeh customer-centric hai).
15% Queries: Analytics (Date range wale).
5% Writes: Naye orders.
Step 3: Options Ko Decode Karein (Why & Where)
Chaliye har option ko simple example se samajhte hain:

Option A: Hash Sharding on order_id
Kya hai? Computer ek mathematical formula (Hash Function) use karta hai order_id par. Jaise order_id = 101 shayad Shard 1 par jaye, aur order_id = 102 Shard 5 par.
Fayda: Data bilkul barabar (evenly) distribute ho jata hai. Koi ek shard overload nahi hota.
Nuksan (The Trap): Agar mujhe "Customer X" ke saare orders dhundhne hain, toh mujhe pata nahi ki uske orders kis-kis shard par bikhre hain. Mujhe saare shards ko query karna padega (Scatter-Gather problem). Yeh slow ho jayega.

Option B: Range Sharding on created_at (Date)
Kya hai? Data ko date ke hisaab se baanta hai.
Shard 1: Jan-Mar 2023 ke orders.
Shard 2: Apr-Jun 2023 ke orders.
Fayda: Agar analytics team puche "Pichle mahine kitne sale hue?", toh wo sirf ek specific shard ko query karegi. Very fast for analytics.
Nuksan (Hotspot Problem): Black Friday ya Sale ke din, naye orders sirf "Current Month" wale shard par aayenge. Baaki shards khali padenge. Current shard par load badh jayega aur wo "melt" (crash) ho sakta hai. Ise Hotspotting kehte hain.

Option C: Directory-based Sharding (Lookup Table)
Kya hai? Ek alag se "Phone Directory" (Lookup Service) hota hai.
Jab request aati hai "Customer X ke orders lao", pehle Directory se pucha jata hai: "Bhai, Customer X kis Shard par hai?"
Directory batata hai: "Wo Shard 3 par hai."
Phir direct Shard 3 se data fetch kiya jata hai.
Fayda: Yeh exactly wahi karta hai jo humein chahiye. Kyunki 80% queries Customer-Centric hain, hum directly usi shard par ja sakte hain jahan wo customer ka data hai. No need to search everywhere.
Nuksan: Directory service khud ek single point of failure ban sakti hai (agar wo down ho gayi toh koi kuch nahi dhundh payega), lekin ise cache karke manage kiya ja sakta hai.

Option D: Consistent Hashing with Virtual Nodes
Kya hai? Yeh Option A (Hashing) ka hi advanced version hai. Isme servers ko ring ki tarah arrange kiya jata hai taaki agar ek server add/remove ho, toh poora data reshuffle na karna pade.
Fayda: Scaling aur rebalancing bahut aasaan hai.
Nuksan: Yeh bhi hashing hi hai. Agar maine order_id par hash kiya, toh wahi Scatter-Gather problem aayegi. Customer ke saare orders alag-alag jagah bikhre honge.

Subscribe :)

SystemDesign #Databases #PostgreSQL #BackendEngineering #SystemDesignInterview #SystemDesignBlog #Redis #Sharding #BlackFridaySystemDesign

Your Smartphone can now deliver best gaming graphics, run AI directly on the device - Qualcomm's Snapdragon 8 Elite

Bhanu partap — Sat, 10 Oct 2026 06:42:43 +0000

We first need to understand the challenges with previous mobile platforms.

Mobile devices have limited power and cooling capacity. When a game renders complex graphics at high frame rates, the GPU has to perform more work. This can increase power consumption and heat, which may cause performance to drop during long gaming sessions.
AI introduces another challenge. Running AI models directly on a smartphone requires sufficient computing power and memory. Some AI features also depend on cloud servers, which can introduce network delays and require an internet connection.

The challenge is not simply to make a smartphone faster. It is to improve performance, graphics, and AI capabilities while managing power consumption and heat.

What Has Qualcomm Added?
Qualcomm is improving three important parts of the mobile architecture: the CPU, GPU, and NPU.

First, the CPU, powered by Qualcomm’s custom Oryon architecture, handles general computing, application logic, and system tasks. Improvements here are designed to support faster processing and smoother application performance.
Second, the Adreno GPU handles graphics rendering for games and other visual applications.

One of the key additions is Adreno Neural Fusion. It combines AI-assisted graphics techniques, including super resolution and frame generation, within a unified graphics pipeline.

Super resolution uses AI to reconstruct a higher-resolution image from a lower-resolution input. Frame generation creates additional frames between rendered frames to make motion appear smoother.

Qualcomm has also introduced Adreno matrix cores for AI-related GPU workloads. For game developers, these capabilities create new opportunities to improve visual quality without requiring every graphical detail to be rendered in the traditional way.

Third, the Hexagon NPU is designed to accelerate AI workloads. Instead of relying entirely on the CPU, applications can use dedicated AI hardware for supported operations.

This helps enable on-device AI, where supported models and tasks can run directly on the smartphone instead of always sending requests to cloud servers.

Qualcomm is also advancing mobile imaging through Intelligent Pixel Control and Advanced Professional Video support on the Extreme platform.

What Will Be the Results?

For gamers, these technologies could deliver sharper graphics and smoother gameplay, depending on game support and device performance.

For game developers, AI-assisted rendering creates more options to improve visual quality while managing the processing demands of mobile hardware.
For AI developers, on-device processing can support responsive AI features with reduced dependence on remote servers. However, the actual benefits depend on model size, memory, software optimisation, and the workload.
For content creators, improvements in image processing and video capabilities could support better mobile photography and more advanced video workflows.
The final experience will still depend on the smartphone’s cooling system, software, game-engine integration, and how developers implement these technologies.

Closing

The biggest change is not simply a faster processor. It is the closer integration of traditional computing, graphics processing, and artificial intelligence.

Qualcomm is giving developers more tools to build advanced mobile experiences.

The real test, however, is how well smartphone manufacturers and developers turn these hardware capabilities into features that people can actually use.

Bhanu Partap,
partapbhanu516@gmail.com