DEV Community: Ashutosh Singh

Idempotency: The Concept That Separates Senior Engineers

Ashutosh Singh — Tue, 23 Jun 2026 08:03:02 +0000

"Just retry it" is the most dangerous advice in distributed systems

You click "Pay $500." The request reaches the server… but the response times out on the way back. Did it work? You have no idea. So you click again.

Now you've been charged twice.

This is the problem idempotency solves - and handling it well is one of the clearest signals of a senior engineer. Let's go deep, with code.

What "idempotent" actually means

An operation is idempotent if performing it once or ten times has the same effect.

GET, PUT, DELETE → naturally idempotent
POST that charges a card, creates an order, or sends an email → not idempotent

The moment a network can fail (always) and a client can retry (always), non-idempotent writes become dangerous.

Idempotency keys: the Stripe pattern

The client generates a unique key per logical operation and sends it with the request:


http
POST /charges
Idempotency-Key: a1b2c3d4-....

Server logic, in pseudocode:


record = store.get(key)
if record exists and completed:
    return record.response        # a retry — don't reprocess
result = process(request)
store.save(key, result)
return result
Simple in theory. The depth is entirely in the edge cases.

Edge case 1 — The concurrent-retry race
Two retries land in the same millisecond. Both run store.get(key) → both see nothing → both call process() → double charge.

A naive "check-then-write" recreates the exact bug you're trying to fix. You need an atomic claim — a unique constraint:


CREATE TABLE idempotency_keys (
  key         TEXT PRIMARY KEY,
  status      TEXT NOT NULL,         -- 'in_progress' | 'completed'
  response    JSONB,
  created_at  TIMESTAMPTZ DEFAULT now()
);

-- Atomically claim the key. Only ONE concurrent request wins.
INSERT INTO idempotency_keys (key, status)
VALUES ($1, 'in_progress')
ON CONFLICT (key) DO NOTHING;
If the insert affects 0 rows, someone else owns the key → you wait/poll for their result or return 409 Conflict. The database's uniqueness guarantee does the hard concurrency work for you.

Edge case 2 — Store the RESPONSE, not just "done"
On a retry you must return the same response the client missed the first time — same charge ID, same status code, same body. Otherwise the client can't reconcile what happened.

So when the first request completes, persist its full response:


UPDATE idempotency_keys
SET status = 'completed', response = $2
WHERE key = $1;
Now any later retry replays that exact response.

Edge case 3 — Partial failures & atomicity
The nastiest case: you charged the card but crashed before saving the result. The key never reaches completed, so a retry charges again.

Two defenses:

Wrap the idempotency record + the side effect in one transaction — only works when the side effect lives in the same database.
For external side effects (charging via a payment processor), record in_progress before the call, and on a retry of an in_progress key, reconcile with the processor (using your key) instead of blindly re-charging.
This is why payment APIs lean heavily on the provider also being idempotent end-to-end.

Edge case 4 — Key scope & expiry
Scope keys per endpoint + account, so the same key string can't collide across users.
Expire them (Stripe uses ~24h) — long enough to cover retries, short enough not to store forever.
Reusing a key with a different payload should error, not silently return the old result.
The mental model: stop chasing "exactly once"
Don't try to guarantee a request runs exactly once. Make it safe to run any number of times.

"Exactly-once delivery" is largely a myth. At-least-once delivery + idempotency = exactly-once effects.

That single reframe is what makes payment, order, and messaging systems reliable — and it's a favourite senior interview probe.

Wrapping up
Idempotency looks like a one-liner ("just store the key") and turns out to be four subtle problems: the race, the response replay, partial-failure atomicity, and key scope/expiry. Nail those and your retries stop being scary.

Disclosure: I work on PrepGrind, a free playground for building & stress-testing systems like this — but the patterns above stand on their own.

How do you handle the concurrent-retry race in your systems — a unique constraint, a distributed lock, or something else? Curious what's worked for others.

The IPL final. 50+ MILLION people streaming the same match at the same second - a real world record set on Hotstar. 🏏

Ashutosh Singh — Thu, 18 Jun 2026 14:49:09 +0000

Hotstar once streamed the Cricket World Cup final to ~59 MILLION people at the same second.

That's not a typo. ~59,000,000 concurrent viewers - a world record.

How does a system survive that? A few things Hotstar actually does:

→ Plan for the "tsunami," not the average. The scary moment isn't steady load — it's a wicket falling, when millions refresh and rejoin in the same few seconds. They design for that spike, not the baseline.

→ Pre-provision capacity. Reactive autoscaling is too slow — servers take minutes to boot, spikes happen in seconds. So Hotstar predicts concurrency and scales up BEFORE the match, keeping warm capacity ready.

→ Segment + multi-CDN fan-out. The live feed is chopped into tiny 2–6s chunks pushed to CDN edges across multiple providers. Millions pull from the nearest edge, not the origin. One encode, served to millions.

→ Graceful degradation ("panic mode"). Under extreme load, Hotstar sheds non-essential features and lowers quality to keep the core stream alive. A slightly lower-res match beats a crashed one.

→ Trade latency for stability. Live runs ~10–30s behind real-time on purpose — that buffer absorbs the spikes.

The big lesson: at this scale you don't fight the load head-on. You predict it, pre-provision for it, push everything to the edge, and degrade gracefully instead of failing.

You can build a live streaming design — origin, transcoding, CDN edge — and push traffic through it to watch where it strains 👇 (free, no signup)
prepgrind.xyz

What would you reach for first to handle 50M+ concurrent viewers?

SystemDesign #SoftwareEngineering #DistributedSystems #InterviewPrep

How Netflix Streams to 320+ Million Users Without Crashing

Ashutosh Singh — Tue, 16 Jun 2026 05:57:13 +0000

Netflix is roughly 15% of all downstream internet traffic. It streams to 320M+ subscribers across the planet, at peak hours, and almost never goes down.

If you've ever sat in a system design interview and your first instinct was "add more servers" — Netflix is the case study that breaks that instinct. Their whole architecture is built on the opposite idea:

Don't bring users to your servers. Bring the data to the users.

Let's break down how they actually do it.

1. The CDN is the product: Open Connect

Most companies rent a CDN (Cloudflare, Akamai, CloudFront). Netflix built its own — Open Connect — and physically ships caching appliances into ISPs' data centers.

So when you hit play in Bangalore or Berlin, the video doesn't come from a Netflix origin across the world. It streams from a box a few miles away, inside your own ISP's network.

Why it matters:

Lower latency + fewer hops → faster start, less buffering.
Less backbone traffic → cheaper for both Netflix and the ISP (a win-win that got ISPs to host the boxes for free).

The lesson: at massive read scale, the network is the bottleneck, not compute.

2. Pre-positioning: cache before the request

Here's the clever part. Netflix knows what's popular in each region, so it pushes those titles to edge servers overnight, during off-peak hours — before anyone presses play.

By the time you're watching at 9pm, the content is already sitting on the box next to you. This is caching taken to its logical extreme: you don't cache on the first request, you predict and pre-warm the cache.

Origin (S3) ──(overnight push)──► Open Connect @ ISP ──► You
cold warm (pre-positioned) fast

3. Adaptive Bitrate Streaming (ABR)

A video isn't stored as one file. It's encoded into many quality levels and chopped into small segments (a few seconds each).

Your player constantly measures your bandwidth and switches quality on the fly — segment by segment. Network dips? It drops to a lower bitrate instead of buffering. Network recovers? It bumps back up.

The principle: degrade gracefully instead of failing. A slightly blurry stream beats a spinning wheel every time.

4. Encode once, serve billions

Transcoding every title into dozens of resolutions and codecs is enormously expensive — so it's done once, offline, in a massive parallel pipeline:

Split the source video into chunks
Encode chunks in parallel across a worker fleet
Reassemble into per-quality streams
Distribute to Open Connect

Heavy, slow work happens on the cold path (offline). The hot path (playback) only ever serves pre-built files. Keeping these two paths separate is a pattern you'll reuse everywhere.

5. Microservices + designing for failure

Behind playback sits a mesh of hundreds of microservices (auth, recommendations, billing, metadata, playback…). At that scale, something is always failing. Netflix's answer is to assume it:

Chaos Monkey randomly kills services in production to prove the system survives.
Circuit breakers, fallbacks, and timeouts so one slow service doesn't cascade.

You don't engineer for "nothing breaks." You engineer for "things break and users don't notice."

What this means for your own designs

If you only remember four things from Netflix:

Move data to the edge (CDN / caching) before you scale compute.
Pre-warm caches for predictable, read-heavy load.
Degrade gracefully (adaptive quality, fallbacks) instead of failing hard.
Separate the cold path from the hot path (offline transcoding vs. live playback).

These show up in almost every large-scale read-heavy system — video, feeds, search, even maps.

Try breaking it yourself

Reading about this is one thing; feeling it is another. If you want to actually build a streaming design — drop in a CDN, edge servers, and an origin, then push live traffic through it and watch where it falls over — that's exactly what we built PrepGrind for. (Disclosure: it's our tool; free to start, no signup.)

What would you reach for first to handle Netflix-scale traffic — CDN, caching, or something else? Curious how others approach it. 👇

PrepGrind — Interactive System Design & DSA Interview Prep Playground

Learn system design and DSA the visual way. Drag-and-drop architecture canvas, live traffic simulation, 33 real case studies, and guided algorithm walkthroughs — free.

prepgrind.xyz

9 Years in Software Engineering - Excited to Join the DEV Community

Ashutosh Singh — Mon, 15 Jun 2026 15:25:51 +0000

Hi everyone

I'm Ashutosh, a Principal Engineer with 8+ years of experience building scalable software systems and leading engineering teams.

Currently, I'm working on large-scale educational platforms and pursuing an M.Tech in AI & Data Science from IIT Patna.

Over the years, I've worked across backend engineering, system design, cloud architecture, and more recently AI-powered applications.

I'm joining DEV to:

Share practical system design lessons
Write about AI and software engineering
Discuss engineering leadership and architecture decisions
Document my journey of building products from scratch

One thing I've learned after 9 years in engineering:

Building software is only half the challenge. Understanding users and solving the right problems is often the harder part.

Looking forward to learning from the community and contributing wherever I can.

What's one engineering topic you'd like to see covered more often?

introductions #softwareengineering #systemdesign #ai