DEV Community: Khushi Patel

How Does Aadhaar Verify Billions of People's Fingerprints in Seconds?

Khushi Patel — Thu, 09 Jul 2026 17:44:45 +0000

Introduction

India's Aadhaar system stores biometric information for over a billion residents. Every day, millions of fingerprint authentication requests are made for banking, government services, SIM verification, and more.

A common question is:

How can Aadhaar verify a fingerprint among more than a billion records in just a few seconds?

The answer is that Aadhaar doesn't compare your fingerprint against every fingerprint in the database. Instead, it uses smart indexing, biometric templates, and a highly optimized distributed architecture.

The Myth

Many people imagine Aadhaar works like this:

Fingerprint
      |
      v
Compare with Person 1
Compare with Person 2
Compare with Person 3
...
Compare with 1.4 Billion People

If this were true, authentication would take hours.

Fortunately, that's not how the system works.

Fingerprints Are Stored as Templates

When you enroll for Aadhaar, the system doesn't store your fingerprint as a normal image.

Instead, it extracts unique features such as:

Ridge endings
Ridge bifurcations
Minutiae points
Relative positions and angles

These features are converted into a compact biometric template, making comparisons much faster than comparing full images.

Authentication Is Usually 1:1 Matching

Most Aadhaar authentications already include your Aadhaar number or Virtual ID.

The process looks like this:

Aadhaar Number + Fingerprint
          |
          v
Find User Record
          |
          v
Compare Submitted Fingerprint
          |
          v
Authentication Success or Failure

Since the system already knows which record to check, it performs one fingerprint comparison, not billions.

What About Identification Without an Aadhaar Number?

In some scenarios, the system may need to identify a person without knowing who they are.

This is called 1:N matching.

Instead of checking every record one by one, the system:

Uses biometric indexes
Narrows the search to likely matches
Searches across multiple servers in parallel
Compares only a small candidate set

This dramatically reduces the search time.

Distributed System Architecture

              Authentication Request
                        |
                        v
                Load Balancer
                        |
        +---------------+---------------+
        |               |               |
        v               v               v
   Biometric       Biometric      Biometric
   Server 1        Server 2       Server 3
        |               |               |
        +---------------+---------------+
                        |
                        v
               Authentication Result

The workload is distributed across many servers, allowing millions of authentication requests to be processed concurrently.

Optimizations That Make It Fast

Modern biometric systems use several techniques:

Biometric templates instead of images
Fast indexing algorithms
Parallel processing
In-memory caching
Distributed databases
Optimized biometric matching engines

Together, these reduce authentication time from minutes to just a few seconds.

Interview Perspective

A common misconception is:

"The system compares your fingerprint against every person in India."

A strong system design answer is:

Most Aadhaar authentications are 1:1 verification, not 1:N identification.
Fingerprints are stored as biometric templates.
Distributed servers process requests in parallel.
Indexing reduces the search space.
Only a tiny number of comparisons are actually performed.

Key Takeaways

Aadhaar does not compare your fingerprint with billions of records for every authentication.
Fingerprints are converted into compact biometric templates.
Most authentications are 1:1 verification using your Aadhaar number or Virtual ID.
Large-scale infrastructure, indexing, and distributed processing allow millions of requests to be handled efficiently.
The speed comes from smart system design—not brute-force searching.

Why Does LinkedIn Show "500+ Connections" Instead of the Exact Number?

Khushi Patel — Mon, 29 Jun 2026 13:56:01 +0000

Introduction

Have you ever noticed that LinkedIn displays "500+ Connections" once someone crosses 500 connections, instead of showing the exact count like 723 or 1,842?

At first glance, this might seem like a simple UI choice, but it's actually a great example of a product and system design decision where engineering, business goals, and user psychology come together.

The Product Perspective

LinkedIn's goal is not to help users compare who has more connections. Instead, it wants to indicate that someone has a strong professional network.

From a networking perspective:

450 Connections → Growing Network
500 Connections → Well Connected Professional
2500 Connections → Also Well Connected Professional

For most users, knowing whether someone has 1,200 or 1,500 connections doesn't provide significant additional value.

So LinkedIn chooses a threshold:

0 - 499  → Show exact number
500+     → Show "500+"

This communicates the important information while reducing unnecessary comparison.

The Engineering Perspective

Imagine LinkedIn has hundreds of millions of users.

Every profile view needs to display:

Name
Headline
Profile picture
Connection count

Showing an exact connection count requires keeping that number constantly updated whenever:

A connection request is accepted
A connection is removed
An account is deleted

For highly connected users, this count changes frequently.

By displaying:

500+

instead of:

LinkedIn reduces the importance of real-time precision.

This gives engineers more flexibility around caching and data synchronization.

A Distributed Systems Trade-off

In system design, there's a concept called:

Precision vs Practical Value

Ask yourself:

Would a user experience change if LinkedIn showed:

12,842

instead of:

500+

Probably not.

Since the exact value provides little additional business value, LinkedIn can optimize for:

Faster profile loads
Better caching
Lower database reads
Simpler distributed updates

This is a classic example of choosing an approximate answer when an exact answer isn't necessary.

Real-World System Design Principle

A common mistake in software engineering is trying to make everything perfectly accurate.

Good system designers ask:

Does the user actually need this precision?

Examples:

Instagram likes may update with slight delays
YouTube subscriber counts are often rounded
Social media follower counts are abbreviated
LinkedIn shows 500+ connections

These systems intentionally sacrifice precision because the business value of exactness is low.

The Deliverable System Design Lesson

One of the most important lessons in system design is:

Build for the requirement, not for perfection.

If the product requirement is:

Show whether a user has a large professional network

then:

500+

solves the problem.

If the requirement were:

Display the exact connection count for analytics purposes

then a different design would be necessary.

Great engineering decisions are often about identifying where precision matters and where it doesn't.

Key Takeaway

LinkedIn shows 500+ connections because the exact number provides little additional value to users after a certain threshold. This decision reduces social comparison, simplifies the user experience, and gives engineers more flexibility in scaling, caching, and synchronizing connection counts.

It's a great example of a system design principle: optimize for what users actually need, not for perfect accuracy everywhere.

How Does Google Docs Handle Two People Editing the Same Line at the Exact Same Time?

Khushi Patel — Thu, 25 Jun 2026 14:23:04 +0000

Introduction

One of the most impressive features of Google Docs is real-time collaboration. Multiple users can edit the same document simultaneously and see changes appear almost instantly.

But what happens when two users edit the exact same line at the exact same moment?

How does Google Docs prevent one user's changes from overwriting another's? Is there a lock on the document? Does someone lose their changes?

The answer lies in a distributed systems technique called Operational Transformation (OT), combined with real-time synchronization architecture.

The Problem

Imagine a document containing:

Hello World

Now consider two users:

User A

Adds:

Hello Beautiful World

User B

At the same time adds:

Hello Amazing World

Both users started editing from the same document version.

The challenge is:

Both changes are valid
Both arrive at the server simultaneously
Neither should overwrite the other
Everyone should eventually see the same document

Why Traditional Database Updates Fail

A typical CRUD system works like this:

Read Data
Modify Data
Update Data

If two users update simultaneously:

User A -> Save
User B -> Save

The second save overwrites the first.

This is known as a:

Lost Update Problem

Final Result:
Hello Amazing World

User A's changes disappear.

For collaborative editors, this is unacceptable.

Google's Solution: Operational Transformation (OT)

Instead of sending the entire document after every edit, Google Docs sends:

Operations

Examples:

Insert "Beautiful" at position 6

Insert "Amazing" at position 6

The server receives operations rather than complete document snapshots.

Understanding Operations

Original text:

Hello World

User A Operation

Insert "Beautiful " at position 6

User B Operation

Insert "Amazing " at position 6

Both operations target the same location.

Without conflict resolution:

Hello Beautiful World

Hello Amazing World

One edit wins.

Google Docs does something smarter.

Operational Transformation in Action

Server receives:

Operation A:
Insert "Beautiful " at position 6

Then receives:

Operation B:
Insert "Amazing " at position 6

The second operation is transformed based on the first.

Instead of:

Insert at position 6

it becomes:

Insert at position 16

because the document has already grown.

Final result:

Hello Beautiful Amazing World

Both users' edits survive.

Real-Time Architecture

          User A
             |
             |
             v
      WebSocket Server
             ^
             |
             |
          User B

Google Docs maintains a persistent connection using:

WebSockets
Real-time event streams
Version tracking

Every edit is sent as a tiny operation rather than a full document.

Step-by-Step Flow

Step 1

Both users load:

Version 100

Step 2

User A sends:

Insert "Beautiful"
Version 100

Step 3

User B sends:

Insert "Amazing"
Version 100

Step 4

Server processes User A first.

Document becomes:

Version 101

Step 5

Server notices User B is based on:

Version 100

while current document is:

Version 101

Now OT transforms User B's operation before applying it.

Step 6

Document updates successfully.

Everyone eventually sees:

Hello Beautiful Amazing World

What About Cursor Positions?

Imagine you're typing while another person inserts text before your cursor.

Without adjustment:

Cursor jumps randomly

Google Docs tracks:

Cursor position
Selection ranges
User presence

When operations are transformed, cursor positions are transformed too.

This is why your typing experience remains smooth even when dozens of users are editing.

Does Google Lock the Document?

No.

Document locking would be terrible for collaboration.

Imagine:

User A is editing line 20

Everyone else would have to wait.

Instead, Google allows:

Concurrent editing

and resolves conflicts intelligently.

This provides a much better user experience.

Modern Alternative: CRDTs

Many newer collaborative systems use:

Conflict-free Replicated Data Types (CRDT)

Used by:

Figma
Notion (parts of architecture)
Modern collaborative editors
Offline-first applications

CRDTs allow clients to merge changes without requiring a central transformation server.

Benefits:

Better offline support
Easier synchronization
High scalability

Google Docs was originally built using Operational Transformation, although modern collaborative systems increasingly adopt CRDT-based approaches.

System Design Architecture

                    +----------------+
                    | Document Store |
                    +----------------+
                            ^
                            |
                            |
+--------+         +----------------+         +--------+
| User A | <-----> | Sync Server    | <-----> | User B |
+--------+         +----------------+         +--------+
                            |
                            |
                    Operational
                   Transformation
                            |
                            v
                    Document State

The Sync Server is responsible for:

Receiving operations
Version tracking
Conflict resolution
Operation transformation
Broadcasting updates

Interview Perspective

A common System Design interview question is:

Design Google Docs.

Expected discussion points:

WebSockets for real-time communication
Operational Transformation or CRDT
Version numbers
Conflict resolution
Document persistence
Cursor synchronization
Horizontal scaling of collaboration servers

Mentioning OT and CRDT immediately demonstrates a strong understanding of collaborative systems.

Key Takeaways

Google Docs does not lock documents during editing.
Users send operations instead of entire document snapshots.
Operational Transformation resolves simultaneous edits.
The server transforms conflicting operations so both changes can survive.
WebSockets provide low-latency synchronization.
Cursor positions are transformed along with document changes.
Modern systems often use CRDTs as an alternative approach.

The magic behind Google Docs is not preventing conflicts—it's intelligently transforming and merging edits so that every user eventually sees the same consistent document.

Why Doesn't an E-Commerce Payment API Get Called Twice When Users Double-Click the Pay Button?

Khushi Patel — Mon, 22 Jun 2026 13:34:15 +0000

Introduction

Imagine you're purchasing a product online. You click the "Pay Now" button, and due to a slow internet connection or impatience, you accidentally click it again.

A common question among developers is:

If two requests are sent, why doesn't the payment get processed twice?

The answer lies in a combination of frontend protection, backend safeguards, database design, and payment gateway architecture.

In this blog, we'll explore how modern e-commerce systems prevent duplicate payments and the system design principles behind it.

The Problem

Consider the following sequence:

User clicks "Pay Now"
Browser sends payment request
User clicks again before the first request completes
Browser sends another payment request

Without proper handling:

Customer may be charged twice
Two orders may be created
Inventory may be deducted twice
Refund processes become necessary

This is a critical financial issue that every payment system must prevent.

First Layer: Frontend Protection

The simplest protection happens in the UI.

Disable the Payment Button

After the first click:

const handlePayment = async () => {
  setLoading(true);
  await makePayment();
};

<button disabled={loading}>
  Pay Now
</button>

What Happens?

User clicks once
Button becomes disabled
Additional clicks are ignored

Why This Isn't Enough

Frontend validation can be bypassed:

Browser refresh
Multiple tabs
Network retries
Malicious requests
Mobile app bugs

Therefore, the backend must still assume duplicate requests can arrive.

Second Layer: Idempotency Keys

This is the most important concept in payment systems.

What Is Idempotency?

An operation is idempotent if performing it multiple times produces the same result.

For payments:

Pay ₹100 once
Pay ₹100 again
Pay ₹100 again

Result:

Only one actual payment is processed.

Generating an Idempotency Key

When a payment request is initiated:

payment_id = "PAY_123456"

Request:

{
  "paymentId": "PAY_123456",
  "amount": 1000
}

The backend stores this key.

First Request

PAY_123456
Status: Processing

Payment starts.

Second Request

Backend receives:

PAY_123456

System checks:

SELECT * FROM payments
WHERE payment_id = 'PAY_123456';

Record already exists.

Instead of processing again:

Return existing response

No second payment occurs.

Architecture Flow

User
  |
  v
Frontend
  |
  v
API Gateway
  |
  v
Payment Service
  |
  +---- Check Idempotency Key
  |
  +---- Already Exists?
          |
       YES --> Return Previous Result
          |
       NO
          |
          v
Process Payment
          |
          v
Store Result

This pattern is used by almost every modern payment platform.

Database-Level Protection

Even if two requests reach the server simultaneously, the database provides another safety layer.

Unique Constraint

CREATE TABLE payments (
  payment_id VARCHAR(100) UNIQUE,
  amount DECIMAL(10,2)
);

Suppose two servers receive:

PAY_123456

at exactly the same moment.

The first insert succeeds:

INSERT INTO payments ...

The second insert fails:

Duplicate Key Exception

Thus, duplicate records cannot exist.

Handling Race Conditions

Consider a distributed environment:

Request A --> Server 1
Request B --> Server 2

Both arrive within milliseconds.

Without coordination:

Server 1 -> Process Payment
Server 2 -> Process Payment

Duplicate charge risk.

Distributed Locking

Many systems use Redis locks.

Flow

Acquire Lock
      |
      v
Process Payment
      |
      v
Release Lock

Example:

LOCK:PAY_123456

Server 1:

Lock Acquired

Server 2:

Lock Exists

Server 2 waits or returns existing result.

Payment Gateway Protection

Payment providers also implement idempotency.

Examples include:

Stripe
Razorpay
PayPal

When sending a request:

POST /payments
Idempotency-Key: PAY_123456

Gateway stores:

PAY_123456

If another request arrives:

Same key detected

Gateway returns the original response instead of charging again.

This creates protection even if your application has a bug.

Real-World Payment Architecture

                User
                  |
                  v
         +----------------+
         |  Frontend App  |
         +----------------+
                  |
                  v
         +----------------+
         | API Gateway    |
         +----------------+
                  |
                  v
         +----------------+
         | Payment Service|
         +----------------+
                  |
      +-----------+------------+
      |                        |
      v                        v
 Redis Lock            Payment Database
      |                        |
      +-----------+------------+
                  |
                  v
          Payment Gateway
                  |
                  v
              Bank

Every layer contributes to preventing duplicate transactions.

Why Multiple Protection Layers Are Needed

A common misconception is:

"Disabling the button solves the problem."

In reality:

Layer	Purpose
Frontend Disable	Prevent accidental clicks
API Idempotency	Prevent duplicate processing
Database Unique Key	Prevent duplicate records
Redis Lock	Prevent race conditions
Payment Gateway Idempotency	Prevent duplicate charges

Payment systems rely on defense in depth, not a single safeguard.

Interview Perspective

A common System Design interview question is:

How would you prevent duplicate payment processing?

Expected answer:

Disable button on frontend
Generate unique payment/order ID
Use idempotency keys
Store payment status in database
Add unique constraints
Use distributed locking for concurrent requests
Leverage payment gateway idempotency support

Discussing all layers demonstrates strong backend and distributed systems knowledge.

Key Takeaways

Double-clicking a payment button can generate multiple requests.
Frontend protection alone is insufficient.
Idempotency keys are the primary mechanism for preventing duplicate payments.
Databases enforce uniqueness through constraints.
Redis locks handle concurrent requests across multiple servers.
Payment gateways provide an additional safety layer.
Modern payment systems use multiple layers of protection to guarantee that a customer is charged only once.

A successful payment architecture is not about stopping duplicate requests it is about ensuring duplicate requests produce only one financial transaction.

If Everything Is Running on Localhost, Why Do We Still Get CORS Errors?

Khushi Patel — Fri, 19 Jun 2026 13:36:32 +0000

One question almost every frontend developer asks at some point:

"My frontend is running on localhost:3000 and my backend is running on localhost:5000. Both are on my own machine. Why am I still getting a CORS error?"

It feels strange because both applications are literally running on the same laptop.

The answer lies in how browsers define an Origin.

What Is an Origin?

An origin is made up of:

Protocol + Domain + Port

For example:

http://localhost:3000

and

http://localhost:5000

have:

Same protocol ✅
Same domain ✅
Different ports ❌

Because the ports are different, the browser treats them as two different origins.

What Happens During a Request?

Imagine your React app is running on:

http://localhost:3000

and it calls:

fetch("http://localhost:5000/users");

From the browser's perspective:

Origin A → localhost:3000
Origin B → localhost:5000

This is considered a cross-origin request.

Why Does the Browser Care?

Imagine if websites could freely call APIs from any origin.

A malicious website could:

Read your banking data
Access your private APIs
Perform actions on your behalf

To prevent this, browsers enforce the Same-Origin Policy.

By default:

Website A cannot access Website B's resources

unless Website B explicitly allows it.

How Does CORS Solve This?

CORS stands for:

Cross-Origin Resource Sharing

It is simply a mechanism that lets the server tell the browser:

"Yes, I trust requests coming from this origin."

Example response header:

Access-Control-Allow-Origin: http://localhost:3000

Now the browser allows the frontend to access the response.

Important Thing To Understand

The backend usually receives the request successfully.

The CORS error is often raised by the browser after receiving the response.

The flow looks like:

Frontend
    ↓
Backend Receives Request ✅
    ↓
Backend Sends Response ✅
    ↓
Browser Blocks Response ❌

That's why you sometimes see the API hit in your backend logs but still get a CORS error in the browser.

Why Doesn't Postman Show CORS Errors?

Because CORS is a browser security feature.

Tools like:

Postman
cURL
Insomnia

don't enforce browser security policies.

So the same request works perfectly there.

The Key Takeaway

Even though both applications are running on the same machine:

localhost:3000
localhost:5000

they are considered different origins because the ports are different.

The browser doesn't care that they're on the same laptop. It only cares about the origin.

That's why CORS exists, and that's why adding the correct Access-Control-Allow-Origin header fixes the issue.

System Design: How Does a UPI Payment Reach the Chai Wala in Just Seconds?

Khushi Patel — Wed, 17 Jun 2026 14:15:46 +0000

Imagine this.

You buy a ₹20 chai from your local chai wala.

You open Paytm, scan his QR code, and make the payment.

But here's the interesting part:

Your Paytm is linked to HDFC Bank
The chai wala's Paytm is linked to SBI
The money moves between two completely different banks

And yet...

Within 2-3 seconds both of you receive a notification.

How does this happen so fast?

Let's follow the journey of that ₹20.

Step 1: You Scan the QR Code

The QR code doesn't contain money.

It usually contains information like:

UPI ID: chaiwala@sbi
Merchant Details

When you enter ₹20 and click Pay, Paytm creates a payment request.

Step 2: Paytm Doesn't Transfer Money

Many people think Paytm sends the money.

It doesn't.

Paytm is simply acting as an interface.

The actual money movement happens between banks through UPI.

So the flow looks like:

You
 ↓
Paytm
 ↓
UPI Network
 ↓
Banks

Step 3: Request Goes To NPCI

The payment request is sent to the UPI infrastructure operated by the National Payments Corporation of India.

Think of NPCI as the traffic controller of India's UPI system.

It knows:

Which bank owns your account
Which bank owns the chai wala's account
Where the request should be routed

Step 4: Your Bank Verifies Everything

NPCI forwards the request to your bank.

Your bank checks:

Is the account active?
Is there enough balance?
Is the UPI PIN correct?
Is the transaction suspicious?

If everything is valid, the bank approves the debit.

Account Balance = ₹1000
Payment = ₹20

Approved ✅

Step 5: Chai Wala's Bank Gets The Credit Request

Now NPCI forwards the request to the chai wala's bank.

His bank verifies the account and credits ₹20.

Chai Wala Balance
₹500
   ↓
₹520

Step 6: Success Message Everywhere

Once both banks confirm:

NPCI marks transaction successful
Paytm receives success response
Chai wala's Paytm receives success response

Within seconds:

You: Payment Successful ✅

Chai Wala: ₹20 Received ✅

But Wait... Did The Money Actually Move?

This is where things get interesting.

Millions of UPI transactions happen every minute.

Banks don't physically transfer money one-by-one for every chai, samosa, or grocery purchase.

Instead, they maintain settlement records.

Throughout the day:

HDFC owes SBI
SBI owes ICICI
ICICI owes Axis

NPCI keeps track of these obligations.

Later, settlement happens through the banking system in bulk.

This is much faster than moving actual money for every transaction individually.

Why Is It So Fast?

Several engineering decisions make this possible:

Parallel Processing

Banks verify requests simultaneously.

High-Speed Network

NPCI operates highly optimized payment infrastructure.

Lightweight Messages

Only transaction information is exchanged, not huge amounts of data.

Deferred Settlement

Banks settle balances later rather than moving money instantly every time.

Simplified Architecture

You
 │
 ▼
Paytm App
 │
 ▼
NPCI UPI Switch
 │
 ├─────────────► Your Bank
 │                    │
 │                    ▼
 │              Debit Money
 │
 ▼
Chai Wala Bank
 │
 ▼
Credit Money
 │
 ▼
Success Response

All of this happens in just a few seconds.

Key Takeaway

When you pay ₹20 to a chai wala:

Paytm is only the interface.
NPCI acts as the traffic controller.
Your bank debits the amount.
The merchant's bank credits the amount.
Settlement between banks happens later.
Optimized infrastructure allows the entire process to complete in seconds.

The next time you hear that familiar "Payment Received" sound from the chai wala's phone, remember:

Behind that simple beep, multiple systems, banks, servers, and networks coordinated in real time to move money across the country.

System Design: How Do We Host an App That Runs a Job Every 5 Minutes?

Khushi Patel — Fri, 12 Jun 2026 12:21:49 +0000

Imagine you're building an app that needs to perform a task every 5 minutes.

Examples:

Send reminder notifications
Check product expiry dates
Generate daily reports
Sync data from third-party APIs
Clean old records from the database

A common beginner question is:

If nobody opens my app, who will trigger these jobs every 5 minutes?

Let's break down how this works in production.

Option 1: Cron Job on a Server

The simplest solution is using a Cron Job.

A cron job is a scheduler built into Linux that can execute commands at specific intervals.

Example:

*/5 * * * * node reminderJob.js

This means:

Every 5 minutes
Run reminderJob.js

Architecture:

┌─────────────┐
│ Linux Server│
└──────┬──────┘
       │
       ▼
  Cron Scheduler
       │
       ▼
   Node.js Job
       │
       ▼
    Database

Works well for:

Small projects
Internal tools
MVPs

Problem With a Single Server

What happens if the server crashes?

Server Down
     ↓
Cron Stops
     ↓
Jobs Missed

For critical applications, this is unacceptable.

Imagine:

Payment settlement jobs
Order processing
OTP cleanup
Subscription renewals

Missing even one execution can create problems.

Option 2: Dedicated Background Worker

Large systems separate API servers and background workers.

Architecture:

        Users
          │
          ▼
    API Servers
          │
          ▼
      Database

          ▲
          │

    Worker Service
          │
          ▼
    Scheduled Jobs

Benefits:

APIs remain fast
Jobs don't affect user requests
Easier to scale independently

Option 3: Kubernetes CronJobs

Modern cloud applications often run on Kubernetes.

Kubernetes provides a resource called CronJob.

Example:

schedule: "*/5 * * * *"

Every 5 minutes Kubernetes automatically creates a container and runs the job.

Architecture:

      Kubernetes
           │
           ▼
      CronJob
           │
           ▼
    New Container
           │
           ▼
       Execute Job

Advantages:

Automatic retries
Auto-healing
Easy deployment
Cloud-native

Preventing Duplicate Execution

Suppose you have 3 servers.

Server A
Server B
Server C

If all three execute the same job:

Send Notification
Send Notification
Send Notification

Users receive the same notification three times.

Not good.

Distributed Locking

To ensure only one server executes the job, companies use distributed locks.

Popular choices:

Redis Lock
Database Lock
ZooKeeper
etcd

Flow:

Worker Starts
      │
      ▼
Acquire Lock
      │
      ▼
Lock Success ?
   /       \
 Yes        No
  │          │
  ▼          ▼
Run Job    Exit

Only one worker gets the lock.

How Big Companies Do It

Companies rarely depend on a single cron job.

Typical architecture:

                Scheduler
                    │
                    ▼
              Message Queue
                    │
        ┌───────────┼───────────┐
        ▼           ▼           ▼
     Worker 1    Worker 2    Worker 3
        │           │           │
        └───────────┼───────────┘
                    ▼
                 Database

Popular technologies:

Cron
Kubernetes CronJobs
Redis
RabbitMQ
Apache Kafka
AWS EventBridge
Google Cloud Scheduler

The scheduler creates tasks.

Workers process those tasks.

This makes the system scalable and fault tolerant.

Example: Expiry Reminder App

Let's say you built an app that stores expiry dates.

Every 5 minutes:

Scheduler triggers a job.
Job finds products expiring soon.
Notification tasks are added to a queue.
Workers send notifications.
Results are stored in the database.

Architecture:

Cron Scheduler
      │
      ▼
Expiry Checker
      │
      ▼
Database Query
      │
      ▼
Message Queue
      │
      ▼
Notification Workers
      │
      ▼
Firebase / APNs

Key Takeaway

A scheduled task running every 5 minutes is usually not handled by user traffic.

Production systems use:

Cron Jobs
Background Workers
Kubernetes CronJobs
Message Queues
Distributed Locks

The bigger the scale, the more important reliability, retries, and duplicate prevention become.

Next time you receive a notification exactly on time, remember:

Somewhere in the cloud, a scheduler woke up, executed a job, and triggered that notification.

How Does Netflix Remember Exactly Where You Stopped Watching?

Khushi Patel — Wed, 10 Jun 2026 14:02:07 +0000

Imagine this:

You're watching a show on your TV. Halfway through an episode, you turn the TV off and leave. A few minutes later, you open Netflix on your phone, and somehow it starts playing from the exact same second where you stopped.

No loading. No searching. No manually finding your spot.

How does Netflix make this happen almost instantly for millions of users?

The Secret: Continuous Progress Syncing

While you're watching a movie or series, Netflix doesn't wait until you finish the episode to save your progress.

Every few seconds, the app sends small updates to Netflix's servers:

User ID
Show ID
Episode ID
Current playback position (e.g., 18 minutes 42 seconds)
Timestamp

A simplified record might look like:

{
  "userId": 12345,
  "showId": 789,
  "episodeId": 12,
  "position": 1122
}

Here, 1122 represents the number of seconds watched.

What Happens When You Close the App?

Suppose you stop watching at 18:42.

Before the TV app closes, Netflix sends one final update containing your latest position.

This information is stored in a highly distributed database that can be accessed from anywhere in the world.

Now Netflix knows exactly where you left off.

Opening Netflix on Another Device

A few minutes later, you open Netflix on your phone.

The app immediately asks Netflix:

Where did this user stop watching?

Netflix returns:

{
  "episodeId": 12,
  "position": "18:42"
}

The phone then starts streaming from that exact point.

To the user, it feels like magic.

Behind the scenes, it's simply fast synchronization between devices and servers.

The Real Challenge: Scale

Doing this for one user is easy.

Netflix has hundreds of millions of users watching content simultaneously on:

TVs
Mobile phones
Tablets
Laptops
Gaming consoles

This means Netflix processes billions of progress updates every day.

Their systems must handle:

Massive write traffic
Low-latency reads
Global availability
Device synchronization
Failure recovery

All while keeping playback smooth.

What If Two Devices Are Watching at the Same Time?

Suppose you're watching on your TV and phone simultaneously.

Which position should Netflix save?

Netflix usually stores timestamps along with progress updates.

The latest valid update wins.

Example:

TV Update:
Position: 10:15
Time: 10:00:01

Phone Update:
Position: 12:30
Time: 10:00:05

Since the phone's update arrived later, Netflix treats that as the newest progress.

Why It Feels Instant

Netflix uses:

Distributed databases
Global data centers
Intelligent caching
Event-driven systems
Real-time synchronization

As a result, when you switch devices, your watch history is already available and ready to use.

System Design Flow

TV App
   │
   │ Progress Update (18:42)
   ▼
Netflix API
   │
   ▼
Distributed Database
   │
   ▼
Mobile App Requests Progress
   │
   ▼
Returns 18:42
   │
   ▼
Playback Resumes

Next Time You Switch Devices...

Remember that Netflix isn't magically tracking your progress.

It's continuously saving tiny updates while you watch and making them available across the world within seconds.

A simple feature for users.

An enormous distributed systems challenge for engineers.

Question for You

If Netflix can sync your watch progress worldwide in seconds, how do you think live-streaming platforms handle 2–5 crore viewers watching the same event simultaneously?

Implement LFU using LRU

Khushi Patel — Thu, 02 Apr 2026 09:27:06 +0000

📌 Core Idea

LFU (Least Frequently Used) means:

Remove the least frequently used key
If multiple keys have same frequency → remove least recently used (LRU)

📦 Data Structures Used

We maintain:

key → node (for O(1) access)
freq → doubly linked list (group nodes by frequency)
minFreq (track minimum frequency in cache)

🧱 Structure Visualization

Freq = 1 → [ most recent .... least recent ]

Freq = 2 → [ most recent .... least recent ]

Freq = 3 → [ most recent .... least recent ]

Each frequency has its own LRU list

🚀 Example Walkthrough

Capacity = 2

Step 1: put(1,10)

Freq 1: [1]

minFreq = 1

Step 2: put(2,20)

Freq 1: 2, 1

minFreq = 1

Step 3: get(1)

Access key 1
Increase its frequency: 1 → 2

Freq 1: [2]

Freq 2: [1]

minFreq = 1

👉 We removed 1 from freq 1 and added to freq 2

Step 4: put(3,30)

Cache is FULL → eviction needed

minFreq = 1
Look at Freq 1 → [2]

👉 Remove 2

After insertion:

Freq 1: [3]

Freq 2: [1]

minFreq = 1

Step 5: get(3)

Move 3 from freq 1 → freq 2

Freq 1: []

Freq 2: [3, 1]

minFreq = 2 (freq 1 is empty now)

Step 6: put(4,40)

Cache full → eviction

minFreq = 2
Freq 2 → [3, 1]

👉 Remove LRU → 1

After insertion:

Freq 1: [4]

Freq 2: [3]

minFreq = 1

🔥 Key Observations

1. Why multiple lists?

Different frequencies → cannot maintain in single list
Each frequency maintains its own LRU order

2. Why minFreq?

Helps directly find which list to evict from in O(1)

3. Why doubly linked list?

O(1) removal
Maintain LRU order inside same frequency

🧠 Mental Model

Think of LFU like books:

Shelf 1 → least used books
Shelf 2 → moderately used
Shelf 3 → most used

When removing:

Pick least used shelf
Remove oldest book from that shelf

🧩 Mapping to Code

Concept	Code
Increase frequency	updateFreq(node)
Remove LRU	list->tail->prev
Track minimum	minFreq
Insert new node	freq = 1

🚨 Common Mistake

When a frequency list becomes empty:

if(freq == minFreq && list is empty)
    minFreq++;

❗ If you don’t update minFreq, eviction will break

Code

class LFUCache {
public:
    class Node {
    public:
        int key, val, freq;
        Node* prev;
        Node* next;

        Node(int k, int v) {
            key = k;
            val = v;
            freq = 1;
        }
    };

    class List {
    public:
        Node* head;
        Node* tail;
        int size;

        List() {
            head = new Node(-1, -1);
            tail = new Node(-1, -1);
            head->next = tail;
            tail->prev = head;
            size = 0;
        }

        void addFront(Node* node) {
            Node* temp = head->next;
            node->next = temp;
            node->prev = head;
            head->next = node;
            temp->prev = node;
            size++;
        }

        void removeNode(Node* node) {
            Node* prevv = node->prev;
            Node* nextt = node->next;
            prevv->next = nextt;
            nextt->prev = prevv;
            size--;
        }
    };

    int cap;
    int minFreq;

    unordered_map<int, Node*> keyNode;        // key -> node
    unordered_map<int, List*> freqList;       // freq -> DLL

    LFUCache(int capacity) {
        cap = capacity;
        minFreq = 0;
    }

    void updateFreq(Node* node) {
        int freq = node->freq;
        freqList[freq]->removeNode(node);

        if (freq == minFreq && freqList[freq]->size == 0) {
            minFreq++;
        }

        node->freq++;

        if (freqList.find(node->freq) == freqList.end()) {
            freqList[node->freq] = new List();
        }

        freqList[node->freq]->addFront(node);
    }

    int get(int key) {
        if (keyNode.find(key) == keyNode.end()) return -1;

        Node* node = keyNode[key];
        updateFreq(node);
        return node->val;
    }

    void put(int key, int value) {
        if (cap == 0) return;

        if (keyNode.find(key) != keyNode.end()) {
            Node* node = keyNode[key];
            node->val = value;
            updateFreq(node);
            return;
        }

        if (keyNode.size() == cap) {
            List* list = freqList[minFreq];
            Node* nodeToRemove = list->tail->prev;

            keyNode.erase(nodeToRemove->key);
            list->removeNode(nodeToRemove);
        }

        Node* newNode = new Node(key, value);
        minFreq = 1;

        if (freqList.find(1) == freqList.end()) {
            freqList[1] = new List();
        }

        freqList[1]->addFront(newNode);
        keyNode[key] = newNode;
    }
};

7-Day Beginner’s Mindmap to Learn GenAI

Khushi Patel — Tue, 15 Jul 2025 04:26:23 +0000

Perfect for anyone curious about ChatGPT, LLMs, image generation & more!
Each day has clear topics + free resources (courses, videos, articles) 🧠✨

👉 Explore the full mindmap here: Link

Follow on Instagram CodeAndCrunch

Understanding Video Streaming and Optimize Media Downloads 📽️📂

Khushi Patel — Tue, 04 Mar 2025 12:48:33 +0000

Video streaming has become an integral part of our digital experience, powering everything from YouTube and Netflix to social media platforms like Facebook, Twitter (X), and Instagram. While streaming is optimized for seamless playback, there are times when users might want to download content for offline access. This raises the question: How does video streaming work, and how can we optimize media downloads while staying within ethical boundaries?

In this post, we’ll break down the core principles of video streaming, explore how video downloads work, and discuss ways to optimize media retrieval efficiently.

How Video Streaming Works ⚒️

Streaming works by delivering media data in small chunks rather than requiring users to download the entire file before playback. Here’s a breakdown of key concepts:

1. Streaming Protocols

HTTP Live Streaming (HLS): Used by Apple devices and many streaming platforms, HLS splits videos into small .tssegments, which are loaded dynamically.
Dynamic Adaptive Streaming over HTTP (DASH): A widely used protocol that adjusts video quality based on network conditions.
Real-Time Messaging Protocol (RTMP): Previously popular for live streaming but now largely replaced by HLS/DASH.

2. Video Encoding & Compression

Codecs (H.264, H.265, VP9, AV1): Used to compress video files while maintaining quality.
Bitrate Adaptation: Streaming platforms adjust video quality based on bandwidth to avoid buffering.

3. Content Delivery Networks (CDNs)

CDNs store video files across multiple servers worldwide to reduce latency and improve load times.

How Media Downloads Work

Downloading a video is different from streaming because it involves retrieving the full file instead of fetching segments dynamically. However, many platforms restrict direct downloads to protect content rights. Here’s how media downloads typically work:

1. Direct File Download

Some platforms provide direct download options (e.g., YouTube Premium for offline viewing).

2. M3U8 Parsing (HLS Streams)

Some media files are segmented into .m3u8 playlists containing multiple .ts files. These need to be reconstructed into a full video. read more

3. API-Based Downloads

Some services provide APIs to fetch media content legally, All in One Downloader All in One Downloader is a great example of this which allow user to download video from multiple social media platform

Conclusion

By leveraging the right technologies and understanding video formats, you can enhance your workflow and make media retrieval more seamless.

What are your thoughts on video downloads and streaming optimization? Have you built any tools to enhance media retrieval? Let’s discuss in the comments! 🚀