DEV Community: Khushi Patel

Real-Time Communication: WebSockets vs. Polling

Khushi Patel — Sat, 31 Oct 2026 14:16:01 +0000

Introduction

Imagine sitting in a café and waiting for your order. You want to know
as soon as it is ready. You could keep asking the cashier, "Is my order
ready?", or ask the cashier to notify you when it is ready. This
illustrates two ways applications receive updates: polling and
WebSockets.

What Is Polling?

Polling is a communication technique in which a client repeatedly sends
requests to a server at regular intervals to check whether new
information is available.

Café Example

Imagine asking the cashier every five seconds whether your order is
ready. Even if it is not ready, you keep asking.

How It Works

The client sends an HTTP request.
The server checks for new information and returns a response.
The client waits for a configured interval.
The client sends another request.
This continues until the update arrives or polling stops.

Short Polling

The client sends requests at fixed intervals, such as every five
seconds.

Advantages: Simple to implement, uses ordinary HTTP, and works when
updates are occasional.

Disadvantages: Requests may be wasted when nothing changes, and
updates may be delayed until the next interval.

Long Polling

The client sends a request, but the server holds it open until new data
is available or a timeout occurs. After receiving a response, the client
sends another request.

Café example: You ask the cashier to let you know when the order is
ready. The cashier waits until there is an update, tells you, and you
ask again if you are still waiting.

Advantages: Can reduce unnecessary empty responses compared with
short polling and can deliver updates soon after they occur.

Disadvantages: The server manages requests that remain open, and
clients need to reconnect after each response. Timeouts and retries need
careful handling.

What Are WebSockets?

WebSockets provide a persistent, two-way communication channel between a
client and server over a single connection. Once established, either
side can send messages at any time without creating a new HTTP request
for every message.

Café Example

Instead of asking the cashier repeatedly, you say, "Please let me know
as soon as my order is ready." The cashier notifies you when it is
ready. You can also talk back through the same channel.

How It Works

The client starts a WebSocket handshake, typically using an HTTP request.
The server accepts the handshake and upgrades the connection to the WebSocket protocol.
The connection stays open while both sides need it.
Client and server can send messages independently.
Either side can close the connection when it is no longer needed.

Simple JavaScript Example

const socket = new WebSocket("wss://example.com/socket");

socket.onopen = () => {
  socket.send(JSON.stringify({ type: "join", roomId: "room-123" }));
};

socket.onmessage = (event) => {
  const message = JSON.parse(event.data);
  console.log("New message:", message);
};

socket.onclose = () => {
  console.log("Connection closed");
};

This is a simplified client example. A production application also needs
authentication, message validation, error handling, reconnection logic,
and authorization checks.

Advantages of WebSockets

Supports two-way communication over one connection.
Enables low-latency updates without repeated polling requests.
Works well for frequent, interactive updates.
Reduces repeated HTTP request overhead when messages are frequent.

Disadvantages of WebSockets

More complex to implement and operate than basic polling.
Servers must manage many long-lived connections.
Reconnection and connection state need careful handling.
Scaling across multiple application servers requires a strategy for routing and distributing messages.

Polling vs. WebSockets

Feature Polling WebSockets

Communication Client repeatedly Two-way communication
requests updates

Connection Separate HTTP requests Persistent connection

Server updates Usually returned in Server can send
response to a client messages whenever
request needed

Update latency Depends on polling Usually low when
interval connection is healthy

Request overhead Repeated requests, Less repeated request
including when nothing overhead for frequent
changes messages

Implementation Simple More complex

Best for Occasional updates Frequent, interactive
updates

WebSockets are not automatically best for every real-time feature. If
updates are rare, polling may be simpler and more cost-effective.

What Are Server-Sent Events (SSE)?

Server-Sent Events (SSE) allow a server to send updates to a client over
a long-lived HTTP connection. Communication is primarily one-way: server
to client.

Café example: The cashier can notify you whenever your order is
ready, but you cannot send messages back through that same notification
channel.

When to Use SSE

Live news feeds
Monitoring dashboards
Notifications
Streaming progress updates from a server

If the client also needs to send frequent messages, use normal HTTP
requests alongside SSE or consider WebSockets.

When Should You Use Each Approach?

Use case Suitable approach

Checking order status every 30 Short polling
seconds

Dashboard with occasional updates Polling or SSE

Live chat application WebSockets

Multiplayer online games WebSockets

Live stock price updates WebSockets or streaming technology

Collaborative document editing WebSockets

One-way live notifications SSE

Background job progress SSE, WebSockets, or polling
depending on requirements

How to Choose the Right Approach

Choose Polling When

Updates are occasional.
A small delay is acceptable.
You want a simple implementation.
The client only needs to check status periodically.

Choose Long Polling When

Updates are infrequent but should be delivered soon after they occur.
You want to avoid frequent empty polling responses.
WebSockets are not available or are not justified.

Choose WebSockets When

You need two-way communication.
Updates are frequent.
Low latency matters.
Users interact continuously, as in chat or multiplayer games.

Choose SSE When

The server needs to stream updates to the client.
The client does not need a persistent two-way channel.
You want an HTTP-based streaming mechanism.

Real-World Example: A Chat Application

Using Polling

The client asks the server every few seconds whether a new message has
arrived. This can work for a basic prototype, but it may cause
unnecessary requests and delayed message delivery.

Using WebSockets

The client establishes a WebSocket connection. When another user sends a
message, the server can push it through the active connection.

A production chat system also needs message persistence, delivery
acknowledgements, offline message handling, authentication,
reconnection, and a strategy for multiple server instances.

Important System Design Considerations for WebSockets

1. Connection Management

Track active connections and clean them up when clients disconnect. Do
not assume an open connection is always healthy.

2. Reconnection

Clients should reconnect after temporary network failures. Use
exponential backoff with jitter so many clients do not reconnect
simultaneously after an outage.

3. Heartbeats

Ping/pong messages can help detect dead or unresponsive connections and
clean up stale connection state.

4. Scaling Across Multiple Servers

A user's WebSocket connection may be held by one application server
while an event is produced on another. A shared message broker or
pub/sub system can distribute events to the server that owns the
connection.

5. Authentication and Authorization

Authenticate clients during connection setup and authorize each
subscription or message. A connected user should not automatically be
allowed to access every room or resource.

6. Load Balancing

Configure infrastructure to support long-lived connections, appropriate
timeouts, and connection limits. Sticky sessions may help in some
designs, but they do not replace a cross-server messaging strategy.

7. Message Delivery

A successful WebSocket send does not necessarily mean the recipient has
processed or permanently stored the message. Applications that need
reliable delivery require acknowledgements, message IDs, persistence,
and retry or replay logic.

Quick Interview Question

Question: You are designing a WhatsApp-like chat application. Would
you use polling or WebSockets?

Sample answer:

"I would choose WebSockets because chat requires low-latency, two-way
communication. Once the connection is established, both users can
exchange messages without repeatedly requesting updates. I would also
design for reconnection, authentication, message delivery, persistence,
and horizontal scaling."

Follow-Up: Are WebSockets Always Better Than Polling?

No. If updates are infrequent and a delay is acceptable, polling can be
simpler and cheaper to operate. The communication pattern should match
the product requirement.

Follow-Up: What If the User Goes Offline?

The server should not rely on an active WebSocket connection to store
messages. Persist messages, track delivery state, and deliver pending
messages when the user reconnects.

Key Takeaway

Short polling: The client repeatedly asks, "Any updates?"
Long polling: The client asks, and the server waits until something changes or a timeout occurs.
WebSockets: Client and server maintain an open, two-way communication channel.
SSE: The server continuously sends updates to the client over a one-way stream.

Choose based on how frequently updates happen, how quickly they must
arrive, whether communication is one-way or two-way, and the complexity
your system can support.

Monolith vs. Microservices Explained Using a Cafe Example

Khushi Patel — Thu, 29 Oct 2026 14:33:14 +0000

What Is a Monolith?

Imagine a café where one kitchen handles everything: taking orders,
preparing food, managing payments, and tracking inventory. All functions
work together as one system.

In software, a monolithic architecture is an application where
different features are built as part of one application and are
typically packaged and deployed together.

For example, an e-commerce application might include user login, product
catalog, shopping cart, payments, and orders. These features can still
be organized into separate modules; the key point is that they are
usually deployed as one application.

What Are Microservices?

Now imagine the café grows into a food court. One team manages orders,
another handles payments, another manages inventory, and another handles
delivery. Each team operates its own service and communicates with other
services when needed.

In software, microservices architecture splits an application into
smaller, independently deployable services, each responsible for a
specific business capability.

An e-commerce application could have user, product, order, payment, and
inventory services. Each service can be deployed and scaled
independently. Services often own their data and communicate through
APIs or messages, depending on the design.

Monolith vs. Microservices

Feature Monolith Microservices

Structure One application Multiple independent
services

Deployment Usually deployed Services can be
together deployed separately

Development Simpler initially More coordination and
infrastructure

Scaling Often scale the Scale individual
application as a whole services

Failure handling A failure can affect Failures can be
the whole application isolated, but not
automatically

Communication Often in-process Network calls or
function calls messages

Data management Often one shared Services own data where
database appropriate

Best fit Early products and many Systems needing
small-to-medium systems independent scaling or
team ownership

Advantages of a Monolith

1. Simpler to Build

The codebase and deployment process are often easier to understand when
starting a new product.

2. Easier Local Development and Testing

Developers can run much of the application in one place, and function
calls within the application do not need network communication.

3. Straightforward Transactions

When related data is stored in one database, transactions across related
operations can be easier to implement.

4. Lower Operational Overhead

You may need fewer deployment pipelines, monitoring configurations, and
infrastructure components.

Disadvantages of a Monolith

1. Scaling Can Be Less Targeted

If order processing receives heavy traffic but inventory does not, you
may have to scale the entire application even when only one part needs
more resources.

2. Releases Can Be Coupled

A small change in one module may require deploying the whole
application, depending on the architecture and deployment setup.

3. The Codebase Can Become Hard to Maintain

Without clear module boundaries, features can become tightly coupled and
changes can affect unexpected parts of the system.

A monolith is not automatically poorly designed. A modular monolith
with clear boundaries can remain maintainable for a long time.

Advantages of Microservices

1. Independent Scaling

If the order service receives much more traffic than the product
service, you can scale the order service without scaling every other
service.

2. Independent Deployment

Teams can release a service separately when its interfaces and
compatibility are managed carefully.

3. Clear Ownership

Teams can own specific business capabilities, code, and operational
responsibilities.

4. Technology Flexibility

Different services can use different technologies when there is a real
reason to do so. This flexibility also introduces more maintenance work.

Disadvantages of Microservices

1. More Operational Complexity

You need to manage service discovery, deployment, configuration,
observability, security, and networking across multiple services.

2. Network Failures

A service call can time out or fail even when the calling service is
healthy. Systems need timeouts, retries, and sometimes circuit breakers.

3. Data Consistency Is Harder

A business operation may involve several services and databases.
Distributed transactions are complex, so systems often use eventual
consistency, events, or sagas where appropriate.

4. Debugging Can Be Harder

A single user request may travel through several services. Distributed
tracing and centralized logs can help teams understand what happened.

5. More Infrastructure and Cost

Each service needs appropriate deployment, monitoring, scaling, and
operational support. For a small team, this overhead may outweigh the
benefits.

When Should You Use a Monolith?

A monolith is often a good choice when: - You are building an MVP or a
new product. - The team is small. - Requirements are changing
frequently. - You want simpler testing, deployment, and debugging. - You
do not yet need independent scaling for different features.

A good starting point is often a modular monolith: one deployable
application, with clear boundaries between business modules.

When Should You Use Microservices?

Microservices may be a better fit when: - Different parts of the
application have very different scaling needs. - Multiple teams need to
release services independently. - The product has clear business
boundaries. - The organization can manage distributed systems,
monitoring, networking, and failure recovery. - The benefits of
independent ownership and deployment outweigh the added complexity.

Do not choose microservices only because the company is growing or
because they sound more modern. Choose them to solve concrete problems.

How Can a Monolith Evolve into Microservices?

Imagine your café becomes extremely popular. Initially, one kitchen
handles everything. Later, online orders increase dramatically, but
inventory management remains small.

You could extract order processing into its own service so it can scale
independently. Over time, other capabilities might be separated when
there is a clear benefit.

A common migration approach is the Strangler Fig pattern: new
functionality is built outside the old system, and capabilities are
gradually moved over until the old implementation can be retired. This
should be done incrementally, with clear service boundaries and a safe
migration plan.

Example: An E-Commerce Application

Monolithic Design

A single application contains login, product catalog, cart, orders,
payments, and inventory modules. The modules may share a database and
are deployed together.

Microservices Design

Separate services manage users, products, orders, payments, and
inventory. The order service may call the payment service and
communicate with inventory through an API or message queue.

This creates independent deployment and scaling options, but also means
teams must handle network failures, duplicate messages, and consistency
between services.

Quick Interview Question

Question: If you were building a new e-commerce application today,
would you choose a monolith or microservices?

Sample answer:

"I would start with a modular monolith unless there were clear
requirements for independent deployment, scaling, or team ownership. It
keeps the initial architecture simpler and helps the team validate the
product. As the application grows, I would extract services based on
measurable bottlenecks and clear business boundaries rather than
splitting everything from day one."

Follow-Up: Can a Monolith Handle Millions of Users?

Yes, depending on the workload, architecture, infrastructure, and
optimization. A monolith can be replicated across multiple servers and
use caching, load balancing, and database scaling. User count alone does
not determine whether microservices are necessary.

Follow-Up: Are Microservices More Reliable?

Not automatically. They can isolate some failures, but they also
introduce network dependencies and more failure modes. Reliability
depends on architecture, observability, timeouts, retries, capacity
planning, and operational practices.

Key Takeaway

Monolith: One application, usually deployed as a unit.
Microservices: Multiple independently deployable services working together.
Best practice: Choose the simplest architecture that meets current needs, keep boundaries clear, and evolve the design as real requirements emerge.

Rate Limiting Explained Using a Cafe Example

Khushi Patel — Fri, 23 Oct 2026 14:04:06 +0000

What Is Rate Limiting?

Imagine a busy café where hundreds of customers arrive at the same time.
The café has limited staff and equipment, so it cannot handle unlimited
orders at once. The manager sets a rule to control how many orders can
be placed within a certain time.

Software systems use the same idea. Rate limiting controls how many
requests a user, application, or IP address can make within a specific
time period.

For example, an API might allow each user to make 100 requests per
minute. If a user exceeds the limit, the system can reject extra
requests, usually with HTTP status code 429 Too Many Requests.

How Does Rate Limiting Work?

Think of the café manager as the rate limiter.

A customer sends an order. In software, this is an API request.
The manager checks how many orders that customer has already placed during the time window.
If the customer is within the limit, the order is accepted.
If the customer has exceeded the limit, the request is rejected or delayed according to the policy.

In a real application, the rate limiter might run in an API gateway,
reverse proxy, application middleware, or dedicated service. It usually
tracks usage by a key such as a user ID, API key, IP address, or
endpoint.

Why Do We Need Rate Limiting?

1. To Protect Servers from Overload

Too many requests can exhaust CPU, memory, database connections, or
other resources. Rate limiting helps control incoming traffic so the
system has a better chance of staying responsive.

2. To Prevent Abuse

A login API without limits could be targeted by someone repeatedly
trying passwords or requesting OTPs. Rate limiting makes excessive
attempts harder. It should be combined with authentication, monitoring,
and abuse detection.

3. To Ensure Fair Usage

Imagine one café customer placing hundreds of orders while everyone else
waits. Per-user limits help prevent one client from consuming all
available capacity.

4. To Control Costs

Some operations are expensive, such as AI generation, sending SMS
messages, or calling paid third-party APIs. Rate limits can help control
usage and reduce unexpected costs.

Where Is Rate Limiting Used?

Login and OTP APIs: Limit repeated login attempts or OTP requests.
Public APIs: Keep usage within published quotas.
Search endpoints: Prevent excessive or automated searches.
Payment APIs: Protect services from excessive requests while preserving correct payment handling.
AI APIs: Limit requests or tokens to manage capacity and cost.
Social media platforms: Limit actions such as posting, commenting, or sending messages.

The limit should fit the operation. A password-reset endpoint may need a
stricter limit than a public read-only endpoint.

Common Rate Limiting Algorithms

1. Fixed Window Counter

Imagine the café resets each customer's order counter at the start of
every minute. A customer can place up to 100 orders during that minute.

The system counts requests in fixed time windows, such as 10:00--10:01
and 10:01--10:02.

Advantages: Simple to implement and efficient.

Disadvantage: Traffic can spike around a window boundary. A user
could make 100 requests just before the minute ends and another 100
immediately after it begins.

2. Sliding Window Log

The system records request timestamps. For each new request, it counts
how many requests occurred in the previous time period, such as the last
60 seconds.

Advantages: Accurate rolling-window enforcement.

Disadvantages: Storing many timestamps can use significant memory
and processing, especially for high-volume clients.

3. Sliding Window Counter

This approach combines counts from the current and previous fixed
windows, weighting the previous window according to how much of it
overlaps the rolling time period.

Advantages: A practical balance between accuracy and memory use.

Disadvantages: It estimates the rolling count rather than tracking
every request timestamp exactly.

4. Token Bucket

Imagine each customer has a bucket containing tokens. Every request
costs one token, and tokens refill at a steady rate. If a token is
available, the request is allowed. If the bucket is empty, the request
is rejected or delayed.

A bucket with a capacity of 10 tokens might allow a burst of 10
requests, while refilling at 2 tokens per second.

Advantages: Allows controlled bursts while limiting the average
request rate.

Disadvantages: Requires choosing the right bucket capacity and
refill rate.

5. Leaky Bucket

Imagine orders entering a container and leaving at a steady pace. If
orders arrive faster than they can be processed, they wait in the
container. If it fills up, new orders are rejected. Depending on
implementation, a leaky bucket may queue requests or reject excess
requests.

Advantages: Smooths incoming traffic and helps control output rate.

Disadvantages: A queue can add latency, and excess requests still
need a defined policy.

Which Algorithm Should You Choose?

Requirement Possible choice

Simple quota with low implementation complexity Fixed Window
Accurate rolling time limit Sliding Window Log
Efficient rolling-window approximation Sliding Window Counter
Allow short bursts while controlling average rate Token Bucket
Smooth traffic into a steady rate Leaky Bucket

There is no single best algorithm for every system. Choose based on
whether bursts should be allowed, how accurate the limit must be,
available storage, and the traffic pattern.

How to Implement Rate Limiting in a Real System

Step 1: Define the Limit

Decide what is being limited and the allowed rate. For example, 100
requests per minute per authenticated user for a search endpoint.

Step 2: Choose the Rate-Limit Key

Common keys include user ID, API key, IP address, or a combination of
user and endpoint. IP-only limits can affect multiple legitimate users
behind the same network, while user-only limits may not protect an
unauthenticated endpoint.

Step 3: Choose Where the Limiter Runs

An API gateway or reverse proxy can apply limits before requests reach
application servers. Application middleware can enforce rules close to
the endpoint. A shared service or data store can coordinate limits
across multiple application instances.

Step 4: Store and Update Usage Safely

If the application runs on multiple servers, a local in-memory counter
on each server may allow a user to exceed the intended global limit. A
shared store such as Redis can help coordinate counters, but updates
must be atomic and the design should account for latency and store
failures.

Step 5: Decide What Happens When the Limit Is Reached

Common policies include rejecting the request, delaying it, or placing
eligible work in a queue. For HTTP APIs, return 429 Too Many Requests.
If useful, include a Retry-After header to indicate when the client
can try again.

Step 6: Monitor and Tune

Track rejected requests, traffic patterns, latency, and usage by
endpoint or client. Adjust limits based on real capacity and legitimate
user behavior.

Example: An API with a Limit of 5 Requests per Minute

Suppose a user can make five search requests per minute.

Requests 1--5: allowed.
Request 6 during the same applicable window: rejected with 429 Too Many Requests.
After enough time has passed for the algorithm's window or tokens to reset or refill, requests can be allowed again.

The exact reset behavior depends on the chosen algorithm. A token bucket
refills gradually rather than necessarily resetting all at once.

Important Design Considerations

Rate Limiting vs. Load Shedding

Rate limiting enforces usage rules, often per user or client. Load
shedding rejects selected work when a system is overloaded to help
protect overall health. A production system may use both.

Rate Limiting vs. Throttling

These terms are sometimes used interchangeably. Rate limiting defines or
enforces a request rate; throttling can also mean slowing down or
delaying requests rather than immediately rejecting them.

Distributed Systems

When requests are served by several application instances, the system
needs a consistent way to apply the limit. A shared data store or
gateway can help, but it introduces its own availability and latency
trade-offs.

Client Behavior

Clients should handle 429 responses sensibly. They can wait before
retrying, respect Retry-After when provided, and use exponential
backoff with jitter where appropriate. Retrying immediately can make
overload worse.

Rate Limits Are Not a Complete Security Solution

Rate limiting helps reduce abuse, but it does not replace
authentication, authorization, fraud detection, input validation, or
monitoring.

Quick Interview Question

Question: An API allows 100 requests per minute. A user sends 100
requests in the first 10 seconds and then sends one more request during
that minute. Should the 101st request be allowed?

Answer: Under a fixed-window limit of 100 requests for that minute,
the 101st request should be rejected. However, the answer can differ
with a token bucket or another policy, so state the algorithm and
time-window behavior.

Follow-up: How would you enforce the limit when the application runs
on 10 servers?

A strong answer mentions a shared rate-limit store or gateway, atomic
updates, a clear key such as user ID or API key, and a plan for what
happens if the shared store is unavailable.

Key Takeaway

Rate limiting is like a café manager controlling how many orders each
customer can place in a given time. It protects system capacity,
supports fair usage, reduces abuse, and controls costs. The right design
depends on the required limit, whether bursts are allowed, and how
requests are distributed across servers.

Message Queues Explained Using a Cafe Example

Khushi Patel — Mon, 19 Oct 2026 13:07:20 +0000

What Is a Message Queue?

Imagine a busy café during lunch time. Many customers place orders at
once, but the café has only four chefs. If the cashier had to wait for a
chef to finish every order before accepting the next one, the line would
move very slowly.

Instead, the cashier writes each order on a ticket and puts it on a
rack. The chefs pick up tickets when they are ready to cook. The cashier
can keep accepting orders while the chefs work through the queue.

That ticket rack is like a message queue in software: it temporarily
stores tasks or messages so the sender and receiver do not need to work
at exactly the same time.

Map the Café to Software

Café Software

Customer's order Message or task
Cashier who creates the ticket Producer
Ticket rack Message queue
Chef who picks up a ticket Consumer or worker
Preparing the order Processing the message
Finished order Successfully processed message

How Does a Message Queue Work?

A producer creates a message, such as "send an order confirmation email."
The producer sends the message to the queue.
The queue stores it until a consumer is ready.
A consumer receives the message and performs the task.
After successful processing, the consumer acknowledges the message so the queue knows it has been handled. The exact acknowledgement and deletion behavior depends on the queue technology and configuration.

The important point is that the producer usually does not have to wait
for the full task to finish.

When Should You Use a Message Queue?

1. When a Task Can Happen in the Background

After a customer places an order, the website should confirm the order
quickly. Sending an email, generating an invoice, and notifying the
warehouse can happen in the background.

Without a queue: The customer request may wait for every task to
finish.

With a queue: The order service saves the order and schedules
follow-up tasks. Background workers process those tasks separately.

Use this when the user does not need to wait for the entire task to
complete.

2. When Traffic Arrives in Sudden Bursts

Imagine 200 orders arrive in one minute, but your workers can process
only 80 orders per minute.

The queue can temporarily hold the extra work. Workers keep processing
at their available rate, and the backlog shrinks when incoming traffic
slows.

A queue buffers traffic; it does not magically increase processing
capacity. If messages arrive faster than they can be processed for a
long time, the backlog will continue growing. You may need more workers,
faster processing, rate limits, or a capacity change.

3. When Two Services Should Not Depend on Each Other Being Available at the Same Time

Suppose your order service is working, but your email service is
temporarily down. A queue can retain email tasks so they can be retried
when the email service recovers, depending on your queue setup.

This reduces the need for the order service to wait for every downstream
service to be available.

4. When You Need Retries for Temporary Failures

A worker might fail because of a temporary network problem or an
unavailable external API. A queue system can support retrying the
message.

Retries should usually have a limit and a delay strategy. After repeated
failures, the message can be moved to a dead-letter queue for
investigation instead of retrying forever.

5. When You Need to Process Many Independent Jobs

Examples include resizing uploaded images, generating reports,
processing payments through a carefully designed workflow, importing
files, sending notifications, or running data pipelines.

Queues are useful when work can be split into independent jobs and
processed by one or more workers.

When Might You Not Need a Message Queue?

The result is needed immediately. If a user asks for their current account balance, the application usually needs to fetch the current result synchronously.
The operation is simple and fast. Adding a queue can create extra infrastructure and complexity without much benefit.
You need an immediate, definitive response. A queue often means the request is accepted for later processing, not that the work has already succeeded.
You cannot safely handle duplicate processing. Many queue systems can deliver a message more than once. Design consumers to be idempotent, meaning processing the same message again does not create an unwanted duplicate effect.

A system can combine both approaches: respond synchronously for the
critical result, then queue non-critical follow-up tasks.

How to Use a Message Queue in a Real Application

Step 1: Identify the Work That Can Be Done Later

Start with a task that does not need to finish before the user receives
the main response. For example: send a confirmation email after an order
is saved.

Step 2: Choose a Queue Technology

Common options include:

RabbitMQ: Often used for task queues and routing messages between services.
Apache Kafka: Often used for high-volume event streams, durable event logs, and consumers that may need to read or replay events.
Cloud-managed queues: Services such as Amazon SQS or Google Cloud Pub/Sub can reduce the work of managing queue infrastructure.

Choose based on your delivery, ordering, throughput, replay, hosting,
and operational requirements rather than popularity alone.

Step 3: Create a Clear Message Format

Keep messages structured and versionable. For example:

{
  "eventType": "OrderPlaced",
  "orderId": "ORD-1042",
  "createdAt": "2026-10-19T12:00:00Z"
}

Avoid putting unnecessary personal or sensitive data in messages. Often,
an identifier such as orderId is enough for a worker to retrieve the
required details securely.

Step 4: Publish the Message

After the order is saved, the order service publishes an OrderPlaced
message. In production systems, think carefully about what happens if
the database save succeeds but publishing the message fails. A common
solution is the transactional outbox pattern, which records the
event in the same database transaction and publishes it reliably
afterward.

Step 5: Build a Consumer or Worker

A separate worker subscribes to or polls the queue, reads the message,
performs the task, and acknowledges success according to the
technology's rules.

For example, an email worker reads the order ID, fetches the order
details, and sends the confirmation email.

Step 6: Make Processing Safe to Retry

A worker can finish its task but fail before its acknowledgement reaches
the queue. The message may then be delivered again.

Use idempotency where possible. For example, store a unique task or
event ID and check whether the email or operation has already been
completed before repeating it. For payments and other critical
operations, use the payment provider's idempotency mechanisms and
carefully designed state transitions.

Step 7: Monitor the Queue

Track: - Number of messages waiting - Age of the oldest message -
Processing rate and failure rate - Retry count and dead-letter queue
size - Consumer health and processing latency

A growing queue can mean traffic has increased, workers are failing, or
processing has become too slow.

Important Concepts to Know

Producer

The component that creates and sends messages. In our café, it is the
cashier placing order tickets on the rack.

Consumer

The component that receives and processes messages. In our café, it is a
chef picking up a ticket.

Acknowledgement

A signal that a message has been processed successfully. The queue's
acknowledgement rules determine when a message can be considered
complete.

Retry

An attempt to process a message again after a temporary failure. Use
bounded retries and appropriate delays.

Dead-Letter Queue

A separate destination for messages that repeatedly fail, allowing
investigation without blocking the normal flow.

Idempotency

The ability to process the same message more than once without causing
the same real-world effect multiple times. This matters because many
systems provide at-least-once delivery, not a guarantee that each
message is delivered exactly once.

Ordering

Some applications require messages to be processed in order, such as
updates to the same account. Ordering guarantees depend on the queue
technology and configuration and may reduce parallelism.

Example: An Online Shopping Order

A customer clicks Place Order.

The order service validates and saves the order.
It publishes an OrderPlaced event.
Separate consumers handle tasks such as sending an email, creating an invoice, notifying the warehouse, and updating analytics.
If the email service is temporarily unavailable, the email task can be retried without requiring the customer to submit the order again.
The system monitors pending messages and failures so problems are visible.

For critical workflows, do not assume that sending a message alone
completes a payment or guarantees delivery. Track each step's status and
design recovery behavior.

Quick Interview Question

Question: Your application receives 200 jobs per minute, but workers
can process only 80 jobs per minute. What happens when you add a message
queue?

Answer: The queue stores the jobs that workers cannot process
immediately, so the application can absorb a short burst. The workers
still process only about 80 jobs per minute. If the incoming rate stays
at 200 per minute, the backlog grows by roughly 120 jobs per minute
until capacity increases or incoming traffic falls.

Follow-up: What would you do if the queue keeps growing?

Possible answers: add more consumers if the work can be parallelized,
optimize slow processing, apply rate limits or backpressure, and
investigate errors or downstream bottlenecks.

Key Takeaway

A message queue is like the café's ticket rack. The producer adds work,
the queue holds it, and consumers process it when they are ready. Use
queues for background work, traffic bursts, service decoupling, and
retryable jobs---but plan for failures, duplicate delivery, monitoring,
and growing backlogs.

CAP Theorem Explained Through a Cafe Example

Khushi Patel — Fri, 16 Oct 2026 12:42:43 +0000

Imagine you own a cafe with two branches. Both branches sell chocolate cakes and share their stock information.

One day, the internet connection between the two branches stops working.

Branch A sells the last chocolate cake, but Branch B does not know about it. Branch B still thinks one cake is available.

This situation helps us understand CAP theorem.

1. C — Consistency

Both branches should have correct, up-to-date information.

Branch A sold the last cake, so Branch B should not promise that same cake to another customer.

In simple words: Don't show incorrect or outdated data when strong consistency is required.

2. A — Availability

The café should keep serving customers.

Even though Branch B cannot contact Branch A, it continues accepting orders using the information it has. However, it might accept an order for a cake that is already sold out.

In simple words: Keep responding to customers, even when the latest information is unavailable.

3. P — Partition Tolerance

The café should handle a broken connection between its branches.

Even if Branch A and Branch B cannot communicate, the system must deal with the network failure.

In simple words: Handle communication failures between servers.

4. What Does CAP Theorem Mean?

When the network connection breaks, the café faces a choice.

Option 1: CP — Consistency + Partition Tolerance

Branch B says, "I can't verify the stock, so I'll wait before confirming this order."

The café protects consistency.
Some customers must wait or receive an error.
Availability is sacrificed for affected requests.

Option 2: AP — Availability + Partition Tolerance

Branch B says, "I'll keep accepting orders even though I can't check Branch A's latest stock."

Customers continue getting responses.
Some information might be outdated or conflicting.
The branches can reconcile their data when communication returns.

5. What About CA?

Consistency and availability can coexist when there is no network partition and the system's design allows it.

But when a partition occurs, a distributed system cannot guarantee both consistency and availability for every affected request.

6. Quick Comparison

Property	Café example	Simple meaning
Consistency	Both branches know the correct cake stock	Keep data correct
Availability	Branch B continues accepting requests	Keep responding
Partition tolerance	Branches handle a broken connection	Handle network failures

7. Interview Question

The internet connection between two café branches is down. Branch A may have sold the last chocolate cake. What should Branch B do if the café must prevent selling the same cake twice?

A. Accept the order immediately using old stock information.
B. Wait or reject the order until stock can be verified safely.
C. Ignore the inventory records.

Answer

Option B is the safest choice if strong consistency is required. Branch B may have to delay or reject the affected request during the network partition.

Key Takeaway

C — Consistency: Keep the data correct.
A — Availability: Keep the service responding.
P — Partition tolerance: Handle a broken connection.

CAP theorem: During a network partition, a distributed system cannot guarantee both consistency and availability for every request. It must make a trade-off for affected operations.

Indexing and Replication Through a Restaurant Example

Khushi Patel — Thu, 15 Oct 2026 07:22:29 +0000

Let's continue our café story, but now imagine the café has become a popular restaurant with thousands of customers, hundreds of menu items, and a large database of orders.

The restaurant faces two problems:

Finding a customer's order quickly takes too much time.
If the main database goes down, the restaurant may stop serving customers.

That's where indexing and replication help.

1. What Is Indexing?

Imagine the restaurant keeps thousands of order records in a large register. When a customer asks for order number 5821, the staff must scan page after page to find it.

This takes time.

Now imagine the register has an index at the back that says:

Order 5821 → Page 240
Order 5822 → Page 241
Order 5823 → Page 242

The staff can jump directly to the right page instead of scanning the entire register.

That is the idea behind database indexing.

In system design

A database index is a data structure that helps the database find rows faster without scanning every row in a table.

For example, a database table might contain millions of orders. An index on order_id can help the database locate a particular order efficiently.

SQL example

CREATE INDEX idx_orders_order_id
ON orders(order_id);

Then a query such as this may use the index:

SELECT *
FROM orders
WHERE order_id = 5821;

The database's query planner decides whether using the index is the best option.

When should you use indexing?

When a column is frequently used to search for records.
When columns are often used to join tables.
When queries frequently filter or sort by certain columns and an appropriate index can help.

What is the downside of indexing?

Imagine the restaurant adds a detailed index for every possible thing in its register. Finding records may become faster, but updating the register becomes more work because the indexes also need updating.

Database indexes have similar trade-offs:

They consume additional storage.
Inserts, updates, and deletes can become slower because indexes must be maintained.
Too many or poorly chosen indexes can hurt performance.

Remember: An index helps you find data faster; it does not automatically make every query faster.

2. What Is Replication?

Now imagine the restaurant's entire order register is kept in one room. If that room becomes inaccessible, staff may not be able to retrieve order information.

To reduce this risk, the restaurant keeps copies of the register in two other rooms.

If the main register becomes unavailable, a copy may still be available.

This is the idea behind database replication.

In system design

Replication means keeping copies of data on multiple database servers or nodes.

A common setup has:

Primary database: Usually accepts writes.
Replica databases: Receive copies of the primary's data and may serve read requests.

How replication works

A customer places an order.
The application writes the order to the primary database.
The change is sent to one or more replicas.
Read requests may be served by the primary or by suitable replicas, depending on the system's design.

Why use replication?

Availability: A replica may help keep the application running if the primary fails.
Read scalability: Read traffic can be distributed across replicas.
Disaster recovery: Copies of data can help recover from some failures, depending on how replication and backups are configured.

What is the downside of replication?

Copies do not always update at exactly the same time.

Imagine the primary register records an order at 12:01, but a replica has not received the new order yet. A staff member checking that replica might temporarily see old information.

This is called replication lag.

Replication also adds operational complexity. Failover, consistency, and replica health must be handled carefully. Replication alone is not a replacement for backups.

3. Types of Replication

A. Primary-Replica Replication

The primary database handles writes, and replicas receive copies of the data. Reads may be distributed to replicas when appropriate.

Restaurant example: The head cashier records new orders in the main register, while other service counters use copies to look up existing orders.

Best used when: The application has many reads and needs to scale read traffic or improve availability.

B. Synchronous Replication

The system waits for confirmation from the required replica or replicas before considering a write complete, according to the configured acknowledgement rules.

Restaurant example: The cashier waits until the required backup register confirms that the new order has been copied before confirming completion.

Trade-off: Stronger guarantees for acknowledged copies can increase write latency and may reduce write availability during network or replica problems.

C. Asynchronous Replication

The primary confirms a write without waiting for replicas to finish copying it. Replicas catch up afterward.

Restaurant example: The cashier confirms the order immediately, and staff copy it into the other registers moments later.

Trade-off: It can make writes faster, but a replica may temporarily return stale data. If the primary fails before changes are replicated, some recent writes may be lost during failover.

4. Indexing vs. Replication

Feature	Indexing	Replication
Main purpose	Find data faster	Keep copies of data on multiple nodes
Restaurant analogy	An index pointing to the right page	Copies of the order register
Helps with	Query performance	Read scaling, availability, and recovery options
Main cost	Extra storage and slower writes due to index maintenance	Extra storage, network traffic, and operational complexity
Does it copy all data to another server?	No	Yes, in a typical full-data replication setup

5. Can We Use Both Together?

Yes! Indexing and replication solve different problems, so they can be used together.

Imagine the restaurant has one primary database and two read replicas. Each database has a useful index on order_id.

Replication provides copies of the data and allows some reads to be distributed.
Indexing helps each database find a particular order efficiently.

Together, they can improve read performance and help the system handle more traffic. Indexes still need to be maintained on replicas, and replication lag must be considered.

6. Interview Question

Your restaurant has 10 million orders. Searching for an order by order_id is slow, and the database is receiving too many read requests.

What would be a sensible approach?

A. Add an index on order_id and consider read replicas for read traffic.
B. Create more indexes on every column without checking query patterns.
C. Replicate the database but leave the slow query unchanged.
D. Remove the database indexes and add more servers.

Answer

Option A is the sensible starting point.

An appropriate index can help speed up searches by order_id. Read replicas can help distribute read traffic when the application can tolerate their consistency characteristics. Check query plans and measure performance before and after making changes.

Key Takeaway

Think of the restaurant's order register:

Indexing is like a register's index that helps you find the right page quickly.
Replication is like keeping copies of the register so multiple service counters can read data and the system has alternatives if one database fails.

Indexing improves how data is found. Replication creates additional copies of data. They address different problems and are often used together in real-world systems.

Understanding Caching Through a Cafe Example

Khushi Patel — Mon, 12 Oct 2026 14:04:10 +0000

Imagine you own a busy café. Every day, customers order popular items like masala chai, cold coffee, and sandwiches.

If the chef has to collect every ingredient from the storeroom and prepare everything from scratch for every order, service will be slow. But if the café keeps popular ingredients ready near the counter, the chef can work faster.

That is the basic idea behind caching in system design.

1. What Is Caching?

Caching means storing frequently accessed data in a temporary, fast storage location so future requests can be served faster.

Café analogy

Storeroom: Database
Ready-to-use ingredients near the counter: Cache
Chef: Application server
Customer order: User request

2. How Does Caching Work?

Suppose a customer asks for cold coffee. The café first checks whether the needed information is already available nearby.

Cache hit

The required data is found in the cache, so the application can return it quickly without querying the database.

Cache miss

The data is not in the cache. The application fetches it from the database and commonly stores it in the cache for future requests.

3. Types of Caching

A. In-Memory Cache

Imagine keeping popular ingredients beside the chef so they can be accessed almost immediately.

In system design, data is stored in RAM, which is typically much faster to access than data stored on disk.

Examples include Redis and Memcached.

Use it when: You need fast access to frequently requested data, such as product details, user sessions, or popular posts.

B. Browser Cache

Imagine a regular customer has a copy of the café menu saved on their phone, so they do not need to request the same menu every time.

In system design, the browser stores resources such as images, CSS, and JavaScript files according to caching rules.

Use it when: You want repeat visitors to load websites faster and reduce unnecessary network requests.

C. CDN Cache

Imagine your café has branches in several cities. Instead of sending every customer to the main café for the menu, each branch keeps a copy of popular menu items.

In system design, a Content Delivery Network (CDN) stores cached content at locations closer to users.

Use it when: You need to serve static content such as images, videos, CSS, and JavaScript to users in different geographical locations.

D. Database Cache

Imagine the café keeps a prepared list of popular orders so the chef does not need to check the main records for every request.

In system design, frequently accessed query results or database pages may be cached to avoid repeated expensive reads. Databases can have internal caches, and applications can also cache query results.

Use it when: The same data is read repeatedly and fetching it from the database is expensive.

4. What Is Cache Expiration (TTL)?

Suppose the café prepares fresh chutney in the morning. It should not keep serving the same chutney indefinitely.

Caching works similarly.

TTL (Time To Live) defines how long a cached item can remain valid before it expires.

Examples:

Café menu data: cache for one hour.
Product prices: use a shorter TTL if prices change frequently.
Static logo: cache for a long period if its URL changes whenever the logo is updated.

The correct TTL depends on how frequently data changes and how much stale data the application can tolerate.

5. The Biggest Challenge: Stale Data

Imagine the café changes the price of cold coffee from ₹100 to ₹120, but the cached menu still shows ₹100.

This is called stale cache data.

Common ways to handle it

TTL: Let cached data expire after a defined duration.
Cache invalidation: Remove or update cached data when the original data changes.
Write-through: Update the cache and underlying data store as part of the write path.
Write-behind: Update the cache first and persist the change to the database later, with appropriate reliability safeguards.

6. What Is Cache Eviction?

Suppose your café counter has room for only 20 ingredient containers. When it becomes full, you must decide which container to remove to make room for a new one.

A cache also has limited memory, so it may need to evict existing entries.

Common eviction strategies

Strategy	Café example
LRU (Least Recently Used)	Remove the ingredient that has not been used for the longest time.
LFU (Least Frequently Used)	Remove the ingredient used the fewest times.
FIFO (First In, First Out)	Remove the ingredient that was added first.

7. When Should You Use Caching?

Scenario	Cache benefit
Popular products	Faster product page loads
Frequently viewed profiles	Fewer database reads
Static images and files	Faster delivery through browser or CDN caches
Repeated expensive calculations	Avoid recalculating the same result
Data that changes constantly	Caching may need short TTLs or careful invalidation

Caching is not always the right choice. It adds complexity, consumes memory, and can return outdated data if freshness is not managed correctly.

8. Interview Question

Your café receives 1,000 requests for its menu every minute. The menu rarely changes, but every request currently queries the database.

What would you do?

A. Increase database queries
B. Cache the menu and set an appropriate expiration or invalidation strategy
C. Delete the database
D. Restart the server after every request

Answer

Option B: Cache the menu and set an appropriate expiration or invalidation strategy.

Caching lets repeated requests be served quickly and reduces database reads. Update or invalidate the cached copy when the menu changes.

Key Takeaway

Think of caching as keeping popular café items ready instead of fetching everything from the storeroom for every order.

In system design, caching helps reduce latency, database load, and repeated computation. The important part is deciding what to cache, where to cache it, how long to keep it, and how to keep it fresh.

Understanding Load Balancing Through a Cafe Example

Khushi Patel — Sat, 10 Oct 2026 15:11:46 +0000

Imagine a busy café with one master chef and four other chefs. Customers keep placing orders. If the master chef prepares every dish alone, customers will wait too long.

Instead, the master chef distributes orders among the four chefs. In system design, this is what a load balancer does: it receives incoming requests and decides which server should handle each one.

The Café Team

Master chef: Load balancer
Four chefs: Servers
Customer orders: Incoming requests
Preparing dishes: Processing requests

1. Round Robin

The master chef assigns orders to chefs one by one in a repeating order.

Café example

Order 1 goes to Chef 1.
Order 2 goes to Chef 2.
Order 3 goes to Chef 3.
Order 4 goes to Chef 4.
Order 5 goes back to Chef 1.

In system design

Requests are distributed sequentially across servers.

Best used when

Servers have similar capacity and requests take roughly the same amount of time.

Limitation

Round Robin does not check whether a chef is already busy. A chef working on a complicated dish may receive another order even when another chef is free.

2. Weighted Round Robin

Now imagine Chef 1 is more experienced and can prepare twice as many dishes as the other chefs. The master chef gives Chef 1 more orders.

Café example

A possible distribution is:

Chef 1 receives 2 orders.
Chef 2 receives 1 order.
Chef 3 receives 1 order.
Chef 4 receives 1 order.

The exact distribution depends on the configured weights.

In system design

Servers receive requests according to assigned weights. A server with a higher weight generally receives a larger share of requests.

Best used when

Servers have different processing capacities.

Limitation

Weights do not necessarily reflect how busy a server is at that moment.

3. Least Connections

Imagine Chef 1 is preparing three dishes, Chef 2 is preparing one, and Chefs 3 and 4 are free. The master chef sends the next order to a chef with the fewest active orders.

Café example

The next order could go to Chef 3 or Chef 4 because both have no active orders.

In system design

The load balancer selects a server with the fewest active connections.

Best used when

Connections or requests can remain active for different lengths of time.

Limitation

The number of active connections does not always represent the real workload. One request may require much more processing than another.

4. Weighted Least Connections

Now combine the previous two ideas. Chef 1 is twice as capable as the other chefs, but the master chef also checks how many active orders each chef has.

The master chef considers both capacity and current connections when choosing a chef.

In system design

The algorithm uses server weights along with active connection counts to help distribute requests.

Best used when

Servers have different capacities and workloads vary.

Limitation

It still relies on connection counts and weights, which may not perfectly represent actual processing effort.

5. Least Response Time

Imagine Chef 1 has only one order but is taking a long time to finish it. Chef 2 has two orders but usually finishes dishes quickly. The master chef considers how quickly each chef is responding and sends the next order to a suitable, faster option.

In system design

Requests are routed based on response time, often combined with active connections. The exact calculation depends on the load balancer.

Best used when

Fast responses and low latency are important.

Limitation

Response times can fluctuate, so a single slow or fast response may not represent long-term performance.

6. IP Hash

Imagine the master chef uses each customer's membership number to consistently assign them to a particular chef. Customer A may usually be assigned to Chef 1, while Customer B may usually be assigned to Chef 3.

In system design

The load balancer hashes the client's IP address to determine which server should receive the request.

Best used when

Consistent client-to-server routing is useful, such as in some session-persistence setups.

Limitation

Traffic can become uneven, and adding or removing servers can change which server a client reaches. IP-based routing may also be less reliable when many users share an IP address.

Quick Comparison

Algorithm	Café rule	Best use
Round Robin	Take turns	Similar servers and requests
Weighted Round Robin	Stronger chef gets more orders	Different server capacities
Least Connections	Choose a chef with fewer active connections	Variable-duration connections
Weighted Least Connections	Consider capacity and active connections	Different capacities and workloads
Least Response Time	Prefer faster-responding chefs	Latency-sensitive applications
IP Hash	Assign clients consistently	Session persistence in suitable setups

Interview Question

Your café has four chefs. Chef 1 is powerful but currently busy. Chef 2 is less powerful but completely free. Which load-balancing algorithm could be a good fit?

A. Round Robin
B. Weighted Round Robin
C. Least Connections
D. IP Hash

Answer

Weighted Least Connections is usually the best fit when both server capacity and current connections matter. Least Connections alone may choose Chef 2 because they have fewer active connections, but it does not account for the difference in capacity.

Key Takeaway

The master chef is the load balancer.
The four chefs are servers.
Customer orders are incoming requests.
The load-balancing algorithm decides which server receives each request.

There is no single best algorithm for every application. Choose based on server capacity, request duration, response time, and whether consistent client routing is needed.

System Design for Beginners: Vertical Scaling vs Horizontal Scaling

Khushi Patel — Thu, 08 Oct 2026 14:22:59 +0000

If you are preparing for software engineering interviews, you have probably heard the term System Design many times.

But what does it actually mean?

System design is not just about drawing boxes and arrows. It is about deciding how different parts of a software system should work together as the number of users, requests and data grows.

For example, building a simple Instagram clone for 100 users is easy. But what happens when millions of users are uploading photos, liking posts and refreshing their feeds at the same time?

That is where system design becomes important.

System design is the process of designing the architecture of a system so it can handle users, traffic, data, failures and future growth.

One of the first concepts you should understand is scaling.

What Is Scaling?

Suppose your application is running on one server.

Users
  |
  v
Server
  |
  v
Database

Everything works fine when you have a small number of users.

But as users increase, your server may start running out of CPU, memory or other resources.

So you need to scale the system.

There are two common ways to do this:

Vertical Scaling
Horizontal Scaling

What Is Vertical Scaling?

Vertical scaling means making your existing server more powerful.

For example, your application currently runs on:

CPU: 4 cores
RAM: 8 GB

You upgrade it to:

CPU: 16 cores
RAM: 64 GB

You did not add another server. You simply made the existing server stronger.

Vertical scaling = Scale up by increasing the resources of one machine.

A simple real life example is upgrading your laptop by adding more RAM or getting a faster processor.

When should you use vertical scaling?

It can be a good choice when:

Your application is small
Traffic is predictable
You need a quick and simple solution
Your workload is difficult to distribute

For example, if an internal company dashboard is used by only 50 people, adding 10 servers would probably be unnecessary.

The Problem With Vertical Scaling

The biggest problem is that there is a limit to how powerful one machine can become.

There is also a single point of failure.

Users
  |
  v
One Server

If that server goes down, your entire application can go down.

This is one reason we often consider horizontal scaling.

What Is Horizontal Scaling?

Horizontal scaling means adding more servers instead of making one server bigger.

For example:

                 Server 1
                /
Users → Load Balancer → Server 2
                \
                 Server 3

Now the traffic is distributed across multiple servers.

Horizontal scaling = Scale out by adding more machines.

Think about a restaurant.

If one cashier is not enough during lunch time, you don't necessarily make that cashier work five times faster. You open more counters.

That is basically horizontal scaling.

What Does a Load Balancer Do?

When you have multiple servers, something needs to decide which server should handle each request.

That's the job of a load balancer.

For example:

User 1 → Server 1
User 2 → Server 2
User 3 → Server 3

Some common load balancing strategies include:

Round Robin
Least Connections
IP Hash

The goal is to distribute traffic so that one server does not become overloaded.

Vertical Scaling vs Horizontal Scaling

Vertical Scaling	Horizontal Scaling
Makes one server more powerful	Adds more servers
Also called scaling up	Also called scaling out
Simpler to implement	More complex
Has hardware limitations	Can scale much further
Can have a single point of failure	Better availability
Good for smaller workloads	Good for large and growing workloads

The easiest way to remember it:

Vertical = Make the machine bigger

Horizontal = Add more machines

Which One Should You Use?

The answer is not always "horizontal scaling".

It depends on the problem.

Example 1: Small Application

Imagine you have an internal dashboard with 100 users.

Your server is using only 30% CPU.

Do you need five servers?

Probably not.

Vertical scaling may be simpler and cheaper.

Example 2: Large E-commerce Platform

Now imagine an e-commerce website normally receives:

100,000 requests/minute

During a big sale, traffic suddenly increases to:

2,000,000 requests/minute

One server may not be enough.

Horizontal scaling makes more sense:

                 Server 1
                /
Users → Load Balancer → Server 2
                \
                 Server 3
                    |
                  Server N

You can add more servers as traffic increases.

One Important Problem With Horizontal Scaling

Horizontal scaling sounds great, but it introduces new challenges.

Suppose a user's session is stored only on Server 1.

User → Server 1

Session stored on Server 1

The next request goes to Server 2.

User → Server 2

Server 2 does not have the session information.

This is why horizontally scaled applications often use:

Stateless servers
Shared session storage
Distributed caches such as Redis

For example:

              Server 1
             /
Users → Load Balancer
             \
              Server 2
                   |
                   v
                 Redis

Now multiple servers can access shared information.

A Common Interview Question

Interviewer:

Your application is running on one server and CPU usage is constantly around 95%. What would you do?

A beginner might immediately say:

"Add another server."

But a better answer is:

"First, I would identify the bottleneck. If the application is CPU bound and a larger machine can handle the workload, vertical scaling could be a quick solution. If traffic is growing and we need better availability and long term scalability, I would consider horizontal scaling."

This shows that you are thinking about requirements and tradeoffs, rather than blindly choosing a solution.

Another Tricky Interview Question

Interviewer:

You have 10 servers, but one server is receiving most of the traffic. You are already using a load balancer. Why could this happen?

Possible reasons include:

The load balancing strategy is not distributing requests evenly
Sticky sessions are being used
Some requests are much heavier than others
Traffic itself is uneven

The important lesson is:

Having multiple servers does not automatically mean your traffic is properly distributed.

The System Design Mindset

When learning system design, don't memorize:

"Horizontal scaling is always better."

Instead, ask:

What is the bottleneck?
How much traffic do we have?
How fast is traffic growing?
Do we need high availability?
Can the workload be distributed?
What will the solution cost?
What happens if a server fails?

System design is about understanding tradeoffs and choosing the right solution for the problem.

Quick Recap

System Design: Designing the architecture of a system so it can handle users, traffic, data, failures and growth.

Vertical Scaling: Make one machine more powerful.

Small Server → Bigger Server

Horizontal Scaling: Add more machines.

1 Server → 5 Servers → 50 Servers

Use vertical scaling when simplicity and a smaller workload are more important.

Use horizontal scaling when traffic is large, growing or requires better availability.

And remember:

Vertical scaling makes one machine stronger. Horizontal scaling makes the system wider.

The goal of system design is not to use the most complicated architecture.

It is to choose the right architecture for the problem you are solving.

learning Generative AI/LLM through practical projects

Khushi Patel — Tue, 01 Sep 2026 13:23:23 +0000

If you want to become good at Generative AI/LLM engineering, watching tutorials is not enough.

The fastest way to understand these technologies is to build projects where you are forced to solve real problems: prompting, context management, retrieval, model adaptation, and evaluation.

Here is a practical project roadmap:

LLM → RAG → Fine-Tuning → Evals

1. Learn LLMs by Building a Lead Triaging Bot

🎯 Project: Lead Triaging Bot

Build an AI system that receives a new customer lead and decides:

Is this a high-quality lead?
What is the customer's intent?
Which product/service are they interested in?
How urgent is the lead?
What should the sales team do next?

Example

Input:

"Hi, I'm looking for an enterprise plan for 200 employees. We need SSO and would like to schedule a demo next week."

Output:

Lead Quality: High
Intent: Enterprise Purchase
Company Size: 200 employees
Urgency: High
Recommended Action: Schedule Demo

What you'll learn

This project gives you a strong foundation in:

LLM APIs
Prompt engineering
System prompts
Structured outputs
Few-shot prompting
JSON responses
Function/tool calling
Temperature and model parameters
Error handling
LLM application architecture

Why this project?

Before jumping into RAG or fine-tuning, you should understand how an LLM behaves without external knowledge or model customization.

This gives you the baseline against which you can later compare RAG and fine-tuning.

2. Learn RAG by Building a Company Knowledge Assistant

🎯 Project: Company Knowledge Assistant

Now take the same LLM and give it access to your own knowledge base.

Upload documents such as:

Company Policies
Product Documentation
HR Policies
FAQs
Technical Documentation
Pricing Documents

Users should be able to ask questions about these documents.

Example

User:

"What is our work-from-home policy?"

RAG pipeline:

User Question
      ↓
Query Embedding
      ↓
Vector Database
      ↓
Retrieve Relevant Documents
      ↓
Context + Question
      ↓
LLM
      ↓
Grounded Answer

RAG works by retrieving relevant information from an external data source and providing that information to the LLM as context. (GitHub)

What you'll learn

Build the project in stages:

Level 1 — Basic RAG

Document loading
Chunking
Embeddings
Vector database
Similarity search
Context injection

Level 2 — Better RAG

Metadata filtering
Hybrid search
Query rewriting
Reranking
Top-K retrieval

Level 3 — Production RAG

Conversation history
Citations
Access control
Streaming
Caching
Observability

Public GitHub project

A great reference is LangChain — RAG From Scratch.

It builds RAG progressively from indexing, retrieval and generation, making it particularly useful for understanding how RAG actually works rather than simply copying a framework implementation. (GitHub)

You can also explore LlamaIndex RAG example for a more application-oriented implementation. (GitHub)

3. Learn Fine-Tuning by Building a Medical Advisor

🎯 Project: Medical Advisor

Important: This should be treated as an educational AI project, not a real medical diagnostic system.

The goal is to take an open-source LLM and adapt it to produce responses in a particular domain and format.

For example, create a dataset containing:

Question
     ↓
Medical Context
     ↓
Expected Response

Then fine-tune an open model on your dataset.

Example

Input:

"What are common symptoms associated with iron deficiency?"

The model should learn to produce a response following your desired structure and style.

What you'll learn

This project teaches:

Dataset preparation
Instruction datasets
Data cleaning
Tokenization
Training/validation splits
Supervised Fine-Tuning (SFT)
LoRA
QLoRA
Model checkpoints
Training metrics
Model comparison
Inference with adapters

Instead of trying to fine-tune a huge model from scratch, start with LoRA/QLoRA. These techniques make experimentation much more practical.

Public GitHub projects

For a simple introduction:

Fine-Tuning LLMs with LoRA and QLoRA

This repository demonstrates LoRA and QLoRA fine-tuning using PyTorch and Hugging Face Transformers. (GitHub)

For a more complete implementation:

LLM Fine-Tuning — SFT, LoRA & QLoRA

It includes dataset loading, tokenization, SFT, LoRA and QLoRA examples. (GitHub)

The important lesson

Don't think:

Fine-tuning = giving the model more knowledge

Instead, think:

Fine-tuning = adapting model behavior, style, format or task performance.

For frequently changing factual knowledge, RAG is often a better solution.

4. Learn Evals by Building an LLM Evaluation System

🎯 Project: Evaluate Your AI Applications

This is the project most beginners skip.

And it is one of the most important.

Suppose your RAG system answers:

"What is the company's leave policy?"

How do you know whether the answer is actually good?

You need an evaluation system.

Build a Lead/RAG Evaluation Bot

Create a test dataset:

Question
Expected Answer
Retrieved Context
Generated Answer

Then evaluate the system automatically.

Evaluate:

Retrieval

Did we retrieve the correct document?
Was the relevant information present?
Was irrelevant information retrieved?

Generation

Is the answer correct?
Is it relevant?
Is it grounded in the retrieved context?
Did the model hallucinate?

Example

Question:
What is our annual leave policy?

Expected:
Employees receive 24 days of annual leave.

Model Answer:
Employees receive 24 days of annual leave.

Evaluation:
Correctness: 1.0
Faithfulness: 1.0
Relevance: 1.0

Now intentionally introduce a bad answer:

Model Answer:
Employees receive 30 days of annual leave.

Evaluation:
Correctness: 0.0
Faithfulness: 0.0

You have now started building an LLM evaluation pipeline.

Public GitHub projects

A good reference is RAG Evaluation Framework.

It separates evaluation into retrieval quality and generation quality and uses LLM-based evaluation with LangChain. (GitHub)

Another useful project is Ragas, which provides metrics and test-data generation for evaluating LLM applications and RAG systems. (GitHub)

You can also study LLM RAG Eval, which focuses specifically on evaluating RAG pipelines. (GitHub)

The Complete Learning Roadmap

Instead of building four unrelated projects, build them as a progression:

                    GENERATIVE AI
                         │
                         ▼
              ┌─────────────────────┐
              │ 1. Lead Triaging Bot│
              │       LLM           │
              └──────────┬──────────┘
                         │
                         ▼
              ┌─────────────────────┐
              │ 2. Knowledge        │
              │    Assistant        │
              │       RAG           │
              └──────────┬──────────┘
                         │
                         ▼
              ┌─────────────────────┐
              │ 3. Medical Advisor  │
              │    Fine-Tuning      │
              └──────────┬──────────┘
                         │
                         ▼
              ┌─────────────────────┐
              │ 4. Evaluation       │
              │    Framework        │
              │       Evals         │
              └─────────────────────┘

What You Should Know After Building All 4

Project	Technology	You Learn
Lead Triaging Bot	LLM	Prompting, structured output, tools
Knowledge Assistant	RAG	Embeddings, retrieval, vector DB, RAG
Medical Advisor	Fine-Tuning	SFT, LoRA, QLoRA, datasets
Evaluation Framework	Evals	Metrics, test datasets, hallucination detection

But the real value comes when you connect them.

Your final architecture can look like:

                         User
                           │
                           ▼
                    Lead / Query
                           │
                           ▼
                   ┌──────────────┐
                   │     LLM      │
                   └──────┬───────┘
                          │
                ┌─────────┴─────────┐
                ▼                   ▼
              RAG              Fine-Tuned
           Knowledge             Model
              │                   │
              └─────────┬─────────┘
                        ▼
                   Final Answer
                        │
                        ▼
                    EVALUATION
                        │
              ┌─────────┼─────────┐
              ▼         ▼         ▼
          Correct?  Relevant?  Grounded?

The key takeaway

Don't learn these technologies as isolated topics.

Build progressively.

Start with an LLM application → add your own knowledge with RAG → adapt the model with fine-tuning → finally build an evaluation layer to measure whether your system actually improved.

That progression takes you from "I know how to call an LLM API" to "I can design, improve and evaluate production-style LLM systems."

How Does Aadhaar Verify Billions of People's Fingerprints in Seconds?

Khushi Patel — Thu, 09 Jul 2026 17:44:45 +0000

Introduction

India's Aadhaar system stores biometric information for over a billion residents. Every day, millions of fingerprint authentication requests are made for banking, government services, SIM verification, and more.

A common question is:

How can Aadhaar verify a fingerprint among more than a billion records in just a few seconds?

The answer is that Aadhaar doesn't compare your fingerprint against every fingerprint in the database. Instead, it uses smart indexing, biometric templates, and a highly optimized distributed architecture.

The Myth

Many people imagine Aadhaar works like this:

Fingerprint
      |
      v
Compare with Person 1
Compare with Person 2
Compare with Person 3
...
Compare with 1.4 Billion People

If this were true, authentication would take hours.

Fortunately, that's not how the system works.

Fingerprints Are Stored as Templates

When you enroll for Aadhaar, the system doesn't store your fingerprint as a normal image.

Instead, it extracts unique features such as:

Ridge endings
Ridge bifurcations
Minutiae points
Relative positions and angles

These features are converted into a compact biometric template, making comparisons much faster than comparing full images.

Authentication Is Usually 1:1 Matching

Most Aadhaar authentications already include your Aadhaar number or Virtual ID.

The process looks like this:

Aadhaar Number + Fingerprint
          |
          v
Find User Record
          |
          v
Compare Submitted Fingerprint
          |
          v
Authentication Success or Failure

Since the system already knows which record to check, it performs one fingerprint comparison, not billions.

What About Identification Without an Aadhaar Number?

In some scenarios, the system may need to identify a person without knowing who they are.

This is called 1:N matching.

Instead of checking every record one by one, the system:

Uses biometric indexes
Narrows the search to likely matches
Searches across multiple servers in parallel
Compares only a small candidate set

This dramatically reduces the search time.

Distributed System Architecture

              Authentication Request
                        |
                        v
                Load Balancer
                        |
        +---------------+---------------+
        |               |               |
        v               v               v
   Biometric       Biometric      Biometric
   Server 1        Server 2       Server 3
        |               |               |
        +---------------+---------------+
                        |
                        v
               Authentication Result

The workload is distributed across many servers, allowing millions of authentication requests to be processed concurrently.

Optimizations That Make It Fast

Modern biometric systems use several techniques:

Biometric templates instead of images
Fast indexing algorithms
Parallel processing
In-memory caching
Distributed databases
Optimized biometric matching engines

Together, these reduce authentication time from minutes to just a few seconds.

Interview Perspective

A common misconception is:

"The system compares your fingerprint against every person in India."

A strong system design answer is:

Most Aadhaar authentications are 1:1 verification, not 1:N identification.
Fingerprints are stored as biometric templates.
Distributed servers process requests in parallel.
Indexing reduces the search space.
Only a tiny number of comparisons are actually performed.

Key Takeaways

Aadhaar does not compare your fingerprint with billions of records for every authentication.
Fingerprints are converted into compact biometric templates.
Most authentications are 1:1 verification using your Aadhaar number or Virtual ID.
Large-scale infrastructure, indexing, and distributed processing allow millions of requests to be handled efficiently.
The speed comes from smart system design—not brute-force searching.

Why Does LinkedIn Show "500+ Connections" Instead of the Exact Number?

Khushi Patel — Mon, 29 Jun 2026 13:56:01 +0000

Introduction

Have you ever noticed that LinkedIn displays "500+ Connections" once someone crosses 500 connections, instead of showing the exact count like 723 or 1,842?

At first glance, this might seem like a simple UI choice, but it's actually a great example of a product and system design decision where engineering, business goals, and user psychology come together.

The Product Perspective

LinkedIn's goal is not to help users compare who has more connections. Instead, it wants to indicate that someone has a strong professional network.

From a networking perspective:

450 Connections → Growing Network
500 Connections → Well Connected Professional
2500 Connections → Also Well Connected Professional

For most users, knowing whether someone has 1,200 or 1,500 connections doesn't provide significant additional value.

So LinkedIn chooses a threshold:

0 - 499  → Show exact number
500+     → Show "500+"

This communicates the important information while reducing unnecessary comparison.

The Engineering Perspective

Imagine LinkedIn has hundreds of millions of users.

Every profile view needs to display:

Name
Headline
Profile picture
Connection count

Showing an exact connection count requires keeping that number constantly updated whenever:

A connection request is accepted
A connection is removed
An account is deleted

For highly connected users, this count changes frequently.

By displaying:

500+

instead of:

LinkedIn reduces the importance of real-time precision.

This gives engineers more flexibility around caching and data synchronization.

A Distributed Systems Trade-off

In system design, there's a concept called:

Precision vs Practical Value

Ask yourself:

Would a user experience change if LinkedIn showed:

12,842

instead of:

500+

Probably not.

Since the exact value provides little additional business value, LinkedIn can optimize for:

Faster profile loads
Better caching
Lower database reads
Simpler distributed updates

This is a classic example of choosing an approximate answer when an exact answer isn't necessary.

Real-World System Design Principle

A common mistake in software engineering is trying to make everything perfectly accurate.

Good system designers ask:

Does the user actually need this precision?

Examples:

Instagram likes may update with slight delays
YouTube subscriber counts are often rounded
Social media follower counts are abbreviated
LinkedIn shows 500+ connections

These systems intentionally sacrifice precision because the business value of exactness is low.

The Deliverable System Design Lesson

One of the most important lessons in system design is:

Build for the requirement, not for perfection.

If the product requirement is:

Show whether a user has a large professional network

then:

500+

solves the problem.

If the requirement were:

Display the exact connection count for analytics purposes

then a different design would be necessary.

Great engineering decisions are often about identifying where precision matters and where it doesn't.

Key Takeaway

LinkedIn shows 500+ connections because the exact number provides little additional value to users after a certain threshold. This decision reduces social comparison, simplifies the user experience, and gives engineers more flexibility in scaling, caching, and synchronizing connection counts.

It's a great example of a system design principle: optimize for what users actually need, not for perfect accuracy everywhere.