DEV Community: Tanu Priya

What Really Happens When You Search Something on Google?

Tanu Priya — Mon, 21 Sep 2026 06:33:32 +0000

You type:

“How does Netflix recommend movies?”

You press Enter.

Within a fraction of a second, Google shows millions of possible results, advertisements, images, videos, maps, and sometimes an AI-generated answer.

It feels almost instant.

But a lot happens between pressing Enter and seeing that results page.

Your request has to travel across the internet, reach Google's infrastructure, be interpreted, matched against an enormous search index, ranked, filtered, and finally turned into a response.

So, what actually happens when you search something on Google?

Let's follow the journey.

The Big Picture

At a high level, a Google search looks something like this:

You type a query
       ↓
Browser sends request
       ↓
DNS + Network
       ↓
Google receives request
       ↓
Query understanding
       ↓
Search index lookup
       ↓
Candidate retrieval
       ↓
Ranking
       ↓
Personalization & context
       ↓
Search result generation
       ↓
Response sent to browser
       ↓
You see the results

And importantly, Google isn't searching the entire internet at that moment.

It is searching its index of the web.

That's one of the most important concepts to understand.

1. You Type a Search Query

Suppose you search:

how does netflix recommendation system work

Your browser needs to send that query to Google's servers.

But before the request can even reach Google, your computer has to figure out where Google's servers are located.

That's where DNS comes in.

2. DNS Finds Google

Your browser needs an IP address to communicate with Google's infrastructure.

You entered:

google.com

But computers communicate using IP addresses.

So your system asks DNS:

What IP address should I use for google.com?

The DNS resolution process may involve:

Browser cache
      ↓
Operating system cache
      ↓
Router / DNS resolver
      ↓
DNS infrastructure
      ↓
Google's IP address

Once an appropriate IP address is available, your browser can establish a connection.

This is similar to looking up someone's phone number before calling them.

3. Your Request Travels Across the Internet

Your search request now needs to travel from your device to Google's infrastructure.

Conceptually:

Your Computer
     ↓
Wi-Fi / Ethernet
     ↓
Router
     ↓
ISP
     ↓
Internet
     ↓
Google Infrastructure

The request isn't necessarily traveling through one fixed path.

Internet routing systems dynamically determine how packets should move through networks.

Your query might travel through multiple routers and networks before reaching Google's infrastructure.

And all of this happens extremely quickly.

4. HTTPS Protects the Connection

Modern Google Search uses HTTPS.

Instead of sending the request as plain text, the connection is encrypted using TLS.

Conceptually:

Browser
   │
   │ Encrypted HTTPS
   ▼
Google

This helps protect data while it travels between your browser and Google's servers.

An important distinction:

Encryption protects the communication channel.

It doesn't mean Google doesn't know what you searched for. Google needs to process the query in order to return search results.

5. Google's Frontend Receives Your Request

Your request reaches Google's infrastructure.

But Google isn't running everything from one giant server.

Instead, Google operates enormous distributed infrastructure across many locations.

A simplified architecture might look like:

                Google
                   │
            Load Balancing
                   │
        ┌──────────┼──────────┐
        ↓          ↓          ↓
      Server     Server     Server
        │          │          │
        └──────────┼──────────┘
                   ↓
          Search Infrastructure

The request can be routed to an appropriate Google server or cluster.

This is one reason large-scale systems rely heavily on:

Load balancing
Distributed systems
Caching
Replication
Geographic distribution
Fault tolerance

6. Google Has Already Crawled the Web

Here's where things get interesting.

When you search Google, Google usually isn't visiting websites one by one at that exact moment.

Instead, Google maintains a massive search index.

Think about the difference between these two approaches.

Approach 1: Search the internet live

Search
  ↓
Visit website 1
  ↓
Visit website 2
  ↓
Visit website 3
  ↓
...
  ↓
Return results

This would be incredibly slow.

Approach 2: Search an index

Internet
   ↓
Crawlers
   ↓
Processed documents
   ↓
Search Index
   ↓
Your Query
   ↓
Results

This is dramatically faster.

7. Googlebot Crawls Websites

Google uses automated crawlers to discover and revisit web pages.

One well-known crawler is Googlebot.

The crawler can discover pages through:

Links
Sitemaps
Previously known URLs
Other discovery mechanisms

For example:

example.com
      │
      ├── /blog
      │
      ├── /products
      │
      └── /about

Google's crawlers can discover these pages and retrieve their content.

But crawling isn't the same thing as ranking.

That's an important distinction.

Crawling
   ↓
Discover content

Indexing
   ↓
Understand and store content

Ranking
   ↓
Determine which results are useful for a query

8. Google Builds a Search Index

Imagine trying to search through the entire internet every time someone enters a query.

That wouldn't scale.

Instead, Google builds and maintains an enormous index.

A simplified representation might look like:

"react"
   ↓
Document A
Document B
Document C
Document D

"next.js"
   ↓
Document B
Document E
Document F

"server components"
   ↓
Document A
Document E

This resembles the idea of an inverted index.

Instead of asking:

Which words exist inside this document?

The search system can efficiently answer:

Which documents contain these words?

This dramatically reduces the amount of work needed during a search.

9. Your Query Gets Understood

Now Google has your query:

how does netflix recommendation system work

The system doesn't simply treat this as five independent words.

It tries to understand the meaning and intent behind the query.

For example:

how does netflix recommendation system work

could be interpreted as:

Topic:
Netflix recommendation systems

Intent:
Learn / understand

Expected content:
Technical explanation

Search systems use sophisticated language understanding techniques to interpret queries.

This is especially important for natural-language searches.

For example:

"best laptop for programming"

is very different from:

"laptop programming error"

Even though they share some words, the intent is different.

10. Query Rewriting Can Happen

Search engines may also transform or expand the query internally.

For example, a user might search:

how fast is js

The system may understand that:

js → JavaScript

and interpret the query accordingly.

Similarly, spelling mistakes, synonyms, entities, and related concepts can influence how a query is processed.

The goal is not simply:

Find pages containing these exact words.

The goal is:

Find information that best satisfies what the user is trying to find.

11. Google Searches the Index

Now the search system needs to find potentially relevant documents.

Suppose Google's index contains billions of documents.

It doesn't want to rank every document individually for every query.

That would be computationally expensive.

Instead, search systems typically use a multi-stage process.

A simplified model looks like:

Query
  ↓
Candidate Retrieval
  ↓
Thousands of Candidates
  ↓
Ranking
  ↓
Top Candidates
  ↓
Final Result

The first stage finds potentially relevant documents.

The next stages determine which ones deserve higher placement.

12. Candidate Retrieval

Imagine your query is:

how does a CDN work

The search system may quickly identify a large set of potentially relevant documents.

For example:

10,000 potentially relevant documents

The system then narrows this down.

Perhaps:

10,000
   ↓
1,000
   ↓
100
   ↓
20
   ↓
10 results

These numbers are only illustrative.

The important concept is retrieval before final ranking.

This architecture allows large-scale search systems to operate efficiently.

13. Ranking Determines the Order

Finding relevant documents isn't enough.

Google also needs to decide:

Which results should appear first?

This is the ranking problem.

Many signals can influence ranking, depending on the query and search context.

Examples can include:

Relevance
Content quality
Page experience
Freshness
Authority
Language
Location
Query intent
Search context

Google has many ranking systems rather than one simple formula.

A useful mental model is:

Query
  +
Content
  +
Context
  +
Many ranking signals
       ↓
Ranking systems
       ↓
Ordered results

The exact weighting and implementation of ranking systems are not publicly disclosed in full.

14. Relevance Matters More Than Keyword Matching

Early search engines relied heavily on matching words.

Modern search is much more sophisticated.

Consider:

"why does my phone battery drain overnight"

A useful result might not contain exactly that sentence.

It might instead discuss:

background processes
battery health
location services
push notifications
sleep settings

The search engine needs to understand that these concepts are related to the user's underlying problem.

This is where modern information retrieval and machine learning become important.

15. Freshness Can Matter

Some queries require recent information.

For example:

Apple stock price

or:

latest iPhone

or:

weather tomorrow

For these searches, older information may be less useful.

Compare that with:

what is recursion

A tutorial written years ago can still be perfectly useful.

So search systems need to understand when freshness matters.

16. Location Can Matter

Search results can also depend on location.

Consider:

pizza near me

A result from another country isn't useful.

Google can use location-related context to provide geographically relevant results.

Conceptually:

Query
  +
Location
  +
Search intent
       ↓
Local results

This is why two people searching for the same local query may not necessarily see identical results.

17. Google May Use Your Search Context

Search results can sometimes be influenced by context associated with the search experience.

Depending on the situation, this can include things such as:

Language
Approximate location
Device context
Previous interactions with Google services
Search settings

Personalization isn't necessarily applied identically to every query.

The exact behavior depends on Google's systems and the user's settings.

18. Special Search Features Are Generated

Google Search isn't limited to ten blue links.

Depending on the query, the results page can contain different types of features.

For example:

Search Query
     │
     ├── Web results
     ├── Images
     ├── Videos
     ├── News
     ├── Maps
     ├── Knowledge panels
     ├── Featured snippets
     ├── Shopping results
     └── Other search features

The system determines which features are relevant to the query.

For example:

"weather in Chennai"

might produce a weather result.

While:

"Taylor Swift"

may produce entity-related information alongside web results.

19. Ads Are a Separate Part of the Search Experience

You may also see advertisements above or around organic search results.

These aren't simply pages that won the organic ranking algorithm.

Google operates a separate advertising system.

A simplified view is:

Search Query
      │
      ├───────────────┐
      ↓               ↓
Organic Search     Ads System
      │               │
      ↓               ↓
Organic Results    Ad Results
      │               │
      └───────┬───────┘
              ↓
        Search Results Page

This distinction is important:

Advertising placement and organic search ranking are different systems.

20. Google Generates the Search Results Page

Once the relevant information has been selected, Google's systems need to construct the response.

The response might include:

Title
URL
Snippet
Images
Videos
Knowledge information
Ads
Related searches
Other features

The browser receives the necessary response data.

21. Your Browser Renders the Page

Now the request has completed its journey back to your device.

Your browser receives the response and begins rendering the interface.

Conceptually:

Google Server
      ↓
HTTP Response
      ↓
Browser
      ↓
HTML / CSS / JavaScript
      ↓
DOM
      ↓
Layout
      ↓
Painting
      ↓
Search Results

The browser then displays the page you see.

And the entire experience feels almost instantaneous.

22. But Google Doesn't Always Start From Scratch

Another important concept is caching.

Large-scale systems use caches extensively.

For example:

Popular Query
     ↓
Cached / Reusable Data
     ↓
Faster Response

Caching can reduce repeated computation and improve response time.

This is one of the fundamental techniques used by large internet systems.

The exact caching architecture Google uses internally is complex and not fully public, but caching is a standard principle in large-scale distributed systems.

23. What Happens If Millions of People Search at Once?

Imagine millions of people searching simultaneously:

"World Cup"
"weather"
"Bitcoin"
"Taylor Swift"
"JavaScript"
"Google"

Google needs to handle enormous amounts of traffic.

This requires distributed infrastructure.

A simplified architecture could look like:

                 Users
                   │
          ┌────────┴────────┐
          ↓                 ↓
       Region A          Region B
          │                 │
     Load Balancers    Load Balancers
          │                 │
       Servers            Servers
          │                 │
          └────────┬────────┘
                   ↓
          Search Infrastructure

Instead of depending on one machine, workloads are distributed across many machines and locations.

24. What If One Server Fails?

Large systems are designed with redundancy.

Suppose:

Server A ❌

The system shouldn't simply stop working.

Instead, traffic can potentially be handled by other infrastructure.

              Request
                 │
          Load Balancing
          /      |      \
         ↓       ↓       ↓
       A ❌      B       C
                ✓       ✓

This is one of the fundamental ideas behind fault-tolerant distributed systems.

25. Why Is Google Search So Fast?

Several technologies and engineering techniques contribute to the speed.

At a high level:

Precomputed Index
       +
Efficient Retrieval
       +
Distributed Infrastructure
       +
Caching
       +
Load Balancing
       +
Optimized Ranking
       +
Fast Networks
       ↓
Fast Search

The key idea is that Google does a huge amount of work before you search.

Crawling and indexing happen ahead of time.

When you finally enter a query, Google can focus on:

Understand
   ↓
Retrieve
   ↓
Rank
   ↓
Generate
   ↓
Respond

instead of crawling the entire internet from scratch.

26. A Simplified End-to-End Architecture

Putting everything together:

                YOU
                 │
                 ▼
          Search Query
                 │
                 ▼
              Browser
                 │
                 ▼
          DNS Resolution
                 │
                 ▼
             Internet
                 │
                 ▼
        Google Infrastructure
                 │
                 ▼
          Load Balancing
                 │
                 ▼
        Query Understanding
                 │
                 ▼
        Search Index Lookup
                 │
                 ▼
       Candidate Retrieval
                 │
                 ▼
             Ranking
                 │
        ┌────────┴────────┐
        │                 │
        ▼                 ▼
   Organic Results      Ads
        │                 │
        └────────┬────────┘
                 ▼
       Search Page Generation
                 │
                 ▼
              Browser
                 │
                 ▼
         Results on Screen

And all of this can happen in a remarkably short amount of time.

27. The Most Important Insight

When you search Google, Google isn't searching the entire internet in real time.

Instead:

Internet
   ↓
Crawling
   ↓
Processing
   ↓
Indexing
   ↓
Search Index
   ↓
Your Query
   ↓
Retrieval
   ↓
Ranking
   ↓
Results

This is one of the most important concepts in understanding how modern search engines work.

28. How This Connects to System Design

Google Search is a great example of large-scale system design because it combines many concepts we've already discussed.

Distributed Systems

Millions of requests need to be processed simultaneously.

Caching

Frequently accessed information can be served efficiently.

Load Balancing

Traffic is distributed across infrastructure.

Database / Indexing

Huge amounts of information need to be organized for fast retrieval.

Ranking Systems

Potentially relevant results need to be ordered.

Fault Tolerance

Individual machines and components can fail without taking down the entire system.

Networking

Requests need to travel quickly across global infrastructure.

Data Processing

Web pages need to be crawled, processed, analyzed, and indexed.

That's why Google Search is much more than a search box.

It's a massive distributed information-retrieval system.

Final Takeaway

The next time you type something into Google, remember that you're not simply asking a computer:

"Find me this webpage."

You're interacting with a massive distributed system.

Your query travels through the network, reaches Google's infrastructure, gets interpreted, and is matched against a huge pre-built index.

Potentially relevant documents are retrieved, ranking systems determine their order, other search features may be generated, and the final response is sent back to your browser.

The simplified journey is:

Search Query
     ↓
DNS
     ↓
Internet
     ↓
Google Infrastructure
     ↓
Query Understanding
     ↓
Search Index
     ↓
Candidate Retrieval
     ↓
Ranking
     ↓
Search Features
     ↓
Browser
     ↓
You

What Really Happens When You Open Google.com in a Browser?

Tanu Priya — Sun, 20 Sep 2026 06:31:32 +0000

You type:

https://www.google.com

You press Enter.

A fraction of a second later, Google appears on your screen.

It feels almost instantaneous.

But behind that single action, your browser may perform DNS resolution, connection establishment, TLS negotiation, HTTP communication, server-side processing, HTML parsing, CSS processing, JavaScript execution, layout calculation, painting, and compositing.

And all of this happens while you are simply waiting to see a webpage.

So what actually happens when you open google.com?

Let's follow the request from your keyboard all the way to the pixels on your screen.

The Complete Journey

At a high level, the journey looks like this:

You type google.com
        ↓
Browser interprets the URL
        ↓
Check browser / OS caches
        ↓
DNS Resolution
        ↓
IP Address discovered
        ↓
Connection established
        ↓
TLS Handshake
        ↓
HTTP Request
        ↓
Google's infrastructure
        ↓
HTTP Response
        ↓
Browser receives HTML
        ↓
HTML → DOM
CSS → CSSOM
        ↓
Render Tree
        ↓
Layout
        ↓
Paint
        ↓
Compositing
        ↓
Pixels appear on screen

But each of these steps contains a lot more interesting engineering.

Let's start from the beginning.

1. You Type `google.com`

You open Chrome, Firefox, Safari, or another browser and type:

google.com

The browser doesn't immediately send those characters to Google.

First, it needs to understand what you entered.

The browser's UI process determines whether the input is:

a URL
a search query
or something else

For example:

google.com

looks like a domain name, while:

best laptops for programming

looks more like a search query.

Modern browsers perform this processing before navigation begins.

In Chromium-based browsers, navigation involves the browser process, networking components, and a renderer process that eventually handles the webpage.

Once the browser determines that you're navigating to a website, the real journey begins.

2. The Browser Needs an IP Address

Computers communicate across networks using IP addresses.

Humans prefer:

google.com

Machines ultimately need something like:

<IP address>

So the browser needs to answer:

"Which IP address belongs to google.com?"

This is where DNS comes in.

DNS stands for:

Domain Name System

You can think of DNS as the internet's distributed phone book.

Instead of remembering:

142.x.x.x

you remember:

google.com

DNS maps the human-readable domain name to an IP address.

3. DNS Resolution

The browser doesn't necessarily perform a complete DNS lookup every time.

There can be multiple layers of caching.

A simplified flow looks like:

Browser
   ↓
OS / DNS cache
   ↓
DNS Resolver
   ↓
Root DNS Servers
   ↓
.com TLD Servers
   ↓
Google's Authoritative DNS
   ↓
IP Address

However, if a cached DNS result is still valid, some of these steps can be avoided.

This is one reason why repeated visits to websites can behave differently from the first visit.

Why caching matters

Imagine if every webpage visit required starting from the root DNS infrastructure.

That would add unnecessary latency.

Instead, DNS responses have a TTL (Time To Live) that allows results to be cached for a period of time.

So the browser may effectively say:

"I already know where google.com is."

and skip much of the lookup process.

Modern DNS can also use encrypted transports such as DNS over HTTPS (DoH) or DNS over TLS (DoT), depending on the system and browser configuration.

4. DNS Gives Us an IP Address

Eventually, the system obtains an address for the destination.

Now the browser knows:

google.com
       ↓
IP address

But knowing where to send the request isn't enough.

The browser still needs to establish communication with the server.

And this is where networking begins.

5. Establishing the Connection

For traditional HTTPS over TCP, the browser establishes a TCP connection to the destination.

TCP provides reliable, ordered delivery of data.

A simplified TCP handshake looks like:

Client                         Server

  SYN  ------------------------>

       <------------------- SYN-ACK

  ACK  ------------------------>

This is commonly called the three-way handshake.

The goal is to establish the connection and synchronize the communication state between both sides.

Only after the connection is established can the browser proceed with the next stage.

But there's another important requirement.

We're visiting:

https://google.com

not:

http://google.com

So the communication needs to be encrypted.

6. The TLS Handshake

HTTPS uses TLS (Transport Layer Security) to protect the connection.

TLS provides important security properties including:

Encryption
Integrity
Server authentication

The browser needs to establish cryptographic keys that can be used to securely communicate with the server.

A simplified TLS flow looks like:

Browser                         Server

ClientHello -------------------->

             <---------------- ServerHello
             <---------------- Certificate
             <---------------- Key Exchange

Certificate verification

Key establishment

Finished ---------------------->

             <---------------- Finished

The actual modern TLS handshake is more nuanced, but conceptually the browser and server are establishing a secure communication channel.

Why does the certificate matter?

Suppose you typed:

https://google.com

You don't want an attacker pretending to be Google.

The server provides a certificate that allows the browser to verify that the server is authorized for the requested domain.

If certificate validation fails, the browser can display a security warning instead of silently trusting the connection.

7. Now We Can Send an HTTP Request

DNS told us where to go.

TCP established reliable communication.

TLS established secure communication.

Now HTTP can finally do its job.

The browser sends an HTTP request.

Conceptually, it looks something like:

GET / HTTP/2
Host: www.google.com
User-Agent: ...
Accept: text/html
Accept-Language: en-US
Cookie: ...

The actual request headers and protocol details vary depending on the browser, connection, cookies, HTTP version, and other factors.

The important part is:

GET /

The browser is essentially saying:

"Give me the resource at the root path."

8. Your Request Travels Across the Internet

The request doesn't magically teleport from your laptop to Google.

It travels through a network.

A simplified model might look like:

Your Browser
     ↓
Your Device
     ↓
Wi-Fi Router
     ↓
ISP
     ↓
Internet Routers
     ↓
Google's Network
     ↓
Google Infrastructure

Every packet can travel through multiple network devices before reaching its destination.

Routers examine addressing information and determine where packets should go next.

This is one of the fascinating things about the internet:

Your browser doesn't need to know the entire route to Google's servers.

The network infrastructure handles routing.

9. The Request Reaches Google's Infrastructure

This is where the story gets much more interesting.

A request to a massive service isn't simply:

Internet → One Server

Large-scale systems typically have layers of infrastructure designed to handle enormous amounts of traffic.

A conceptual architecture could look like:

                 Internet
                    ↓
              Edge Network
                    ↓
             Load Balancing
                    ↓
          Frontend / Web Servers
                    ↓
          Application Services
                    ↓
        Internal Services / Storage

The exact architecture of Google's production systems is complex and not something we can reduce to one simple diagram.

But the important distributed-systems lesson is:

A large website is usually an ecosystem of services, not one giant computer.

10. The Server Processes the Request

Once the request reaches the appropriate service, the server determines what response should be returned.

Depending on the website, this might involve:

Authentication
Cookies
Request routing
Feature flags
Personalization
Application logic
Cache lookup
Database access
Internal APIs
Other services

For a simple static website, the server might be able to return cached content almost immediately.

For a dynamic application, the request may trigger much more work.

This is why backend architecture matters.

The user sees:

Google loaded.

But the backend may have performed many operations before the browser receives the response.

11. The Server Sends an HTTP Response

Eventually, the browser receives an HTTP response.

Conceptually:

HTTP/2 200
Content-Type: text/html
Content-Encoding: ...
Cache-Control: ...

followed by the response body.

The response body could contain HTML such as:

<!DOCTYPE html>
<html>
  <head>
    ...
  </head>

  <body>
    ...
  </body>
</html>

The browser now has something important:

HTML.

But the page isn't "finished" yet.

This is where browser rendering begins.

12. The Browser Starts Parsing HTML

The browser's rendering engine receives the HTML.

It doesn't simply wait for the entire document and then process everything at once.

HTML can be processed progressively as data arrives.

Modern browsers can parse streamed HTML incrementally, allowing rendering work to begin before the entire response has arrived.

The HTML parser turns markup into a structure called the:

DOM

DOM stands for:

Document Object Model

For example:

<body>
  <h1>Hello</h1>
  <p>Welcome!</p>
</body>

can be represented conceptually as:

Document
   |
   └── body
        |
        ├── h1
        |    └── "Hello"
        |
        └── p
             └── "Welcome!"

This tree structure allows JavaScript and browser APIs to interact with the document.

13. HTML Isn't Enough

HTML tells the browser:

"What elements exist?"

But it doesn't fully describe how those elements should look.

That's the job of CSS.

For example:

h1 {
  font-size: 40px;
  margin-bottom: 20px;
}

The browser needs to download and process the CSS.

It constructs another representation called the:

CSSOM

CSSOM stands for:

CSS Object Model

So now we have:

HTML
 ↓
DOM

CSS
 ↓
CSSOM

14. DOM + CSSOM

The browser combines information from the DOM and CSSOM to determine what should actually be rendered.

Conceptually:

              HTML
                ↓
               DOM
                \
                 \
                  → Render information
                 /
                /
              CSSOM
                ↑
                CSS

This produces the information needed for rendering.

A simplified representation is often called the render tree.

The render tree contains the visual information necessary for the browser to determine what appears on the screen.

15. Then Comes Layout

The browser now needs to answer:

Where exactly should every element appear?

Suppose you have:

<div>
  <h1>Hello</h1>
  <p>This is a paragraph.</p>
</div>

The browser needs to calculate things such as:

h1:
x = 40
y = 20
width = 300
height = 50

p:
x = 40
y = 90
width = 500
height = 80

This process is called:

Layout

or sometimes:

Reflow

The browser calculates the geometry of visible elements.

This includes:

Position
Width
Height
Margins
Padding
Line wrapping
Font metrics
Relationships between elements

And this can become computationally expensive on complex pages.

16. Painting the Page

Once the browser knows where everything goes, it needs to determine what should actually be drawn.

This stage is called:

Paint

The browser creates drawing instructions for things such as:

Text
Backgrounds
Borders
Shadows
Images
Gradients

Conceptually:

DOM + CSS
    ↓
Layout
    ↓
Paint instructions

But we're still not quite done.

17. Compositing

Modern browsers don't necessarily draw the entire webpage as one giant image.

Different parts of a page can be handled as separate compositing layers.

For example:

Layer 1 → Background
Layer 2 → Main content
Layer 3 → Fixed header
Layer 4 → Animation

The browser can then composite these layers together.

Conceptually:

Painted Layers
      ↓
Compositor
      ↓
GPU / Display Pipeline
      ↓
Screen

This is one reason certain CSS operations can be significantly cheaper to animate than operations that repeatedly trigger expensive layout work.

18. JavaScript Enters the Picture

Modern websites aren't just HTML and CSS.

JavaScript can:

Modify the DOM
Respond to user interactions
Fetch APIs
Track state
Update the UI
Start animations
Load additional resources
Communicate with servers

For example:

button.addEventListener("click", () => {
  console.log("Clicked!");
});

When JavaScript changes the page, the browser may need to perform additional rendering work.

For example:

JavaScript
    ↓
DOM change
    ↓
Style calculation
    ↓
Layout
    ↓
Paint
    ↓
Composite

Not every JavaScript operation triggers every stage, but expensive DOM and style changes can cause significant work.

19. One Page Can Trigger Many Requests

Here's an important realization.

You didn't just request:

google.com

The initial HTML can reference many additional resources.

For example:

HTML
 ├── CSS
 ├── JavaScript
 ├── Images
 ├── Fonts
 ├── API requests
 └── Other resources

The browser discovers these resources while processing the document and requests them as needed.

This means a webpage is often not:

1 request → 1 response

It is closer to:

Navigation request
       ↓
HTML
       ↓
 ┌─────┼─────┬─────┐
 ↓     ↓     ↓     ↓
CSS   JS   Images Fonts
       ↓
   API Requests
       ↓
   More Data

That's why network waterfalls in Chrome DevTools can become surprisingly large.

20. HTTP/2 and HTTP/3 Make This More Efficient

Modern web communication isn't limited to the old HTTP/1.1 model.

HTTP/2 introduced features such as:

Multiplexing
Header compression
Stream prioritization mechanisms

This allows multiple requests and responses to share a connection more efficiently.

HTTP/3 goes further by using QUIC, which runs over UDP and incorporates transport and TLS functionality into the protocol stack.

Conceptually:

Older model:

HTTP
 ↓
TCP
 ↓
IP


HTTP/2:

HTTP/2
  ↓
 TCP
  ↓
 IP


HTTP/3:

HTTP/3
  ↓
 QUIC
  ↓
 UDP
  ↓
 IP

The important point is that the web's transport layer has evolved significantly to reduce latency and improve performance.

21. Caching Can Change Everything

Now imagine you've already visited Google.

The browser may already have cached some resources.

Instead of downloading everything again:

Browser
   ↓
"Do I already have this?"
   ↓
YES
   ↓
Use cached resource

Caching can exist at multiple layers:

Browser Cache
      ↓
Operating System / Network Cache
      ↓
CDN / Edge Cache
      ↓
Server Cache
      ↓
Database / Storage

A good caching strategy can dramatically reduce:

Network requests
Server work
Latency
Bandwidth usage

This is one of the most important performance principles in web development.

22. The Browser Finally Shows You the Page

After all this:

DNS
 ↓
Connection
 ↓
TLS
 ↓
HTTP
 ↓
Server
 ↓
HTML
 ↓
DOM
 ↓
CSSOM
 ↓
Render Tree
 ↓
Layout
 ↓
Paint
 ↓
Composite
 ↓
Pixels

you finally see the webpage.

To you, it looked like:

Type → Enter → Website

But internally, an enormous amount of work happened.

23. What Happens When You Click Something?

The process doesn't end when the page appears.

Suppose you click a button.

The browser might perform:

Mouse Event
     ↓
JavaScript Event Handler
     ↓
State Change
     ↓
DOM Update
     ↓
Style Calculation
     ↓
Layout
     ↓
Paint
     ↓
Composite

Or perhaps the button triggers an API request:

Click
  ↓
JavaScript
  ↓
fetch("/api/data")
  ↓
HTTP Request
  ↓
Server
  ↓
Database / Service
  ↓
HTTP Response
  ↓
JavaScript
  ↓
UI Update

So the same concepts repeat throughout the lifetime of the application.

24. Where Performance Problems Come From

Now the entire process becomes useful from an engineering perspective.

A webpage can be slow for completely different reasons.

DNS is slow

The browser spends more time resolving the domain.

Connection establishment is slow

Network latency or connection setup adds delay.

Server response is slow

The backend takes too long to generate the response.

HTML is large

The browser has more data to download and parse.

JavaScript is expensive

The main thread can become blocked.

CSS is expensive

Style calculation and layout can become costly.

Images are huge

Downloading and decoding them takes time.

Too many requests

The browser has more network work to coordinate.

Third-party scripts are expensive

Analytics, ads, widgets, and other external code compete for network and CPU resources.

This is why web performance isn't simply:

"Make your JavaScript faster."

Performance is an end-to-end problem.

25. The Browser Is Doing Multiple Things at Once

One of the biggest misconceptions about browsers is thinking they execute the webpage as one simple sequence.

Modern browsers are complex, multi-process systems.

A simplified architecture might look like:

                 Browser
                    |
        ┌───────────┼───────────┐
        ↓           ↓           ↓
   Browser       Network      Storage
   Process        Work         Work
        |
        ↓
   Renderer Process
        |
   ┌────┼────┐
   ↓    ↓    ↓
 HTML  CSS   JS
   |
   ↓
 Rendering Pipeline
   |
   ↓
   GPU
   |
   ↓
 Screen

Chromium's architecture separates browser-level responsibilities from renderer responsibilities, with communication between processes handled through IPC.

This architecture provides important isolation and allows browsers to manage complex web applications more safely and efficiently.

26. The Entire Journey in One Diagram

Here's the complete mental model I use:

                 USER
                  │
                  ▼
          Types google.com
                  │
                  ▼
           Browser UI
                  │
                  ▼
          URL Interpretation
                  │
                  ▼
           DNS Resolution
                  │
                  ▼
             IP Address
                  │
                  ▼
        Connection Establishment
                  │
                  ▼
            TLS Handshake
                  │
                  ▼
           HTTP Request
                  │
                  ▼
        ┌───────────────────┐
        │ Google's Network  │
        │                   │
        │ Edge / Routing     │
        │ Load Balancing     │
        │ Application Logic  │
        │ Internal Services  │
        └───────────────────┘
                  │
                  ▼
          HTTP Response
                  │
                  ▼
             HTML
                  │
          ┌───────┴────────┐
          ▼                ▼
        DOM              CSSOM
          │                │
          └───────┬────────┘
                  ▼
             Render Tree
                  │
                  ▼
                Layout
                  │
                  ▼
                Paint
                  │
                  ▼
             Compositing
                  │
                  ▼
                 GPU
                  │
                  ▼
              SCREEN

And that entire pipeline happens incredibly quickly.

27. The Most Important Mental Model

If you're preparing for frontend, backend, or system-design interviews, don't memorize a list of terms.

Understand the dependency chain:

DNS
 ↓
Where is the server?

Connection
 ↓
How do I communicate with it?

TLS
 ↓
Can I communicate securely?

HTTP
 ↓
What resource do I want?

Server
 ↓
How should the request be processed?

HTML
 ↓
What exists on the page?

CSS
 ↓
How should it look?

JavaScript
 ↓
How should it behave?

Layout
 ↓
Where does everything go?

Paint
 ↓
What should be drawn?

Compositing
 ↓
How should the layers be combined?

GPU / Display
 ↓
How do those instructions become pixels?

Once you understand this chain, many seemingly unrelated web-development concepts start making sense.

28. Why This Question Is So Important

"What happens when you open Google.com?" sounds like a beginner question.

It isn't.

A strong answer can touch almost every major area of web development:

Frontend
   ├── HTML
   ├── CSS
   ├── JavaScript
   └── Rendering

Networking
   ├── DNS
   ├── TCP
   ├── TLS
   ├── HTTP
   └── IP

Backend
   ├── Servers
   ├── APIs
   ├── Caching
   └── Databases

System Design
   ├── Load Balancing
   ├── Distributed Systems
   ├── CDNs
   └── Scalability

Performance
   ├── Latency
   ├── TTFB
   ├── Rendering
   └── Network Optimization

Security
   ├── HTTPS
   ├── TLS
   ├── Certificates
   └── Origin Isolation

One URL can therefore become a tour through almost the entire web stack.

Final Takeaway

The next time you type:

google.com

don't think:

"The website opens."

Think:

URL
 ↓
DNS
 ↓
IP
 ↓
Connection
 ↓
TLS
 ↓
HTTP
 ↓
Server Infrastructure
 ↓
HTML
 ↓
DOM
 ↓
CSSOM
 ↓
JavaScript
 ↓
Layout
 ↓
Paint
 ↓
Compositing
 ↓
Pixels

What looks like a single action to the user is actually a coordinated interaction between your browser, operating system, DNS infrastructure, network, security protocols, servers, application services, and rendering engine.

And that is one of the most beautiful things about the web:

A few characters typed into an address bar can trigger an entire distributed system.

How Do Big Backend Applications Scale?

Tanu Priya — Sat, 19 Sep 2026 10:23:04 +0000

A backend rarely starts as a complicated distributed system.

It usually starts with something much simpler:

User → API Server → Database

A few users send requests. The server processes them. The database stores the data. Everything is easy to understand.

Then the application becomes popular.

The number of users increases. Requests arrive faster. Some database queries become expensive. Large files need processing. Emails and notifications have to be sent. One server starts running out of capacity.

So you add another server.

Then another.

Then a cache.

Then a queue.

Then database replicas.

Eventually, you might have dozens or hundreds of machines working together.

This raises a fundamental question:

How does a simple backend gradually evolve into a system capable of serving millions of requests?

The answer isn't simply "add more servers."

Backend scaling is a process of identifying bottlenecks, separating workloads, distributing work, and accepting carefully chosen trade-offs.

The architecture becomes more complicated because the problems become more complicated.

What Does It Actually Mean to Scale?

Before looking at specific technologies, it helps to define what we're trying to improve.

A backend typically has several important characteristics.

Latency

How long does one request take?

For example:

GET /products/123
→ 80 ms

Lower latency generally means the individual request completes faster.

Throughput

How much work can the system handle over a period of time?

For example:

10,000 requests / second

Throughput is about the amount of work the system can process.

Concurrency

How many operations are happening at the same time?

A server might have thousands of active connections even if only a fraction of them are actively consuming CPU at a particular moment.

Availability

How often is the system operational and able to serve requests?

A system that works extremely quickly but frequently goes offline isn't very useful.

Scalability

Scalability is about how well the system continues to handle increasing workload as demand grows.

These concepts are related, but they aren't the same.

You can improve latency without increasing throughput.

You can increase throughput while making individual requests slower.

You can increase capacity while accidentally making the system harder to operate.

That's why system design is fundamentally about trade-offs.

The First Version: Keep It Simple

Imagine you're building a small e-commerce application.

The first version might look like:

Users
  ↓
API Server
  ↓
Database

The API server might handle:

Authentication
Validation
Business logic
Database queries
API responses

The database handles persistent state.

For a small application, this architecture can work extremely well.

There is a tendency in system design discussions to jump immediately to technologies such as Kubernetes, Kafka, Redis, microservices, and distributed databases.

But complexity has a cost.

Every additional component means another system that must be:

Deployed
Monitored
Secured
Debugged
Upgraded
Paid for

If your application has 100 users, introducing twenty distributed components probably creates more problems than it solves.

So the first scaling principle is:

Start with the simplest architecture that satisfies the current requirements.

The architecture should evolve when the workload demands it.

What Happens When Traffic Increases?

Suppose the application becomes popular.

More users mean more requests.

Initially, the server may handle everything comfortably.

Eventually, resources begin to approach their limits.

You might observe:

CPU       → 90%
Memory    → 85%
Database  → High load
Latency   → Increasing
Errors    → Increasing

But these numbers don't tell you what to change.

A server can be slow because of CPU-intensive work.

Or because it is waiting for the database.

Or because it is making too many network calls.

Or because a particular query is poorly optimized.

Or because a downstream service is slow.

This is why scaling should begin with measurement, not infrastructure.

You want to answer:

Where is the time and capacity actually going?

That leads to one of the most important ideas in system design:

Don't scale everything. Scale the bottleneck.

Step 1: Optimize Before You Scale

Imagine one endpoint is taking 2 seconds to respond.

The first instinct might be:

"We need more servers."

But suppose the request contains a database query that scans millions of rows.

Adding more API servers won't fix that query.

You might instead need:

A better index
A more efficient query
Pagination
Smaller result sets
Connection pooling
Reduced database round trips

This is an important distinction.

There are two broad ways to handle more workload:

Do less work

Examples:

Cache repeated results
Optimize database queries
Avoid unnecessary computation
Reduce payload sizes

Do more work in parallel

Examples:

Add application servers
Add database replicas
Add workers
Partition workloads

Good architectures often use both.

Step 2: Add Caching

One of the most common forms of unnecessary work is repeatedly calculating or retrieving the same information.

Imagine a product page:

GET /api/products/123

Suppose thousands of users request the same product.

Without caching, every request may reach the database.

Request
   ↓
API
   ↓
Database
   ↓
Response

If the product doesn't change frequently, doing this thousands of times is wasteful.

A cache allows frequently accessed data to be stored temporarily.

Request
   ↓
Cache
   ↓
Database if needed

If the data exists in the cache, that's a cache hit.

If it doesn't, that's a cache miss.

A common approach is called cache-aside:

Application checks the cache.
If data exists, return it.
Otherwise, query the database.
Store the result in the cache.
Return the result.

Redis is commonly used for this type of workload.

Why Is Caching So Powerful?

Imagine a database receives 100,000 identical reads.

If 90,000 of those requests can be served from a cache, the database only needs to handle a fraction of the original workload.

That can improve:

Database capacity
Response latency
Application throughput

But caching introduces a difficult problem:

Stale Data

Suppose the database says:

Price = ₹999

while the cache still contains:

Price = ₹899

Which value should the user receive?

This is why cache design involves concepts such as:

TTL
Cache invalidation
Cache-aside
Write-through caching
Cache consistency

Caching is therefore not simply a performance feature.

It is also a data-consistency decision.

And this is a recurring pattern in system design:

Performance improvements often introduce consistency or operational trade-offs.

Step 3: One Server Is No Longer Enough

Eventually, the application server itself may become the bottleneck.

Perhaps CPU usage is consistently high.

Perhaps memory usage is approaching the machine's limit.

Perhaps there are simply too many concurrent requests.

At this point, horizontal scaling becomes useful.

Instead of one server:

Users
  ↓
Server

we run multiple instances:

Users
  ↓
Load Balancer
  ↓
+--------+--------+
|        |        |
Server   Server   Server

The load balancer distributes requests among the available instances.

Now the application can increase capacity by adding instances.

This is horizontal scaling.

Why Stateless Servers Matter

Multiple servers create an important question:

Where does application state live?

Suppose Server 1 stores a user's session in its local memory.

The user's next request goes to Server 3.

Server 3 doesn't have that session.

Now the system has a problem.

This is why scalable application servers are generally designed to be stateless.

Instead of keeping important shared state only in local memory, that state can be stored in an external system.

For example:

Server 1 ─┐
Server 2 ─┼──→ Shared State
Server 3 ─┘

The shared system might be a database, Redis, or another appropriate storage mechanism.

Now any application instance can process the request.

This has an important operational benefit.

If a server disappears, another server can replace it without losing application state.

Servers become relatively disposable compute units.

That property is extremely useful in horizontally scaled environments.

Step 4: The Database Becomes the Bottleneck

Now imagine we have five application servers.

Users
  ↓
Load Balancer
  ↓
Server Server Server Server Server
  ↓
Database

The application layer has scaled.

But there is still one database.

Every server is sending queries to it.

Eventually, the database may become the limiting component.

Before introducing database replication or sharding, investigate the fundamentals.

Indexes

Indexes can dramatically reduce the amount of data a database needs to inspect for supported query patterns.

Query Optimization

A query that scans unnecessary rows or performs expensive joins can consume significant resources.

Connection Pooling

Creating database connections repeatedly can be expensive.

Connection pools allow applications to reuse connections.

Pagination

Returning 1 million rows when the user only needs 20 is unnecessary work.

Avoiding N+1 Queries

Fetching one record and then performing another query for every related record can create hundreds or thousands of database calls.

These optimizations are often more valuable than immediately adding infrastructure.

The lesson is simple:

A slow database query is still a slow query after you add more API servers.

Step 5: Database Replication

After optimizing the database, the workload may still be too large.

Many applications have a read-heavy workload.

Think about an online store.

A product might be viewed thousands of times while its price or description changes relatively infrequently.

In such cases, database replication can distribute read traffic.

A simplified architecture looks like:

             Application
                 |
          +------+------+
          |             |
          ↓             ↓
       Primary      Read Replica
       Writes          Reads

The primary handles writes.

Replicas can serve appropriate read queries.

This can reduce pressure on the primary database.

But there is an important trade-off.

Replication Lag

Suppose a user updates their profile.

The write reaches the primary database immediately.

The replica may receive that change slightly later.

If the next request reads from the replica, the user might temporarily see the old information.

This creates a consistency question:

Does this operation require the latest data, or is slightly stale data acceptable?

For some applications, a small amount of replication lag is acceptable.

For others, it may not be.

Therefore, replication isn't simply about making a database faster.

It also requires designing how reads and writes interact.

Step 6: Move Slow Work Out of the Request

Now consider a different problem.

Suppose a user requests a report.

Generating that report requires:

Querying millions of records
Processing the data
Creating a file
Uploading the file
Sending an email

If all of this happens inside the HTTP request, the user may be waiting for a very long time.

Instead, the API can create a background job.

User
 ↓
API
 ↓
Queue
 ↓
Worker

The API can respond quickly while the worker performs the expensive operation asynchronously.

This pattern is useful for:

Email sending
Video processing
Image processing
Report generation
Notifications
Data exports
Scheduled processing

The queue creates a boundary between request processing and background processing.

Why Queues Help During Traffic Spikes

Queues are particularly useful when work arrives in bursts.

Imagine a system normally receives:

100 jobs/minute

Then a marketing campaign causes:

10,000 jobs

If the system tries to process all 10,000 immediately, downstream resources may become overwhelmed.

A queue lets the system absorb the burst.

Workers can process jobs at a sustainable rate.

This is closely related to backpressure.

Instead of forcing every component to operate at the speed of the fastest producer, the queue provides a buffer.

But now we have another set of engineering problems.

What if a worker crashes halfway through a job?

What if the job is delivered twice?

What if the job keeps failing?

These lead to concepts such as:

Retries
Acknowledgements
Idempotency
Dead-letter queues
Visibility timeouts
Retry backoff

Idempotency: An Important Property of Background Jobs

Suppose a worker processes:

Send ₹1,000 payment

The worker completes the payment but crashes before acknowledging the message.

The queue may deliver the same job again.

If processing the job twice creates two payments, you have a serious problem.

An idempotent operation can safely be repeated without producing an incorrect additional effect.

For example, instead of blindly creating a payment every time, the system might associate the operation with a unique transaction ID and ensure that the same transaction cannot be applied twice.

This is an example of why distributed systems require more than just adding components.

Once work can be retried, duplicated, delayed, or reordered, the application has to account for those possibilities.

Step 7: Scale Workers Independently

Once background processing has been separated from API requests, the two workloads can scale independently.

Suppose you have:

10 API servers
3 workers

Later, video processing becomes much more popular.

You might need:

10 API servers
30 video workers

There is no reason to increase API capacity simply because video processing became expensive.

Different workloads have different resource requirements.

An API may be mostly network and database bound.

A video encoder may be CPU intensive.

A machine-learning worker may require large amounts of memory or specialized hardware.

Independent scaling allows each workload to receive the resources it actually needs.

Step 8: When Does a Monolith Become a Problem?

As the application grows, another kind of scaling problem can appear.

The problem isn't necessarily CPU or memory.

It is organizational and architectural complexity.

Imagine one huge application containing:

Users
Orders
Payments
Search
Notifications
Analytics
Files

The application may still run perfectly well.

But as the codebase grows, different teams may need to deploy different areas independently.

One module may require very different scaling characteristics from another.

This is where modular architecture becomes important.

A good first step is often a modular monolith.

Keep the application as one deployable unit, but clearly separate its internal domains.

For example:

Application
├── Users
├── Orders
├── Payments
├── Search
└── Notifications

This gives the codebase clear boundaries without immediately introducing distributed-system complexity.

Only when those boundaries need independent deployment, scaling, ownership, or infrastructure should you consider separating them into services.

Step 9: Microservices and Their Cost

A microservices architecture might eventually look like:

                 API Gateway
                      |
        +-------------+-------------+
        |             |             |
      Users         Orders       Payments
      Service       Service       Service

Each service can potentially have its own:

Deployment lifecycle
Scaling strategy
Team ownership
Data storage
Technology choices

This can be useful.

But there is a major trade-off.

Inside a monolith:

Orders → Payment Module

might be a local function call.

With microservices:

Orders Service
      ↓
Network
      ↓
Payment Service

Now you have network latency, timeouts, retries, authentication, serialization, and partial failures.

The system can fail in ways that never existed inside the monolith.

For example:

Orders Service → Working
Payment Service → Working
Network → Failed

Both services are healthy, but the request still fails.

This is the fundamental cost of distribution.

Microservices don't remove complexity. They move complexity into the network and operations.

That's why microservices should solve a specific problem rather than being treated as the default architecture for every large application.

Step 10: Observability Becomes Essential

As the architecture becomes distributed, debugging becomes harder.

Imagine a request travels through:

Load Balancer
      ↓
API
      ↓
Cache
      ↓
Orders Service
      ↓
Payment Service
      ↓
Database

The request takes 4 seconds.

Where did the delay happen?

Was the cache slow?

Was the database slow?

Did the payment service take 3 seconds?

Did a network request time out and retry?

Without observability, answering these questions becomes difficult.

This is why larger systems rely heavily on:

Logs

Detailed records of events.

Metrics

Numerical measurements such as:

Request rate
Error rate
CPU
Memory
Latency
Database connections
Queue depth

Traces

A way to follow one request as it moves through multiple components.

Observability is therefore not just something added after the system is built.

For distributed systems, it becomes part of the system itself.

Step 11: Design for Failure

A system with one server has a relatively small number of failure modes.

A distributed system has many more.

A cache can fail.

A database replica can fall behind.

A queue can become full.

A worker can crash.

A service can time out.

A network connection can fail.

A deployment can introduce an error.

So as the architecture grows, a new question becomes increasingly important:

What happens when something fails?

Consider a cache.

If the cache disappears, can the application still read from the database?

Consider a worker.

If it crashes halfway through a job, can another worker safely retry it?

Consider a database replica.

If it becomes unavailable, can the application temporarily route reads elsewhere?

Consider a service.

If it becomes slow, should the caller wait indefinitely?

These questions lead to concepts such as:

Timeouts
Retries
Circuit breakers
Failover
Health checks
Rate limiting
Graceful degradation

Reliability is not the absence of failure.

It is the ability of the system to behave sensibly when failures occur.

A Realistic Scaling Journey

Let's put everything together with an e-commerce example.

Stage 1: Small Application

You start with:

Users
 ↓
API
 ↓
Database

Simple, inexpensive, easy to operate.

Stage 2: Repeated Reads

Product pages become popular.

The database receives the same reads repeatedly.

You introduce caching.

Users
 ↓
API
 ↓
Cache
 ↓
Database

Stage 3: More Traffic

The API server becomes overloaded.

You add multiple instances behind a load balancer.

Users
 ↓
Load Balancer
 ↓
API API API

Stage 4: Database Pressure

The database becomes the next bottleneck.

You optimize queries, add indexes, and improve connection management.

If read traffic is still high, you introduce replicas.

Stage 5: Background Work

The application starts generating invoices, processing images, and sending notifications.

You move those tasks into a queue.

API
 ↓
Queue
 ↓
Workers

Stage 6: Independent Workloads

Image processing becomes much heavier than normal API traffic.

You scale image workers independently.

Stage 7: Organizational Growth

The engineering organization becomes larger.

Payments, orders, search, and notifications have different requirements.

You introduce stronger module boundaries.

Some modules may eventually become independent services.

Notice what happened.

The final architecture wasn't designed on day one.

It evolved because each new problem required a different solution.

That is how many real systems grow.

The Architecture Is a Set of Trade-Offs

There is no architecture that is simultaneously:

Cheapest
Fastest
Simplest
Most available
Most consistent
Most scalable
Easiest to operate

Every decision changes something else.

Caching can improve latency but introduce stale data.

Replication can increase read capacity but introduce replication lag.

Queues can protect the API from expensive background work but introduce asynchronous behavior.

Microservices can provide independent deployment and scaling but introduce distributed-system complexity.

Horizontal scaling can increase capacity but requires stateless application design and shared infrastructure.

This is why system design is fundamentally about trade-offs.

You aren't searching for the perfect architecture.

You're choosing the architecture whose trade-offs make sense for the problem.

A Better Way to Think About Scaling

Instead of memorizing:

Caching
Load Balancer
Replication
Queue
Microservices

think in terms of the problems they solve.

Problem	Possible Solution
Repeated expensive reads	Caching
One server cannot handle traffic	Horizontal scaling
Requests depend on local state	Externalize shared state
Database queries are slow	Indexing and query optimization
Too many database reads	Replication / caching
Long-running requests	Asynchronous processing
Background workload increases	Worker scaling
Large codebase becomes difficult to manage	Modular architecture
Independent deployment is required	Service boundaries
Distributed system is difficult to debug	Observability
Components fail independently	Resilience mechanisms

This is much more useful than memorizing architecture diagrams.

When you encounter a new system-design problem, ask:

What is the bottleneck?

Then:

What is the smallest architectural change that can remove it?

The Complete Mental Model

A mature backend might eventually look something like this:

                         Users
                           |
                           v
                     Load Balancer
                           |
                  +--------+--------+
                  |        |        |
                  v        v        v
               Server   Server   Server
                  |
             +----+----+
             |         |
             v         v
           Cache    Database
                       |
                 +-----+-----+
                 |           |
                 v           v
              Primary    Read Replica

                       |
                       v
                     Queue
                    / | \
                   /  |  \
                  v   v   v
              Worker Worker Worker

And parts of the application may eventually become independent services.

But this diagram is not a blueprint.

It is a collection of patterns.

You might use only three of them.

You might use ten.

You might replace some entirely.

The right architecture depends on:

Traffic
Data volume
Latency requirements
Consistency requirements
Availability requirements
Team structure
Budget
Operational maturity
Failure tolerance

The Most Important Lesson

The biggest mistake in system design is thinking that a large architecture is automatically a better architecture.

It isn't.

A simple architecture that reliably handles 10,000 users is better than an unnecessarily complicated architecture that creates operational problems.

And a simple architecture that works today may not be sufficient tomorrow.

That's okay.

Architecture is allowed to evolve.

The goal isn't to predict the future perfectly.

The goal is to build a system that can evolve when the future arrives.

So the backend scaling journey can be summarized as:

More Users
     ↓
More Work
     ↓
Measure
     ↓
Find the Bottleneck
     ↓
Reduce Work OR Add Capacity
     ↓
Measure Again
     ↓
Repeat

Sometimes reducing work means caching.

Sometimes it means optimizing a query.

Sometimes adding capacity means more servers.

Sometimes it means read replicas.

Sometimes it means more workers.

And sometimes the problem isn't performance at all—it is code ownership or organizational complexity.

That's when modularity and service boundaries become important.

Final Takeaway

You don't start with a distributed system.

You grow into one.

A backend might begin as:

Server + Database

Then evolve into:

Load Balancer
+ Multiple Servers
+ Cache
+ Database Replication
+ Queue
+ Workers

And eventually, parts of the system may become independent services.

But every step should have a reason.

Before adding a new component, ask two questions:

What problem does this solve?

and:

What new complexity does this introduce?

If you can answer both, you're not just adding technologies.

You're designing a system.

And that is the real skill behind scaling big backend applications.

What Happens When Your Backend Gets 1 Million Requests?

Tanu Priya — Fri, 18 Sep 2026 08:27:13 +0000

An API can look perfectly fine during development.

You test it locally, everything responds quickly, and a single server seems more than enough.

Then traffic grows.

Suddenly, the server is handling thousands of requests at the same time. CPU usage increases, database queries pile up, response times get worse, and eventually requests start failing.

At that point, the solution usually isn't just a bigger server.

You need to change how the system is designed.

A common architecture looks like this:

                         Users
                           |
                           v
                    Load Balancer
                    /     |     \
                   v      v      v
                Server  Server  Server
                   \      |      /
                    \     |     /
                      Cache
                        |
                        v
                     Database

For work that doesn't need to happen during the request, a queue can be added:

API Server
    |
    v
  Queue
    |
    +----> Worker
    +----> Worker
    +----> Worker

This is the point where backend development starts becoming system design.

The First Problem: One Server

Start with the simplest architecture:

Users
  |
  v
Server
  |
  v
Database

Every request reaches the same server.

That server handles authentication, business logic, database queries, API responses, and potentially background work too.

As traffic increases, its resources become limited:

CPU       → High
Memory    → High
Latency   → Increasing
Errors    → Increasing

You can move to a larger machine.

That's vertical scaling.

But eventually, a single machine still has a ceiling.

The other option is to add more servers.

Horizontal Scaling

Instead of:

Users
  |
  v
One Server

we can have:

             Load Balancer
              /    |    \
             v     v     v
          Server Server Server

This is horizontal scaling.

The load balancer distributes incoming requests across available servers.

For example, if three servers are handling traffic:

10,000 requests/sec
        |
        v
Load Balancer
   /     |     \
3333   3333   3334

The actual distribution won't always be perfectly equal. It depends on the load-balancing strategy and the workload.

Common strategies include:

Round Robin
Least Connections
Weighted Routing
IP Hash

The important idea is simple:

Spread the work instead of depending on one machine.

Keeping Servers Stateless

Multiple servers introduce another problem.

Suppose a user logs in through Server 1.

If the login state exists only in Server 1's memory, the next request could reach Server 3.

Login
  ↓
Server 1
  ↓
Session stored locally

Next request
  ↓
Server 3
  ↓
Session not found

This is one reason scalable applications generally avoid keeping important shared state only in a server's local memory.

Instead, shared state can be stored in something accessible to all instances:

Server 1 ─┐
Server 2 ─┼──> Redis / Database
Server 3 ─┘

Now any server can handle the next request.

The Database Becomes the Next Bottleneck

Adding more backend servers doesn't automatically solve the whole problem.

Consider:

             Load Balancer
            /      |      \
        Server   Server   Server
            \      |      /
                 Database

Now three servers can process requests, but all of them are still querying the same database.

The database may become the bottleneck.

This leads to an important system-design principle:

When you remove one bottleneck, another one often becomes visible.

So instead of sending every request to the database, we can reduce unnecessary database work.

Caching

Suppose thousands of users request the same product:

GET /api/products/123

Without caching:

Request
   ↓
Server
   ↓
Database
   ↓
Response

With a cache:

Request
   ↓
Server
   ↓
Cache
   |
   +---- Hit → Response
   |
   +---- Miss
          ↓
       Database
          ↓
        Cache
          ↓
       Response

If the requested data is already in the cache, the database doesn't need to process another query.

A simple example:

1,000 requests
      ↓
    Cache
      ↓
900 cache hits
      ↓
100 database requests

The numbers are just an example. The actual benefit depends on the application's traffic and cache-hit rate.

Redis is commonly used for this kind of workload.

But Caching Creates Another Problem

What happens when the database changes?

Suppose the cache contains:

product:123 → ₹999

Then the database is updated:

₹999 → ₹899

The cache may still contain the old value.

Now the application can return stale data.

This is why caching isn't simply:

"Put Redis in front of the database."

You also need to think about:

TTL
Invalidation
Cache-aside
Write-through
Stale data

Caching can reduce database load significantly, but it introduces another system that needs to be designed and maintained.

Not Every Task Belongs in the Request

Some operations are expensive but don't need to finish before the user gets a response.

For example:

Upload video
     ↓
Process video
     ↓
Generate thumbnails
     ↓
Send notifications

Doing all of this inside the API request can make the endpoint slow.

Instead:

Client
  |
  v
API Server
  |
  v
Queue
  |
  +----> Worker 1
  +----> Worker 2
  +----> Worker 3

The API creates a job and puts it into the queue.

Workers process those jobs separately.

This works well for things such as:

Emails
Notifications
Image processing
Video processing
Reports
Data exports

The queue acts as a buffer between incoming requests and background processing.

A More Complete Architecture

Putting the pieces together:

                         Users
                           |
                           v
                    Load Balancer
                           |
              +------------+------------+
              |            |            |
              v            v            v
          Server 1     Server 2     Server 3
              |            |            |
              +------------+------------+
                           |
                    +------+------+
                    |             |
                    v             v
                  Cache        Database
                                  |
                                  v
                             Read Replica

              Server
                 |
                 v
               Queue
                 |
          +------+------+ 
          |      |      |
          v      v      v
       Worker  Worker  Worker

Each component has a different responsibility:

Load Balancer
→ Distributes requests

Servers
→ Run application logic

Cache
→ Avoids repeated expensive reads

Database
→ Stores persistent data

Queue
→ Holds asynchronous work

Workers
→ Process background jobs

That's the important part of this architecture.

You're not adding technologies randomly.

You're adding them because a specific part of the system needs help.

What Happens During a Traffic Spike?

Imagine your application normally receives:

10,000 requests/minute

Then traffic suddenly jumps:

100,000 requests/minute

A scalable architecture might handle the increase like this:

Traffic Spike
     ↓
Load Balancer
     ↓
Multiple Servers
     ↓
Cache handles repeated reads
     ↓
Database handles remaining queries
     ↓
Queue absorbs background work
     ↓
Workers process jobs

If the infrastructure supports autoscaling, more application instances can be started as demand increases.

But simply adding servers isn't enough.

You have to understand where the actual bottleneck is.

The Database Has Its Own Scaling Problems

At larger workloads, the database may need additional techniques:

Indexing
Connection Pooling
Query Optimization
Read Replicas
Partitioning
Sharding

For example:

                 Database
                /        \
               v          v
           Primary     Read Replica
            Writes         Reads

But these aren't default requirements for every application.

If a database handles your workload comfortably, adding replicas or sharding can add unnecessary complexity.

The architecture should follow the workload.

One Million Requests Doesn't Tell the Whole Story

The number 1 million requests sounds huge, but request count alone doesn't tell us how difficult the system is.

Consider:

1 million requests
→ Simple cached reads

versus:

1 million requests
→ Complex queries
→ Multiple database writes
→ External API calls
→ File processing

These are very different workloads.

When designing for scale, ask:

How many requests per second?

How expensive is each request?

How much data does each request access?

How much traffic can be cached?

How many requests reach the database?

Which operations can run asynchronously?

What happens when a server fails?

These questions are much more useful than simply asking:

"Can my backend handle 1 million requests?"

The Mental Model

When traffic increases, don't immediately think:

"I need a bigger server."

Think:

"Where is the bottleneck?"

Then solve that specific problem.

More Traffic
     ↓
Load Balancer
     ↓
More Servers
     ↓
Cache
     ↓
Database
     ↓
Queue + Workers

The load balancer distributes traffic.

Multiple servers increase application capacity.

The cache reduces repeated work.

The database stores persistent data.

The queue moves expensive work out of the request path.

Workers process that work independently.

That's the transition from building an API to engineering a system.

A backend that works for 100 users and a backend that works for millions of users can use the same basic code.

What changes is how you design the system around that code.

How Do Backend Developers Secure APIs?

Tanu Priya — Thu, 17 Sep 2026 05:51:08 +0000

Building an API is only half the job.

Once an API is exposed to real users, you have to think about what happens when someone sends unexpected input, calls an endpoint they shouldn't, sends thousands of requests, or gets access to a secret.

That's why API security isn't a single feature. It's a collection of layers that protect different parts of the application.

A typical request might go through:

Request
   ↓
Authentication
   ↓
Authorization
   ↓
Validation
   ↓
Rate Limiting
   ↓
Business Logic
   ↓
Database
   ↓
Response

Let's look at the important parts.

Authentication

The first question is:

Who is making this request?

For example:

GET /api/profile
Authorization: Bearer <token>

The backend verifies the credentials and identifies the user.

If authentication fails:

Invalid / missing credentials
        ↓
401 Unauthorized

Authentication establishes identity, but it doesn't decide what that user can access.

That's authorization's job.

Authorization

Suppose two users are logged in:

Alex → user
Sarah → admin

Alex sends:

DELETE /api/users/100

The backend shouldn't simply check whether Alex is logged in.

It should also check whether Alex has permission to delete that user.

Authenticated user
       ↓
Permission check
       ↓
Allowed?
   ├── No → 403 Forbidden
   └── Yes → Continue

This check must happen on the backend.

Hiding a button in the frontend is not authorization.

A user can always send the API request directly.

Validate Client Input

Never assume that data from the frontend is trustworthy.

A client might send:

{
  "name": 123,
  "age": -50,
  "role": "admin"
}

even if your frontend never allows those values.

The backend should validate:

Required fields
Data types
Formats
Lengths
Allowed values
Business rules

A useful rule is:

Frontend validation improves the user experience. Backend validation protects the application.

SQL Injection

One classic API vulnerability is SQL injection.

This is dangerous:

const query =
  `SELECT * FROM users WHERE email = '${email}'`;

User input is being inserted directly into the SQL statement.

Instead, use parameterized queries:

const result = await db.query(
  "SELECT * FROM users WHERE email = $1",
  [email]
);

The database can then treat the supplied value as data rather than SQL syntax.

The same principle applies when using ORMs: use their parameterized query mechanisms rather than constructing raw SQL from untrusted input.

XSS

XSS stands for Cross-Site Scripting.

Consider a comment system:

User input
   ↓
API
   ↓
Database
   ↓
Browser

If the application later renders that content as executable HTML or JavaScript, an attacker could potentially run code in another user's browser.

Common defenses include:

Safe rendering
Output encoding
Avoiding unsafe HTML injection
Content Security Policy

The important lesson is that data stored in your database isn't automatically safe.

It may have originally come from an untrusted user.

CSRF

CSRF stands for Cross-Site Request Forgery.

It is particularly relevant when authentication uses cookies because browsers can automatically send cookies with requests.

A simplified attack looks like:

User is logged in
      ↓
Authentication cookie
      ↓
Malicious website
      ↓
Forged request
      ↓
Your API

Depending on the application, protections can include:

SameSite cookies
CSRF tokens
Origin validation

The right defense depends on how authentication and browser requests are designed.

Rate Limiting

An attacker can automate requests to endpoints such as:

Login
OTP verification
Password reset
Search
Expensive API operations

Without limits:

Attacker
   ↓
Thousands of requests
   ↓
Backend
   ↓
Database

Rate limiting can stop excessive traffic:

Too many requests
       ↓
429 Too Many Requests

Rate limiting isn't only about preventing brute-force attacks. It can also protect backend resources from accidental or malicious traffic spikes.

Protect Your Secrets

Backend applications often use sensitive credentials:

Database passwords
JWT signing keys
API keys
Cloud credentials
Payment provider secrets
OAuth secrets

Don't put them directly into source code:

const secret = "my-secret-key";

Use environment configuration or a dedicated secret-management system instead:

const secret = process.env.JWT_SECRET;

Also remember that secrets can leak through more than source code.

Watch out for:

Git repositories
Logs
Error messages
Frontend bundles
Debug endpoints
Screenshots

A secret is only useful if it stays secret.

HTTPS and Encryption

API traffic should use HTTPS.

Without HTTPS:

Client
   ↓
Network
   ↓
Server

With HTTPS:

Client
   ↓
Encrypted TLS connection
   ↓
Server

HTTPS protects data while it travels between the client and server.

This is especially important for:

Passwords
Cookies
Tokens
Personal data
Payment information

Encryption protects the connection, but it doesn't decide whether the authenticated user has permission to access a resource.

Security Headers

HTTP security headers can provide additional browser protections.

Common examples include:

Content-Security-Policy
Strict-Transport-Security
X-Content-Type-Options
Referrer-Policy

For example, Content Security Policy can restrict where a browser is allowed to load scripts and other resources.

These headers don't replace secure application code, but they add another layer of defense.

Don't Return More Data Than Necessary

Suppose your database contains:

id
name
email
password_hash
internal_notes
created_at

Your API probably doesn't need to return all of it.

Instead of:

SELECT * FROM users;

select only what the client needs:

SELECT id, name, email
FROM users;

The same applies to error messages.

Avoid exposing:

Database credentials
SQL queries
Internal file paths
Stack traces
Private service information

The client usually needs a simple error, while detailed information can remain in protected server logs.

Don't Trust IDs From the Client

Consider:

GET /api/orders/123

The user is authenticated.

That's not enough.

The backend should check whether that particular order belongs to that user.

Otherwise, someone might change:

/api/orders/123

to:

/api/orders/124

and access another user's data.

The check should be:

Authenticated user
       ↓
Requested resource
       ↓
Does the user have access?
       ↓
Allow / Reject

This is a common authorization problem in APIs.

Logging and Monitoring

Security also means being able to notice suspicious activity.

Useful events to track include:

Failed logins
Permission failures
Password reset attempts
Rate-limit violations
Administrative actions
Unusual request patterns

For example:

User 42
DELETE /api/users/100
Authorization denied

But don't casually log sensitive credentials.

Avoid storing things such as:

Passwords
Access tokens
Refresh tokens
API keys

Logs can become a security risk themselves if they contain secrets.

Defense in Depth

At this point, the bigger picture becomes clear.

API security isn't about finding one perfect security mechanism.

It's about having several layers:

                    Request
                       ↓
                 Rate Limiting
                       ↓
                Authentication
                       ↓
                 Authorization
                       ↓
                  Validation
                       ↓
                 Business Logic
                       ↓
                    Database
                       ↓
                   Response

And around the application:

HTTPS
Security Headers
Secret Management
Logging
Monitoring

Each layer deals with a different type of problem.

For example:

SQL Injection
→ Parameterized queries

XSS
→ Safe rendering + output handling

CSRF
→ Appropriate CSRF protections

Brute Force
→ Rate limiting

Broken Authorization
→ Server-side permission checks

Exposed Secrets
→ Secret management

If one layer fails, another layer can still limit the damage.

That's the basic idea behind defense in depth.

A Practical Checklist

Before putting an API into production, ask:

Authentication
[ ] Are protected endpoints authenticated?

Authorization
[ ] Are permissions checked on the server?
[ ] Can users access other users' resources?

Validation
[ ] Are request bodies validated?
[ ] Are query parameters validated?

Database
[ ] Are queries parameterized?

Rate Limiting
[ ] Are login and expensive endpoints protected?

Secrets
[ ] Are credentials outside the source code?
[ ] Are secrets excluded from logs?

Transport
[ ] Is HTTPS enabled?

Browser Security
[ ] Are cookies configured appropriately?
[ ] Are CSRF protections considered?

Monitoring
[ ] Are suspicious activities detectable?

This isn't a complete security audit, but it's a good starting point.

The Mental Model

When securing an API, think about the request one question at a time:

Who is making the request?
        ↓
Authentication

What can they access?
        ↓
Authorization

Is the input valid?
        ↓
Validation

Are they sending too many requests?
        ↓
Rate Limiting

Can the connection be intercepted?
        ↓
HTTPS

Can browser behavior be abused?
        ↓
CSRF / Security Headers

Are credentials protected?
        ↓
Secret Management

Can suspicious behavior be detected?
        ↓
Logging / Monitoring

The main lesson is simple:

API security isn't a feature you add once. It's a collection of decisions made throughout the backend.

Authentication tells you who the user is.

Authorization controls what they can access.

Validation keeps unexpected input out of your application logic.

Rate limiting controls abusive traffic.

HTTPS protects data in transit.

And careful secret management prevents credentials from becoming an easy attack path.

The most useful question isn't:

"Is my API secure?"

It's:

"What could go wrong at each stage of this request, and what prevents it?"

That's the mindset that helps turn a working API into a secure one.

Cookies vs Sessions vs JWT: What's the Difference?

Tanu Priya — Wed, 16 Sep 2026 08:22:51 +0000

When you build a login system, you'll quickly run into three terms:

Cookies, sessions, and JWTs.

They are often mentioned together, which makes it easy to assume that they are three different ways of doing the exact same thing.

They're not.

A cookie is mainly a way for a browser to store and send data.

A session is server-side state that can be used to remember a logged-in user.

A JWT is a signed token that can carry information about an authenticated user.

The confusion usually starts because these things can also be used together.

For example:

Browser
   ↓
Cookie
   ↓
Session ID
   ↓
Server-side Session

Or:

Browser
   ↓
Cookie
   ↓
JWT

So instead of asking:

"Which one is better?"

it's more useful to understand what each one actually does and then decide which combination fits your application.

First, What Problem Are We Solving?

Let's start with a normal login.

A user sends:

POST /api/login

with:

{
  "email": "alex@example.com",
  "password": "mypassword"
}

The backend verifies the credentials.

Let's say they are correct.

The server now knows:

User ID: 42

But what happens when the user makes another request?

GET /api/profile

HTTP doesn't automatically remember that the previous request was a successful login.

The application needs a way to recognize the user again.

That's the problem these authentication mechanisms help solve.

Cookies: How Does the Browser Remember Something?

Let's start with cookies because they are the easiest to misunderstand.

A cookie is not an authentication system by itself.

It's a small piece of data that a browser can store and send back to a website.

A server can send:

Set-Cookie: session_id=abc123

The browser stores it.

Later, when making another request to the relevant site, the browser can send:

Cookie: session_id=abc123

That's the basic idea.

You can think of a cookie as a container for a value that the browser knows how to send back to the server.

That value could be:

session_id
authentication-related data
preferences
language settings
analytics identifiers

So:

Cookie ≠ Session
Cookie ≠ JWT

A cookie is a transport/storage mechanism.

What you put in it determines how it's being used.

Sessions: Keep the Login State on the Server

Now let's look at sessions.

After a successful login, the server can create a session:

Session ID: abc123
User ID: 42

The actual session information stays on the server.

Conceptually:

abc123 → User 42

The browser only needs to keep the session identifier.

That's where cookies often come in.

The server sends:

Set-Cookie: session_id=abc123

The browser stores it.

On a later request:

GET /api/profile
Cookie: session_id=abc123

the server reads the session ID and looks up the corresponding session.

The flow is:

Login
  ↓
Verify credentials
  ↓
Create session
  ↓
Store session on server
  ↓
Send session ID in cookie
  ↓
Browser stores cookie
  ↓
Browser sends cookie later
  ↓
Server finds session
  ↓
User identified

This is why you will often see cookies and sessions used together.

The cookie carries the identifier.

The session holds the state.

A Simple Example

Imagine a server has this session store:

abc123 → User 42
xyz789 → User 91

Alex logs in and receives:

session_id=abc123

When Alex requests:

GET /api/profile
Cookie: session_id=abc123

the server can do:

abc123
  ↓
Session Store
  ↓
User 42
  ↓
Fetch profile

The browser never needs to know what is stored inside the server-side session.

That's one of the useful properties of this approach.

So Where Does JWT Come In?

JWT stands for JSON Web Token.

Instead of giving the client a random session ID that points to server-side authentication state, the server can issue a signed token containing claims.

A simplified payload might look like:

{
  "sub": "42",
  "role": "user",
  "exp": 1760003600
}

The server signs the token.

The resulting JWT looks roughly like:

Header.Payload.Signature

The client can then send it with future API requests.

A common format is:

Authorization: Bearer eyJ...

The backend receives the token and verifies its signature and relevant claims.

Conceptually:

Login
  ↓
Verify credentials
  ↓
Create JWT
  ↓
Sign JWT
  ↓
Send JWT to client
  ↓
Client sends JWT
  ↓
Server verifies JWT
  ↓
Identify user

That's the basic JWT model.

The Important Difference

At this point, the three terms should start looking different.

	What it does
Cookie	Stores and sends data through the browser
Session	Keeps authentication state on the server
JWT	Carries signed claims that a server can verify

This is why comparing them as if they were direct alternatives can be confusing.

They're not really at the same layer.

For example, you can have:

Cookie + Session

or:

Cookie + JWT

or:

Authorization Header + JWT

Those are all different architectures.

Cookie + Session

This is a common pattern for traditional web applications.

The architecture looks like:

             Browser
                |
             Cookie
                |
          Session ID
                |
                ↓
        Server-side Session
                |
             User ID

For example:

Cookie:
session_id=abc123

Server:

abc123 → User 42

The browser carries the identifier.

The server owns the actual session state.

One practical advantage is that the server can invalidate the session directly.

For example:

Logout
  ↓
Invalidate session
  ↓
Session ID no longer works

This can make server-side session management convenient.

Cookie + JWT

A JWT doesn't have to be stored in JavaScript-accessible browser storage.

It can also be placed in a cookie.

For example:

Set-Cookie: access_token=eyJ...; HttpOnly; Secure

The browser then sends the cookie with appropriate requests.

The architecture becomes:

Browser
   ↓
Cookie
   ↓
JWT
   ↓
Server verifies JWT

Here the cookie is handling browser storage/transmission while the JWT is the authentication credential.

Again:

Cookie ≠ JWT

They are doing different jobs.

JWT in the Authorization Header

Another common pattern is to send the JWT explicitly:

GET /api/profile
Authorization: Bearer eyJ...

This is common when working with APIs where the client is responsible for attaching the token.

The server receives it and performs verification.

Request
   ↓
Authorization header
   ↓
JWT
   ↓
Verify token
   ↓
Identify user
   ↓
Process request

This pattern is common in applications where the frontend, mobile app, or another client communicates with a backend API.

But having an API does not automatically mean JWT is required.

That's an important distinction.

What Does "Stateless JWT" Mean?

One reason JWTs are popular is that they can support a stateless authentication model.

With a traditional session:

Session ID
    ↓
Server
    ↓
Session Store
    ↓
User

The server needs access to the session state.

With a self-contained JWT:

JWT
 ↓
Verify signature
 ↓
Read claims
 ↓
Identify subject

The server can potentially authenticate the request without looking up a traditional session record.

This can be useful when an application has multiple backend servers:

                 Load Balancer
                 /     |     \
                /      |      \
          Server A  Server B  Server C
                \      |      /
                 \     |     /
                 Verify JWT

Each server can verify the token using the appropriate key.

However, calling JWT authentication "stateless" doesn't mean the entire authentication system has zero state.

Applications may still need state for:

Refresh tokens
Revocation
Device sessions
Security events
User sessions

So stateless JWT authentication is a design choice, not a guarantee that the whole system becomes state-free.

What About Security?

This is where the choice becomes more interesting.

For browser applications, cookies have security-related attributes such as:

HttpOnly
Secure
SameSite

For example:

Set-Cookie: session_id=abc123;
HttpOnly;
Secure;
SameSite=Lax

HttpOnly prevents ordinary JavaScript from reading the cookie directly.

Secure restricts the cookie to HTTPS connections.

SameSite controls when the browser sends the cookie in cross-site situations.

These options are useful, but they don't magically make authentication secure.

The application still needs to consider things like:

HTTPS
CSRF
XSS
Token theft
Session fixation
Password security
Session expiration
Authorization

The correct security setup depends on the application.

What About localStorage?

You may have seen JWTs stored like this:

localStorage.setItem("token", token);

It is convenient.

But there's an important trade-off.

JavaScript running on the page can read values stored in localStorage.

So if an attacker manages to execute malicious JavaScript through an XSS vulnerability, a token stored there may be accessible to that script.

This doesn't mean:

localStorage = always insecure

or:

cookies = automatically secure

Security is more complicated than that.

It means you should understand what each storage mechanism exposes and choose based on your application's threat model.

What Happens During Logout?

This is one of the places where sessions and JWTs behave differently.

With a server-side session:

Logout
  ↓
Invalidate session
  ↓
Session ID becomes useless

The server can immediately stop recognizing that session.

With a self-contained access JWT, the server may continue accepting the token until it expires unless the application has an additional revocation mechanism.

That's one reason JWT-based systems often use short-lived access tokens.

For example:

Access token
   ↓
15-minute lifetime

A refresh mechanism can then be used to obtain another access token when necessary.

A common flow is:

Login
  ↓
Access Token + Refresh Token
  ↓
Use Access Token
  ↓
Access Token expires
  ↓
Use Refresh Token
  ↓
Get new Access Token

Refresh-token handling needs its own security design.

Scaling Sessions

Now imagine your application grows.

Initially:

Users
  ↓
Server
  ↓
Database

Later:

             Load Balancer
                  ↓
          ┌───────┼───────┐
          ↓       ↓       ↓
       Server A Server B Server C

With server-side sessions, you need to decide where those sessions live.

For example:

Server A ──┐
Server B ──┼──→ Shared Session Store
Server C ──┘

A shared store allows different servers to find the same session.

This adds infrastructure, but it is a perfectly reasonable architecture.

With JWTs, each server can potentially validate the token independently:

Server A → Verify JWT
Server B → Verify JWT
Server C → Verify JWT

That can simplify some scaling scenarios.

But it doesn't eliminate the other problems involved in distributed authentication.

You still have to think about:

Key management
Token expiration
Refresh tokens
Revocation
Authorization

So Which One Should You Use?

This is the part where generic tutorials often go wrong.

They say:

"Use JWT because it's modern."

That's not a useful architectural decision.

Start with the application.

A traditional web application

If most of your application is server-rendered and the server already owns the user's session state, a session-based approach can be a straightforward choice:

Browser
  ↓
Secure Cookie
  ↓
Session ID
  ↓
Server Session

A frontend + backend API

If you have separate clients communicating with an API, token-based authentication may fit naturally:

Web App
Mobile App
Desktop App
      ↓
     API
      ↓
Authentication

JWT can be useful here, but it isn't mandatory.

Cookies and sessions can also be used with APIs.

A distributed system

If several services need to verify the same authentication information, signed tokens can be convenient:

             API Gateway
                  ↓
       ┌──────────┼──────────┐
       ↓          ↓          ↓
   Service A  Service B  Service C
       ↓          ↓          ↓
       └──── Verify identity ──┘

Again, the architecture determines whether that benefit actually matters.

The Questions I Would Ask Before Choosing

Instead of starting with:

"Should I use JWT?"

I'd ask:

Who are the clients?

Is the application browser-based?

Do we need server-side session state?

How should logout work?

Do we need immediate credential revocation?

How long should authentication credentials live?

How will refresh work?

Where will credentials be stored?

Do multiple services need to verify the identity?

What threats does the application need to handle?

These questions tell you much more than simply choosing a technology because it's popular.

The Mental Model

If you remember only one thing from this article, remember this:

Cookie
→ How the browser stores and sends a value

Session
→ Authentication state maintained by the server

JWT
→ A signed token containing claims

And they can be combined:

Cookie + Session

or:

Cookie + JWT

or:

Authorization Header + JWT

They're not mutually exclusive.

That's the part that usually clears up the confusion.

Final Takeaway

Cookies, sessions, and JWTs are often discussed together, but they solve different problems.

A cookie is a mechanism for storing and sending data through the browser.

A session is server-side state that can be used to remember an authenticated user.

A JWT is a signed token that carries claims which a server can verify.

The right architecture depends on what you're building.

A small server-rendered application might be perfectly comfortable with cookies and server-side sessions.

A system serving multiple types of clients might benefit from token-based authentication.

A distributed system might find signed tokens useful for passing verifiable identity information between services.

None of those choices makes one technology universally better than the others.

The better question isn't:

"JWT or sessions?"

It's:

"Where should authentication state live, how should the client carry it, and how should the server verify and revoke it?"

Once you start thinking in those terms, cookies, sessions, and JWTs stop looking like competing technologies and start looking like different pieces of an authentication architecture.

How Does JWT Authentication Work? A Complete Guide!

Tanu Priya — Tue, 15 Sep 2026 10:23:19 +0000

You enter your email and password, click Login, and suddenly the application knows who you are.

A few seconds later, you open your profile and the backend already knows which user's data to return.

But HTTP doesn't work like that by itself.

Every request is independent. So after the login request finishes, how does the server recognize the next request?

One popular solution is JWT authentication.

JWT stands for JSON Web Token. It gives the client a signed piece of information that can be sent with later requests so the server can verify the request's identity.

A simplified flow looks like this:

Login
  ↓
Verify credentials
  ↓
Create JWT
  ↓
Send JWT to client
  ↓
Client sends JWT with API requests
  ↓
Server verifies JWT
  ↓
Request continues

The interesting part is what happens inside that flow.

Why Does Authentication Need a Token?

Imagine you log into an application:

POST /api/login

with:

{
  "email": "alex@example.com",
  "password": "mypassword"
}

The backend verifies the credentials.

At this point, it knows:

This is user 42.

But the next request could be:

GET /api/profile

There is nothing inherent in HTTP that tells the server:

"This is the same person who successfully logged in two seconds ago."

The application needs a way to carry that authentication information from one request to another.

That's where the token comes in.

After successful authentication, the server issues a JWT:

Login
  ↓
JWT issued
  ↓
Client keeps JWT
  ↓
JWT sent with future requests

The token becomes part of the application's authentication mechanism.

What Is Inside a JWT?

A JWT usually looks like this:

xxxxx.yyyyy.zzzzz

There are three sections:

Header.Payload.Signature

For example:

eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9
.
eyJzdWIiOiI0MiIsInJvbGUiOiJ1c2VyIn0
.
signature

These three parts have different jobs.

Header

The header describes the token.

A simplified example:

{
  "alg": "HS256",
  "typ": "JWT"
}

alg specifies the signing algorithm and typ identifies the token type.

Payload

The payload contains claims.

For example:

{
  "sub": "42",
  "role": "user",
  "iat": 1760000000,
  "exp": 1760003600
}

Here:

sub → subject / user identifier
role → application-specific role
iat → issued-at time
exp → expiration time

These claims allow the server to carry useful information about the token and its subject.

There is an important security detail here.

A normal JWT payload is not encrypted.

It is encoded so that the token can be represented as a compact string.

Anyone who obtains the token can generally decode its header and payload.

So don't put things like passwords, private keys, or other secrets inside the payload.

Signature

The signature is what protects the integrity of the signed token data.

Conceptually:

Header + Payload
       ↓
Signing algorithm
       +
Secret/private key
       ↓
Signature

The server can later verify the signature.

If someone changes:

{
  "role": "user"
}

to:

{
  "role": "admin"
}

without having the appropriate signing key, the signature will no longer match.

The server can reject the token.

So the signature doesn't hide the payload.

It helps answer:

Has the signed token data been changed, and does it have a valid signature?

What Happens When You Log In?

Let's follow the request from beginning to end.

The client sends:

POST /api/login
Content-Type: application/json

with:

{
  "email": "alex@example.com",
  "password": "mypassword"
}

The backend might do something like:

Receive credentials
       ↓
Find user
       ↓
Verify password
       ↓
Credentials valid?
       ↓
Create JWT
       ↓
Sign JWT
       ↓
Return token

The token might contain:

{
  "sub": "42",
  "exp": 1760003600
}

The server signs it using the configured key.

The response could then contain:

{
  "accessToken": "eyJ..."
}

The client can now use that access token when calling protected APIs.

The Next API Request

Now the frontend wants Alex's profile.

It sends:

GET /api/profile
Authorization: Bearer eyJ...

The backend receives the request and extracts the token.

It then verifies it.

Conceptually:

Request
  ↓
Extract JWT
  ↓
Verify signature
  ↓
Check expiration
  ↓
Validate required claims
  ↓
Identify user
  ↓
Continue request

If the token represents user 42, the application can use that identity when processing the request.

For example:

JWT
 ↓
sub = 42
 ↓
User 42
 ↓
Fetch profile
 ↓
Return profile

Notice something important here.

The user doesn't send their password with every request.

The JWT is the credential being presented for that request.

JWT Authentication in Express

In a Node.js application, JWT verification is often placed in middleware.

A simplified example:

function authenticate(req, res, next) {
    const header = req.headers.authorization;

    if (!header || !header.startsWith("Bearer ")) {
        return res.status(401).json({
            message: "Authentication required"
        });
    }

    const token = header.slice(7);

    try {
        const payload = jwt.verify(
            token,
            process.env.JWT_SECRET
        );

        req.user = payload;
        next();
    } catch {
        return res.status(401).json({
            message: "Invalid or expired token"
        });
    }
}

A protected route can then use the middleware:

app.get(
    "/api/profile",
    authenticate,
    getProfile
);

The request now has a predictable path:

GET /api/profile
       ↓
Authentication middleware
       ↓
Verify JWT
       ↓
Attach user to request
       ↓
Controller
       ↓
Database
       ↓
Response

The controller doesn't need to repeatedly implement JWT verification.

That responsibility stays in the authentication layer.

Authentication Is Still Different From Authorization

This is where many beginners mix things up.

Suppose the JWT tells the application that the current user is:

User ID: 42
Role: user

That establishes identity.

But now the user requests:

DELETE /api/users/100

The backend still has another question:

Is user 42 allowed to delete another user?

That's authorization.

So the complete process is:

JWT verification
      ↓
Who is this?
      ↓
User 42
      ↓
Authorization check
      ↓
Can User 42 perform this action?
      ↓
Allow / Reject

For example:

function requireAdmin(req, res, next) {
    if (req.user.role !== "admin") {
        return res.status(403).json({
            message: "Forbidden"
        });
    }

    next();
}

And:

app.delete(
    "/api/users/:id",
    authenticate,
    requireAdmin,
    deleteUser
);

JWT helps with authentication.

It does not automatically give the user permission to do everything.

What Does Stateless JWT Authentication Mean?

You've probably heard that JWT authentication is stateless.

The idea is that the server doesn't necessarily need to maintain a traditional server-side session for every access token.

With a session-based approach:

Client
  ↓
Session ID
  ↓
Server
  ↓
Session Store
  ↓
User

With a self-contained JWT:

Client
  ↓
JWT
  ↓
Server verifies token
  ↓
Claims identify subject

This can be convenient in systems with multiple backend instances.

Imagine:

                 Load Balancer
                      ↓
          ┌───────────┼───────────┐
          ↓           ↓           ↓
       Server A    Server B    Server C
          ↑           ↑           ↑
          └──── Verify JWT ───────┘

Each server can verify the token using the appropriate key.

There is no requirement for every request to first find a traditional session record.

But there's a catch.

Stateless doesn't mean there is no state anywhere in the authentication system.

Applications may still maintain state for refresh tokens, revocation, device sessions, security events, or other security controls.

Access Tokens and Refresh Tokens

Access tokens are often short-lived.

For example:

Access Token
↓
Expires in 15 minutes

If the access token expires, the user shouldn't necessarily have to log in again.

A common design is to use a refresh token.

Login
  ↓
Access Token + Refresh Token
  ↓
Access Token used for APIs
  ↓
Access Token expires
  ↓
Refresh Token
  ↓
New Access Token

This gives the application a way to keep a user signed in while limiting the lifetime of an individual access token.

Refresh tokens need careful handling because a stolen refresh token can have a much longer useful lifetime.

That's why real authentication systems often add things like refresh-token rotation and revocation.

What Happens When the JWT Expires?

JWTs commonly contain an expiration claim:

{
  "sub": "42",
  "exp": 1760003600
}

Once that time has passed, the token should no longer be accepted as a valid access token.

The API might respond with:

401 Unauthorized

The client can then use its refresh mechanism if one exists:

API request
    ↓
Access token expired
    ↓
401
    ↓
Refresh authentication
    ↓
New access token
    ↓
Retry request

This is one reason short-lived access tokens are useful.

If an access token is stolen, its useful lifetime can be limited.

It doesn't make theft harmless, but it can reduce the window of exposure.

Where Should You Store a JWT?

There isn't one universal answer.

JWTs are commonly transported using:

Authorization header

or:

Cookie

For browser applications, cookies can provide useful security controls.

For example:

Set-Cookie: access_token=...;
HttpOnly;
Secure;
SameSite=Lax

HttpOnly prevents normal page JavaScript from directly reading the cookie.

Secure tells the browser to send it only over HTTPS.

SameSite controls when the browser sends the cookie in cross-site situations.

Cookies aren't automatically safer in every situation, though. They introduce considerations such as CSRF protection and cross-origin configuration.

The right approach depends on the application's architecture and threat model.

What If Someone Steals the JWT?

A valid access token should be treated like a credential.

If an attacker gets one, they may be able to make authenticated API requests until the token expires or is otherwise invalidated.

For example:

Attacker
   ↓
Stolen JWT
   ↓
Authorization: Bearer <token>
   ↓
API
   ↓
Authenticated request

That's why protecting the token matters.

Common security practices include:

HTTPS
Short-lived access tokens
Secure refresh-token handling
Signature verification
Expiration checks
Issuer/audience validation when applicable
Restricted signing algorithms
Avoiding sensitive information in payloads

JWT doesn't provide all of these protections automatically.

The application has to implement the authentication system correctly.

JWT Is Not Encryption

This is probably the most common JWT misconception.

People see:

Header.Payload.Signature

and assume the contents are hidden.

They're not.

A normal signed JWT provides integrity, not confidentiality.

Think of it this way:

Encoding
→ Makes data representable as a token

Signing
→ Helps detect modification

Encryption
→ Hides the data

JWT signing and encryption solve different problems.

If the application needs sensitive information to remain confidential, it needs an appropriate encryption mechanism rather than simply putting that information into a signed JWT.

401 vs 403

JWT authentication also makes the difference between 401 and 403 easier to understand.

A 401 Unauthorized response generally means the request doesn't have valid authentication credentials.

For example:

No token
Invalid token
Expired token

In simple terms:

401
→ "I can't authenticate you."

A 403 Forbidden response means the server knows who the user is, but the user isn't allowed to perform the requested action.

Authenticated user
        ↓
Not enough permissions
        ↓
403

So:

401 → Authentication problem
403 → Authorization problem

JWT Doesn't Replace Good API Security

It can be tempting to think:

"We're using JWT, so our API is secure."

That's not how it works.

JWT only solves part of the problem.

A production authentication system still needs to think about:

Password storage
HTTPS
Token lifetime
Refresh tokens
Token theft
Revocation
Key management
Authorization
CSRF
XSS
Rate limiting
Account recovery
Brute-force protection

The exact list depends on the application.

JWT is a tool, not the entire security architecture.

JWT vs Sessions

JWT and sessions are often compared as if one has to replace the other.

In reality, both are valid approaches.

A session-based system might look like:

Login
  ↓
Create session
  ↓
Session ID in cookie
  ↓
Server looks up session

A JWT-based system might look like:

Login
  ↓
Create signed JWT
  ↓
Client receives token
  ↓
Client sends token
  ↓
Server verifies token

Sessions make some server-side operations, such as immediate session invalidation, straightforward.

JWTs can be useful when authentication information needs to be verified across multiple services without relying on a shared session store.

But JWT systems can become complicated once you add refresh tokens, revocation, key rotation, and multiple token types.

So don't choose JWT simply because it is popular.

Start with the requirements of the application.

The Complete JWT Flow

Putting everything together:

                    LOGIN
                      ↓
             Email + Password
                      ↓
                  Backend
                      ↓
             Verify credentials
                      ↓
                 Create JWT
                      ↓
                 Sign JWT
                      ↓
              Send to client
                      ↓
             Future API request
                      ↓
              Send JWT with request
                      ↓
               Verify JWT
                      ↓
             Validate its claims
                      ↓
               Identify user
                      ↓
             Check permissions
                      ↓
               Business logic
                      ↓
                  Database
                      ↓
                  Response

There are really two separate decisions happening:

Authentication
      ↓
"Who is making this request?"

Authorization
      ↓
"What is this user allowed to do?"

A Practical Mental Model

When you encounter JWT authentication in a project, think about it in this order:

1. User logs in
       ↓
2. Backend verifies credentials
       ↓
3. Backend issues a signed JWT
       ↓
4. Client sends the JWT with API requests
       ↓
5. Backend verifies the token
       ↓
6. Backend identifies the user
       ↓
7. Authorization checks permissions
       ↓
8. Request is allowed or rejected

And remember what each part actually means:

JWT
├── Header
│   └── Token metadata
│
├── Payload
│   └── Claims
│
└── Signature
    └── Integrity/authenticity check

The easiest mistake to make is thinking that JWT is simply a secure replacement for a password.

It isn't.

A JWT is a signed token carrying claims. The server uses it to authenticate requests without requiring the user's password on every request.

Once you understand that, the rest of the system starts making sense.

Login creates the authentication context.
The JWT carries that context between requests.
The server verifies the JWT.
Authorization decides what the authenticated user can actually do.

That's the core of JWT authentication.

Authentication vs Authorization: What Is the Difference?

Tanu Priya — Mon, 14 Sep 2026 09:34:52 +0000

You log into an application with your email and password.

The application checks your credentials and lets you in.

Now you open the admin dashboard.

The application checks something else: are you actually allowed to use the admin dashboard?

These two checks are related, but they aren't the same.

Authentication → Who are you?

Authorization  → What are you allowed to do?

A simple real-world example makes the difference easier to understand.

When you show your ID at an office entrance, security verifies that you are actually you.

That's authentication.

Once you're inside, your ID might allow you to enter some rooms but not others.

That's authorization.

The same idea is used inside web applications.

Authentication: Who Are You?

Authentication is about verifying a user's identity.

A typical login request might look like:

POST /api/login

with:

{
  "email": "alex@example.com",
  "password": "mypassword"
}

The backend receives the credentials and verifies them.

Conceptually:

Email + Password
       ↓
    Backend
       ↓
Find user
       ↓
Verify password
       ↓
Credentials valid?
       ↓
Authenticated

If everything checks out, the server now knows:

User ID: 42
Email: alex@example.com

But there is a problem.

HTTP is stateless.

The next request doesn't automatically know that the previous login succeeded.

That's where sessions, cookies, and tokens come in.

Sessions: Remembering the User

One common solution is a server-side session.

After successful login, the server creates a session:

User 42
   ↓
Session created
   ↓
Session ID = abc123

The server stores something similar to:

abc123 → User 42

The browser then needs to send that session ID with future requests.

This is commonly done using a cookie.

Login
  ↓
Create session
  ↓
Send session cookie
  ↓
Browser stores cookie
  ↓
Future requests include cookie

For example:

Set-Cookie: session_id=abc123

Later:

Cookie: session_id=abc123

The server can look up the session and determine which user is making the request.

So the flow becomes:

Browser
   ↓
Cookie
   ↓
Session ID
   ↓
Server
   ↓
User 42

The browser doesn't need to send the user's password again.

What About Tokens?

Another common approach is token-based authentication.

After login, the server generates a token and gives it to the client.

Login
  ↓
Verify credentials
  ↓
Generate token
  ↓
Send token

The client can then include the token with API requests.

A common format is:

Authorization: Bearer <token>

The backend receives the token and verifies it.

If the token is valid, the backend can identify the user and continue processing the request.

This approach is particularly common with APIs and applications where the client and backend are separate systems.

Sessions vs Tokens

The difference is mainly about where authentication state is maintained.

With a traditional session:

Client
  ↓
Session ID
  ↓
Server
  ↓
Session data

With a self-contained token:

Client
  ↓
Token
  ↓
Server validates token

Neither approach is universally better.

The choice depends on the application, security requirements, infrastructure, and how authentication needs to be managed.

Authorization: What Can the User Do?

Now let's say Alex has successfully logged in.

Authentication is complete.

But Alex tries:

GET /api/admin/users

Being logged in doesn't automatically give Alex access to this endpoint.

The backend needs to ask:

Does this user have permission to access this resource?

For example:

Alex
Role: user

/admin/users
     ↓
Access denied

An administrator might have:

Sarah
Role: admin

/admin/users
     ↓
Access allowed

That's authorization.

The complete flow looks like:

Request
   ↓
Authentication
   ↓
Who is the user?
   ↓
Authorization
   ↓
Is the user allowed?
   ↓
Business logic

This distinction becomes especially important when building APIs.

Authentication and Authorization in Middleware

In a Node.js application, authentication and authorization are often separated into middleware.

For example:

function authenticate(req, res, next) {
    const token = req.headers.authorization;

    if (!token) {
        return res.status(401).json({
            message: "Authentication required"
        });
    }

    const user = verifyToken(token);

    if (!user) {
        return res.status(401).json({
            message: "Invalid token"
        });
    }

    req.user = user;

    next();
}

The middleware identifies the user and attaches that information to the request.

Then authorization can happen separately:

function requireAdmin(req, res, next) {
    if (req.user.role !== "admin") {
        return res.status(403).json({
            message: "Access denied"
        });
    }

    next();
}

The route can combine both:

app.get(
    "/admin/users",
    authenticate,
    requireAdmin,
    getUsers
);

Now the request goes through two different checks.

GET /admin/users
       ↓
authenticate
       ↓
Who is this?
       ↓
requireAdmin
       ↓
Can they access this?
       ↓
getUsers

That's a much cleaner design than putting all of the security logic inside every controller.

401 vs 403

You'll often see two status codes when working with protected APIs.

401 Unauthorized

The server doesn't have valid authentication information.

For example:

No token
Invalid token
Expired credentials

Think:

"I don't know who you are."

403 Forbidden

The server knows who you are, but you don't have permission to perform the operation.

For example:

Logged in as normal user
        ↓
Trying to delete another user
        ↓
403 Forbidden

Think:

"I know who you are, but you can't do this."

A simple way to remember it:

401 → Authentication problem

403 → Authorization problem

Roles and Permissions

Authorization is often implemented using roles.

For example:

User
Admin
Moderator
Editor

An admin might be allowed to:

Create users
Delete users
Manage settings
View reports

while a normal user might only be able to:

View their profile
Create comments
View posts

For larger applications, permissions can be more granular:

users:read
users:create
users:delete

posts:read
posts:update
posts:delete

This makes it possible to give users exactly the permissions they need instead of relying entirely on broad roles.

Don't Trust the Frontend for Authorization

This is one of the most important points.

Suppose your frontend hides the delete button from normal users:

if (user.role === "admin") {
    showDeleteButton();
}

That's good for the UI.

But it isn't a security mechanism.

A user can still manually send the request:

DELETE /api/users/123

directly to your backend.

That's why authorization must happen on the server.

Frontend
   ↓
API Request
   ↓
Backend
   ↓
Authentication
   ↓
Authorization
   ↓
Allow / Reject

The frontend decides what the user sees.

The backend decides what the user is actually allowed to do.

Where Passwords Fit In

Authentication often starts with a password, but passwords should not be stored directly in the database.

Instead, applications use password-hashing algorithms designed for secure password storage.

Conceptually:

Password
   ↓
Password hashing
   ↓
Stored password hash

During login, the submitted password is verified against the stored hash.

The application should not need to retrieve the original password.

This is one of the basic security boundaries every authentication system needs to get right.

Putting Everything Together

A typical protected API request might look like this:

                    LOGIN
                      ↓
              Verify credentials
                      ↓
                Authentication
                      ↓
            Session / Cookie / Token
                      ↓
              Future API Request
                      ↓
                Authentication
                      ↓
                 Authorization
                      ↓
                 Controller
                      ↓
                  Database
                      ↓
                  Response

Each part has a different job.

Authentication
→ Establish the user's identity

Authorization
→ Decide what that user can access

Sessions, cookies, and tokens help the application maintain or carry authentication information between requests.

They don't decide what the user is allowed to do.

The Mental Model

Whenever you're building a protected API, ask two questions:

1. Who is making this request?

2. Is this user allowed to perform this action?

The first question is authentication.

The second is authorization.

For example:

User logs in
     ↓
Authentication
     ↓
"I know who you are."
     ↓
User requests admin dashboard
     ↓
Authorization
     ↓
"I know who you are,
but are you allowed here?"

That's the core difference.

Authentication establishes identity. Authorization controls access.

Once you understand that distinction, sessions, cookies, tokens, roles, permissions, 401, and 403 all start fitting into the same picture.

And when you're designing a backend, keeping authentication and authorization as separate responsibilities makes the security model much easier to reason about.

Database Indexing: Why Can an Index Make Queries Faster?

Tanu Priya — Sun, 13 Sep 2026 07:53:00 +0000

Imagine your users table has 10 million rows.

Your application frequently runs:

SELECT *
FROM users
WHERE email = 'alex@example.com';

The database returns the result, but how did it find one user among millions?

It could check the rows one by one.

Or, it could use an index.

An index gives the database a more efficient way to find data without scanning the entire table every time.

But indexes aren't magic. They consume storage, add overhead to writes, and aren't useful for every query.

Understanding that trade-off is what makes indexing useful from a developer's perspective.

1. What Is a Database Index?

A database index is an additional data structure that helps the database locate rows more efficiently.

Think about a book.

If you want to find a particular topic, you don't normally read every page. You use the index to find where that topic appears.

A database index follows a similar idea:

Without Index

Query
  ↓
Scan rows
  ↓
Find matching rows
  ↓
Return result

With an appropriate index:

Query
  ↓
Search Index
  ↓
Locate relevant rows
  ↓
Read required data
  ↓
Return result

The important idea is:

An index can reduce the amount of data the database needs to examine.

That's the real reason indexes can make queries faster.

2. What Happens Without an Index?

Suppose we have:

users

id    name      email
-----------------------------
1     Rahul     rahul@mail.com
2     Priya     priya@mail.com
3     Alex      alex@mail.com
4     John      john@mail.com
...

Now run:

SELECT *
FROM users
WHERE email = 'alex@mail.com';

If there isn't a useful index on email, the database may perform a sequential scan.

Conceptually:

Row 1 → Check
Row 2 → Check
Row 3 → Match
Row 4 → Check
...

With a small table, this isn't a big deal.

But imagine the table contains:

10,000 rows
1 million rows
100 million rows

Scanning a large portion of the table for every lookup can become expensive.

3. What Changes When We Add an Index?

We can create an index:

CREATE INDEX idx_users_email
ON users(email);

Now the database has another structure that can help it search for email values.

Conceptually:

Query
  ↓
Email Index
  ↓
alex@mail.com
  ↓
Row location
  ↓
User row

Instead of searching through potentially millions of table rows, the database can first search the much more structured index.

That's the fundamental advantage.

The database isn't necessarily processing the same amount of data faster.

It's potentially processing much less data.

4. How Does an Index Search So Quickly?

Many traditional database indexes use tree-based structures, particularly B-tree-style indexes.

You can think about a simplified tree like this:

                 50
              /      \
            25        75
           /  \      /  \
         10   40    60   90

If you're searching for 60, you don't need to inspect every value.

You start at the top:

60 > 50
  ↓
Go right

60 < 75
  ↓
Go left

60 found

Real database indexes are much more sophisticated and optimized for storage systems, but the underlying idea is useful:

Organize values so large portions of the search space can be eliminated quickly.

5. Indexes Usually Don't Store the Whole Table

An index isn't simply a second copy of your table.

Conceptually, an index might contain:

email              → row location

alex@mail.com      → location
john@mail.com      → location
priya@mail.com     → location
rahul@mail.com     → location

The exact structure depends on the database and index type.

The index provides information that helps the database locate the underlying data.

So you can think of your database as:

Database
   |
   ├── Table
   |
   └── Index

The table stores the actual records.

The index provides an additional access path to those records.

6. An Index Doesn't Guarantee Faster Queries

This is an important detail.

Suppose your table contains only 20 rows.

You create an index:

CREATE INDEX idx_users_email
ON users(email);

The database might still choose a table scan.

Why?

Because scanning 20 rows could be cheaper than using the index and then fetching the corresponding table rows.

The database's query optimizer decides which execution strategy appears cheaper.

Conceptually:

Query
  ↓
Query Optimizer
  ↓
Evaluate possible plans
  ↓
Choose execution plan

So:

Creating an index gives the database an option. It doesn't force the database to use it.

7. How Do You Know What the Database Is Doing?

Instead of guessing, inspect the execution plan.

For example, PostgreSQL supports:

EXPLAIN
SELECT *
FROM users
WHERE email = 'alex@example.com';

You can also inspect actual execution behavior with:

EXPLAIN ANALYZE
SELECT *
FROM users
WHERE email = 'alex@example.com';

This can help you understand:

Was an index used?

Was a sequential scan performed?

How many rows were expected?

How many rows were actually processed?

How much time did the operation take?

This is much more useful than simply assuming an index is helping.

8. Indexes Are a Trade-Off

If indexes only had benefits, we'd put them on every column.

But they don't.

Indexes require additional:

Storage
Memory
Write work
Maintenance

Suppose you insert a new user:

INSERT INTO users
(name, email)
VALUES
('Alex', 'alex@example.com');

The database needs to update the table.

If email is indexed, it also needs to maintain that index.

Conceptually:

INSERT
  ↓
Update Table
  ↓
Update Index

If a table has many indexes, writes may require more index maintenance.

So indexing creates a trade-off:

Faster Reads
     ↕
More Write Overhead

Good database design is about finding the right balance.

9. Which Columns Should You Index?

Don't start by looking at the schema and saying:

"This column looks important. Let's index it."

Start with your queries.

Suppose your application frequently executes:

SELECT *
FROM users
WHERE email = ?;

An index on email could be useful.

If it frequently executes:

SELECT *
FROM orders
WHERE user_id = ?;

an index on user_id may be worth considering.

If it frequently executes:

SELECT *
FROM products
ORDER BY created_at DESC
LIMIT 20;

an index involving created_at might help.

The key question is:

How does the application actually access the data?

Indexes should follow real access patterns.

10. Primary Keys and Indexes

Primary keys are common candidates for efficient lookups.

For example:

CREATE TABLE users (
    id BIGINT PRIMARY KEY,
    name VARCHAR(100),
    email VARCHAR(255)
);

A database generally creates or enforces an index-like structure associated with the primary key, depending on the database system.

That makes queries such as:

SELECT *
FROM users
WHERE id = 42;

a natural use case for indexed access.

This is one reason primary keys are so commonly used for direct record retrieval.

11. Unique Indexes

Indexes can also help enforce data integrity.

Suppose every user must have a unique email:

CREATE UNIQUE INDEX idx_users_email
ON users(email);

Now the index isn't only useful for finding users.

It also helps enforce the rule that two users cannot have the same email value under that uniqueness constraint.

So an index can contribute to both:

Performance
+
Data Integrity

12. Composite Indexes

Real applications often filter using multiple columns.

Consider:

SELECT *
FROM orders
WHERE user_id = 42
AND status = 'completed';

You might consider:

CREATE INDEX idx_orders_user_status
ON orders(user_id, status);

This is a composite index.

It indexes a combination of columns rather than just one.

Conceptually:

(user_id, status)

Composite indexes can be extremely useful when they match common query patterns.

But there's an important detail.

13. Column Order Matters

Consider:

CREATE INDEX idx_orders_user_status
ON orders(user_id, status);

The index is organized around:

user_id
   ↓
status

A query using:

WHERE user_id = 42

aligns naturally with the leading column.

A query using only:

WHERE status = 'completed'

is a different situation.

This is why composite indexes should be designed based on the queries they are intended to support.

Don't simply combine every column that appears in your queries.

Think about the actual access patterns.

14. Indexes Can Help With Sorting

Indexes aren't only useful for WHERE conditions.

Consider:

SELECT *
FROM products
ORDER BY created_at DESC
LIMIT 20;

Without a suitable access path, the database may need to:

Read rows
   ↓
Sort rows
   ↓
Return 20

With an appropriate index, the database may be able to access rows in a useful order.

Conceptually:

Index
  ↓
Rows already accessible in useful order
  ↓
Return required rows

Whether the database actually chooses this strategy depends on the query and execution plan.

15. Indexes Can Help With Range Queries

Indexes are also useful for many range queries.

For example:

SELECT *
FROM products
WHERE price >= 1000
AND price <= 5000;

An ordered index on price can help locate the relevant range.

Conceptually:

Price Index

500
1000  ← Start
1500
2000
3000
4000
5000  ← End
6000

Instead of searching the entire table, the database can navigate toward the relevant part of the index.

This is useful for things such as:

Prices
Dates
Timestamps
Numeric values
IDs

16. Not Every Query Benefits From an Index

Consider:

SELECT *
FROM users
WHERE status = 'active';

Suppose the table contains 10 million users and 9 million are active.

An index on status may not provide much benefit for this particular query because most of the table matches anyway.

The database may decide that scanning the table is more efficient.

This is related to selectivity.

A condition is highly selective when it narrows the possible rows significantly.

For example:

email = 'alex@example.com'

might match one row.

While:

status = 'active'

might match millions.

But don't treat selectivity as an absolute rule.

The optimizer considers many factors when choosing a plan.

17. Indexes Don't Fix Bad Queries

Suppose you create an index on:

user_id

and then execute:

SELECT *
FROM orders
WHERE user_id = 42;

What if user 42 has 10 million orders?

The index can help locate those orders, but the database still has to process and return a huge result.

The problem isn't necessarily the missing index anymore.

The query itself may be asking for too much data.

A better API might use:

SELECT id, total, created_at
FROM orders
WHERE user_id = 42
ORDER BY created_at DESC
LIMIT 50;

Indexing and query design need to work together.

18. Indexes and Pagination

This becomes particularly important for APIs.

A common pagination approach is:

SELECT *
FROM products
ORDER BY id
LIMIT 20 OFFSET 20000;

For large offsets, the database may still need to process or skip many preceding rows.

An alternative is cursor or keyset pagination:

SELECT *
FROM products
WHERE id > 20000
ORDER BY id
LIMIT 20;

With a suitable index, the database can start from a known position.

This is a good example of a broader lesson:

Performance comes from good query design plus appropriate indexing.

An index isn't a substitute for understanding how your data is accessed.

19. Too Many Indexes Can Become a Problem

Imagine this table:

users

id
name
email
phone
age
city
status
created_at

You might create an index on every column.

It sounds optimized.

But now every relevant write may require maintaining several index structures.

You also consume more storage.

And some indexes might never be used by important queries.

So the goal isn't:

Maximum Indexes

It's:

Useful Indexes

A good index is one that solves a real access problem.

20. A Practical Example

Imagine an e-commerce application.

The orders table contains:

50 million rows

The application frequently runs:

SELECT id, total, created_at
FROM orders
WHERE user_id = ?
ORDER BY created_at DESC
LIMIT 20;

As the table grows, the endpoint becomes slower.

Instead of immediately adding more backend servers, you inspect the database execution plan.

You discover that the database is doing far more work than necessary.

Now you can evaluate whether an index designed around:

user_id
created_at

matches the query pattern.

After making the change, you check the execution plan again and benchmark the query.

That's the right way to approach indexing.

Not:

"The query is slow, so add an index."

But:

Slow Query
    ↓
Inspect Plan
    ↓
Understand the Work
    ↓
Design Appropriate Index
    ↓
Measure Again

21. Indexing Is Part of System Design

Database indexing might look like a small implementation detail.

But database performance affects the entire application.

Consider:

User
  ↓
API Request
  ↓
Backend
  ↓
Database Query
  ↓
Slow Scan

If the query becomes slower:

Database latency increases
        ↓
API latency increases
        ↓
Requests stay active longer
        ↓
More concurrent requests
        ↓
Higher resource usage

A well-designed index can sometimes reduce that chain by making the database work more efficiently.

That's why database optimization is part of system design.

A Simple Mental Model

When you see a slow query, don't immediately think:

"I need an index."

Instead ask:

What is the database doing?
        ↓
Is it scanning too much data?
        ↓
Can an index provide a better access path?
        ↓
Does the query itself need improvement?
        ↓
What does EXPLAIN show?
        ↓
Did performance actually improve?

And remember the fundamental trade-off:

                INDEX
                  |
        ┌─────────┴─────────┐
        ↓                   ↓
   Faster Reads        Additional Cost
                            |
                   ┌────────┼────────┐
                   ↓        ↓        ↓
                Storage   Writes  Maintenance

The biggest misconception about indexing is that an index simply makes the database "search faster."

A better way to think about it is:

An index gives the database a more efficient access path to the data.

If that access path matches the query, the database may avoid scanning, sorting, or processing large amounts of unnecessary data.

But every index comes with a cost.

So good indexing isn't about creating indexes everywhere.

It's about understanding your application's most important queries, inspecting how the database executes them, and creating indexes that reduce expensive work without creating unnecessary overhead elsewhere.

The best index isn't the one you can create. It's the one your workload actually needs.

What Actually Happens When You Run a Database Query?

Tanu Priya — Sat, 12 Sep 2026 10:28:23 +0000

You write something that looks almost too simple:

SELECT *
FROM users
WHERE id = 42;

The database returns a row, and from the application's perspective, the job is done.

But the database didn't simply "look through the table" and return the answer.

Behind that one query, the database may parse SQL, check permissions, build an execution plan, choose an index, read pages from memory or disk, filter rows, perform joins or sorting, and finally return the result to your application.

Understanding this process makes databases much easier to reason about.

It also explains why some queries take milliseconds while others take seconds.

Let's follow a query from the moment your application sends it until the result comes back.

1. The Application Sends a Query

Imagine your backend executes:

const result = await db.query(
  "SELECT * FROM users WHERE id = 42"
);

The application sends the query to the database server.

The simplified flow is:

Application
     ↓
Database Connection
     ↓
SQL Query
     ↓
Database Server

The database receives the SQL statement as text or through a database protocol.

At this point, the database still has to figure out what the query means and how it should execute it.

2. The Database Receives the Request

The database server receives the query through an existing or newly established connection.

In a production application, there is often a connection pool between your application and database.

Instead of creating a completely new database connection for every query, the application can reuse existing connections.

Conceptually:

Application
     ↓
Connection Pool
     ↓
Database

This matters because creating and maintaining connections has overhead.

A connection pool allows multiple requests to efficiently share a controlled number of database connections.

3. The Database Parses the SQL

Now the database needs to understand the query.

Consider:

SELECT name
FROM users
WHERE id = 42;

The database parses the SQL syntax and builds an internal representation of the statement.

It needs to understand things such as:

SELECT → What data do we want?
FROM   → Which table?
WHERE  → Which conditions?

Conceptually:

SQL Query
   ↓
Parser
   ↓
Internal Representation

If the SQL syntax is invalid, execution stops here.

For example:

SELEC name FROM users;

would result in a syntax error.

The database cannot execute something it cannot understand.

4. The Database Checks Permissions

The database also needs to determine whether the requesting user is allowed to perform the operation.

For example, a database user might have permission to:

SELECT

but not:

DELETE

If the application attempts an operation it isn't authorized to perform, the database can reject the query.

So the flow is more like:

Query
  ↓
Parse
  ↓
Check Permissions
  ↓
Continue

This is one reason database credentials and permissions should be carefully configured.

Your application should generally have only the database privileges it actually needs.

5. The Query Optimizer Enters the Picture

This is where things become interesting.

Suppose you run:

SELECT *
FROM users
WHERE email = 'alex@example.com';

The database has multiple ways to execute this query.

It could scan every row:

User 1
User 2
User 3
User 4
...
User 1,000,000

Or, if there is an index on email, it could use the index to find the relevant row much more efficiently.

The database needs to decide:

What is the most efficient way to execute this query?

That's the job of the query optimizer.

6. What Is a Query Execution Plan?

The optimizer creates an execution plan.

The plan describes how the database intends to execute the query.

For example:

Query
 ↓
Use index on email
 ↓
Find matching row
 ↓
Fetch row from table
 ↓
Return result

Another query might produce:

Query
 ↓
Scan table
 ↓
Filter rows
 ↓
Sort results
 ↓
Return result

The database chooses a plan based on information such as:

available indexes
estimated number of rows
filtering conditions
table statistics
join relationships
sorting requirements
database configuration

The exact optimizer behavior depends on the database engine.

7. Full Table Scan

Let's say you run:

SELECT *
FROM users
WHERE age = 25;

If there is no useful index, the database may need to inspect many rows.

Conceptually:

Users Table

Row 1 → age 21
Row 2 → age 32
Row 3 → age 25 ✓
Row 4 → age 41
Row 5 → age 25 ✓
...

The database checks rows to determine which ones satisfy:

age = 25

This is often called a table scan or sequential scan, depending on the database.

For a small table, this may be completely fine.

For a table containing hundreds of millions of rows, scanning a huge amount of data can become expensive.

8. Indexes Change the Game

Now suppose you create:

CREATE INDEX idx_users_email
ON users(email);

Then you execute:

SELECT *
FROM users
WHERE email = 'alex@example.com';

The database may be able to use the index instead of scanning the entire table.

Conceptually:

Without Index

Query
 ↓
Scan many rows
 ↓
Find matching row

With an appropriate index:

Query
 ↓
Index
 ↓
Locate matching value
 ↓
Fetch row

This can dramatically reduce the amount of data the database needs to inspect.

But indexes aren't magic.

They also consume storage and add work to inserts, updates, and deletes.

9. Why Not Create an Index on Everything?

It might sound logical to create an index for every column.

But that's usually a bad idea.

Suppose a table has:

id
name
email
age
city
phone
created_at
status

You could theoretically create indexes on all of them.

But indexes have costs.

When you insert a new row:

INSERT
  ↓
Update Table
  ↓
Update Relevant Indexes

More indexes can mean more maintenance work.

They also consume storage.

So indexing is a trade-off:

Faster Reads
     ↕
More Storage + Write Overhead

Good database design means choosing indexes based on actual query patterns.

10. The Database Reads Data Pages

A database doesn't usually think about data exactly like your application does.

It manages data in units such as pages or blocks, depending on the database system.

Conceptually:

Database
 ├── Page 1
 ├── Page 2
 ├── Page 3
 ├── Page 4
 └── ...

Rows are stored within these structures.

When the database needs data, it needs to access the relevant pages.

And this leads to one of the most important performance concepts:

Memory is much faster than storage.

11. Database Memory and Buffer Cache

Databases try to avoid reading from disk unnecessarily.

Frequently accessed data can remain in memory through mechanisms such as a database buffer pool or cache.

Conceptually:

Query
  ↓
Memory?
  |
  ├── Yes → Use cached page
  |
  └── No
       ↓
      Storage
       ↓
     Load page
       ↓
      Memory

If the required data is already in memory, the database can avoid a slower storage read.

This is one reason repeated queries can behave very differently from a first query, depending on the workload and cache state.

12. The Database Doesn't Always Read the Entire Row

Suppose your query is:

SELECT name
FROM users
WHERE id = 42;

You only asked for:

name

The database doesn't necessarily need to return every column to the application.

This is one reason it's often better to avoid:

SELECT *

when you only need a few columns.

Instead:

SELECT name, email
FROM users
WHERE id = 42;

This makes the requested data explicit and can reduce the amount of data that needs to be processed or transferred.

13. What Happens With Multiple Conditions?

Consider:

SELECT *
FROM products
WHERE category = 'laptops'
AND price < 50000;

Now the database needs to evaluate multiple conditions.

It may use:

Index
   ↓
Find candidate rows
   ↓
Check category
   ↓
Check price
   ↓
Return matching rows

The optimizer determines the actual execution strategy.

This is why two queries that look similar to a developer can have very different performance characteristics.

14. Joins Make Queries More Interesting

Suppose you have:

users
orders

and want to find orders belonging to a particular user.

You might write:

SELECT users.name, orders.total
FROM users
JOIN orders
ON users.id = orders.user_id
WHERE users.id = 42;

Now the database has to combine information from two tables.

Conceptually:

Users
   ↓
Match user_id
   ↓
Orders
   ↓
Filter
   ↓
Result

The database has different algorithms for performing joins, and the optimizer chooses a strategy based on the available information and estimated cost.

This is one of the strengths of relational databases.

15. Sorting Also Costs Work

Consider:

SELECT *
FROM products
ORDER BY price DESC;

The database needs to produce the rows in price order.

Depending on the query and available indexes, it may need to perform a sort.

Conceptually:

Rows
 ↓
Read
 ↓
Sort
 ↓
Return

Sorting a small number of rows is usually cheap.

Sorting millions of rows can be much more expensive.

This is why queries involving large datasets should be designed carefully.

16. LIMIT Can Reduce Unnecessary Work

Suppose you only need the latest 20 posts.

Instead of:

SELECT *
FROM posts
ORDER BY created_at DESC;

you might use:

SELECT *
FROM posts
ORDER BY created_at DESC
LIMIT 20;

Now the database knows that the application only needs a limited number of results.

With a suitable index and execution plan, this can be significantly more efficient than retrieving a huge result set.

This is especially important for APIs that power feeds, search results, dashboards, and admin panels.

17. Aggregations Require Computation

Consider:

SELECT COUNT(*)
FROM orders
WHERE user_id = 42;

The database needs to calculate the result.

Other examples include:

SUM()
AVG()
MIN()
MAX()
COUNT()

For example:

SELECT
    category,
    COUNT(*)
FROM products
GROUP BY category;

The database now needs to:

Read data
   ↓
Group rows
   ↓
Calculate counts
   ↓
Build result

Aggregation queries can become expensive when they operate on large datasets.

18. Transactions Change How Queries Are Executed

Now consider:

BEGIN;

UPDATE accounts
SET balance = balance - 1000
WHERE id = 1;

UPDATE accounts
SET balance = balance + 1000
WHERE id = 2;

COMMIT;

These queries are part of one transaction.

The database needs to ensure that the transaction follows its consistency and durability rules.

If something goes wrong before the transaction commits:

BEGIN
 ↓
UPDATE
 ↓
ERROR
 ↓
ROLLBACK

The database can undo the transaction according to its transactional model.

Transactions are essential for operations where multiple changes need to behave as one logical unit.

19. Concurrency Makes Databases Harder

Imagine two users try to purchase the last available product at exactly the same time.

Both requests might execute:

Check inventory
     ↓
Inventory = 1
     ↓
Purchase

If the system isn't designed correctly, both requests might believe the product is available.

Databases provide mechanisms such as:

locks
isolation levels
MVCC
transactions

to manage concurrent operations.

The exact behavior depends on the database engine and configuration.

This is where database design becomes much more than simply writing SQL.

20. MVCC and Concurrent Reads

Many modern relational databases use some form of Multi-Version Concurrency Control (MVCC).

The basic idea is that readers and writers can often work concurrently without every read blocking every write.

Conceptually:

Transaction A
     ↓
Reads version 1

Transaction B
     ↓
Creates version 2

Depending on the database's isolation model, Transaction A can continue seeing a consistent view while Transaction B modifies newer data.

The exact implementation differs between databases, but the larger idea is important:

Databases need sophisticated mechanisms to handle many operations happening at the same time.

21. The Database Produces the Result

Once the database has completed the execution plan, it produces the result.

For example:

Query
 ↓
Parse
 ↓
Optimize
 ↓
Execute
 ↓
Read Data
 ↓
Filter / Join / Sort
 ↓
Result

The result might be:

id | name  | email
-------------------------
42 | Alex  | alex@mail.com

The database then sends the result back through the database connection.

22. The Backend Receives the Result

Your backend receives the database result:

const result = await db.query(
    "SELECT name, email FROM users WHERE id = 42"
);

The application can then transform the database representation into an API response.

For example:

{
  "id": 42,
  "name": "Alex",
  "email": "alex@example.com"
}

The client doesn't need to know how the database produced the result.

The backend acts as the boundary between the API and the data layer.

23. The Entire Journey

Let's put everything together.

Suppose your application executes:

SELECT name
FROM users
WHERE id = 42;

A simplified journey is:

Application
     ↓
Connection Pool
     ↓
Database Server
     ↓
Parse SQL
     ↓
Check Permissions
     ↓
Query Optimizer
     ↓
Execution Plan
     ↓
Index / Table Scan
     ↓
Memory / Storage
     ↓
Filter Rows
     ↓
Build Result
     ↓
Send Result
     ↓
Application

One SQL statement can therefore trigger a surprisingly large amount of work.

24. Why Some Queries Are Slow

When someone says:

"The database is slow."

that doesn't necessarily mean the database itself is the problem.

The issue could be:

Missing Index
       ↓
Large Table Scan
       ↓
Too Many Rows
       ↓
Expensive Join
       ↓
Large Sort
       ↓
Lock Contention
       ↓
Slow Storage
       ↓
Connection Pool Exhaustion

Or the query might simply be asking the database to do an enormous amount of work.

This is why database performance tuning should start with understanding what the database is actually doing.

25. EXPLAIN Shows the Database's Plan

Most major relational databases provide tools for inspecting query execution plans.

For example, PostgreSQL supports:

EXPLAIN
SELECT *
FROM users
WHERE email = 'alex@example.com';

You can also use:

EXPLAIN ANALYZE
SELECT *
FROM users
WHERE email = 'alex@example.com';

The output can help you understand things such as:

Which index was used?
Was a table scan performed?
How many rows were estimated?
How many rows were actually processed?
How much time did each operation take?

This is one of the most useful tools for understanding slow queries.

Instead of guessing:

"Maybe I need an index."

you can inspect what the database is actually doing.

26. Query Optimization Is About Reducing Work

A useful mental model is:

A fast query usually isn't doing less work by accident. It has been designed so the database can avoid unnecessary work.

For example:

Bad Pattern
Query
 ↓
Scan 10 million rows
 ↓
Filter
 ↓
Sort
 ↓
Return 20 rows

A better execution strategy might be:

Query
 ↓
Use Index
 ↓
Find relevant rows
 ↓
Return 20 rows

The goal isn't simply:

"Make the database faster."

It's:

"Make the database do less unnecessary work."

27. Why Database Indexes Are So Important

Suppose your API frequently executes:

SELECT *
FROM users
WHERE email = ?;

If email is frequently used for lookups, an index may make sense.

Without an index:

Query
 ↓
Potentially inspect many rows

With an appropriate index:

Query
 ↓
Index lookup
 ↓
Relevant row(s)

But indexes should be based on actual workload.

An index that helps one query may not help another.

Database optimization is therefore closely connected to understanding how your application accesses its data.

28. The Database Is Not Just Storage

A common beginner mental model is:

Backend
   ↓
Database
   ↓
Save data

A database is much more than a place to store rows.

It is responsible for things such as:

Parsing
Query Planning
Indexing
Data Retrieval
Transactions
Concurrency
Consistency
Durability
Caching
Recovery

That's why databases are such complex pieces of software.

Your SQL query may be only a few lines long, but the database engine underneath it is doing significant work.

29. What Happens in a Production Application?

A real application may look more like:

                 Client
                    ↓
                Backend
                    ↓
              Connection Pool
                    ↓
                Database
                    ↓
        ┌───────────┼───────────┐
        ↓           ↓           ↓
      Cache       Indexes      Storage
                    ↓
                 Query
                    ↓
               Execution
                    ↓
                Response

And there may be additional infrastructure such as:

Read Replicas
Connection Proxies
Monitoring
Backups
Replication
Sharding

This is where database queries connect directly to system design.

A query that takes 5 milliseconds on a small dataset may behave very differently when the database has billions of rows and thousands of concurrent requests.

30. A Query Is a Request for Work

The most useful way to think about a database query is not:

"I'm asking the database for some data."

Instead think:

"I'm asking the database to perform a specific computation over stored data."

For:

SELECT name
FROM users
WHERE age > 18
ORDER BY name
LIMIT 20;

the database has to:

Find users
   ↓
Filter age > 18
   ↓
Sort by name
   ↓
Take 20
   ↓
Return result

The SQL statement describes what you want.

The database decides how to produce it.

That separation is one of the most important concepts in database systems.

A Simple Mental Model

Whenever you execute:

SELECT *
FROM users
WHERE id = 42;

think:

Application
     ↓
Connection
     ↓
Database
     ↓
Parse Query
     ↓
Check Permissions
     ↓
Build Execution Plan
     ↓
Choose Index / Scan
     ↓
Read Memory / Storage
     ↓
Filter / Join / Sort if needed
     ↓
Build Result
     ↓
Return Result
     ↓
Application

And when a query becomes slow, don't immediately blame the database.

Ask:

What is the database actually doing?

Is it scanning millions of rows?

Is it missing an appropriate index?

Is it sorting a huge dataset?

Is a join producing far more rows than expected?

Is the database waiting on locks?

Is the application exhausting its connection pool?

Is the query simply doing too much work?

Tools such as execution plans can help answer these questions.

The biggest lesson is simple:

SQL describes what you want. The database figures out how to get it.

Once you understand that process, indexes, query optimization, transactions, joins, caching, connection pooling, and database scaling stop feeling like isolated concepts.

They become different ways of controlling how much work the database has to perform to answer your application's questions.

SQL vs NoSQL: Which Database Should You Use?

Tanu Priya — Fri, 11 Sep 2026 09:49:45 +0000

Choosing a database is one of those decisions that looks simple when you're starting a project.

You might hear:

"Use PostgreSQL."

Someone else says:

"MongoDB is easier."

Another developer recommends:

"Just use Redis."

And suddenly a simple application has turned into a database debate.

The reality is that SQL vs NoSQL is not about finding the universally better database.

It's about understanding how your application stores, reads, updates, and relates data.

The right database depends on your data model, consistency requirements, query patterns, scale, and how the application is expected to evolve.

Let's break down the actual differences.

1. What Is a Database?

Before comparing SQL and NoSQL, let's understand the problem a database solves.

Imagine an e-commerce application.

You need to store:

Users
Products
Orders
Payments
Reviews
Inventory

Without a database, you would need some way to persist all this information and retrieve it efficiently.

A database provides:

Application
     ↓
Database
     ↓
Persistent Data

The backend can ask questions such as:

"Find this user."

"Give me all products in this category."

"Which orders belong to this customer?"

"How many products are in stock?"

Different databases are optimized for different types of questions.

That's where SQL and NoSQL start to differ.

2. What Is SQL?

SQL databases are relational databases.

SQL stands for Structured Query Language, which is commonly used to interact with relational databases.

Popular relational databases include:

PostgreSQL
MySQL
MariaDB
SQL Server
Oracle Database

The data is generally organized into tables.

For example:

Users
-------------------------
id | name  | email
-------------------------
1  | Alex  | alex@mail.com
2  | Sam   | sam@mail.com

And:

Orders
-------------------------
id | user_id | total
-------------------------
101| 1       | 4999
102| 2       | 2999

The user_id can establish a relationship between orders and users.

This relational model is one of the defining characteristics of SQL databases.

3. What Is NoSQL?

NoSQL is a broad category of databases that don't primarily use the traditional relational table model.

Popular NoSQL databases include:

MongoDB
Redis
Cassandra
DynamoDB
Couchbase

But an important point is that NoSQL doesn't mean one specific database structure.

Different NoSQL databases use different models.

For example:

Document
Key-Value
Wide-Column
Graph

MongoDB is primarily a document database.

Redis is primarily a key-value/data-structure store.

Cassandra uses a wide-column model.

So saying:

"NoSQL is just MongoDB."

would be incorrect.

4. SQL Uses Tables and Relationships

Suppose you're building a blogging platform.

You might have:

Users
Posts
Comments

A relational design could look like:

Users
  |
  | 1-to-many
  ↓
Posts
  |
  | 1-to-many
  ↓
Comments

The database can represent these relationships using keys.

For example:

SELECT posts.title, users.name
FROM posts
JOIN users
ON posts.user_id = users.id;

This ability to query relationships is one of the strengths of relational databases.

You don't have to store the same user information repeatedly inside every post.

Instead, you store the relationship.

5. NoSQL Often Stores Data Differently

A document database might store a product like:

{
  "id": 42,
  "name": "Mechanical Keyboard",
  "price": 4999,
  "tags": [
    "keyboard",
    "mechanical",
    "gaming"
  ]
}

Another product could have different fields:

{
  "id": 43,
  "name": "Monitor",
  "price": 15999,
  "refreshRate": 144
}

This can make document databases convenient when records naturally have different shapes.

Instead of forcing every record into the same set of columns, documents can have more flexible structures.

6. Schema: Fixed vs Flexible

One of the common differences between SQL and NoSQL is how schemas are handled.

A SQL table might define:

CREATE TABLE users (
    id INT PRIMARY KEY,
    name VARCHAR(100),
    email VARCHAR(255)
);

The structure is explicitly defined.

A document database can allow documents with different fields:

{
  "name": "Alex",
  "email": "alex@example.com"
}

and:

{
  "name": "Sam",
  "email": "sam@example.com",
  "github": "samdev"
}

This flexibility can be useful when application data changes frequently.

But flexible schema doesn't mean:

"No structure is required."

You still need to think carefully about how your data is modeled.

Otherwise, flexibility can turn into inconsistency.

7. SQL Is Strong When Relationships Matter

Consider a banking system.

You might have:

Customers
Accounts
Transactions
Loans
Payments

These entities are strongly related.

A transaction belongs to an account.

An account belongs to a customer.

A payment may be associated with an account or loan.

You often need queries involving multiple related entities.

For example:

Find all transactions
for a particular customer
during a particular month.

Relational databases are very good at expressing this type of structured relationship.

This is one reason SQL databases are common for systems involving complex relationships and transactional data.

8. SQL and Transactions

Transactions are another major reason developers choose relational databases.

Suppose you transfer:

₹1,000

from Account A to Account B.

You don't want this to happen:

Account A → -₹1,000

Account B → unchanged

The operation should behave as one logical unit.

Conceptually:

BEGIN
   ↓
Debit Account A
   ↓
Credit Account B
   ↓
COMMIT

If something goes wrong:

BEGIN
   ↓
Debit Account A
   ↓
ERROR
   ↓
ROLLBACK

The database can maintain transactional guarantees so the system doesn't end up with partially completed operations.

This is extremely important for many financial and business applications.

9. ACID Properties

Relational databases are commonly associated with ACID transactions.

ACID stands for:

Atomicity
Consistency
Isolation
Durability

A simplified understanding:

Atomicity

The transaction succeeds as a unit or is rolled back.

Consistency

The database moves from one valid state to another while respecting its rules.

Isolation

Concurrent transactions should not incorrectly interfere with each other.

Durability

Once committed, the data should persist despite failures within the database system's durability guarantees.

You don't need to memorize these immediately.

The important idea is:

Transactions help keep related operations consistent.

10. Does NoSQL Not Support Transactions?

This is a common misconception.

NoSQL does not mean:

"No transactions."

Many modern NoSQL databases support transactional operations, although the exact capabilities and trade-offs differ by database.

The more useful question is:

What transactional model does this database provide, and does it fit my application's requirements?

Don't choose a database based on old stereotypes.

Look at the actual database and its guarantees.

11. NoSQL Can Be Useful for Flexible Data

Imagine you're building a content platform where different types of content have different structures.

A video might contain:

{
  "title": "System Design Basics",
  "duration": 420,
  "resolution": "1080p"
}

An article might contain:

{
  "title": "Understanding Redis",
  "author": "Alex",
  "readingTime": 8
}

A document-oriented database can naturally represent these different shapes.

This can reduce friction when your data model is evolving or naturally document-oriented.

But again, flexible data models still require good design.

12. SQL vs NoSQL Is Also About Query Patterns

One of the biggest mistakes is asking:

"Which database is faster?"

That's usually the wrong first question.

Instead ask:

What queries will my application perform most often?

For example:

Find user by email

or:

Get all orders for a user

or:

Get latest 20 posts

or:

Find products by category and price range

The database should be selected based partly on these access patterns.

A database that is excellent for one workload might be a poor choice for another.

13. SQL Query Example

Suppose you need all orders for a user.

You might write:

SELECT *
FROM orders
WHERE user_id = 42
ORDER BY created_at DESC;

The database can use indexes to make this query efficient.

For a more complex query:

SELECT users.name, orders.total
FROM users
JOIN orders
ON users.id = orders.user_id
WHERE users.id = 42;

SQL is designed to express these kinds of relational queries.

This is one of the reasons SQL remains extremely popular.

14. NoSQL Query Example

In a document database, you might store data in a structure designed around how the application reads it.

For example:

{
  "userId": 42,
  "name": "Alex",
  "orders": [
    {
      "orderId": 101,
      "total": 4999
    },
    {
      "orderId": 102,
      "total": 2999
    }
  ]
}

Now retrieving the user's orders can potentially involve reading one document.

This is an important NoSQL design principle:

Model data around the queries your application needs.

Instead of always normalizing everything into separate tables, some NoSQL systems encourage denormalizing data when it makes common reads more efficient.

15. Normalization vs Denormalization

SQL systems often encourage normalization.

Instead of storing:

User Name
User Email
User Name
User Email
User Name
User Email

in many places, you might store the user once and reference them using an ID.

Users
   ↓
user_id
   ↓
Orders

This reduces unnecessary duplication.

NoSQL systems may sometimes favor denormalization.

For example:

{
  "orderId": 101,
  "customer": {
    "name": "Alex",
    "email": "alex@example.com"
  },
  "total": 4999
}

Now the order contains customer information directly.

The trade-off is that duplicated data can become harder to keep synchronized.

So neither approach is automatically better.

The correct choice depends on how the application reads and updates data.

16. Scaling SQL Databases

SQL databases can scale very well.

A common architecture might start as:

Application
     ↓
PostgreSQL

As traffic grows, you might introduce:

Application
     ↓
Connection Pool
     ↓
Database

Then potentially:

Application
     ↓
Read/Write Layer
     ↓
 ┌───────────────┐
 ↓               ↓
Primary       Read Replicas

You can also use:

indexes
caching
read replicas
partitioning
sharding
connection pooling

So:

"SQL can't scale."

is simply not true.

The real question is how you scale the particular database and workload.

17. Scaling NoSQL Databases

Many NoSQL systems are designed with horizontal scaling in mind.

Conceptually:

             Application
                  ↓
             NoSQL Cluster
          ┌───────┼───────┐
          ↓       ↓       ↓
        Node 1  Node 2  Node 3

Data can be distributed across multiple nodes depending on the database's architecture and configuration.

This can be particularly useful for workloads involving:

very large datasets
high write volume
geographically distributed traffic
predictable access patterns

But horizontal scalability is not a free advantage.

Distributed systems introduce complexity around consistency, partitioning, replication, failure handling, and operations.

18. SQL vs NoSQL: Consistency Trade-offs

One of the deeper database decisions is consistency.

Suppose a user changes their profile.

You might expect:

Write
 ↓
Read
 ↓
Immediately see new value

Some distributed architectures may allow replicas to temporarily contain different versions of the data.

You might have:

Primary
   ↓
Replication
   ↓
Replica

For some applications, slight replication delay is acceptable.

For others, it may not be.

For example:

Social media feed

might tolerate some delay.

But:

Bank account balance

has much stricter requirements.

The important question is:

How consistent does this data need to be?

19. SQL vs NoSQL Is Not a Religion

Developers sometimes turn database selection into a technology argument.

"PostgreSQL is better."

"No, MongoDB is better."

"SQL is outdated."

"NoSQL is unreliable."

These arguments usually miss the point.

A database is a tool.

The right question is:

What does the application need?

Not:

Which database is trendy?

20. When SQL Is Usually a Strong Choice

SQL is often a good fit when your application has:

complex relationships
structured data
strong transactional requirements
complex queries
reporting requirements
joins across entities
well-defined business rules

Examples could include:

Banking
Accounting
ERP
Inventory
E-commerce
Booking systems
Order management

These systems often benefit from relational modeling and strong transaction support.

21. When NoSQL Can Be a Strong Choice

NoSQL can be a good fit when you have:

flexible or rapidly changing document structures
very high-scale distributed workloads
access patterns that fit a document/key-value/wide-column model
large amounts of data
workloads where horizontal distribution is important

Examples might include:

Event data
Activity feeds
Catalog-like documents
High-volume telemetry
Caching
Distributed key-value workloads

But the exact database still matters.

MongoDB, Redis, Cassandra, and DynamoDB solve different problems.

22. You Don't Have to Choose Only One

This is an important point.

Real systems often use multiple data stores.

For example:

                Application
                     |
          ┌──────────┼──────────┐
          ↓          ↓          ↓
      PostgreSQL   Redis    Object Storage
          |
       Primary Data

PostgreSQL might store:

Users
Orders
Payments
Products

Redis might store:

Sessions
Cache
Rate Limits
Temporary Data

Object storage might contain:

Images
Videos
Documents

Each system has a different responsibility.

This is sometimes called polyglot persistence.

The goal isn't to force every type of data into one database.

The goal is to use the appropriate storage system for each workload.

23. Don't Add a Database Just Because You Can

Using multiple databases sounds powerful.

But every additional system adds operational complexity.

For example:

PostgreSQL
Redis
MongoDB
Kafka
Elasticsearch

means your team now needs to understand, monitor, secure, back up, upgrade, and troubleshoot all of them.

If PostgreSQL solves the problem effectively, adding MongoDB just because it is popular may make the architecture worse.

A good rule is:

Use another database when it solves a real problem, not because it sounds more scalable.

24. Database Choice Should Start With the Data Model

Before choosing a database, ask:

What data am I storing?

Users
Orders
Products
Events
Sessions
Logs

How are these entities related?

User → Orders
Order → Products
Product → Reviews

How will the application query the data?

By ID?
By email?
By time?
By category?
By relationship?

How often does the data change?

Frequently?
Occasionally?
Mostly read-only?

How important is consistency?

Eventually consistent?
Strong consistency?
Transactional?

How much data and traffic do I expect?

1,000 users?
1 million?
100 million?

These questions are much more useful than simply asking:

"SQL or NoSQL?"

25. A Practical Decision Framework

You can use a simple process.

Start with:

Do I have strong relationships
between entities?
        |
        ├── Yes → Consider SQL
        |
        └── No
             ↓
Do I need flexible documents
or a specialized NoSQL model?
             |
             ├── Yes → Consider NoSQL
             |
             └── No → Evaluate based on workload

Then ask:

What queries will dominate?

What consistency is required?

What scale is expected?

What operational complexity can the team handle?

The answers should drive the decision.

26. The Database Is Part of System Design

Choosing a database isn't an isolated decision.

It affects:

API Design
     ↓
Data Model
     ↓
Caching
     ↓
Scaling
     ↓
Consistency
     ↓
Reliability
     ↓
Operations

For example, choosing a relational database may influence how you model relationships.

Choosing a distributed NoSQL system may influence how you design partition keys.

Choosing Redis for caching introduces cache invalidation and expiration decisions.

The database becomes part of the overall architecture.

27. A Common Beginner Mistake

A common mistake is choosing a database before understanding the application.

For example:

"I learned MongoDB, so I'll use MongoDB."

or:

"Everyone uses PostgreSQL, so I'll use PostgreSQL."

Neither is a strong architectural decision.

Instead:

Requirements
     ↓
Data Model
     ↓
Access Patterns
     ↓
Consistency Requirements
     ↓
Scale
     ↓
Database Choice

The database should follow the problem.

Not the other way around.

SQL vs NoSQL at a Glance

Requirement	SQL	NoSQL
Structured data	Strong fit	Depends on database
Complex relationships	Strong fit	Often more application-managed
Joins	Strong	Usually limited/different
Transactions	Strong support	Depends on database
Flexible schema	Possible, but typically more structured	Often a strength
Horizontal scaling	Possible	Often a core design goal
Complex queries	Strong	Depends on database
Document-oriented data	Possible	Often a strong fit
Key-value workloads	Usually not the primary strength	Strong fit for some systems
Reporting	Often strong	Depends on database

This table is intentionally simplified.

Different databases have different capabilities, and modern SQL and NoSQL systems overlap more than many beginner comparisons suggest.

The Bigger Lesson

The real SQL vs NoSQL question isn't:

"Which database is better?"

It's:

"Which data model, query model, consistency model, and scaling strategy best fit my application?"

If your application has:

Users
Orders
Payments
Inventory

and those entities have strong relationships and transactional requirements, a relational database may be a natural starting point.

If your application primarily works with flexible documents, massive distributed workloads, or specialized access patterns, a NoSQL database may be a better fit.

And sometimes the answer is:

SQL + NoSQL

because different parts of the system have different requirements.

A Simple Mental Model

Before choosing a database, think through:

                Requirements
                     ↓
                 What data?
                     ↓
              How is it related?
                     ↓
              How will I query it?
                     ↓
            How consistent must it be?
                     ↓
              How much will it scale?
                     ↓
              Choose the database

Don't start with:

SQL vs NoSQL

Start with:

What problem am I trying to solve?

That's the mindset that leads to better database decisions.

A database isn't good because it is popular.

A database is good when its data model, guarantees, query capabilities, performance characteristics, and operational requirements match the problem you're solving.

And that's the real difference between choosing a database because you've heard of it and choosing one because your system actually needs it.

How Does a Backend Actually Connect to a Database?

Tanu Priya — Thu, 10 Sep 2026 07:41:20 +0000

Every time your API reads or writes data, there is a chain of components working behind the scenes: API → Backend → ORM/Database Driver → Connection Pool → Database → Result. The database driver handles communication, connection pools prevent your application from creating a new connection for every request, and ORMs make database operations easier without eliminating SQL underneath. Understanding this flow is essential once you move beyond building simple APIs.

You write:

const user = await prisma.user.findUnique({
  where: { id: 42 }
});

And somehow a database returns the correct user.

It looks like one line of code.

It isn't.

Behind that line, your application has to communicate with another system, obtain a database connection, send a query, wait for the database to execute it, receive the result, and return that connection so another request can use it.

And when your application goes from 10 users to 10,000 users, those details stop being implementation trivia.

They become architecture.

This is the part of backend development that most tutorials hide.

Let's look at what actually happens.

The Request That Started It All

Imagine you're building a blogging platform.

A user opens their profile.

The browser sends:

GET /api/users/42

Your backend receives the request and needs to return:

{
  "id": 42,
  "name": "Alex",
  "email": "alex@example.com"
}

The obvious mental model is:

API → Database → Result

The real system is closer to:

Client
   ↓
API
   ↓
Backend
   ↓
ORM / Database Driver
   ↓
Connection Pool
   ↓
Database
   ↓
Query Result
   ↓
Backend
   ↓
API Response

That middle section is where most of the interesting engineering happens.

1. The Database Driver: Your Backend's Translator

Your Node.js, Python, Java, or Go application doesn't automatically know how to communicate with PostgreSQL or MySQL.

It needs a database driver.

For example, a Node.js application can use the PostgreSQL pg driver.

Conceptually:

Application
     ↓
Database Driver
     ↓
PostgreSQL

A simplified example:

import pg from "pg";

const { Pool } = pg;

const pool = new Pool({
  connectionString: process.env.DATABASE_URL
});

The driver handles the database-specific communication.

Your application says:

"Run this query."

The driver takes care of communicating with the database.

This separation is useful because your application doesn't need to understand every low-level detail of the database protocol.

2. Then There Is the Query

Once your application can communicate with the database, it needs to tell the database what it wants.

For example:

SELECT id, name, email
FROM users
WHERE id = 42;

The database receives that query and does the actual database work.

It has to determine:

Which table is involved?
Which rows match?
Should an index be used?
How should the query be executed?
What data needs to be returned?

The database then sends the result back to your application.

For example:

[
  {
    "id": 42,
    "name": "Alex",
    "email": "alex@example.com"
  }
]

Your backend turns that into an API response.

So:

HTTP Request
     ↓
Backend
     ↓
SQL Query
     ↓
Database
     ↓
Result
     ↓
Backend
     ↓
HTTP Response

That's the basic version.

But there's a scalability problem.

3. Why You Can't Just Open a New Connection for Every Request

Suppose your API receives 1,000 requests.

A naive approach would be:

Request 1 → Open Connection → Query → Close
Request 2 → Open Connection → Query → Close
Request 3 → Open Connection → Query → Close
...

It sounds reasonable.

It isn't.

Creating database connections consumes resources. Doing it repeatedly adds unnecessary overhead and can put significant pressure on the database.

Instead, production applications generally reuse connections.

That's what connection pooling is for.

4. Connection Pools: The Unsung Hero of Backend Systems

A connection pool maintains a set of reusable database connections.

Instead of:

Request
   ↓
Create Connection
   ↓
Query
   ↓
Destroy Connection

you get:

Request
   ↓
Borrow Connection
   ↓
Run Query
   ↓
Return Connection

A pool might look like:

             Connection Pool

        ┌─────────────────────┐
        │ Connection 1        │
        │ Connection 2        │
        │ Connection 3        │
        │ Connection 4        │
        └─────────────────────┘
                  │
                  ↓
               Database

Imagine four requests arrive.

They can use the available connections:

Request A → Connection 1
Request B → Connection 2
Request C → Connection 3
Request D → Connection 4

When Request A finishes:

Connection 1 → Available Again

Another request can reuse it.

The connection doesn't need to be recreated.

5. More Connections Doesn't Mean More Speed

This is one of those backend lessons that becomes important only after you start dealing with production systems.

You might think:

"If 10 connections are good, 100 connections must be better."

Not necessarily.

Your database has finite resources.

Too many connections can create:

memory pressure
CPU overhead
contention
increased latency
unnecessary database load

So a connection pool isn't just a performance trick.

It's also a concurrency control mechanism.

For example:

const pool = new Pool({
  connectionString: process.env.DATABASE_URL,
  max: 10
});

The correct pool size depends on your workload, database, server resources, and deployment architecture.

The important lesson is:

Your backend shouldn't be allowed to create an unlimited number of database connections.

6. Where Does an ORM Fit?

Now let's make the architecture more realistic.

Most modern applications don't write SQL for every simple database operation.

You might use an ORM such as Prisma.

Instead of:

SELECT *
FROM users
WHERE id = 42;

you write:

const user = await prisma.user.findUnique({
  where: {
    id: 42
  }
});

Much cleaner.

But don't let the abstraction fool you.

The database hasn't disappeared.

The architecture is now:

Your Application
       ↓
      ORM
       ↓
Database Driver
       ↓
Connection Pool
       ↓
Database

The ORM gives you a more convenient interface for working with the database.

7. ORM Doesn't Mean "No SQL"

This is probably the most important misconception to clear up.

If you use Prisma, Sequelize, TypeORM, or another ORM, you're not avoiding databases.

You're avoiding manually writing every database query.

The underlying system still needs database operations.

So when you write:

prisma.user.findUnique({
  where: { id: 42 }
});

the database still has to:

Receive Query
     ↓
Understand Query
     ↓
Create Execution Plan
     ↓
Read Data
     ↓
Return Result

The ORM simply provides a higher-level way of expressing what you want.

That's why SQL knowledge doesn't become irrelevant when you learn an ORM.

If anything, it becomes more useful when things stop working as expected.

8. Raw SQL vs ORM: The Tradeoff

This isn't really an argument about which technology is "better."

It's about how much control versus abstraction you need.

Raw SQL

SELECT
    users.name,
    COUNT(posts.id) AS post_count
FROM users
LEFT JOIN posts
    ON users.id = posts.author_id
GROUP BY users.id;

You control the query directly.

That's valuable for:

complex joins
aggregations
database-specific features
performance tuning
advanced queries

But you're also responsible for writing and maintaining the SQL.

ORM

const users = await prisma.user.findMany({
  include: {
    posts: true
  }
});

This can make everyday development much faster.

You may also get:

type safety
migrations
schema modeling
autocomplete
relationship handling
developer tooling

The tradeoff is abstraction.

The ORM decides how your high-level operation gets translated into database operations.

9. The Approach I'd Actually Recommend

For most application development:

Use the ORM where it makes the code simpler.

But don't treat it as a black box.

If you encounter:

a slow query
unexpected database load
a complicated join
an inefficient query
a database-specific requirement

drop down a level.

Think of it like this:

                 Your Application
                        │
                        ↓
              ┌──────────────────┐
              │       ORM        │
              └──────────────────┘
                    │        │
             Simple Queries  │
                    │        ↓
                    │     Raw SQL
                    │        │
                    └────┬───┘
                         ↓
                  Database Driver
                         ↓
                  Connection Pool
                         ↓
                      Database

The goal isn't to eliminate SQL.

The goal is to use the right abstraction for the job.

10. What Happens When the Database Is Slow?

Here's where understanding the architecture starts paying off.

Your endpoint suddenly takes three seconds.

You look at the backend code:

const user = await prisma.user.findUnique({
  where: { id: 42 }
});

Nothing obviously looks wrong.

So where is the bottleneck?

It could be:

API
 ↓
Backend
 ↓
ORM
 ↓
Connection Pool
 ↓
Database

Maybe:

the connection pool is exhausted
the query is slow
an index is missing
the database is overloaded
too much data is being returned
another query is consuming resources

This is why knowing only the ORM isn't enough.

You need to understand the layers underneath it.

11. Indexes: When the Database Has Too Much Data to Search

Imagine your users table contains 10 million rows.

You run:

SELECT *
FROM users
WHERE email = 'alex@example.com';

Without an appropriate index, the database may need to inspect a large amount of data to find the matching record.

With an index, the database can often locate the relevant record much more efficiently.

Conceptually:

Without useful index:

Database
 ↓
Check many rows
 ↓
Find matching row


With index:

Database
 ↓
Index
 ↓
Matching row

This is why database performance isn't simply about writing faster backend code.

Sometimes the bottleneck is inside the database itself.

12. What About Multiple Operations?

Suppose a user publishes a post.

Your application needs to:

Create Post
    ↓
Update User Statistics
    ↓
Create Activity Record

What happens if:

Create Post          ✅
Update Statistics    ❌

Now your data may be inconsistent.

This is where transactions become important.

Conceptually:

BEGIN
  ↓
Create Post
  ↓
Update Statistics
  ↓
Create Activity
  ↓
COMMIT

If something fails:

BEGIN
  ↓
Create Post
  ↓
Update Statistics
  ↓
ERROR
  ↓
ROLLBACK

Instead of treating each operation independently, the database can treat the group as one logical unit.

This is another reason understanding databases matters even if an ORM handles the syntax for you.

13. Things Get More Interesting When You Scale

Imagine your application grows.

You start with:

1 Backend Server
       ↓
   Database

Then traffic increases.

You add more servers:

                 Load Balancer
                /      |      \
               ↓       ↓       ↓
            Server  Server  Server
               \       |       /
                \      |      /
                 Database

Now there's something easy to overlook:

Each application server may have its own connection pool.

Suppose you configure:

20 connections per server

and deploy:

5 backend servers

You could potentially have:

5 × 20 = 100

database connections.

So scaling your application servers can also increase the number of connections reaching your database.

This is why database configuration cannot be considered separately from application architecture.

14. The Production Stack Is More Than "Backend + Database"

A beginner's architecture often looks like:

Frontend
   ↓
Backend
   ↓
Database

A production system can look more like:

                    Client
                      ↓
                Load Balancer
                      ↓
              Backend Servers
                      ↓
                ORM / Driver
                      ↓
               Connection Pool
                      ↓
          ┌───────────┴───────────┐
          ↓                       ↓
       Cache                  Database
                                  ↓
                               Indexes
                                  ↓
                              Storage

And depending on the application, you may also introduce:

queues
background workers
read replicas
monitoring
caching layers
database migrations
transaction management

The database is not an isolated component.

It's part of a larger system.

15. Common Mistakes Developers Make

Creating a connection for every request

This defeats one of the main benefits of connection pooling.

Setting the pool size arbitrarily high

More connections can eventually make the database less stable, not more performant.

Assuming the ORM automatically creates efficient queries

ORMs are useful abstractions, but developers still need to understand the queries their applications generate.

Ignoring indexes

A perfectly written API can still be slow because the database is doing unnecessary work.

Never learning SQL

This becomes painful when you need to debug a slow query or understand why the database isn't behaving as expected.

Treating the database as a simple storage box

A database isn't just where your JSON happens to live.

It is an engine that manages querying, indexing, concurrency, consistency, transactions, and much more.

The Mental Model That Actually Matters

If you remember only one diagram from this article, make it this one:

                    API Request
                         ↓
                  Backend Server
                         ↓
                   ORM / Driver
                         ↓
                  Connection Pool
                         ↓
                     Database
                         ↓
                   Query Result
                         ↓
                  Backend Server
                         ↓
                  API Response

Each layer solves a different problem.

Layer	Responsibility
API	Communicates with clients
Backend	Handles application logic
ORM	Provides a high-level database interface
Driver	Communicates with the database
Connection Pool	Reuses and manages connections
Database	Stores and processes data

Once you understand this pipeline, a lot of backend concepts stop feeling disconnected.

Indexes make database lookups more efficient.

Transactions protect consistency.

Connection pools manage database concurrency.

ORMs provide abstraction.

SQL gives you direct control.

Caching reduces unnecessary database work.

They're all pieces of the same system.

Final Take

The interesting thing about database communication is that the code you write is often the smallest part of the story.

You might write:

const user = await prisma.user.findUnique({
  where: { id: 42 }
});

But underneath that line is an entire pipeline:

Application
    ↓
ORM
    ↓
Database Driver
    ↓
Connection Pool
    ↓
Database
    ↓
Query Execution
    ↓
Result

When an application is small, you can get away with treating that pipeline as a black box.

When the application grows, you can't.

A slow API might actually be a slow query.

A database outage might actually be an exhausted connection pool.

A scaling problem might actually be too many connections across multiple servers.

And a seemingly harmless ORM operation might generate far more database work than you expected.

That's why learning backend development isn't just about learning frameworks.

It's about understanding what those frameworks are doing underneath the surface.

The best backend developers don't just know how to query a database. They understand what happens between the API request and the row coming back.

And once you understand that journey, the next question naturally follows:

How does the database itself find, organize, and protect all those rows?

That's where tables, relationships, primary keys, foreign keys, indexes, normalization, transactions, and database design begin.

DEV Community: Tanu Priya

What Really Happens When You Search Something on Google?

The Big Picture

1. You Type a Search Query

2. DNS Finds Google

3. Your Request Travels Across the Internet

4. HTTPS Protects the Connection

5. Google's Frontend Receives Your Request

6. Google Has Already Crawled the Web

Approach 1: Search the internet live

Approach 2: Search an index

7. Googlebot Crawls Websites

8. Google Builds a Search Index

9. Your Query Gets Understood

10. Query Rewriting Can Happen

11. Google Searches the Index

12. Candidate Retrieval

13. Ranking Determines the Order

14. Relevance Matters More Than Keyword Matching

15. Freshness Can Matter

16. Location Can Matter

17. Google May Use Your Search Context

18. Special Search Features Are Generated

19. Ads Are a Separate Part of the Search Experience

20. Google Generates the Search Results Page

21. Your Browser Renders the Page

22. But Google Doesn't Always Start From Scratch

23. What Happens If Millions of People Search at Once?

24. What If One Server Fails?

25. Why Is Google Search So Fast?

26. A Simplified End-to-End Architecture

27. The Most Important Insight

28. How This Connects to System Design

Distributed Systems

Caching

Load Balancing

Database / Indexing

Ranking Systems

Fault Tolerance

Networking

Data Processing

Final Takeaway

What Really Happens When You Open Google.com in a Browser?

The Complete Journey

1. You Type google.com

2. The Browser Needs an IP Address

3. DNS Resolution

Why caching matters

4. DNS Gives Us an IP Address

5. Establishing the Connection

6. The TLS Handshake

Why does the certificate matter?

7. Now We Can Send an HTTP Request

8. Your Request Travels Across the Internet

9. The Request Reaches Google's Infrastructure

10. The Server Processes the Request

11. The Server Sends an HTTP Response

12. The Browser Starts Parsing HTML

DOM

13. HTML Isn't Enough

CSSOM

14. DOM + CSSOM

15. Then Comes Layout

16. Painting the Page

17. Compositing

18. JavaScript Enters the Picture

19. One Page Can Trigger Many Requests

20. HTTP/2 and HTTP/3 Make This More Efficient

21. Caching Can Change Everything

22. The Browser Finally Shows You the Page

23. What Happens When You Click Something?

24. Where Performance Problems Come From

DNS is slow

Connection establishment is slow

Server response is slow

HTML is large

JavaScript is expensive

CSS is expensive

Images are huge

Too many requests

1. You Type `google.com`