In the previous part of Under the Hood, we explored how Express middleware processes an incoming request before it reaches the route handler.
We ended with an important question:
When our application asks MongoDB for a user, how does MongoDB actually find that data?
You may have written code like this:
const user = await User.findOne({
email: "aryan@example.com"
});
It looks simple.
We give MongoDB an email and it gives us a user.
But imagine a collection containing:
10 documents
100 documents
10,000 documents
1,000,000 documents
100,000,000 documents
Does MongoDB check every document one by one?
If it did, database queries would become extremely slow as our application grew.
So what actually happens?
How does MongoDB decide what to search?
What is an index?
Why is an indexed query usually faster?
And how can we check whether MongoDB actually used an index?
Let's go Under the Hood.
1. Where MongoDB Fits in Our Application
Before looking inside MongoDB, let's connect it with what we learned in the previous parts.
Suppose a client sends:
GET /users/123
The request travels through our application:
Client
↓
HTTP Request
↓
Node.js
↓
Express
↓
Middleware
↓
Route Handler
↓
MongoDB
↓
Query Result
↓
Route Handler
↓
HTTP Response
↓
Client
For example:
app.get("/users/:id", async (req, res) => {
const user = await User.findOne({
_id: req.params.id
});
res.json(user);
});
The Express route receives the request.
Then our application asks the database for a document.
But there is an important detail.
Our Node.js application does not directly manipulate MongoDB's internal data structures.
It communicates with MongoDB through a MongoDB driver.
For Node.js applications, MongoDB provides an official Node.js driver for connecting to MongoDB and performing database operations.
If we use Mongoose, Mongoose sits between our application and the MongoDB driver.
Conceptually:
Express Route
↓
Mongoose
↓
MongoDB Node.js Driver
↓
MongoDB Server
So when we write:
User.findOne(...)
there is much more happening underneath.
2. What Is a MongoDB Document?
MongoDB is a document database.
Instead of storing data primarily as rows and columns, MongoDB stores data as documents inside collections.
A document looks similar to JSON:
{
_id: ObjectId("..."),
name: "Aryan",
email: "aryan@example.com",
age: 21
}
MongoDB actually stores documents as BSON, which is a binary representation of JSON with additional data types.
A collection can contain many documents:
users
├── Document 1
├── Document 2
├── Document 3
├── Document 4
└── ...
For example:
{
_id: 1,
name: "Aryan",
email: "aryan@example.com"
}
and:
{
_id: 2,
name: "Rahul",
email: "rahul@example.com"
}
MongoDB can then query these documents using its Query API.
3. What Happens When We Run findOne()?
Consider:
const user = await User.findOne({
email: "aryan@example.com"
});
At a high level, the process looks like this:
Application
↓
Mongoose
↓
MongoDB Driver
↓
MongoDB Server
↓
Query
↓
Query Planner
↓
Index or Collection Scan
↓
Matching Document
↓
Result
↓
Application
The interesting part is:
Query
↓
Query Planner
↓
How should MongoDB find this data?
MongoDB doesn't simply receive the query and blindly scan everything.
The server interprets the query and builds a query plan describing how it should retrieve the data.
4. What Is a Query?
A query tells MongoDB what data we want.
For example:
{
email: "aryan@example.com"
}
means:
Find documents where the
"aryan@example.com".
In MongoDB:
db.users.find({
email: "aryan@example.com"
});
Or if we only want one document:
db.users.findOne({
email: "aryan@example.com"
});
We can also query using conditions.
For example:
db.users.find({
age: {
$gt: 18
}
});
This means:
Find users whose age is greater than 18.
MongoDB's Query API supports CRUD operations as well as aggregation pipelines for more complex data processing.
But knowing what we want is only half the problem.
MongoDB also needs to figure out:
What is the most efficient way to find it?
That's where the query planner comes in.
5. The Query Planner
Imagine we have one million users.
Our query is:
db.users.findOne({
email: "aryan@example.com"
});
MongoDB has multiple possible ways to find the document.
For example:
Option 1:
Check every document
Option 2:
Use an index on email
Option 3:
Use another available index
MongoDB's query optimization process determines a suitable plan for executing the query.
Conceptually:
Query
↓
Query Planner
↓
Choose a Plan
↓
Execute Plan
↓
Return Results
The plan might involve:
Index
↓
Find matching keys
↓
Fetch documents
Or it might involve:
Collection
↓
Check documents
↓
Find matches
The second approach is called a collection scan.
6. What Is a Collection Scan?
Suppose we have these documents:
Document 1 → email: user1@gmail.com
Document 2 → email: user2@gmail.com
Document 3 → email: user3@gmail.com
Document 4 → email: aryan@example.com
Document 5 → email: user5@gmail.com
And we run:
db.users.findOne({
email: "aryan@example.com"
});
Without a useful index, MongoDB may need to inspect documents to find the matching one.
Conceptually:
Document 1 → ❌
Document 2 → ❌
Document 3 → ❌
Document 4 → ✅
With a large collection, this can require examining many documents.
MongoDB's explain output represents this kind of operation with a COLLSCAN stage.
The problem becomes obvious:
Small collection
↓
Not a big problem
Huge collection
↓
Potentially expensive
So how can we avoid searching through the entire collection?
7. Enter Indexes
An index is a separate data structure that helps MongoDB locate documents more efficiently.
Think about a book.
Suppose you want to find information about:
MongoDB
You could read every page from beginning to end.
Or you could use the index at the back of the book.
The index might tell you:
MongoDB → Page 142
You can immediately jump closer to the information you need.
Database indexes work with the same basic idea:
Without Index
Collection
↓
Document
↓
Document
↓
Document
↓
Document
↓
Search...
With an index:
Query
↓
Index
↓
Find matching key
↓
Locate document
↓
Return result
MongoDB stores index values separately from the collection so that read operations can use the index to identify relevant documents instead of scanning the entire collection.
8. Creating an Index
Suppose our application frequently searches users by email.
We can create an index:
db.users.createIndex({
email: 1
});
Here:
email
is the indexed field.
And:
1
means ascending index order.
We can then run:
db.users.findOne({
email: "aryan@example.com"
});
MongoDB can use the email index to locate matching documents.
MongoDB recommends creating indexes for fields used frequently in queries, while also considering the cost indexes add to write operations.
9. Why Don't We Index Everything?
If indexes make queries faster, it might seem like we should create an index for every field.
But that isn't a good idea.
Indexes have a cost.
Imagine a document:
{
name: "Aryan",
email: "aryan@example.com",
age: 21,
city: "Jaipur"
}
If we create indexes on:
name
email
age
city
MongoDB now has more index structures to maintain.
When we insert or update data, the database may also need to update the relevant indexes.
So:
More indexes
↓
Potentially faster reads
+
More work during writes
+
More storage
MongoDB's documentation specifically notes that indexes improve read performance but add overhead to writes because indexes also need to be updated.
The goal isn't:
Create as many indexes as possible.
The goal is:
Create indexes that support the queries your application actually performs.
10. What Happens When MongoDB Uses an Index?
Let's return to:
db.users.findOne({
email: "aryan@example.com"
});
Suppose we have:
Users Collection
1,000,000 documents
And:
Index on email
Conceptually, MongoDB can do something like:
Query
↓
email = "aryan@example.com"
↓
Email Index
↓
Matching index entry
↓
Corresponding document
↓
Result
Instead of checking every document:
Document 1
Document 2
Document 3
Document 4
...
Document 1,000,000
the index helps narrow down where MongoDB needs to look.
The exact execution plan can vary depending on the query, available indexes, data distribution, and MongoDB's query optimizer.
That's why we shouldn't assume an index was used just because one exists.
We can check.
11. How Do We Know If MongoDB Used the Index?
MongoDB provides explain() for examining query plans and execution statistics.
For example:
db.users
.find({
email: "aryan@example.com"
})
.explain("executionStats");
The output contains information about how MongoDB executed the query.
We can inspect things such as:
Execution time
Documents examined
Index keys examined
Indexes used
Query stages
MongoDB's documentation specifically recommends explain("executionStats") for understanding whether a query used an index and how many documents or index keys were examined.
12. COLLSCAN vs IXSCAN
Two important terms you may see in an explain plan are:
COLLSCAN
and:
IXSCAN
COLLSCAN
Means MongoDB performed a collection scan.
Conceptually:
Collection
↓
Scan documents
↓
Check conditions
↓
Return matches
IXSCAN
Means MongoDB used an index scan.
Conceptually:
Index
↓
Scan relevant index entries
↓
Locate matching data
↓
Return results
MongoDB's explain documentation uses COLLSCAN for collection scans and IXSCAN when an index is selected by the query planner.
So if you run:
db.users
.find({
email: "aryan@example.com"
})
.explain("executionStats");
and see:
COLLSCAN
MongoDB performed a collection scan.
If you see:
IXSCAN
an index scan was involved.
13. A Simple Experiment
Let's make this concrete.
Suppose we have:
db.users.insertMany([
{
name: "Aryan",
email: "aryan@example.com"
},
{
name: "Rahul",
email: "rahul@example.com"
},
{
name: "Priya",
email: "priya@example.com"
}
]);
Now query:
db.users.find({
email: "aryan@example.com"
});
Before creating an email index, we can inspect the query:
db.users
.find({
email: "aryan@example.com"
})
.explain("executionStats");
We may see a collection scan:
COLLSCAN
Now create an index:
db.users.createIndex({
email: 1
});
Run the query again:
db.users
.find({
email: "aryan@example.com"
})
.explain("executionStats");
The plan may now contain:
IXSCAN
The exact explain output depends on the MongoDB version and query, but the important idea is that explain() lets us inspect what MongoDB actually did rather than guessing.
14. totalDocsExamined vs nReturned
One useful part of executionStats is:
nReturned
and:
totalDocsExamined
Imagine our query returns:
nReturned = 1
but MongoDB examined:
totalDocsExamined = 1,000,000
That tells us something important.
We only needed one document, but MongoDB examined a huge number of documents to find it.
With a suitable index, we might see a much smaller number of documents examined.
MongoDB's explain output exposes these execution statistics specifically so developers can understand how efficiently a query was executed.
15. What Is a Compound Index?
Sometimes we don't query using only one field.
For example:
db.users.find({
city: "Jaipur",
age: 21
});
We could create a compound index:
db.users.createIndex({
city: 1,
age: 1
});
This index contains multiple fields.
Conceptually:
Compound Index
city
↓
age
↓
Document location
Compound indexes can support queries involving the indexed fields and their index prefixes. The order of fields in a compound index matters.
For example:
{
city: 1,
age: 1
}
can support queries using:
city
or:
city + age
But the index is not equivalent to having an index beginning with age.
This is why index design matters.
16. MongoDB Doesn't Just "Search the Database"
A common mental model is:
Application
↓
MongoDB
↓
Search database
↓
Return result
But a better mental model is:
Application
↓
Query
↓
MongoDB Server
↓
Query Planner
↓
Choose Execution Plan
↓
┌──────────────────┐
│ │
Index Collection
Scan Scan
│ │
└────────┬─────────┘
↓
Matching Documents
↓
Results
↓
Application
This is much closer to what happens conceptually.
MongoDB's server interprets the query, builds a plan, executes it, and returns the results.
17. Where Does Mongoose Fit?
If you're building a MERN application, you may not write MongoDB queries directly using db.users.find().
You might use Mongoose.
For example:
const user = await User.findOne({
email: "aryan@example.com"
});
Here:
Your Code
↓
Mongoose
↓
MongoDB Driver
↓
MongoDB
Mongoose provides a higher-level developer experience with models and schemas.
But eventually, the database still needs to process a MongoDB query.
So this:
User.findOne({
email: "aryan@example.com"
});
doesn't mean Mongoose itself searches through one million users.
The actual database operation is performed by MongoDB.
18. What Happens During a Real API Request?
Let's put everything together.
Suppose the client sends:
GET /users/123
Our Express route:
app.get("/users/:id", async (req, res) => {
const user = await User.findOne({
_id: req.params.id
});
res.json(user);
});
The complete flow looks like this:
Client
↓
HTTP Request
↓
Node.js
↓
Express
↓
Middleware
↓
Route Handler
↓
Mongoose Query
↓
MongoDB Driver
↓
MongoDB Server
↓
Query Planner
↓
Choose Plan
↓
Index / Scan
↓
Matching Document
↓
Query Result
↓
Mongoose
↓
Route Handler
↓
HTTP Response
↓
Client
Now our previous parts connect together.
19. Why _id Is Special
Every MongoDB document has an _id field that uniquely identifies the document within its collection.
For example:
{
_id: ObjectId("..."),
name: "Aryan",
email: "aryan@example.com"
}
MongoDB automatically creates a unique index on _id for a collection.
So a query such as:
db.users.findOne({
_id: someId
});
can efficiently locate the document using the _id index.
This is one reason IDs are commonly used when retrieving a specific document.
20. What About Sorting?
Suppose we want the newest users:
db.users
.find({})
.sort({
createdAt: -1
});
If this operation is common, an index involving createdAt may help support the query and sorting.
For more complex queries, index field order becomes important.
For example:
db.users.createIndex({
city: 1,
createdAt: -1
});
can support query patterns involving the index's prefix and can also help with sorting depending on the query shape.
This is why indexes aren't simply:
"Make an index on everything."
They should be designed around the queries the application actually performs. MongoDB's indexing guidance recommends mapping application query patterns to appropriate indexes.
21. Indexes Are Not Magic
An index does not automatically make every query fast.
For example, a query might match a very large percentage of a collection.
In that situation, using an index may not provide much benefit.
MongoDB considers query selectivity when determining how useful an index is. Highly selective queries eliminate many possible documents and can benefit strongly from indexes.
So:
Good index
+
Good query
+
Appropriate data distribution
↓
Better performance
The important lesson is:
Indexes should be designed around real query patterns, not added blindly.
22. What Happens When Data Changes?
There is another side to indexes.
Suppose we have:
db.users.createIndex({
email: 1
});
Now we insert:
{
name: "Aryan",
email: "aryan@example.com"
}
MongoDB has to store the document.
But it also has to maintain the index.
Conceptually:
Insert Document
↓
Store Document
+
Update Index
This is why indexes improve many reads but add overhead to writes.
So database performance is always about trade-offs.
23. A Practical Backend Example
Imagine we are building a login API:
POST /login
The client sends:
{
"email": "aryan@example.com",
"password": "..."
}
Our Express route might look like:
app.post("/login", async (req, res) => {
const { email, password } = req.body;
const user = await User.findOne({
email: email
});
if (!user) {
return res.status(401).json({
message: "Invalid credentials"
});
}
// Password verification...
res.json({
message: "Login successful"
});
});
Now think about the database query.
If the application has millions of users and frequently searches by:
email
an appropriate index can be extremely important.
We could create:
db.users.createIndex({
email: 1
});
Then MongoDB can use the index to efficiently locate candidate documents for the query.
This is a simple example of how application-level decisions and database-level decisions are connected.
24. How to Think About Database Performance
When a query becomes slow, don't immediately think:
"MongoDB is slow."
Instead, ask:
What query are we running?
↓
How many documents does it examine?
↓
Is an appropriate index available?
↓
Which execution plan was selected?
↓
How many documents are returned?
↓
Is the query selective?
MongoDB provides tools such as explain() to investigate these questions.
This is much more useful than simply adding random indexes.
25. Common Mistakes
❌ Creating an index on every field
More indexes aren't automatically better.
Indexes consume storage and can add write overhead.
❌ Assuming an index is always used
Creating an index doesn't mean every query will use it.
Use:
.explain("executionStats")
to inspect the actual query plan.
❌ Ignoring query patterns
Indexes should be based on how your application actually queries data.
For example, if your application frequently searches:
{
email: "..."
}
then an index on email makes more sense than an index on a field that is rarely queried.
❌ Thinking MongoDB simply searches every document
MongoDB has a query planning and execution process.
It can use indexes or perform collection scans depending on the query and available plans.
❌ Adding indexes without measuring
Instead of guessing, inspect your queries using:
.explain("executionStats")
Look at:
nReturned
totalDocsExamined
totalKeysExamined
executionTimeMillis
These statistics help us understand what MongoDB actually did.
26. The Mental Model
If you remember only one thing from this article, remember this:
Application
↓
Query
↓
MongoDB
↓
Query Planner
↓
Execution Plan
↓
┌─────────────────┐
│ │
Index Collection
Scan Scan
│ │
└────────┬────────┘
↓
Matching Documents
↓
Results
↓
Application
And when we use an index:
Query
↓
Index
↓
Find relevant entries
↓
Retrieve documents
↓
Return results
The key idea is:
MongoDB doesn't just "look for your data." It chooses a way to find it.
27. The Bigger Picture
Let's connect the first five parts of Under the Hood.
Part 1
What Happens When You Enter a URL?
↓
Part 2
How HTTP Requests Work
↓
Part 3
How Node.js Handles Requests
↓
Part 4
How Express Middleware Works
↓
Part 5
How MongoDB Finds Your Data
Now we can understand the journey much better:
Browser
↓
URL
↓
HTTP Request
↓
Node.js
↓
Express
↓
Middleware
↓
Route Handler
↓
MongoDB Query
↓
Query Planner
↓
Index / Collection Scan
↓
Document
↓
HTTP Response
↓
Browser
Each part is responsible for a different stage of the journey.
28. Conclusion
MongoDB queries may look simple from our application code:
const user = await User.findOne({
email: "aryan@example.com"
});
But behind that single line, several things happen.
The application sends a query through the MongoDB driver.
MongoDB receives and interprets the query.
The query planner determines an execution plan.
MongoDB may use an index to locate relevant documents or perform a collection scan.
The matching documents are then returned to the application.
The most important concepts to remember are:
- MongoDB stores data as BSON documents.
- Applications communicate with MongoDB through drivers.
- Mongoose provides a higher-level way to work with MongoDB from Node.js applications.
- MongoDB uses a query planning process to determine how to execute a query.
- A collection scan checks documents in the collection.
- An index provides a separate data structure that can help locate relevant documents.
-
IXSCANindicates index scanning in an explain plan. -
COLLSCANindicates a collection scan. -
explain("executionStats")helps us understand how a query was executed. - Indexes improve many reads but add overhead to writes.
The bigger picture is now:
Client
↓
HTTP Request
↓
Node.js
↓
Express
↓
Middleware
↓
Route Handler
↓
MongoDB Query
↓
Query Planner
↓
Index / Collection Scan
↓
Document
↓
HTTP Response
↓
Client
But finding the user is only one part of building a real application.
The next question is:
How does our application know whether the user making the request is actually allowed to access that data?
That's where authentication comes in.
That's what we'll explore in the next part of Under the Hood.
Top comments (0)