Imagine you are searching for one particular student's record in a database containing millions of students. Without any special mechanism, the database may need to examine a large number of rows to find the required record.
This raises an important question:
How can a database find the required data quickly when a table contains millions or even billions of records?
One of the most important techniques used by database systems to improve query performance is indexing.
In this article, we will understand what a database index is, why it is needed, how it works, different types of indexes, and when using an index can actually become a disadvantage.
What Is a Database Index?
A database index is a data structure that helps the database find specific rows more efficiently.
A simple way to understand an index is to compare it with the index of a book.
Suppose you want to find a particular topic in a 500-page book. You could start from page one and check every page until you find it.
That would take time.
Instead, you can look at the book's index, find the topic, and directly go to the relevant page.
A database index works on a similar idea.
Instead of searching through every row in a table, the database can use an index to locate the relevant rows more efficiently. Both MySQL and PostgreSQL documentation describe indexes as a way to make finding and retrieving specific rows faster.
Why Do We Need Indexes?
Consider a students table:
CREATE TABLE students (
id INT,
name VARCHAR(100),
email VARCHAR(100),
age INT
);
Suppose the table contains 10 million students.
Now imagine running:
SELECT *
FROM students
WHERE email = 'rahul@example.com';
If there is no suitable index, the database may need to examine many rows to determine which one matches the condition.
As the amount of data grows, inefficient searches can become expensive.
Now suppose we create an index on email:
CREATE INDEX idx_students_email
ON students(email);
The database now has an additional structure that can help it locate rows based on the email value.
MySQL specifically recommends considering indexes for columns used in conditions such as WHERE, while also warning that unnecessary indexes consume space and add work to data modifications.
How Does an Index Work?
At a high level, an index stores information that allows the database engine to locate matching records more efficiently.
For example:
Students Table
ID Name Email
1 Amit amit@gmail.com
2 Rahul rahul@gmail.com
3 Priya priya@gmail.com
4 Neha neha@gmail.com
An index on email can maintain an organized structure associated with those email values and the corresponding table rows.
Conceptually:
Email Index
amit@gmail.com → Row 1
neha@gmail.com → Row 4
priya@gmail.com → Row 3
rahul@gmail.com → Row 2
Instead of treating the entire table as the only place to search, the database can use the index to narrow down where the required row is located.
The exact internal implementation depends on the database system and index type.
B-Tree Indexes
One of the most common index structures is the B-tree.
For example, MySQL documentation describes B-tree indexes as structures that can efficiently find specific values and ranges of values, including conditions involving operators such as =, >, <=, BETWEEN, and IN.
A simplified representation might look like:
[50]
/ \
[20] [80]
/ \ / \
[10] [30] [60] [90]
The database can navigate through this structure rather than checking every possible value sequentially.
This is one reason indexes can significantly improve the performance of suitable queries.
Creating an Index
The general SQL syntax is:
CREATE INDEX index_name
ON table_name(column_name);
For example:
CREATE INDEX idx_email
ON students(email);
Now the database has an index named idx_email associated with the email column.
You can then run queries such as:
SELECT *
FROM students
WHERE email = 'rahul@example.com';
The database optimizer can decide whether using the index is beneficial for that query.
Indexes on Multiple Columns
Indexes don't have to contain only one column.
For example:
CREATE INDEX idx_name_age
ON students(name, age);
This is called a multi-column or composite index.
It can be useful when queries frequently filter or search using a combination of columns.
For example:
SELECT *
FROM students
WHERE name = 'Rahul'
AND age = 20;
However, the order of columns in a composite index matters. Therefore, creating an index should be based on the actual queries your application performs rather than simply adding as many columns as possible.
Database systems such as MySQL and PostgreSQL provide specific support for multi-column indexes.
Primary Key and Indexes
Primary keys are also closely related to indexing.
Consider:
CREATE TABLE students (
id INT PRIMARY KEY,
name VARCHAR(100),
email VARCHAR(100)
);
Database systems can use an index or index-like structure associated with the primary key to efficiently locate records.
For example:
SELECT *
FROM students
WHERE id = 500000;
The database can use the primary-key structure to efficiently locate the record.
The exact implementation differs between database systems and storage engines, so it is important not to assume that every DBMS internally handles indexes in exactly the same way.
The Advantages of Indexing
1. Faster Data Retrieval
The biggest advantage of indexing is faster retrieval for queries that can effectively use the index.
For example:
SELECT *
FROM users
WHERE email = 'user@example.com';
A suitable index can help the database find matching rows efficiently.
2. Better Query Performance
Indexes can improve the performance of many SELECT queries, particularly when they reduce the amount of data the database needs to examine. MySQL's documentation specifically discusses indexes as an important optimization for SELECT operations.
3. Useful for Large Tables
As tables grow, efficient ways of locating data become increasingly important.
An index can help prevent every query from having to inspect the entire table when an indexed lookup is appropriate.
Do Indexes Always Make Queries Faster?
No.
This is one of the most important things to understand about indexing.
Adding an index to every column is not a good strategy.
Indexes themselves require:
- Storage space
- Maintenance
- Additional work when data changes
For example, if you have:
INSERT INTO students ...
the database may need to update relevant indexes as well as the table data.
Similarly, UPDATE and DELETE operations can require index maintenance.
MySQL explicitly notes that unnecessary indexes waste space and increase the cost of INSERT, UPDATE, and DELETE operations. PostgreSQL likewise describes index overhead and recommends using indexes sensibly.
So the goal is not:
Create as many indexes as possible.
The goal is:
Create useful indexes for the queries your application actually needs.
When Should You Create an Index?
Indexes are particularly worth considering for columns frequently used in operations such as:
WHERE
JOIN
ORDER BY
For example:
SELECT *
FROM orders
WHERE customer_id = 100;
If this type of query is executed frequently on a large table, an index on customer_id may be useful.
MySQL's documentation also highlights the importance of indexes for queries involving joins and foreign keys.
However, whether an index actually improves a particular query depends on factors such as the data distribution, query structure, table size, and database optimizer.
How Do We Know Whether an Index Is Being Used?
Database systems provide tools for examining query execution plans.
For example, MySQL provides the EXPLAIN statement.
You can write:
EXPLAIN
SELECT *
FROM students
WHERE email = 'rahul@example.com';
The resulting execution plan provides information about how MySQL intends to execute the query and can help you investigate whether an index is being considered or used. MySQL's documentation specifically recommends EXPLAIN for examining query plans and index usage.
This is important because creating an index does not automatically mean every query will use it.
A Simple Real-World Example
Imagine an e-commerce application with a table containing millions of orders.
orders
id
customer_id
product_id
order_date
amount
status
Suppose the application frequently runs:
SELECT *
FROM orders
WHERE customer_id = 1050;
Without a suitable index, the database may have to inspect a large amount of data.
We could create:
CREATE INDEX idx_customer_id
ON orders(customer_id);
Now the database has an additional structure that can help it locate orders belonging to customer 1050.
This can be particularly valuable when the table is large and the query is executed frequently.
Different Types of Indexes
Different database systems support different index types.
For example, PostgreSQL 18 documents several index types, including:
- B-tree
- Hash
- GiST
- SP-GiST
- GIN
- BRIN
It also supports concepts such as multicolumn, unique, partial, and expression indexes.
You don't need to learn every index type before understanding the fundamentals.
For beginners, it is better to first understand:
- What an index is
- Why it improves certain queries
- B-tree indexes
- Single-column indexes
- Composite indexes
- Index trade-offs
- How to inspect query plans
Common Mistakes When Using Indexes
Mistake 1: Indexing every column
More indexes aren't automatically better.
They consume storage and increase maintenance work.
Mistake 2: Ignoring actual query patterns
Indexes should be designed around how your application accesses its data.
Mistake 3: Assuming every query uses an index
The database optimizer decides how a query should be executed.
Mistake 4: Never checking the execution plan
Using EXPLAIN can help you understand what the database is actually doing.
Mistake 5: Forgetting write performance
Indexes can improve reads while adding work to writes.
Indexing: The Trade-Off
The most important idea to remember is that indexing involves a trade-off.
INDEXING
│
┌────────┴────────┐
↓ ↓
Faster Reads Extra Cost
│
┌──────┴──────┐
↓ ↓
Storage Write overhead
Good database design tries to find the right balance.
You don't want a database with no useful indexes, but you also don't want dozens of unnecessary indexes.
Conclusion
Database indexing is one of the fundamental techniques used to improve database performance.
An index provides an additional data structure that can help the database locate rows more efficiently than scanning the entire table in suitable situations.
However, indexes are not free. They require storage and can increase the work required for INSERT, UPDATE, and DELETE operations.
Therefore, good indexing is not about creating the maximum number of indexes. It is about understanding your application's queries and creating indexes that provide meaningful benefits.
Once you understand indexing, the next concepts worth exploring are query execution plans, composite indexes, transactions, normalization, and query optimization.
The better you understand how a database finds data, the better you can design applications that continue to perform well as their data grows.
Top comments (0)