Searching billions of documents for a phrase and getting ranked results in tens of milliseconds looks like magic. It is not. It comes down to two ideas working together: an index that maps words to documents instead of scanning documents for words, and a way to spread that index across machines so each holds only a slice. Understand both and full-text search stops being mysterious.
The core problem
A database scans rows. If you ask a plain database to find every document containing a word, it reads documents and checks them, which is linear in the amount of data. That is fine for exact key lookups and hopeless for free-text search across huge corpora. You need the opposite mapping. Instead of "given a document, what words does it have", you want "given a word, which documents have it". That inversion is the whole trick.
The second problem is size. One machine cannot hold the index for billions of documents, and one machine cannot serve the query load. So the index has to be split across nodes, and a query has to find the right nodes and combine their answers.
Key design decisions
Build an inverted index. At index time, each document is broken into tokens by an analyzer that lowercases, splits on word boundaries, and often strips or stems words. For every token, the engine keeps a posting list: the set of document ids that contain it, often with positions for phrase matching. A query for a word becomes a direct lookup of its posting list, not a scan. A multi-word query intersects or unions posting lists, which is fast because the lists are sorted.
Store the index in immutable segments. New documents go into small new segments rather than editing existing ones. Segments are immutable, which makes them cache-friendly and safe to read without locks. A background process merges small segments into larger ones over time. A delete is just a marker; the document is removed for real during a later merge.
Split an index into shards. An index is divided into shards, each a self-contained inverted index over a subset of documents. Shards spread across nodes, so both storage and query work are distributed. The number of shards is chosen up front because it determines how documents map to shards.
Route by hashing the document id. To decide which shard a document belongs to, the engine hashes its routing key, usually the id, modulo the shard count. The same hash decides where to look on read, so a lookup by id goes straight to one shard. A full-text query, which could match anywhere, scatters to all shards, each returns its top ranked hits, and a coordinating node gathers and merges them into the final ranking. This scatter-gather is why fixing the shard count early matters: change it and the hash no longer points at the right place.
The trade-offs
The inverted index makes reads fast and writes heavier. Indexing a document means analyzing it and updating many posting lists, and results are not visible until a refresh exposes the new segment. That is why search is near-real-time, not instant: there is a small, tunable delay between writing a document and being able to find it. You trade write latency and immediacy for very fast reads.
Immutable segments trade update efficiency for read speed and simplicity. Updates and deletes accumulate as new segments and delete markers, and merging them costs I/O and CPU in the background. Skip the merges and query speed degrades as segment count grows.
Sharding trades operational simplicity for scale. Too few shards and you cannot spread load; too many and every query pays coordination overhead across all of them, and each tiny shard wastes overhead. Because the shard count is baked into routing, getting it wrong means reindexing to change it.
How the real system does it
The real design analyzes text into tokens, stores an inverted index in immutable, periodically merged segments, splits each index into a fixed number of shards, and routes by hashing the routing key so id lookups hit one shard while full-text queries scatter to all shards and gather ranked results. Replicas of each shard add read capacity and failover.
The general lesson: pick the data structure that matches the question. Databases scan because they answer "what is in this row"; search inverts the index because it answers "which rows contain this term". The right structure, spread the right way, is what makes the impossible query fast.
I wrote the full breakdown, with diagrams and the data model, here: https://www.systemdesign.academy/interview/design-elasticsearch
Top comments (0)