There's a ticket that every product team gets eventually. It says something like "search is broken" or "why can't users find anything?" or my favorite, "can we make search not suck?"
The search box was built on PostgreSQL. It worked fine for two years. Then the catalog grew, users started expecting Google-level typo tolerance, and the PM asked for faceted filters. Now the LIKE query takes 4 seconds and someone's googling "Elasticsearch vs Solr."
I've been on the team that migrates. It's not fun, but it's predictable. Here's why it happens and what you're actually signing up for.
The Moment Your Database Stops Being Good Enough
Every search feature starts the same way:
SELECT * FROM products WHERE name LIKE '%headset%';
This works. With 10,000 products and a reasonable index, it returns in milliseconds. Nobody complains.
Then the requirements compound:
- "Can we add typo tolerance? Users type 'wireles' and get nothing back."
- "Can we search across name, description, brand, and category simultaneously?"
- "Can we rank results by relevance instead of returning them in insertion order?"
- "Can we add filters (price range, brand, in-stock) that work with the text search?"
- "Can we show facet counts (42 results in Electronics, 18 in Gaming)?"
Each of these is solvable individually in PostgreSQL. Together, at scale, they compound into a query that no amount of indexing will save. I ran EXPLAIN ANALYZE on our compound search query once it hit a million records — the planner was doing a sequential scan because the trigram GIN index couldn't handle the combined text + filter + sort. Response time: 3.8 seconds. The PM's exact words were "that's not search, that's a loading screen."
PostgreSQL's full-text search (tsvector / tsquery) buys you time. It handles tokenization, stemming, and ranking. It's genuinely good for moderate-scale use cases — maybe up to a few million documents if your queries aren't too complex. But the moment you need fuzzy matching, synonym expansion, language-aware analysis, and relevance tuning together, you've outgrown what a relational database was designed to do.
The progression almost always looks like:
LIKE '%term%' → PostgreSQL FTS → Elasticsearch
The question isn't whether you'll make this jump. It's when your product requirements force it.
What Elasticsearch Actually Does Differently
Databases scan documents looking for matches. Elasticsearch flips this — it pre-builds a map from term → documents that contain it. So instead of asking "does this document contain 'headset'?" a million times, it asks "which documents contain 'headset'?" once.
Three documents:
Doc 1: "Wireless gaming headset with noise cancellation"
Doc 2: "Gaming keyboard with RGB lighting"
Doc 3: "Wireless ergonomic mouse"
ES stores:
wireless → [1, 3]
gaming → [1, 2]
headset → [1]
keyboard → [2]
mouse → [3]
Search for "wireless gaming headset" = intersect the posting lists: [1,3] ∩ [1,2] ∩ [1] = Doc 1. Done. No scanning. This is why it stays fast at 100 million documents — you're looking up terms in a map, not scanning rows.
The tricky part that took me a while to internalize: all this work happens at write time. When a document is indexed, ES tokenizes it, stems it, lowercases it, and slots every term into the inverted index. Queries are fast because indexing was slow (relatively). You're trading write cost for read speed.
Analyzers — Why Search Feels Smarter Than It Is
The reason "running shoes" matches a document containing "run" isn't magic. It's the analyzer pipeline that runs during indexing:
Tokenization: split text into individual terms
Lowercasing: "Running" → "running"
Stop word removal: drop "with", "the", "a"
Stemming: "running" → "run", "cancellation" → "cancel"
Both the indexed document AND the search query pass through the same analyzer. So when you search for "running," it's stemmed to "run," which matches documents that were also stemmed to "run" at index time.
This is also why Elasticsearch handles typos — Levenshtein distance (edit distance) allows fuzzy matching. "elsticsearch" is 1 edit away from "elasticsearch," so the engine can suggest or match the correct term.
The analyzer is the most underrated configuration in Elasticsearch, and honestly the documentation around custom analyzers is frustratingly sparse for how important they are. A poorly configured analyzer is the #1 reason search results feel bad — I spent two days debugging "why don't users find products by brand" before realizing the analyzer was stripping the brand name as a stop word. Getting it right requires understanding your data: English prose needs stemming, product SKUs need keyword (no analysis), and multi-language catalogs need per-field language analyzers.
Relevance Scoring — Where Databases Fully Give Up
Finding matches is easy. Ranking them is where search gets genuinely hard.
Three documents match "python performance":
Doc A: "Python Performance Optimization Guide"
Doc B: "Python Python Python Performance"
Doc C: "Performance Tuning for Large-Scale Python Applications"
Which one should rank first? ES uses BM25, which basically asks: how often does the term appear in this document, how rare is it across all documents, and how long is the document? A match in a short, focused title beats the same match buried in a 10,000-word page. And keyword stuffing (Doc B) actually gets penalized because BM25 applies diminishing returns to repeated terms.
You can't do this in SQL. Not without pulling every matching row into memory and rescoring in application code — at which point your database is just an expensive SELECT *.
The one thing nobody tells you: BM25 is the default and it works well out of the box, but the moment you need business-specific ranking (boost products that are in-stock, penalize items with bad reviews, promote sponsored listings), you're writing custom function_score queries. Those get complex fast and are genuinely hard to debug when results feel "off." We had a ranking bug once where out-of-stock items ranked first because the boost function was multiplicative on a field that happened to be zero for in-stock products. Took a full day to figure out.
The Architecture: ES Sits Beside Your Database, Not Instead Of It
This trips up newcomers. ES is not your primary datastore. It's a read-optimized copy.
Writes → PostgreSQL (source of truth)
↓
CDC / Events / Dual writes
↓
Elasticsearch (searchable view)
↓
Reads ← Search API
Your database handles transactions and correctness. ES handles discovery and ranking. Data flows into ES through CDC (Debezium), event pipelines (Kafka consumers), or direct application writes (simple but you'll eventually hit inconsistency bugs when one write succeeds and the other fails).
One gotcha: ES is near-real-time with a default 1-second refresh interval. Write a document, query immediately — it might not be there yet. For most search UIs nobody notices. But if you have a flow where a user creates something and lands on a page that searches for it, you'll get a bug report within the first week.
How it scales: a single Lucene index gets too big, so ES splits it into shards — each an independent Lucene index on potentially different nodes. Search hits all shards in parallel, a coordinator merges the top results. Horizontal scaling = add nodes, redistribute shards. Underneath, Lucene uses immutable segments (new data → new segment, never modify old ones, merge in background). This is why reads are fast (no locking on immutable data) and crash recovery is clean.
What Queries Actually Look Like
This is the part most "intro to ES" articles skip. Here's what you're actually writing when you use Elasticsearch:
Simple multi-field search:
GET /products/_search
{
"query": {
"multi_match": {
"query": "wireless gaming headset",
"fields": ["name^3", "description", "brand^2", "category"],
"type": "best_fields",
"fuzziness": "AUTO"
}
}
}
The ^3 boosts matches in the name field 3x over description. fuzziness: AUTO handles typos. This single query does what would take a complex stored procedure in PostgreSQL — multi-field search, fuzzy matching, field-weighted relevance scoring.
Search with filters and facets:
GET /products/_search
{
"query": {
"bool": {
"must": [
{ "multi_match": { "query": "headset", "fields": ["name^3", "description"] } }
],
"filter": [
{ "term": { "in_stock": true } },
{ "range": { "price": { "gte": 50, "lte": 200 } } }
]
}
},
"aggs": {
"brands": { "terms": { "field": "brand.keyword", "size": 10 } },
"price_ranges": {
"range": {
"field": "price",
"ranges": [
{ "to": 50 },
{ "from": 50, "to": 100 },
{ "from": 100, "to": 200 },
{ "from": 200 }
]
}
}
}
}
The bool query separates relevance (must — affects scoring) from filtering (filter — yes/no, no scoring, cached). Aggregations give you facet counts in the same request. This is the "show me wireless headsets, $50-200, in stock, with brand and price facets on the sidebar" query — one round trip.
One thing that tripped me up early: that brand.keyword field. If you try to aggregate on an analyzed text field, ES throws an error or gives garbage results. You need the .keyword sub-field (not analyzed, exact match) for terms aggregations. The mapping needs to define both:
"brand": {
"type": "text",
"fields": { "keyword": { "type": "keyword" } }
}
Text for searching, keyword for filtering and aggregating. Once you know this it's obvious, but I lost a few hours to it the first time.
Custom ranking with function_score:
GET /products/_search
{
"query": {
"function_score": {
"query": { "multi_match": { "query": "headset", "fields": ["name", "description"] } },
"functions": [
{
"filter": { "term": { "is_sponsored": true } },
"weight": 1.5
},
{
"field_value_factor": {
"field": "review_score",
"modifier": "log1p",
"missing": 1
}
}
],
"boost_mode": "multiply"
}
}
}
This boosts sponsored products by 1.5x and factors in review scores with logarithmic dampening (so a product with 4.8 stars doesn't infinitely dominate over 4.5). This is where relevance tuning lives — and where it gets hard to debug when results feel wrong.
The Alternatives Landscape (2025+)
Elasticsearch isn't the only option anymore. The landscape has shifted, and for some use cases the newer tools are genuinely better:
Typesense — we used this for internal documentation search and it was running in an afternoon. Typo tolerance and relevance work out of the box. No JVM, no cluster management, single binary. Great for product/site search under 100M documents. Falls short on complex aggregations and log analytics — it's not trying to be ES, and that's fine.
Meilisearch — similar positioning to Typesense. Rust-based, developer-friendly, sub-50ms search without tuning. Good until you need sharding or serious multi-tenancy. I'd pick it for a startup's product search without hesitation.
OpenSearch — AWS's fork after Elastic changed the license. Functionally near-identical to ES 7.x, API-compatible. If you're already on AWS and want managed infrastructure (OpenSearch Service) without licensing headaches, this is the pragmatic choice. Migration from ES is mostly a find-and-replace in your client config.
Algolia — fully managed, zero infrastructure. Response times are absurd (single-digit milliseconds). But pricing is per search operation, which gets expensive fast at scale. Good for teams that value DX and speed-to-market over cost efficiency. Less control over relevance tuning than self-hosted ES.
PostgreSQL FTS — don't dismiss it. For under 2-3M documents with straightforward requirements (tokenized search + basic ranking, no fuzzy), it's genuinely sufficient. You avoid an entire operational category. We should've stuck with it longer on one project instead of jumping to ES prematurely.
My take: if your dataset is under 10M documents and your requirements are "text search + filters + typo tolerance" — look at Typesense or Meilisearch first. You'll be in production in a day instead of a week. If you need complex aggregations, log analytics, custom scoring pipelines, or you're north of 100M documents — Elasticsearch (or OpenSearch) still wins. Nothing else handles that combination at that scale.
The Cost
Nobody talks about this in the "why Elasticsearch is amazing" posts:
- You're now running two data stores. Two things to monitor, two things that can fail independently, and sync logic between them that will have bugs.
- Mapping changes require reindexing. Change an analyzer on a field? Every existing document keeps the old index structure. You need to build a new index from scratch and swap.
- At 500M documents, that reindex takes hours. Hope you set up the alias pattern (
products_v1→products_v2swap) from the beginning. Retrofitting it into a system that assumed a single index name is a bad week. - Shard management is a dark art. Too few shards = can't distribute load. Too many = overhead per shard kills performance. The default of 1 shard is almost never right for production, and the "correct" number depends on your document count, document size, and query patterns.
It's worth the cost if your product genuinely needs good search. Just know what you're signing up for operationally.
When You Don't Need Elasticsearch
Not every search box justifies a cluster:
- < 100K documents with simple queries: PostgreSQL FTS handles this fine. Seriously, don't add infrastructure for a problem you don't have.
- Exact-match lookups only: if users search by SKU, order ID, or email — that's a database index, not a search engine. A B-tree does this in microseconds.
- Strong transactional consistency required: if a write must be visible in search results within the same request, ES's 1-second refresh lag will hurt you. We learned this one the hard way with a status update flow where users would update a record and immediately search for it with the new status — and not find it.
- Tiny product catalog: 500 products? Load them into memory on startup, filter in Java. You don't need distributed infrastructure for this.
The decision should be: "have we outgrown what our database can do for this specific access pattern?" If yes, add ES. If you're speculating that you might need it someday — wait until you actually do.
The Progression Every Team Goes Through
Elasticsearch isn't the best database (it isn't one), and it isn't even the search algorithm (Lucene does that part). It's infrastructure that bridges the gap between what users expect from a search box and what a relational database can deliver.
That gap widens with every feature request. And at some point, probably around the third PM ticket about "search quality," you'll start evaluating ES. When you do, plan for the alias-based reindexing pattern from day one, budget for the operational overhead, and don't index more fields than you actually search on. The cluster will thank you.
Further reading:
- Elasticsearch: The Definitive Guide (official, somewhat dated but conceptually solid)
- Relevant Search — Turnbull & Berryman (the best book on search relevance engineering)
- Apache Lucene (the actual engine underneath)
- BM25 scoring
Top comments (0)