You type a query into a box. Three milliseconds later the thing you wanted is sitting at the top of the results, and it feels like the machine read your mind.
It did not. Your words got torn into pieces, weighed against billions of other queries, matched against an inverted index, scored by algorithms juggling dozens of relevance signals, and reassembled into a ranked list, all before you finished blinking. The magic is a very fast, very elaborate act of misdirection, and we have spent thirty years getting better at the trick. It is genuinely delightful once you see how the trick works, and by the end of this post you will.
Now large language models and semantic search have walked on stage, and everyone has decided the trick is obsolete. Most people have this backwards. Before you can talk sensibly about where search is going, you have to be honest about what search already is.
I have built search engines for two decades: large-scale comparison shopping sites, then Amazon CloudSearch fifteen years ago, and the long road from there into Amazon OpenSearch Service. I have watched several revolutions arrive. This one is real. The reaction to it is mostly confused. So let me walk you through what search actually does, why the old parts are not going anywhere, and what genuinely changes when the thing typing the query stops being human.
The story the industry tells about “traditional” search
AI-era commentary often assumes lexical search, the kind where you match words to words, is a quaint pre-AI relic we are finally evolving past. That take is the kind of wrong that gets a production system operator paged at 3 a.m.
Start with the workhorse, the inverted index. Picture the index at the back of a cookbook. Instead of reading all 400 recipes to find the ones with chocolate, you flip to the back, look up “chocolate,” and get a list of page numbers. The inverted index does the same thing for every term in every field across billions of documents, which is the difference between a cookbook and a library that answers you instantly.
Building that index is more than stuffing words into a table. The engine parses text into meaningful units, because “New York” should be matched like one term and not two. It applies language rules, so “running” and “run” match together. It weighs rare terms over common ones, because matching “ergonomic” tells you far more than matching “the.” All of that careful work happens up front, at indexing time, so your query can be cheap later. Expensive now so it is fast forever is a good bargain, and it is not primitive.
Here is the deeper point, search was never really about matching. Matching is table stakes. Search is about relevance, and not all matches are equal. Pretending they are is how you (incorrectly, usually) rank a laptop sticker above an actual laptop. Decades of unglamorous, load-bearing refinement live in that ranking: BM25, TF-IDF, field boosting, recency weighting. This is also why a search engine is not just a database with a nicer query box. Databases optimize for transactional correctness, and they were never built to apply information science to rank results by relevance. Search engines trade away some of those transactional guarantees on purpose, in exchange for thousands of queries per second at millisecond latency and a scoring layer whose entire job is relevance. “Just add search to the database” is a kluge that demos well and dies in production.
Where embeddings and LLMs actually fit
So if lexical search is not primitive, where do embeddings and large language models fit? They are not the replacement. They are the extension, and understanding exactly what they add is where it gets fun.
Lexical search is excellent at explicit terms and genuinely bad at synonyms it was never told about. Search for “laptop” against a document that only says “notebook computer” and it shrugs. Embeddings fix that, and it helps to be precise about how, because the honest explanation is more interesting than the hand-wavy one. An embedding is a projection learned from word co-occurrence. “laptop” and “notebook computer” land near each other in vector space not because the model knows what either thing is, but because the two phrases show up surrounded by the same words across the training corpus. It is correlation over usage, not comprehension. That distinction matters, and it is also kind of wonderful: you get results that feel like understanding out of pure statistics about which words keep company with which.
That correlation is genuinely useful, because it retrieves documents that share no exact terms with the query. Embeddings do not evict the inverted index, though. They move in next door, and the combination has a name: hybrid search. Lexical matching gives you precision, the exact hits on product names and SKUs where being approximately right is being wrong. Embedding-based matching gives you recall, the related documents whose surrounding words overlap even when the query terms do not appear. You want both, because a system with only one fails in a way the other would have caught. OpenSearch Service runs both over the same documents and combines the scores.
One rule survives every wave of this, and if you take one practical thing from this post, take this one: start with the document, not the model. The document is what you retrieve and what you store, so define it first. If you want to retrieve products, your document is a product. It sounds too obvious to say, yet it is easy to skip, because there is always an existing database schema right there, ready to be shoehorned into a search index. Get the document right and everything downstream falls into place: which fields are searchable, which need exact matching, whether you need vectors at all.
The searcher is changing, and search is the foundation underneath
The next wave is already here, and it reframes what search is for. For thirty years, the thing on the other end of the query was a person. That single assumption shaped every design decision. A person is impatient, so you optimize for a fast first page. A person scans a ranked list and clicks one or two results, so a strong top ten is the whole game. A person tolerates “good enough,” because human eyes do the final filtering for free. Ten blue links works because a human knows which link to pick.
An agent is a different animal. It does not scan, it consumes: it fires queries in a loop and reads the whole result set, not the first few. An agent has infinite patience, so the human problem, getting the one right answer to the top before someone gives up, mostly goes away. What it lacks is taste. It cannot glance at result seven and think “close, but not that one.” So the game shifts. Ranking, the careful ordering of a list for a glancing human, matters less. Bringing back the right matches, without polluting the set with plausible-looking wrong ones, matters more. Ranking does not vanish, since something still has to choose which candidates fit the context window, but that reranking step is a topic for the follow-up post.
Agents also change the shape of the query. Instead of one fuzzy human query hoping a good ranking sorts it out, a task planner decomposes the work into tightly scoped retrievals, each sub-agent asking for exactly the matches its step needs. The agent will act on what comes back without a human eye to catch a bad match, so the results have to be precise, structured, and self-describing.
The abstraction worth holding onto is simple: the fundamentals of retrieval do not change, but the consumer does, and the consumer sets the goal. Search is the foundation the agentic wave is built on, not a casualty of it, because any agent that looks something up before it acts is standing on a search engine.
Agent-driven search is a big topic, and it deserves more than a section at the tail of a post about fundamentals. Retrieval for agents, result formats an agent can trust, and how retrieval-augmented generation changes when the “user” is a loop and not a person, all get their own post soon. For now, the useful move is simply to notice that “who is searching” is now a real design variable, and to stop assuming the answer is always a human.
Where this is heading
The interface to search is changing far faster than the machinery underneath. Chat feels revolutionary precisely because it hides the complexity, but pull the cover off and it is still retrieval, ranking, and relevance scoring, now with embedding-based correlation stirred in and, increasingly, an agent rather than a person reading the output.
The teams that build the best search over the next few years will not be the ones chasing every new model on release day. They will be the ones who understand the fundamentals, know where a semantic layer earns its keep, and ask who is actually on the other end of the query. So define your documents clearly. Use hybrid search. Measure relevance, not just latency. The tools keep changing, and that is the fun of it. The job, closing the gap between what a searcher wants and the information that answers it, does not.
Top comments (0)