How does a computer know that "car" and "automobile" mean almost the same thing, while "car" and "carpet" — nearly identical as strings — mean nothing alike? The answer is embeddings, and they quietly power most of modern AI.
If you understand embeddings, a huge amount of AI stops being mysterious: semantic search, RAG, recommendations, clustering, deduplication. They're all the same trick wearing different clothes. Here's the trick.
Meaning as coordinates
An embedding turns a piece of content — a word, a sentence, an image, a product — into a vector: a list of numbers, often a few hundred or a few thousand of them. You can think of that vector as a set of coordinates placing the content in a high-dimensional "meaning space."
The whole point is where things land. A good embedding model places things with similar meaning close together and unrelated things far apart. "Car" and "automobile" end up as near-neighbors. "Car" and "carpet" end up in completely different neighborhoods, despite sharing four letters. The model learned this from seeing how words and sentences are actually used, so proximity in the space corresponds to closeness in meaning, not spelling.
Measuring closeness
Once meaning is geometry, "how similar are these two things?" becomes "how close are these two vectors?" The standard measure is cosine similarity — the angle between two vectors. Point in nearly the same direction, similarity near 1: strongly related. Point at right angles, similarity near 0: unrelated. It's a genuinely simple idea doing genuinely heavy lifting.
Semantic search vs keyword search
This is where embeddings earn their keep. Traditional keyword search matches strings: search "how to fix a slow laptop" and it hunts for those exact words. A document titled "speeding up a sluggish computer" — a perfect answer — might not match at all.
Semantic search embeds the query and the documents into the same space and returns the nearest documents by meaning. Now "slow laptop" and "sluggish computer" sit close together, and the right document surfaces even with zero shared keywords. You're searching by intent, not vocabulary. To make this fast at scale, the document vectors live in a vector database built to find nearest neighbors in milliseconds across millions of items. It's a pattern I reach for constantly across the systems I build.
The pipeline in practice
- Embed your content once, offline, and store the vectors.
- At query time, embed the query with the same model.
- Find the nearest vectors and return the content they point to.
That's the retrieval half of RAG, the matching engine behind recommendations, and the core of any "search by meaning" feature — one mechanism, many products.
Where it goes wrong
- Model mismatch: query and documents must be embedded by the same model, or their coordinates aren't comparable.
- Chunk size: embed a whole long document into one vector and you blur its meaning into mush; the granularity of what you embed decides what you can retrieve.
- Domain gap: a general model may not understand your specialized jargon well — sometimes you need one tuned to your domain.
Seeing it in a real system
Retrieval over an embedded knowledge store is exactly how you give an AI grounded, relevant context instead of guesses. In my write-up "Saturday MK1: an AI assistant that's a system, not a prompt", one of the engines is a knowledge base that retrieves relevant information for each request — semantic retrieval as a first-class component of the agent, not an afterthought.
Once you see meaning as geometry, the magic drains out of a lot of AI and something better takes its place: a mental model you can actually build with. More of mine at www.divyakush.com.
Divyakush Punjabi · Full-Stack & AI Engineer
Portfolio · GitHub · LinkedIn
Top comments (0)