What Are Embeddings?: How AI Knows a Cat and a Kitten Are Related ⭐

This is a very simplified version of what an embedding is and is meant for beginners. Felt that I had to put a disclaimer here for the ai bros.

The question embeddings answer

How does a computer "know" that a cat and a kitten are related, that a dog is closer to a cat than to a car, and that a car is closer to a truck than to either a cat or a dog?? Computers don't understand meaning, they only understand numbers. So the trick is turning the meaning of text into numbers that math can work with.

An embedding is simply a long list of numbers that represents the meaning of a piece of text. Modern embeddings often contain hundreds or even thousands of numbers, but you don't need to understand each number individually. Pieces of text with similar meaning end up close together in this mathematical space.

A map of meaning

Imagine every word or phrase being placed on a giant map. Similar meanings end up close together, while unrelated end up far away. A model places words near each other based on how they are used in huge amounts of text, so "cat" and "kitten" end up near each other, "dog" and "puppy" are also close together and nearby to cats too, and something unrelated like a "car" sits in a completely different region.

In reality, embeddings live in far more bigger structure than a map, meaning in multiple dimensions, usually hundreds or thousands, which is impossible to actually draw (or even understand with my small 3d brain). But the two-dimensional version captures the core idea: distance yields relationships.

Where the numbers actually come from

An embedding isn't assigned by a person deciding "cat should be at this coordinate". During the training phase a model gradually learns how to convert text into embeddings based on the contexts it sees across enormous amounts of data. Words used in similar contexts end up near each other, because that's all the model ever sees: patterns in language.

This is also why embeddings capture relationships that go beyond simple synonyms. Classic examples show that the relationship between "king" and "queen" is mathematically similar to the relationship between "man" and "woman".

Why this matters for search and recommendations

Traditional keyword search matches exact words. Searching for an "affordable laptop" in a keyword-based system might miss a product like a "budget-friendly notebook" because the words don't literally match, even we (the humans) know that the meaning is identical.

Embedding-based search fixes this problem. Instead of matching words, it converts your search query into a point in the giant map and finds the nearest neighbors/points. This is why modern search and recommendation systems can surface something relevant even when you didn't use the "right" word.

Words aren't the only thing that can be embedded

Despite using words like "cat" as examples, embeddings aren't limited to single words. Entire sentences, paragraphs, documents, images, and even audio can all be represented as embeddings. The idea stays exactly the same: similar meaning ends up close together in the giant map.

In previous articles we talk about RAG and how this has a retrieval step. A simplified description of the retrieval step by adding the embeddings to the calculation will be: Your question is converted into an embedding, the documents have been embedded already, and the system simply retrieves the ones that are closest in that embedding map.

Where this shows up if you're building products

If you're building search, recommendations, finding duplicate content, or anything that needs to group or match content by meaning rather than exact text, embeddings are usually the tool for it, not string matching anymore.

A few practical things worth knowing:

• Documents are usually embedded once and stored. When a user searches, only the new query needs to be embedded, making retrieval much faster.
• Similarity isn't the same as correctness. Two pieces of text can be very similar while still answering the wrong question. That's why many systems rank, filter, or validate results after the similarity search.
• The embedding model matters. Different embedding models place things differently, and a model tuned for one domain may not embed specialised or technical content as usefully as one tuned for that domain.

Resources

If you'd like to dive deeper into embeddings and how they're used in modern AI systems, these are excellent starting points:

• Anthropic – Embeddings Documentation
  
  https://platform.claude.com/docs/en/build-with-claude/embeddings
  
  A practical guide explaining what embeddings are, how they work, and how they're used for semantic search, recommendations, and retrieval.

• OpenAI – Embeddings Guide & FAQ
  
  https://help.openai.com/en/articles/6824809-embeddings-faq
  
  Covers OpenAI's embedding models, common questions, distance metrics, and best practices.

• Word2Vec: Efficient Estimation of Word Representations in Vector Space (2013)
  
  https://arxiv.org/abs/1301.3781
  
  The classic paper by Tomas Mikolov and colleagues that popularised learned word embeddings.

• Pinecone – Dense Vector Embeddings Explained
  
  https://www.pinecone.io/learn/series/nlp/dense-vector-embeddings-nlp/
  
  A visual, beginner-friendly explanation of dense vectors, embeddings, and semantic search, with examples beyond Word2Vec.

• Jay Alammar – The Illustrated Word2vec
  https://jalammar.github.io/illustrated-word2vec/
  A visual, intuitive explainer on word embeddings, in the same style as his famous "Illustrated Transformer."

• Embeddings & Vector Search in 2026: The Engineer's Complete Guide
  https://jobsbyculture.com/blog/embeddings-vector-search-guide-2026
  Covers embedding models, ANN algorithms (HNSW, IVF), and building production RAG pipelines.

• How to Generate Better Embeddings for Vector Search
  https://pr-peri.github.io/llm/2026/02/12/generate-embeddings-vector-search.html
  Practical guide to improving retrieval quality via chunking, cleaning, metadata, hybrid search, and proper evaluation (Recall@K, MRR).

• Best Embedding Models 2026: Tested on 50K Documents
  https://pecollective.com/tools/best-embedding-models/
  Compares OpenAI, Cohere, Voyage AI, and Jina embedding models on real retrieval benchmarks, not just MTEB scores.