A customer asks a support agent, "How do I get my money back?" The right page is titled "Refund and cancellation terms". The agent's search says "0 results". Keyword search needs the same letters, and these two texts share no words.
Embeddings fix this. Here is the idea in plain language, with one worked example using made-up numbers.
The idea in three lines
- An embedding turns a sentence into a short list of numbers, like an address on a map.
- Sentences with similar meaning get nearby addresses, even with no shared words.
- A vector database stores the addresses and quickly finds the nearest ones.
Picture a city. Cafes cluster in one district and banks in another. Here the places are sentences and the districts are topics. Nobody types in a list of synonyms. An embedding model learns from a huge amount of text which sentences are used in similar situations.
How a search by meaning works
There are two parts.
- In advance, once: cut documents into chunks (short passages on one topic), turn each chunk into a vector, and store it with its text and labels such as source, date and who may read it.
- Every question: turn the question into a vector with the same model, then fetch the nearest chunks.
Worked example: measuring "near"
Give each address two numbers: (money, login). Real ones have hundreds of numbers, but the idea is the same.
- Question "get my money back" is at (8, 2).
- Refund page is at (9, 1).
- Password page is at (1, 9).
The distance is the square root of (gap one squared + gap two squared).
Refund page:
- Gaps: 9 - 8 = 1 across, and 2 - 1 = 1 up.
- Square and add: 1 + 1 = 2. The square root of 2 is about 1.4.
Password page:
- Gaps: 8 - 1 = 7 across, and 9 - 2 = 7 up.
- Square and add: 49 + 49 = 98. The square root of 98 is about 9.9.
1.4 is far smaller than 9.9, so the refund page is nearest and it matches, with no shared words.
Trap 1: closeness is not permission
Say the agent asks, "Who can see employee salaries?" The question lands next to the HR salary pages, because they are very similar in meaning. The tool returns an HR page the agent is not allowed to read.
The map only knows what is similar. It has no idea who is asking.
The fix is to store a label on every chunk saying who may read it, and to drop the chunks the asker may not read before results are shown.
Trap 2: the re-embed trap
A manager reads about a newer embedding model and says "just switch it on". But each model draws its own map. An address from one map means nothing on another.
If you embed new questions with the new model and keep old addresses for stored chunks, search gets worse or random, because the two maps do not line up. Switching the model means re-embedding every chunk. That costs time and money, so plan it first.
Caveats
- The numbers above are illustrative. Real addresses have hundreds of numbers, and you cannot draw them.
- Some tools score so that bigger means closer, so check which way yours points.
- Large systems search "approximately" to be fast, and may miss a close match.
- Test on your real, messy documents. In one example from the chapter, search looked great on a few clean documents, but on the full set old policy versions ranked first. Old and new versions say almost the same thing, so both sit close to the question. Date and version labels fixed it. A better model would not have.
Go further
This article is a short version of a free, interactive chapter with a clickable meaning map, a 20-snippet search you can try, and a quiz: Embeddings and vector databases. No signup needed. It is part of Deployed Lab, a course on AI engineering and the forward deployed engineer role.
Top comments (0)