If you have used semantic search, a recommendation engine, or any "chat with your documents" feature, you have used embeddings. They are the way modern AI systems represent meaning as numbers, and the idea is simpler than the name suggests.
In this walkthrough we will cover what an embedding actually is, then run a small Java program that shows the two things that make embeddings useful: measuring similarity and doing arithmetic on meaning. The code was compiled and run on JDK 21 (Temurin 21.0.13), and the outputs shown are real runs.
Core concepts
An embedding is just a vector. A list of floating point numbers, like [0.12, -0.44, 0.87, ...]. An embedding model turns a piece of text (a word, a sentence, a whole document) into one of these vectors. The numbers are not chosen by hand; the model learns them from large amounts of text, so that the position of each vector captures something about the meaning of the text.
Distance means relatedness. As the OpenAI embeddings guide puts it: the distance between two vectors measures their relatedness. Texts with similar meaning end up close together; unrelated texts end up far apart. The usual way to measure this is cosine similarity, which looks at the angle between two vectors: 1.0 means pointing the same way, 0 means unrelated, -1 means opposite.
Real embeddings are big. Our demo below uses 4 numbers per word so we can see what is happening. A production model like text-embedding-3-small returns 1,536 numbers per input, and text-embedding-3-large returns 3,072.
This animation sums up the whole idea:
Why should you care? Because with a decent embedding model, "close in vector space" is often a good proxy for "similar meaning", and that powers a lot of features: search ranked by relevance instead of keyword, clustering documents by topic, recommendations, anomaly detection, and classification by nearest label. It is a proxy, not a guarantee: how well similarity tracks meaning depends on the model, the data it was trained on, and the similarity metric you choose, and it can break down on text that is very different from that training data.
Walkthrough: embeddings from first principles in Java
We will hand-build tiny 4D "embeddings" for six words. Each dimension pretends to be one learned feature: royalty, humanness, masculinity, fruitiness. A real model discovers these features on its own; we pick them so the mechanics are visible. One honest limitation up front: because we choose the numbers ourselves, this demo shows the mechanics of similarity and vector arithmetic, not learned semantic behavior. It does not independently reproduce what trained models do; it makes their math visible.
import java.util.LinkedHashMap;
import java.util.Map;
public class EmbeddingsDemo {
// Hand-built 4D "embeddings". Real models learn these from data;
// ours are chosen by hand so we can see the mechanics.
// dimensions: [royalty, humanness, masculinity, fruitiness]
static final Map<String, double[]> VECTORS = new LinkedHashMap<>();
static {
VECTORS.put("king", new double[]{1.0, 0.8, 0.6, 0.0});
VECTORS.put("queen", new double[]{1.0, 0.8, 0.0, 0.0});
VECTORS.put("man", new double[]{0.0, 0.8, 0.6, 0.0});
VECTORS.put("woman", new double[]{0.0, 0.8, 0.0, 0.0});
VECTORS.put("apple", new double[]{0.0, 0.0, 0.0, 1.0});
VECTORS.put("orange", new double[]{0.0, 0.0, 0.1, 0.9});
}
static double cosineSimilarity(double[] a, double[] b) {
double dot = 0, normA = 0, normB = 0;
for (int i = 0; i < a.length; i++) {
dot += a[i] * b[i];
normA += a[i] * a[i];
normB += b[i] * b[i];
}
return dot / (Math.sqrt(normA) * Math.sqrt(normB));
}
static void printSimilarity(String a, String b) {
System.out.printf("cos(%s, %s) = %.3f%n", a, b,
cosineSimilarity(VECTORS.get(a), VECTORS.get(b)));
}
static String nearest(double[] v, String... exclude) {
String best = null;
double bestSim = -2;
for (var e : VECTORS.entrySet()) {
boolean skip = false;
for (String x : exclude) if (e.getKey().equals(x)) skip = true;
if (skip) continue;
double sim = cosineSimilarity(v, e.getValue());
if (sim > bestSim) { bestSim = sim; best = e.getKey(); }
}
return best + " (similarity " + String.format("%.3f", bestSim) + ")";
}
public static void main(String[] args) {
System.out.println("--- Cosine similarity between word vectors ---");
printSimilarity("king", "queen");
printSimilarity("king", "man");
printSimilarity("king", "woman");
printSimilarity("king", "apple");
printSimilarity("apple", "orange");
System.out.println();
System.out.println("--- Vector arithmetic: king - man + woman ---");
double[] king = VECTORS.get("king"), man = VECTORS.get("man"), woman = VECTORS.get("woman");
double[] result = new double[4];
for (int i = 0; i < 4; i++) result[i] = king[i] - man[i] + woman[i];
System.out.println("result vector: " + java.util.Arrays.toString(result));
System.out.println("nearest word: " + nearest(result, "king", "man", "woman"));
}
}
Real output from my run:
--- Cosine similarity between word vectors ---
cos(king, queen) = 0.906
cos(king, man) = 0.707
cos(king, woman) = 0.566
cos(king, apple) = 0.000
cos(apple, orange) = 0.994
--- Vector arithmetic: king - man + woman ---
result vector: [1.0, 0.8, 0.0, 0.0]
nearest word: queen (similarity 1.000)
Two things to notice:
-
Similarity matches intuition.
kingis closest toqueen(0.906), less close tomanandwoman, and completely unrelated toapple(0.000). Meanwhileappleandorangesit on top of each other (0.994). No string comparison anywhere; it all falls out of the geometry. -
Arithmetic works on meaning.
king - man + womanproduces exactly the vector[1.0, 0.8, 0.0, 0.0], which is ourqueenvector, and a nearest-neighbor search findsqueenwith similarity 1.000. This mirrors the famous finding of the original word2vec paper (Mikolov et al., 2013), which showed that vectors learned from a large corpus exhibit this kind of regularity. The difference matters: our queen match is exact because we built the vectors to make it so, while word2vec's result was empirical, emerging from vectors learned from data. The demo illustrates the arithmetic; it does not prove the learned behavior.
That nearest-neighbor search at the end is not a toy detail: it is the core of semantic search and RAG. You store the embedding of every document, embed the query, and return the documents with the highest cosine similarity, exactly the retrieval step described in the OpenAI guide.
Where do real embeddings come from?
You do not hand-pick the numbers like we did. An embedding model is trained on a large text corpus so that words and documents used in similar contexts get similar vectors. Today you typically get embeddings by sending text to a model: an API such as OpenAI's embeddings endpoint (text-embedding-3-small returns 1,536 dimensions) or an open model you run yourself. The input is text, the output is the float vector, and from that point on everything is the math we just ran.
Where you will use them
- Semantic search: rank documents by cosine similarity to the query embedding, not by keyword overlap.
- RAG (retrieval augmented generation): fetch the most similar document chunks and hand them to the LLM as context.
- Recommendations and clustering: group or suggest items whose embeddings are close.
- Classification and anomaly detection: nearest label wins; outliers are the vectors far from everything else.
Wrap-up
An embedding is a learned vector of floats where distance encodes meaning. Cosine similarity gives you "how related are these two texts" as one number, nearest-neighbor search over stored embeddings gives you semantic search and RAG, and vector arithmetic can even solve analogies. The whole mechanism fits in the small Java program above; the only thing a real model adds is learning the numbers instead of choosing them.
Sources
- OpenAI documentation: Vector embeddings
- Mikolov et al., 2013: Efficient Estimation of Word Representations in Vector Space (word2vec)
Written with AI assistance; code examples were compiled and run on JDK 21 before publishing.
Top comments (0)