Slice the last 1400 dimensions off a normal embedding and search quality collapses. Do the exact same slice on a Matryoshka embedding and you keep most of your recall while cutting index size and query cost by 20x. Same operation, a Python array slice, wildly different outcome. The difference is entirely in how the model was trained, and if you don't know which kind of model produced your vectors, you will get burned.
Matryoshka Representation Learning (MRL) is the trick behind OpenAI's text-embedding-3 dimensions parameter and the truncatable outputs of models like nomic-embed-text-v1.5 and the newer Gemini and Qwen embedding families. This post is about what MRL actually does to the vector space, how to exploit it for adaptive retrieval, and the specific ways it breaks.
TL;DR
- Matryoshka embeddings pack a coarse-to-fine representation into a single vector: the first 64 dims are a usable embedding, the first 256 a better one, all 1536 the best. You truncate by slicing and re-normalizing.
- The magic is in the loss, not the architecture. MRL sums the contrastive loss over many nested prefixes during training, forcing the earliest dimensions to carry the most information.
- Truncating a non-Matryoshka embedding destroys recall because information is spread uniformly across dimensions with no ordering. Never slice a vector unless the model card says MRL.
- The payoff is adaptive retrieval: shortlist millions of docs with a cheap 64-dim ANN pass, then re-rank the top candidates with full-dimension vectors. 5-10x less memory, near-identical top-k.
- Combine truncation with binary quantization for compounding wins, but validate recall on your data — the accuracy cliff is dataset-dependent, not a fixed dimension.
What makes a Matryoshka embedding different from a normal one?
A standard embedding model distributes semantic information across all output dimensions with no particular structure. Dimension 5 is no more important than dimension 1400. If you keep only the first 64 of 1536 dims, you throw away roughly 96% of the signal, scattered arbitrarily, and cosine similarities become noise.
Matryoshka embeddings are trained so that information is front-loaded. The name comes from Russian nesting dolls: a smaller usable embedding lives inside the larger one. The first m dimensions of a d-dimensional MRL vector form a coherent embedding for any m in a predefined set like {64, 128, 256, 512, 768, 1024, 1536}. You get the smaller representation for free by slicing:
import numpy as np
def truncate(vec: np.ndarray, dim: int) -> np.ndarray:
# Slice to the target prefix, then RE-NORMALIZE.
# Cosine/dot-product ANN assumes unit vectors; the prefix
# of a unit vector is not unit-length.
sub = vec[..., :dim]
return sub / np.linalg.norm(sub, axis=-1, keepdims=True)
full = model.encode("how do I rotate a KV cache") # shape (1536,)
small = truncate(full, 64) # shape (64,)
The re-normalization line is the one people forget. A prefix of a unit vector has norm < 1, and dot-product indexes (which most vector DBs use for cosine) will silently return distorted scores if you feed in un-normalized prefixes.
How does the Matryoshka loss force the front dimensions to matter?
The architecture is an ordinary transformer encoder. All the work happens in the objective. During training, instead of computing the contrastive loss once on the full d-dim output, MRL computes it separately on each nested prefix and sums them:
L_MRL = Σ_{m ∈ {64,128,...,d}} w_m · L_contrastive( truncate(z, m) )
Every prefix has to independently solve the retrieval task. Gradients from the 64-dim term flow only through the first 64 coordinates, so those coordinates are pressured to be maximally discriminative on their own. The 128-dim term reuses those first 64 and refines with the next 64, and so on. The result is a spectral-like ordering: early dimensions capture dominant semantic axes, later ones add fine detail. It behaves like a learned, task-aware PCA baked into the forward pass — but unlike PCA you don't need a separate projection matrix or a fitted basis; the ordering is intrinsic to the vector.
Two practical consequences fall out of this:
- Training cost barely moves. The extra loss terms are cheap matrix ops on already-computed activations. That's why labs ship MRL by default now — it's nearly free to add.
- The full-dimension quality is essentially unchanged versus a non-MRL baseline. You are not trading peak accuracy for truncatability; you're getting the nested property as a bonus.
Why does truncating a normal (non-MRL) embedding wreck recall?
Because there is no ordering to exploit. In a conventionally trained model, discriminative information is smeared roughly uniformly across dimensions — the variance per dimension is flat, and no prefix was ever asked to stand on its own. Chop it and you delete random pieces of the signal. Empirically, slicing a non-MRL 1536-dim model to 256 dims can drop recall from the high-90s to coin-flip territory on hard queries, while an MRL model at 256 dims loses only a few points.
This is the failure mode I see most: someone reads that OpenAI supports a dimensions parameter, assumes truncation is a universal property of embeddings, and starts slicing vectors from a self-hosted model that was never trained with MRL. Retrieval quality craters and they blame the vector DB. Check the model card. If it doesn't explicitly say Matryoshka or list supported truncation dimensions, treat the full dimension as the only valid one.
One subtlety even for OpenAI's API: passing dimensions=256 returns a vector that is already truncated and re-normalized server-side. If you instead pull the full text-embedding-3-large vector and slice locally, remember to normalize yourself, and slice only to dimensions the model actually supports — arbitrary cuts between the trained checkpoints degrade faster than the advertised ones.
How do you use this in production? Adaptive retrieval.
The highest-leverage use of MRL is a two-pass "funnel" over a large corpus. Store full-dimension vectors once, but build the hot ANN index on a cheap truncated prefix. Shortlist fast, then re-rank precisely.
# Ingest: store full vectors, index a 64-dim prefix.
for doc in corpus:
v = model.encode(doc.text) # 1536-dim, unit norm
doc.full = v.astype(np.float32) # kept for re-rank
ann_index.add(truncate(v, 64), doc.id) # tiny, fits in RAM
def search(query, k=10, shortlist=200):
q = model.encode(query)
# Pass 1: cheap, high-recall shortlist over the whole corpus.
cand_ids = ann_index.search(truncate(q, 64), shortlist)
# Pass 2: exact re-rank of a few hundred candidates at full dim.
scored = [(cid, float(q @ store[cid].full)) for cid in cand_ids]
scored.sort(key=lambda x: -x[1], reverse=False)
return scored[:k]
The economics: a 64-dim float32 index is 24x smaller than 1536-dim. For 10M documents that's roughly 61 GB down to 2.6 GB — the difference between "spills to disk across a cluster" and "fits in one machine's RAM." Pass 1 touches every document but does it cheaply; pass 2 does the expensive full-precision dot product on only the ~200 survivors. Because MRL's 64-dim prefix still gets the right documents into the shortlist (high recall even if the ordering within it is rough), the full-dim re-rank recovers the precise top-k. You get near-full-dimension nDCG at a fraction of the memory and query FLOPs.
Tune shortlist to your recall target. If the 64-dim pass has, say, 95% recall@200, widen the shortlist or bump the index prefix to 128 dims until the re-ranked top-k stabilizes.
Can you stack Matryoshka with quantization?
Yes, and the gains multiply. Truncation cuts the number of dimensions; quantization cuts the bits per dimension. They're orthogonal.
A common aggressive stack: binary-quantize a truncated prefix for the shortlist (1 bit/dim, Hamming distance search), then re-rank with full-dimension float32 or int8 vectors. A 1024-dim binary vector is 128 bytes and searches via XOR + popcount — brutally fast — while your full vectors sit in a secondary store for the re-rank pass. The relevant order of operations: shortlist with the smallest/cheapest representation that still clears your recall bar, re-rank with the richest one you can afford on a few hundred candidates.
The catch is that the errors compound. Truncation loses fine-grained axes; binary quantization loses magnitude. Do both too aggressively and the shortlist starts missing relevant docs that the re-rank pass can never recover — you can't re-rank a document that wasn't retrieved. Recall lost in pass 1 is gone.
Where does Matryoshka truncation actually break?
The accuracy cliff is dataset-specific. There is no universal "safe" dimension. On a corpus with well-separated topics, 64 dims might be plenty; on a corpus of near-duplicate legal clauses where distinctions live in subtle late dimensions, even 512 dims may blur documents together. The coarse prefixes capture dominant axes of variation — if your hard cases differ only along minor axes, truncation erases exactly the signal you need. Always measure recall on your own queries before committing to a dimension.
Symmetry matters. Query and document vectors must be truncated to the same dimension and both re-normalized. Mixing a 64-dim query against 256-dim stored vectors is a dimension mismatch that either errors out or, worse in a hand-rolled setup, silently compares the wrong subspaces.
Not all "supports dimensions" models are true MRL. Some models expose a projection head or PCA-fitted reduction that only works at specific sizes and isn't a genuine nested representation. With those, arbitrary slicing fails even though a dimensions parameter exists. Trust the trained checkpoints, not interpolation between them.
Downstream re-rankers assume full vectors. If you feed a cross-encoder or a fusion step the truncated shortlist scores as if they were final, you inherit the truncation error. Keep truncation confined to the ANN shortlist stage and re-rank with full fidelity.
So: can you truncate embedding dimensions without losing recall?
Only if the model was trained with Matryoshka Representation Learning. MRL front-loads semantic information into the earliest dimensions by summing the contrastive loss over many nested prefixes during training, so slicing a vector to its first m coordinates (and re-normalizing) yields a smaller, still-coherent embedding. That enables adaptive retrieval — a cheap low-dimension ANN shortlist over the whole corpus followed by a full-dimension re-rank — cutting index memory and query cost by an order of magnitude with minimal recall loss. Apply the same slice to a conventionally trained embedding and recall collapses, because information there has no ordering to preserve. Check the model card for Matryoshka support, always re-normalize after slicing, and validate the recall-versus-dimension trade-off on your own data, since the accuracy cliff depends entirely on how separable your corpus is.
Top comments (0)