Every OpenAI embedding you store in Postgres is 6 KB. A hundred million documents is 600 GB of vectors before you build a single index, and an HNSW index wants to live in RAM. That is how a RAG prototype turns into a database bill nobody planned for.
The fix sounds simple: store fewer bits per number. pgvector already has halfvec (16 bits) and binary_quantize (1 bit). VectorChord ships RaBitQ. TurboQuant from Google Research is the new name everyone mentions. Each one promises a smaller index with "minimal recall loss".
I spent most of this year building TurboQuant and RaBitQ into Postgres as a Rust extension. The most useful thing I learned is this: whether compression is safe depends on your embedding model, not on the codec. Here is what that looks like in numbers, the mistake that cost me months, and how to check your own data in three minutes.
The same trick, three very different results
I took pgvector's recommended setup for binary quantization (search 1-bit codes, then rerank the best 100 with the full vectors) and ran it on three public datasets. Recall@10 means: of the 10 results exact search would return, how many did we get back.
| Dataset | Recall@10, binary + rerank |
|---|---|
| OpenAI ada-002 embeddings (DBpedia) | 0.992 |
| GloVe word vectors | 0.348 |
| SIFT image descriptors | 0.055 |
On OpenAI embeddings it is almost free: a 32x smaller search for 1% of the results. On SIFT it returns the wrong answer 95% of the time. SIFT values are all positive, so one sign bit per number says almost nothing.
Your embeddings are not DBpedia. A vendor benchmark on someone else's vectors tells you very little about yours, and nothing in your monitoring will flag that 20% of your search results quietly got worse. You find out when users stop trusting the answers.
What each option costs, on real embeddings
On 100,000 OpenAI ada-002 embeddings, exhaustive search over the codes (so this is the compression alone, before any index), averaged over three random seeds:
| Option | Bytes per vector | Smaller by | Recall@10 |
|---|---|---|---|
| float32 | 6,144 | 1x | 1.000 |
| halfvec | 3,072 | 2x | 0.999 |
| RaBitQ 4-bit | 776 | 8x | 0.972 |
| TurboQuant 4-bit | 772 | 8x | 0.953 |
| RaBitQ 1-bit | 200 | 31x | 0.834 |
| binary_quantize | 192 | 32x | 0.707 |
| RaBitQ 1-bit, rerank top 100 | 200 searched | 31x | 0.999 |
| TurboQuant 3-bit, rerank with 8-bit | 580 searched | 11x | 0.996 |
Three practical takeaways:
- halfvec is free. If your index fits in memory at half size, stop here.
- At one bit, the codec matters. RaBitQ keeps 13 more points of recall than a plain sign bit for 8 extra bytes per row, because it centres the data and corrects its own estimate.
- Search small, rerank precise. The 100 best candidates from tiny codes almost always contain the true top 10. Reranking them gets you back to 0.99+ while the part you search stays 10 to 30 times smaller.
In Postgres, today
You don't need to write an extension any more. Same 100,000 embeddings, real indexes in Postgres 18, each at the first setting that reaches 0.95 recall:
| Index | Size | Recall@10 | p50 latency |
|---|---|---|---|
| pgvector HNSW, float32 | 781 MiB | 0.959 | 2.34 ms |
| pgvector HNSW, halfvec | 391 MiB | 0.957 | 2.31 ms |
| pgvector HNSW, binary + rerank | 48 MiB | 0.963 | 2.83 ms |
| VectorChord, default | 812 MiB | 0.981 | 2.58 ms |
VectorChord, rerank_in_table
|
31 MiB | 0.957 | 1.69 ms |
Two things I would not have guessed:
-
VectorChord's default index is bigger than plain HNSW. It keeps a second copy of every vector inside the index for reranking. One option,
rerank_in_table = true, stores only the compressed codes: 25x smaller, and the fastest index in this test. VectorChord warns that this mode gets slow when the table has to come from disk; here the table fit in memory, so test it at your real size. - An HNSW graph costs more than the codes in it. The binary index uses about 500 bytes per row for 192 bytes of codes. The rest is neighbour lists. It is still 16x smaller than float32.
The mistake that cost me months
TurboQuant comes in two flavours. One spends all its bits on the codes. The other spends one bit on a correction that makes inner products mathematically unbiased. Unbiased sounded strictly better, so that is what I shipped in my extension.
It isn't, for search. Same total bits per number, 100,000 GloVe vectors:
| Bits | With the correction bit | All bits on the codes |
|---|---|---|
| 5 | 0.806 | 0.925 |
| 4 | 0.667 | 0.867 |
| 3 | 0.478 | 0.753 |
Up to 28 points of recall lost. Search only needs the right order of results for one query, and the uncorrected codes shrink every score by the same factor, which never changes the order. The correction removes that harmless shrink and adds noise, and noise is exactly what reorders neighbours.
A second detail was worth up to 11 more points: after decoding, rescale each vector back to unit length. The decoded lengths wobble by about 1% from row to row, and that wobble leaks straight into the ranking.
Both bugs looked like "compression is just lossy". Neither was. When your compressed recall is far below the paper, suspect the estimator before you blame the bits.
Which one should you use?
- Under a few million vectors: keep float32 or halfvec. Memory is cheap at that size.
-
Memory is the bill, recall must stay high: a 1-bit index with a rerank from the table. pgvector's binary + rerank or VectorChord with
rerank_in_table, whichever wins on your data. - Constant inserts, data that drifts: TurboQuant. Nothing to train, a vector is compressed the moment it arrives.
- Very large, slow-changing collections: RaBitQ with IVF (VectorChord), with an error bound that tells you how many candidates to rerank.
- Changing embedding models: measure again. Your old compression may not survive the new model.
Check your own data in three minutes
I turned all of this into shrinkvec, an open-source tool. Point it at a table and it samples your vectors, computes the exact answers, measures every option above, builds the real indexes in a scratch schema and prints the SQL for the smallest one that keeps your recall:
pip install 'shrinkvec[pg]'
shrinkvec run --dsn "$DATABASE_URL" --table documents --column embedding --target 0.95
Or try it with nothing installed, on real OpenAI embeddings with Postgres, pgvector and VectorChord inside one container:
docker run --rm -p 8080:8080 ghcr.io/ahmad-codex/shrinkvec:demo
Put --min-recall 0.95 in CI and a model or codec change that hurts search fails the build instead of reaching users.
Every number in this post comes from it. The TurboQuant and RaBitQ implementations are checked against the error figures their papers publish and against the reference libraries (turbovec and the official rabitqlib) on the same vectors. The method and every raw result are in the repository, and one script regenerates all of them.
Compression doesn't replace a good index. It decides what fits in memory, and that is usually what decides the bill. Measure it on your data before you ship it.
References: TurboQuant (Zandieh et al., ICLR 2026) · RaBitQ (Gao and Long, SIGMOD 2024) · Extended RaBitQ (Gao et al., SIGMOD 2025) · pgvector · VectorChord · turbovec · shrinkvec
Ahmad Shawar builds LLM and vector search systems in .NET, TypeScript and Rust. GitHub · LinkedIn
Top comments (0)