Product Quantization part 2- how querry gets fetched

yashasvi.shukla — Sat, 25 Jul 2026 05:16:49 +0000

in last blog i have talked about how document is stored via the process of product quantization - which makes querrying faster .
we saw how an approximately 6 GB vector dataset containing one million vectors was compressed to a much smaller size.

step 6

continuing from the last blog

when querry comes in first it's converted into embeddings - from that same model which have converted the doc into embedding each embedding is converted into 96 chunks of 16 dimensions

step 7

we pick querry's chunk 1st/96 - measure the distance against each of chunk 1's 256 centroids which we made at step 3 ( check prev blog )

we store all the distance we have measured for querry chunk 1 against all 256 centroids in a LOOKUP table_1 (256 entries).

step 8 - distance measuring against each centroid for chunk 2 ,chunk 3 .....chunk 96

we repeat the same process for
querry's chunk 2nd/96 - measured against chunks 2's 256 centroids ( made at step 3) - LOOKUP_table_2 with 256 entries is created for this chunk 2 also

then qeurry's chunk 3/96 - LOOKUP_table_3 created
...
till querry's chunk 96/96 - LOOKUP_table_96 created

we do it once for a querry we recieve every time

step 9 important step here every thing combines and gives us the result

things are bit complex here read carefully 2-3 times -

remember these 2 points
we have compression of 96 array reference (MADE in 5th step ), during compression stage .
we have a 96 lookup_table created at prev step(step 8) .

we pick 1st vector array from 1M vectors (see step 5)
it have 96 array reference
at array reference 1/96 the code was e.g 17
we check lookup table_1 for no.17 - we have a calculated distance already in prev. step we store that distance

then array reference 2/96 the code was lets say 4 ,
we check with lookUP_table_2 in lookup table for id 4 we save the distance

then compressed array reference 3/96 we check with lookup_table_3 we store the distance
..
..
we continue this for array reference 96/96 we check till lookup_table_96 we store the distance

step 10 addition

we add all the distances we measured in prev step for vector 1

step 11

now just like we have done the operation in step 09
we are going to repeat that operations for vector 2 till vector 1Million.
and store the total distance for each vecotr 2 , vector 3 , ....... vector 1 Million.

step 12

now we do have sum of distances from each vector ,
now we check vector 1 - is close or far ,
then for vector 2 - is it close or far ,
..
..
this is continued for 1m vector .
and closest one are the answers for the querry.

step 13 complete summary

Product Quantization Explained

yashasvi.shukla — Tue, 21 Jul 2026 08:54:02 +0000

While building my agentic architecture, I had to choose between pgvector, ChromaDB, Qdrant, and Pinecone. I started studying where each one performs well and where it falls short.

One thing I kept coming across was that pgvector starts consuming a lot of memory once your dataset grows to millions of vectors. That's when people often switch to Qdrant.

I became curious: what is Qdrant doing under the hood that makes it so much more memory-efficient?

That question led me to Product Quantization (PQ).

pgvector works well for smaller datasets, but once you have millions of vectors, memory usage increases rapidly because every vector is stored in full precision. Qdrant handles the same scale much more efficiently using a compression technique called Product Quantization (PQ). Understanding how PQ works took me much longer than I expected.

1st step

What an embedding actually is. One document → 1 vector of N numbers (e.g. 1536). Each number is a coordinate along a learned direction — not spatial like x/y/z, but the mathematical generalization of it.

Why compression is needed at all. we have now 1 million vectors × 1536 floats × 4 bytes = ~6GB just for one field, in memory its a large number . This is the "why should I care" bridge before you go into HOW.

2nd step

we pick 1st vector and slice it into 96 sub-vectors of 16 numbers
This split is arbitrary—it isn't intrinsic to the data. It's simply a design choice that makes clustering computationally tractable.

keep in mind

keep in mind 1st vector's chunk 1 is in dimension 1-16,
similarly 2nd vector's chunk 1 is also in dimension 1-16,
it goes on 1MNth vector - chunk 1 is in dimension 1-16
i.e chunk 1 of each vector is in same dimension i.e (1-16)

similarly 1st vector's chunk 2 is in dimension 17-32
2nd vector's chunk 2 is also in dimension 17-32
till 1millionTH vector chunk 2 will be in dimension 17-32
.
.
.
similarly 1st vector's chunk 96 is in dimension 1521-1536
2nd vector's chunk 2 is also in dimension 1521-1536
till 1millionTH vector chunk 2 will be in dimension 1521-1536

idea here is each similar chunk from each vector lives in similar dimensional space

3rd step

Next, we take Chunk 1 from every vector. Since each Chunk 1 represents dimensions 1–16, we now have 1 million 16-dimensional chunks. We then run k-means clustering on these chunks.

K-means groups together chunks that are close to each other in the vector space (i.e., the distance between them is small). For each group, it computes the average of all the chunks in that group. This average becomes the cluster centroid.

We repeat this process until 256 cluster centroids are created for Chunk 1.

now we repeat the same for chunk 2 of ALL 1M vectors
then for chunk 3 ........till chunk 96

we have 256 cluster for each 96 chunks

so far we had 1 million vectors , each vector of 1536 dimension ,
we each vector into 96chunks * 16spaces

we took each chunk 1 formed 256 clusters
we took each chunk 2 formed 256 clusters
.
.
we took chunk 96 formed 256 clusters

now

the compression begins

4th step

remember at second step we have divided each vector into 96 chunks

now read carefully
**

we pick up vector 1

see step 2
we pick chunk 1 (from step 2) we compare it with chunk

1's 256 centroids (which we made in prev step)
and check chunk 1 is closest to which centroid among these 256
centroids

say it was close to cluster 33 , we store this id 33

we are still at vector 1
now we pick chunk 2 (from step 2 ) we we compare it with chunk

2's 256 centroids (which we made in prev step) not any other chunk's
centroids

we found it was close to say cluster 221 , we store this id 221

we repeat the same process till 96th chunk

so we get a compressed pq 96 ids for vector 1
[33,221,88.......144]

5th step **
now we move on to vector 2
we repeat the same process
we pick up **chunk 1 (from step 2) we compare it with chunk

1's 256 centroids (which we made in step #3)
and check chunk 1 is closest to which centroid among these 256
centroids

say it was close to cluster 17 , we store this id 17

we are still at vector 2
now we pick chunk 2 (from step 2 ) we we compare it with chunk

2's 256 centroids (which we made in step 3) not any other chunk's
centroids

we found it was close to say cluster 4 , we store this id 4
we got ids for vector 2
[17,4,190......,9]

we repeat the same process till 96th chunk

and we repeat this for vector 3 then vector 4 till..... 1M vectors

Compression = replace values with IDs. For a given vector, chunk 1's actual 16 numbers are replaced with the ID of the nearest centroid in chunk 1's codebook (using Euclidean distance). That ID is a number between 0 and 255, so it fits in a single byte instead of storing 16 float32 values (64 bytes).

Repeat this for all 96 chunks, and the vector shrinks from 6144 bytes (1536 × 4) to 96 bytes—roughly 64× smaller.

6144 bytes → 96 bytes (~64× compression)

Of course, something is lost. Product Quantization is lossy, so distance calculations become approximate rather than exact. The tradeoff is lower memory usage at the cost of some recall. Systems like Qdrant mitigate this by rescoring the top-K candidates using the original full-precision vectors, recovering much of the lost accuracy.

in next blog i would talk about after compressing , how we are going to fetch fast if any querry comes in

i have given my many hours understanding it don't expect you would understand in 1st go read it several times ,you would surely understand in 1/50th time i took to understand after reading this .

DEV Community: yashasvi.shukla