Vector Product Quantization
Large vector collections eat memory quickly. One billion vectors with 768 dimensions need roughly 3 terabytes of storage in raw form. Product quantization, often shortened to PQ, compresses vectors into tiny codes while keeping search useful.
The Zip File Analogy
Think about a photo reduced to a smaller file size. The picture still shows the same scene, but fine details soften. Product quantization does the same for vectors. It shrinks the size dramatically and accepts a small loss in precision.
The Core Idea
PQ splits each long vector into short pieces. It replaces each piece with the ID of the closest "typical piece" from a shared catalog. Instead of storing many decimal numbers, the database stores a short list of catalog IDs.
PRODUCT QUANTIZATION IN FOUR STEPS
Original vector (8 numbers shown for simplicity):
[0.9, 0.1, 0.4, 0.7, 0.2, 0.8, 0.5, 0.3]
Step 1: SPLIT into 4 sub-vectors
[0.9, 0.1] [0.4, 0.7] [0.2, 0.8] [0.5, 0.3]
Step 2: MATCH each piece to the closest catalog entry
piece 1 -> entry #12
piece 2 -> entry #87
piece 3 -> entry #5
piece 4 -> entry #203
Step 3: STORE only the IDs
[12, 87, 5, 203]
Step 4: Discard the original decimals
The Codebook
The catalog of typical pieces is called a codebook. The system builds one codebook per sub-vector position by running a clustering method on training data. Each codebook usually holds 256 entries, so each ID fits inside a single byte.
CODEBOOK FOR SUB-VECTOR POSITION 1
Entry #0 -> [0.10, 0.95]
Entry #1 -> [0.85, 0.20]
Entry #2 -> [0.40, 0.40]
...
Entry #255 -> [0.70, 0.65]
Memory Savings Example
Original vector: 768 numbers x 4 bytes = 3,072 bytes
PQ with 96 sub-vectors, 1 byte each = 96 bytes
Compression = 3,072 / 96 = 32 times smaller
1 billion vectors:
Raw: about 3 TB
With PQ: about 96 GB
A reduction from terabytes to gigabytes changes what hardware you need. A dataset that once required a large cluster might fit inside a single powerful server.
How Search Works With PQ
The query stays uncompressed. The engine precomputes the distance between each query piece and every codebook entry, then stores those results in small lookup tables. Scoring a stored vector becomes a series of table lookups and additions.
Query piece 1 vs codebook 1 -> table of 256 distances
Query piece 2 vs codebook 2 -> table of 256 distances
...
Score for stored code [12, 87, 5, 203]:
table1[12] + table2[87] + table3[5] + table4[203]
Lookups run much faster than full decimal math, so PQ speeds up search while it saves memory.
The Accuracy Trade-Off
| More Sub-Vectors | Fewer Sub-Vectors |
|---|---|
| Higher accuracy | Lower accuracy |
| Larger codes | Smaller codes |
| Less compression | More compression |
Compression discards detail, so recall drops slightly. Teams recover most of the loss with a technique called re-scoring. The engine retrieves the top 100 candidates using compressed codes, then re-ranks them with the original full vectors stored on cheaper disk space.
PQ Combined With Other Indexes
- IVF with PQ: IVF narrows the search to a few clusters, and PQ compresses the vectors inside them. This pairing handles very large datasets.
- HNSW with PQ: The graph guides the search while compressed vectors reduce memory pressure.
Other Compression Options
Scalar quantization offers a simpler alternative. It converts each 4-byte decimal into a 1-byte whole number. The result shrinks memory by 4 times with minimal accuracy loss. Binary quantization goes further and keeps only one bit per dimension.
When to Use PQ
- Your dataset exceeds what your memory budget supports.
- You accept a small recall drop in exchange for lower cost.
- You can run a recall test against a flat index to verify quality.
Product quantization makes huge vector collections affordable. Measure recall before and after compression, and keep the original vectors available for re-scoring when accuracy matters most.