Vector Cost Optimization
Vector databases consume memory, storage, and compute. Costs stay small during a demo and grow quickly in production. Smart design choices reduce the bill without hurting search quality.
Where the Money Goes
COST SOURCES
+------------------------------------------+
| 1. Memory (vectors + index in RAM) | often the largest
| 2. Storage (disk, backups, replicas) |
| 3. Compute (queries, index builds) |
| 4. Embedding (model calls for new data) |
| 5. Network (data transfer) |
+------------------------------------------+
Estimate Memory First
Raw vector memory = vectors x dimensions x 4 bytes
5,000,000 vectors x 768 dimensions x 4 bytes
= 15,360,000,000 bytes
= about 15.4 GB
Graph indexes such as HNSW add extra overhead on top.
Replicas multiply the total (2 replicas = roughly 2x memory).
Run this calculation before you choose a server size or a managed plan. Surprises appear when teams skip it.
Cost Levers
| Lever | How It Saves Money | Trade-Off |
|---|---|---|
| Smaller embedding dimensions | Cuts memory and compute directly | Slight loss of detail |
| Scalar quantization | About 4 times less memory | Small recall drop |
| Product quantization | Up to 32 times less memory | Larger recall drop, needs re-scoring |
| Disk-based indexes | Moves data from RAM to cheaper disk | Slower queries |
| Fewer replicas | Lowers storage and memory | Less availability |
| Result caching | Skips repeated searches | Stale results if data changes |
Reduce Dimensions
Some embedding models support shorter output vectors with small quality loss. Dropping from 1536 to 768 dimensions halves memory use. Test recall at several sizes and pick the smallest size that meets your quality goal.
1536 dimensions -> 100% memory
768 dimensions -> 50% memory
384 dimensions -> 25% memory
Apply Quantization
Quantization compresses stored vectors. Scalar quantization is the easiest first step because it needs little configuration and keeps accuracy high. Move to product quantization for very large datasets, and use re-scoring with original vectors to protect quality.
Tier Your Data
Not every vector deserves premium storage. Fresh and popular content needs fast access. Old, rarely searched content can live on slower storage.
DATA TIERS
HOT (last 30 days, frequent queries)
-> fast memory, full precision
WARM (30 days to 1 year)
-> compressed vectors, disk-backed index
COLD (older than 1 year)
-> archive storage, rebuild on demand
Control Embedding Costs
- Embed once: Store vectors and never recompute unchanged content.
- Batch requests: Send many texts per call instead of one at a time.
- Detect changes: Save a hash of each document and re-embed only when the hash changes.
- Chunk wisely: Avoid excessive overlap that multiplies vector count.
Remove Waste
- Delete duplicate chunks and near-identical documents.
- Remove vectors for content that no longer exists.
- Drop unused test collections left from experiments.
- Store only the metadata fields you actually filter on or display.
Right-Size Your Infrastructure
- Monitor memory and processor usage for a few weeks before you commit to a size.
- Scale down during quiet hours when your platform supports it.
- Compare managed service pricing against self-hosting, including the engineering time that self-hosting requires.
- Choose a database that matches your scale, since a small project rarely needs a large distributed cluster.
Measure Before and After
Every optimization changes quality, speed, or both. Keep a fixed test set of real queries. Record recall, response time, and monthly cost before each change, then compare the numbers afterward. A change that saves money but drops recall below your target fails the test.
Cost control comes from small, measured decisions. Estimate memory early, compress with care, tier your data, and verify quality after every change.