Vector Reranking
Vector search retrieves candidates quickly, but the top results do not always appear in the best order. Reranking adds a second, more careful pass that reorders the candidates by true relevance. This two-step design raises answer quality in search engines and AI assistants.
The Hiring Analogy
A company receives 500 job applications. A quick screening reduces the pile to 30 promising candidates. Senior managers then interview those 30 in depth and pick the top 5. The quick screening resembles vector retrieval. The interviews resemble reranking.
Two-Stage Search
TWO-STAGE PIPELINE
Million documents in the database
|
v
STAGE 1: Vector retrieval (fast, approximate)
|
v
Top 50 candidates
|
v
STAGE 2: Reranker (slow, precise)
|
v
Top 5 final results
Stage 1 filters a huge collection down to a manageable list. Stage 2 spends more effort on each candidate because the list is small.
Why Vector Search Needs Help
A bi-encoder embedding model converts the query and each document into vectors independently. The model never sees the query and the document together, so it can miss subtle connections. Compressing a long document into one vector also discards detail.
Bi-Encoder vs Cross-Encoder
BI-ENCODER (used in retrieval)
Query -> Model -> vector Q \
compare vectors
Document -> Model -> vector D /
CROSS-ENCODER (used in reranking)
[Query + Document together] -> Model -> relevance score
The model reads both texts at once and judges how well they match.
| Feature | Bi-Encoder | Cross-Encoder |
|---|---|---|
| Speed | Very fast | Slower |
| Precomputed vectors | Yes | No |
| Accuracy | Good | Higher |
| Role | Retrieve candidates | Reorder candidates |
| Scales to millions | Yes | No, only small lists |
A Concrete Example
Query: "Can I return a gift without a receipt?"
After vector retrieval:
1. Store locations and opening hours
2. Return policy for online orders
3. Gift card terms and conditions
4. Returns without proof of purchase
5. Receipt lookup by phone number
After reranking:
1. Returns without proof of purchase
2. Receipt lookup by phone number
3. Return policy for online orders
4. Gift card terms and conditions
5. Store locations and opening hours
The reranker reads the full question and notices that "without a receipt" matches "without proof of purchase". The correct page climbs from fourth place to first place.
Types of Rerankers
- Cross-encoder models: Small, dedicated models trained to score query and document pairs.
- Hosted reranking services: Providers offer reranking through an API call.
- Language model rerankers: A large language model reads candidates and ranks them. This option offers flexibility at higher cost and delay.
Balancing Speed, Cost, and Quality
| Setting | Effect of Increasing It |
|---|---|
| Candidates sent to reranker | Higher recall, slower response, higher cost |
| Final results returned | More context, but possible noise |
| Reranker model size | Better accuracy, slower response |
Practical Guidelines
- Retrieve 25 to 100 candidates, since the reranker cannot rescue documents that retrieval missed.
- Return 3 to 10 results after reranking.
- Measure response time and set a limit your users accept.
- Cache reranking results for repeated queries.
- Compare ranking quality with and without the reranker on real questions.
When Reranking Pays Off
Reranking helps most in AI assistants that feed retrieved text to a language model. Cleaner, more relevant context leads to better answers. Skip reranking for simple, small collections where plain vector search already performs well.
Reranking adds precision on top of speed. Combine a fast retriever with a careful reranker, and your system delivers both.