Vector Troubleshooting

Vector systems fail in quiet ways. Search still returns results, but the results look wrong, arrive slowly, or miss documents you know exist. A clear diagnostic routine saves hours of guessing.

The Doctor Analogy

A doctor asks about symptoms, runs tests, and narrows down the cause before prescribing a treatment. Apply the same discipline. Describe the symptom precisely, test one cause at a time, and change one setting at a time.

Symptom Guide

Symptom	Likely Causes	First Check
Results look random or unrelated	Different embedding models for documents and queries, wrong distance metric	Confirm one model name and version on both sides
Correct document never appears	Poor chunking, content missing, restrictive filter	Search without filters and look up the document by ID
Queries run slowly	No index, high search width, memory pressure	Check the index type and memory use
Recall dropped after a data load	Index needs a rebuild, many deletions	Compare against a flat index test
Memory keeps climbing	Growing data, replicas, large index settings	Calculate expected memory and compare
New documents stay invisible	Failed inserts, delayed indexing	Read insert responses and wait for indexing to finish
Filtered searches return too few results	Filter too strict, missing metadata	Count how many vectors match the filter alone

A Step-by-Step Diagnosis

DIAGNOSIS FLOW

  Problem reported
        |
        v
  1. Reproduce it with one exact query
        |
        v
  2. Is the target document in the database?
     No  -> fix ingestion
     Yes -> continue
        |
        v
  3. Does a flat (exact) search find it?
     No  -> embedding or chunking problem
     Yes -> index or filter problem
        |
        v
  4. Does the result appear with filters removed?
     Yes -> filter or metadata problem
     No  -> index settings problem

Problem 1: Mismatched Embeddings

This mistake produces the strangest results. Documents embedded with one model and queries embedded with another live in different spaces. Check the vector dimension first, since a mismatch there usually raises an error. Matching dimensions can still hide different models, so verify the model name in your configuration and in your stored metadata.

Problem 2: Bad Chunks

Chunks that cut sentences in half, mix topics, or contain only page headers produce weak vectors. Print the actual text of the top results. Reading the chunks often reveals the issue within minutes.

CHECKLIST FOR CHUNK QUALITY

  [ ] Each chunk covers one idea
  [ ] Chunks start and end at sentence boundaries
  [ ] No repeated headers or footers
  [ ] Section titles included for context
  [ ] Chunk size fits the embedding model limit

Problem 3: Wrong Distance Metric

Each embedding model recommends a metric, usually cosine similarity. A collection created with Euclidean distance when the model expects cosine can lower result quality. Read the model documentation and match the setting.

Problem 4: Slow Queries

Confirm that an approximate index exists on the collection.
Lower the search width and measure the change in speed and recall.
Check whether vectors spill from memory to disk.
Review result size, because returning large payloads slows responses.
Look at traffic spikes and add replicas if needed.

Problem 5: Degraded Index After Many Changes

Graph indexes can lose quality after heavy deletion and updates. Compare recall with a flat search on a sample of queries. A noticeable gap suggests rebuilding or optimizing the index.

Keep a Debugging Log

DEBUG LOG ENTRY

  Date:        2026-11-13
  Symptom:     Refund questions return shipping documents
  Hypothesis:  Chunks too large, topics mixed
  Change:      Chunk size 800 -> 300 words
  Result:      Recall@5 improved from 62% to 81%
  Decision:    Keep change

Prevent Problems Early

Run a fixed test set after every deployment.
Record the model name, chunk settings, and index settings with each collection.
Validate vector dimensions and metadata at insert time.
Keep a small flat index for comparison tests.

Calm, ordered troubleshooting beats random tweaking. Reproduce the issue, isolate the layer, change one thing, and measure again.

Previous lesson

Back to course

Next lesson