Vector Semantic Search

Semantic search finds content based on meaning instead of exact keyword matches. This topic walks you through building a complete semantic search system from scratch, using the concepts covered in earlier topics.

What Makes Search "Semantic"

KEYWORD SEARCH (traditional)              SEMANTIC SEARCH (vector-based)
─────────────────────────────────         ─────────────────────────────────
Query: "affordable laptop"                Query: "affordable laptop"
                                           
Searches for exact words:                 Searches for meaning:
"affordable" AND "laptop"                 budget computers, cheap notebooks,
                                           low-cost PCs, value laptops
                                           
Misses: "budget notebook computer"        Finds: "budget notebook computer"
(no matching keywords)                    (similar meaning, different words)

The Full Architecture

SEMANTIC SEARCH SYSTEM ARCHITECTURE
─────────────────────────────────────────────────────────────

INDEXING PHASE (done once, repeated when data changes)
┌──────────┐    ┌──────────────┐    ┌──────────────┐
│ Raw Data │ →  │ Embedding    │ →  │ Vector       │
│ (docs)   │    │ Model        │    │ Database     │
└──────────┘    └──────────────┘    └──────────────┘

QUERY PHASE (happens on every user search)
┌──────────┐    ┌──────────────┐    ┌──────────────┐    ┌──────────┐
│ User     │ →  │ Embedding    │ →  │ Vector       │ →  │ Ranked   │
│ Query    │    │ Model (same!)│    │ Database     │    │ Results  │
└──────────┘    └──────────────┘    └──────────────┘    └──────────┘

Step 1: Prepare Your Documents

Real-world documents are often too long to embed as a single vector effectively. Break long documents into smaller chunks before embedding.

def chunk_text(text, chunk_size=500, overlap=50):
    """Split text into overlapping chunks"""
    words = text.split()
    chunks = []
    start = 0

    while start < len(words):
        end = start + chunk_size
        chunk = " ".join(words[start:end])
        chunks.append(chunk)
        start += chunk_size - overlap   # overlap preserves context across chunks

    return chunks

document = "Your long article text goes here..."
chunks = chunk_text(document)
print(f"Created {len(chunks)} chunks")

The overlap between chunks prevents important context from being cut off at chunk boundaries. A sentence split across two chunks still retains some surrounding context in each.

Step 2: Generate Embeddings for All Chunks

from openai import OpenAI

client = OpenAI(api_key="YOUR_KEY")

def get_embedding(text):
    response = client.embeddings.create(
        model="text-embedding-3-small",
        input=text
    )
    return response.data[0].embedding

embedded_chunks = []
for i, chunk in enumerate(chunks):
    vector = get_embedding(chunk)
    embedded_chunks.append({
        "id": f"chunk-{i}",
        "values": vector,
        "metadata": {"text": chunk, "source": "article-1"}
    })

Step 3: Store Vectors in the Database

from pinecone import Pinecone

pc = Pinecone(api_key="YOUR_PINECONE_KEY")
index = pc.Index("semantic-search-demo")

index.upsert(vectors=embedded_chunks)
print("All chunks stored")

Step 4: Build the Search Function

def semantic_search(query, top_k=5):
    # Convert query to a vector using the SAME embedding model
    query_vector = get_embedding(query)

    # Search the vector database
    results = index.query(
        vector=query_vector,
        top_k=top_k,
        include_metadata=True
    )

    # Format results
    search_results = []
    for match in results["matches"]:
        search_results.append({
            "text": match["metadata"]["text"],
            "score": round(match["score"], 4),
            "source": match["metadata"]["source"]
        })

    return search_results

Step 5: Run a Search

results = semantic_search("How does artificial intelligence work?")

for r in results:
    print(f"Score: {r['score']} | Source: {r['source']}")
    print(f"Text: {r['text'][:150]}...")
    print("---")

Improving Search Quality

Add a Relevance Threshold

Filter out results below a minimum similarity score to avoid showing irrelevant matches.

def semantic_search_filtered(query, top_k=5, min_score=0.7):
    results = semantic_search(query, top_k)
    return [r for r in results if r["score"] >= min_score]

Combine with Metadata Filters

Narrow results to a specific category, date range, or author before ranking by similarity.

results = index.query(
    vector=query_vector,
    top_k=5,
    filter={"category": {"$eq": "tutorials"}},
    include_metadata=True
)

Re-rank Top Results

For maximum accuracy, retrieve more candidates than you need (such as 20), then use a specialized re-ranking model to reorder them before showing the final top 5 to the user. Re-ranking models are slower but more precise than vector similarity alone.

Common Pitfalls to Avoid

A well-built semantic search system handles typos, synonyms, and rephrased queries naturally — capabilities that keyword search systems struggle with by design.