Vector Pinecone

Pinecone is a fully managed vector database. You interact with it through an API — no servers to set up, no infrastructure to maintain. This topic walks you through creating an index, inserting vectors, and running your first similarity search.

Core Concepts Before You Code

PINECONE STRUCTURE:
───────────────────────────────────────────────────
Project
  └── Index (one vector space with a fixed dimension)
        └── Namespace (logical partition within an index)
              └── Vectors (id + values + optional metadata)

Example:
  Project: my-app
    Index: products (dimension=1536, metric=cosine)
      Namespace: electronics
        Vector: {id: "prod-001", values: [0.23, 0.87, ...], metadata: {name: "Headphones"}}
        Vector: {id: "prod-002", values: [0.45, 0.12, ...], metadata: {name: "Speaker"}}
      Namespace: clothing
        Vector: {id: "prod-101", values: [...], metadata: {name: "T-Shirt"}}

Step 1: Create a Free Account and Get an API Key

Go to pinecone.io and create a free account. Once logged in, navigate to API Keys in the left sidebar and copy your key. Store it as an environment variable — never paste it directly into your code.

# In your terminal or .env file:
PINECONE_API_KEY=your-api-key-here

Step 2: Install the Pinecone SDK

pip install pinecone

Step 3: Create an Index

An index holds all your vectors. You set the dimension (must match your embedding model) and distance metric at creation time — these cannot change later.

from pinecone import Pinecone, ServerlessSpec

pc = Pinecone(api_key="YOUR_API_KEY")

pc.create_index(
    name="my-first-index",
    dimension=1536,          # Must match your embedding model
    metric="cosine",         # cosine, euclidean, or dotproduct
    spec=ServerlessSpec(
        cloud="aws",
        region="us-east-1"
    )
)

index = pc.Index("my-first-index")

Step 4: Generate Embeddings and Upsert Vectors

First generate embeddings using your chosen model, then upsert them into Pinecone. Upsert means insert-or-update — if the ID already exists, Pinecone updates it.

from openai import OpenAI

client = OpenAI(api_key="YOUR_OPENAI_KEY")

# Your documents
documents = [
    {"id": "doc-1", "text": "Python is a programming language"},
    {"id": "doc-2", "text": "Dogs are loyal pets"},
    {"id": "doc-3", "text": "Machine learning uses algorithms"},
]

# Generate embeddings
vectors_to_upsert = []
for doc in documents:
    response = client.embeddings.create(
        model="text-embedding-3-small",
        input=doc["text"]
    )
    embedding = response.data[0].embedding  # List of 1536 floats

    vectors_to_upsert.append({
        "id": doc["id"],
        "values": embedding,
        "metadata": {"text": doc["text"]}  # Store original text for retrieval
    })

# Upsert in batches (max 100 per batch)
index.upsert(vectors=vectors_to_upsert)
print("Vectors stored successfully")

Step 5: Query the Index

To search, embed your query using the same model, then pass the resulting vector to Pinecone.

# Embed the search query
query_text = "What are good programming languages?"
query_response = client.embeddings.create(
    model="text-embedding-3-small",
    input=query_text
)
query_vector = query_response.data[0].embedding

# Search Pinecone
results = index.query(
    vector=query_vector,
    top_k=3,                    # Return top 3 matches
    include_metadata=True       # Include the stored metadata
)

# Display results
for match in results["matches"]:
    print(f"ID: {match['id']}")
    print(f"Score: {match['score']:.4f}")
    print(f"Text: {match['metadata']['text']}")
    print("---")

Expected Output

ID: doc-1
Score: 0.8921
Text: Python is a programming language
---
ID: doc-3
Score: 0.7234
Text: Machine learning uses algorithms
---
ID: doc-2
Score: 0.2103
Text: Dogs are loyal pets
---

The query about programming languages scores highly against "Python is a programming language" and moderately against machine learning. The unrelated "Dogs are loyal pets" entry scores low — exactly as expected.

Batch Upsert for Large Datasets

Always upsert in batches when dealing with more than a few hundred vectors. Pinecone accepts up to 100 vectors per upsert call.

def batch_upsert(index, vectors, batch_size=100):
    for i in range(0, len(vectors), batch_size):
        batch = vectors[i:i + batch_size]
        index.upsert(vectors=batch)
        print(f"Upserted batch {i // batch_size + 1}")

batch_upsert(index, vectors_to_upsert)

Key Pinecone Limits on the Free Tier

The free tier is generous enough to build and test a complete application. Once you move to production with millions of vectors, upgrade to a paid tier based on the number of vectors and query volume you need.