Vector Chatbot Memory
Language models forget everything outside the current conversation window. A chatbot that served you yesterday starts today with a blank mind. Vector memory fixes this problem. The chatbot saves important facts as vectors and retrieves the relevant ones when a new message arrives.
The Notebook Analogy
Picture an assistant who keeps a notebook. Before every meeting, the assistant flips to the pages that relate to the topic and skips the rest. Vector memory works the same way. It never reads every past message. It pulls only the notes that match the current moment.
Short-Term and Long-Term Memory
| Memory Type | Where It Lives | What It Holds |
|---|---|---|
| Short-term | Inside the model prompt | The last several messages of the current chat |
| Long-term | Vector database | Facts, preferences, and summaries from many chats |
How Memory Retrieval Works
MEMORY LOOP
User: "Plan my weekend meals"
|
v
Embed the message into a vector
|
v
Search memory database
|
v
Retrieved memories:
- "User follows a vegetarian diet"
- "User dislikes mushrooms"
- "User cooks for two people"
|
v
Add memories to the prompt
|
v
Chatbot answers with personal context
|
v
Save new useful facts back to memory
What to Store
Storing every sentence floods the database with noise. Store information that stays useful across sessions.
- Preferences: Favorite formats, tone, and language.
- Stable facts: Job role, project names, and tools in use.
- Decisions: Choices the user made and the reasons behind them.
- Summaries: Short recaps of long conversations.
Memory Record Structure
MEMORY RECORD
id: "mem_0042"
text: "User prefers short answers with examples"
vector: [0.14, 0.62, -0.33, ...]
metadata: {
"user_id": "u_981",
"created": "2026-03-04",
"type": "preference",
"importance": 4
}
The user_id field keeps each person's memories separate. Always filter by user ID during search so one person never sees another person's data.
Smarter Ranking With Recency and Importance
Similarity alone can surface outdated memories. Blend three signals for better results.
FINAL SCORE = (similarity x 0.6) + (recency x 0.2) + (importance x 0.2)
Memory A: very similar, 2 years old, low importance
Memory B: fairly similar, 2 days old, high importance
Memory B often wins.
Adjust the weights after you observe real conversations.
Summarizing Old Conversations
Long chat histories grow expensive. A summarization step compresses older messages into short notes.
- Wait until a conversation ends or reaches a length limit.
- Ask a language model to write a short summary of key facts.
- Embed the summary and store it with metadata.
- Archive or delete the raw messages according to your retention policy.
Handling Changing Facts
People change jobs, cities, and preferences. When new information contradicts an old memory, update or replace the old record instead of keeping both. Keep a timestamp so the system can prefer the newest fact.
Privacy and Control
- Consent: Tell users what the chatbot remembers.
- Visibility: Let users view their stored memories.
- Deletion: Offer a clear way to remove a single memory or all memories.
- Sensitive data: Avoid storing passwords, payment details, or health records unless your system meets the legal requirements.
- Retention: Delete memories after a defined period.
Common Mistakes
- Saving every message instead of meaningful facts.
- Forgetting to filter by user, which risks data leaks.
- Retrieving too many memories and crowding the prompt.
- Never updating outdated facts.
Vector memory turns a forgetful chatbot into a helpful long-term assistant. Store selectively, rank with care, and give users full control over their data.