RAG Embeddings
An embedding is a list of numbers that represents the meaning of a piece of text. This idea sits underneath every vector search operation covered in the previous topic, and it deserves a closer look on its own.
Turning Words Into Numbers
A computer cannot compare "meaning" directly the way a human brain does. It needs numbers to do math. An embedding model reads a sentence and outputs something like a row of three hundred numbers. Sentences with similar meaning produce number rows that sit close together, even when the actual words used are completely different.
A Flavor Profile Analogy
Picture rating a dish on five scales: sweetness, spiciness, saltiness, sourness, and bitterness. Two dishes with nearly identical scores probably taste similar, even if their names differ completely. An embedding works the same way, except it scores meaning across hundreds of scales instead of five.
From Sentence to Number Row
| Sentence | Simplified Score Example |
|---|---|
| "The cat sat on the mat" | High score on "animals," low score on "finance" |
| "The dog lay on the rug" | High score on "animals," low score on "finance," close to the cat sentence |
| "The bank raised interest rates" | High score on "finance," far from both animal sentences |
How Distance Becomes a Search Result
Once every chunk of text has a number row, finding an answer becomes a distance problem. The question gets its own number row. The system checks which stored chunks sit closest to that row, then returns those chunks as the search result. Close numbers mean close meaning, and close meaning means a strong match.
Measuring Distance Between Meanings
Where Embeddings Come From
A specialized embedding model creates these number rows. This model differs from the main chat model. Its only job is converting text into meaningful numbers, and it runs quickly because the task stays narrow and focused compared to the broader work a full chat model performs.
Common Beginner Confusion
| Myth | Reality |
|---|---|
| Embeddings store the exact words | Embeddings store meaning, not exact text |
| One embedding model fits every language equally well | Some models specialize in specific languages or domains |
| Bigger number rows always mean better results | Quality depends on training data, not just row size |
A Practical Example
A cooking app stores thousands of recipes as embeddings. A user searches "quick dinner with chicken." No recipe title contains that exact phrase, yet the search still surfaces a fast chicken stir-fry recipe, because its overall meaning sits close to the question in the embedding space. The user gets a useful match without needing to guess the exact recipe title.