Search used to mean matching words. You typed a keyword, the database looked for that keyword in an index, and the results were the rows that contained it. That model works well when you know the exact words you are looking for. It fails the moment the question is about meaning instead of spelling. A document about "feline behavior" contains no word "cat" in the title, but it is exactly what a search for "cats" should return. Vector databases exist to close that gap. They search by meaning, and understanding how they do it starts with understanding embeddings.
An embedding is a list of numbers, usually a few hundred to a few thousand long, that represents the meaning of a piece of text, an image, or an audio clip. A model such as a transformer reads the input and produces the vector. The key property is geometric: inputs with similar meaning produce vectors that point in similar directions. The numbers themselves are not human readable, and that is fine, because nobody reads them directly. What matters is the geometry. Two vectors close together mean two inputs that mean similar things. Two vectors far apart mean the opposite.
Similarity between two vectors is usually measured with cosine similarity, which is the cosine of the angle between them. Vectors pointing in the same direction score near 1. Vectors at right angles score near 0. Vectors pointing in opposite directions score near negative 1. The formula is simple, and implementing it in a few lines makes the idea concrete.
import numpy as np def cosine_similarity(a, b): a = np.array(a, dtype=float) b = np.array(b, dtype=float) return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))) cat = [0.9, 0.1, 0.8, 0.2] feline = [0.85, 0.15, 0.75, 0.25] car = [0.1, 0.9, 0.2, 0.8] print(cosine_similarity(cat, feline)) # close to 1, similar meaning print(cosine_similarity(cat, car)) # close to 0, unrelated meaning Other distance measures exist, and the choice is worth a sentence each. Euclidean distance measures straight line distance between vector endpoints, which is sensitive to magnitude as well as direction. Dot product is fast and common when vectors are normalized, because for unit vectors it equals cosine similarity. Most systems normalize embeddings to unit length at insert time, which makes the three measures behave consistently and keeps the math cheap. The important decision is to pick one measure and use it consistently across inserts and queries, because a database that stores vectors measured one way and queries them another way returns nonsense.
The obvious search algorithm is brute force: compare the query vector against every stored vector and keep the top matches. That is exact, and it is also unusable at scale. A million vectors of 768 dimensions means roughly 768 million multiply operations per query, and the cost grows linearly with the collection. This is the problem vector databases solve. They trade a small amount of accuracy for a large amount of speed, using approximate nearest neighbor search, usually shortened to ANN.
The two index families you will meet most often are IVF and HNSW, and they take opposite approaches to the same tradeoff. IVF, inverted file index, clusters the vectors into groups at build time, using k-means or a similar algorithm. At query time it finds the clusters nearest to the query vector and only searches inside those. Searching ten clusters out of a thousand skips ninety nine percent of the work, at the cost of possibly missing a vector that sits just across a cluster boundary. The number of clusters probed is a tuning knob: probe more for accuracy, probe fewer for speed.
HNSW, hierarchical navigable small world, builds a graph instead. Each vector is a node, connected to its nearest neighbors, and the graph is layered so that the top layers are sparse and the bottom layer contains every node. A query starts at the top layer and greedily walks toward the query vector, dropping down a layer each time it gets closer. It is like navigating a map by zooming in: the top layer gets you to the right city, the next layer to the right neighborhood, the bottom layer to the right street. HNSW is usually faster and more accurate than IVF for the same memory, and it is the default in most systems today, including pgvector. The tradeoff is build time and memory, because the graph has to be constructed and held in memory.
The storage question is where the market gets interesting. A dedicated vector database such as Pinecone, Weaviate, or Qdrant is built around ANN search from the ground up. A general purpose database with a vector extension, most famously PostgreSQL with pgvector, adds vector search to a database you already run. The practical difference is less about raw search speed than about what else you need. If your application already lives in Postgres, and your vector collection is in the hundreds of thousands rather than the hundreds of millions, pgvector keeps your data in one place, in one transaction, with one backup strategy. A minimal example shows how little ceremony it takes.
CREATE EXTENSION IF NOT EXISTS vector; CREATE TABLE documents ( id SERIAL PRIMARY KEY, title TEXT NOT NULL, content TEXT NOT NULL, embedding vector(768) ); CREATE INDEX ON documents USING hnsw (embedding vector_cosine_ops); SELECT id, title FROM documents ORDER BY embedding <=> '[0.9, 0.1, 0.8, ...]' LIMIT 5; The <=> operator is cosine distance, and the HNSW index makes that query fast without scanning the whole table. The pattern is the same everywhere: store the vector beside the metadata, filter by the metadata with ordinary SQL, and rank by vector distance. That combination, metadata filtering plus vector ranking, is what real applications need. A pure vector store that cannot filter by tenant id or date forces you to do that filtering in application code, which defeats the purpose.
The most common application today is retrieval augmented generation, or RAG. The pattern is straightforward. Documents are split into chunks, each chunk is embedded and stored, a user question is embedded the same way, the nearest chunks are retrieved, and those chunks are passed to a language model along with the question. The model answers from the retrieved context instead of from memory alone. The vector database is the memory of the system, and its quality determines the quality of the answers. Bad chunking, stale embeddings, or a missing metadata filter will produce confident answers built on the wrong documents, and no amount of prompt engineering fixes that.
Chunking deserves more attention than it usually gets, because it is where most RAG systems quietly fail. A chunk that is too large dilutes the meaning: one paragraph about refunds buried in a page about everything else will match weakly. A chunk that is too small loses context: a sentence that says "it costs fifty dollars" is useless without knowing what "it" is. A common starting point is chunks of a few hundred tokens with a small overlap between neighbors, and then measuring retrieval quality on real questions instead of guessing. Overlap matters because meaning does not respect chunk boundaries, and a sentence split in half embeds badly on both sides.
There are real limits to know before adopting the technology. Embeddings drift: when the embedding model is upgraded, old vectors are not comparable with new ones, and the honest fix is re-embedding the collection, which is a batch job worth planning for. High dimensional indexes are approximate by design, so recall is never one hundred percent, and systems that need exact results should keep a brute force path for small collections. Memory is a cost driver: an HNSW graph for a hundred million vectors needs serious RAM, and that cost is the reason many teams keep vectors in Postgres until the collection outgrows it. None of these are reasons to avoid vector search. They are reasons to size the solution to the collection you actually have.
A good starting rule is simple. If you already run Postgres and your collection is under a few million vectors, start with pgvector. You get transactions, backups, and filtering for free, and you can migrate later if the collection outgrows it. If you need multi tenancy at huge scale, managed infrastructure, or hybrid search that combines keyword and vector ranking natively, a dedicated vector database is the better home. Either way, measure recall on your own data. Two systems with the same index can return different results on different data, and the benchmark that matters is your questions against your documents.
Vector search is not magic, and that is good news. It is geometry applied to meaning, with an index that trades a little accuracy for a lot of speed. Once you see the embedding, the distance metric, and the index as three separate decisions, the whole field becomes approachable, and picking a database becomes a sizing question instead of a leap of faith.
