Vector Databases and How Semantic Search Works

Traditional databases store and retrieve data by exact match or range comparison. A WHERE name = 'foo' query works because equality is well-defined. But meaning is not equal or unequal—it is similar or dissimilar in continuous degrees. Vector databases are purpose-built to query by similarity, making them the storage layer for semantic search, recommendation systems, and retrieval-augmented generation.

What Is a Vector?

An embedding model converts text (or an image, audio clip, or any other input) into a dense numeric vector—an array of typically hundreds or thousands of floating-point numbers. The model is trained such that inputs with similar meaning map to vectors that are close together in the high-dimensional space.

For example, the sentences "the dog ran across the park" and "a canine sprinted through the garden" should produce vectors that are close to each other, while "federal interest rate policy" should produce a vector far from both.

For a deeper grounding in how embeddings work and how to generate them, see Embeddings: The Foundation of AI Search.

Similarity Metrics

Vector databases measure closeness using one of a few distance or similarity functions:

• Cosine similarity: Measures the angle between two vectors, ignoring magnitude. The most common choice for text embeddings normalized to unit length. Ranges from -1 (opposite) to 1 (identical).
• Dot product: Equivalent to cosine similarity for unit-normalized vectors; faster to compute on some hardware.
• Euclidean distance (L2): Measures straight-line distance in vector space. Used when magnitude carries meaning.

For text search, cosine similarity or dot product on L2-normalized embeddings is the standard.

Approximate Nearest Neighbor Search

A naive similarity search scans every stored vector and computes distance to the query vector. At millions of vectors, this is too slow for interactive applications. Vector databases use Approximate Nearest Neighbor (ANN) algorithms to trade a small amount of accuracy for orders-of-magnitude speed.

The dominant ANN algorithms:

• HNSW (Hierarchical Navigable Small World): Builds a layered graph structure. Excellent recall, fast queries, high memory usage. Used by Qdrant, Weaviate, and pgvector.
• IVF (Inverted File Index): Clusters vectors into buckets, searches only nearby buckets. Lower memory than HNSW, slightly lower recall.
• FAISS: A library from Meta that implements many ANN algorithms; used as the engine inside several hosted services.

In practice you tune two parameters: ef_construction (quality during index build) and ef_search (quality vs. speed at query time).

Metadata Filtering

Pure vector search retrieves the most semantically similar items globally. Production applications typically need filtered search: "find the most similar product description, but only within category X and published after 2024."

Vector databases handle this with metadata filters that run alongside the ANN search. The implementation matters:

• Pre-filter: Apply the metadata condition first, then run ANN on the filtered subset. Accurate but slow when the filtered set is small relative to the index.
• Post-filter: Run ANN globally, then discard results that fail the filter. Fast but can return fewer than k results if many match the filter.
• Filtered HNSW (Qdrant, Weaviate): The ANN traversal respects the filter during graph traversal, balancing accuracy and speed.

Hybrid Search

Semantic search misses exact matches. If a user searches for a product SKU or a specific error code, a semantic model may not retrieve the right document because the embedding encodes meaning, not spelling. Hybrid search combines vector similarity with a keyword (BM25) index and blends their scores.

ts
// Pseudocode — most managed vector DBs expose a hybrid search parameter
const results = await vectorDB.search({
  query: userQuery,
  vector: queryEmbedding,
  hybridAlpha: 0.5, // 0 = pure keyword, 1 = pure vector
  topK: 10,
});

Hybrid search is a significant quality improvement for RAG pipelines. See Retrieval-Augmented Generation (RAG), Explained for how retrieval quality affects the full pipeline.

Choosing a Vector Store

Managed services

• Pinecone: Fully managed, serverless tier available, strong ecosystem. Good default for teams that want zero ops.
• Weaviate Cloud: Managed Weaviate with hybrid search, modules, and multi-tenancy built in.
• Qdrant Cloud: Managed Qdrant. Strong filtered ANN, good performance benchmarks.

Self-hosted / open source

• Qdrant: Fast, written in Rust, excellent payload filtering, easy Docker deployment.
• Weaviate: Feature-rich, supports multiple modalities, built-in vectorization.
• Chroma: Simple Python-first store, good for prototyping.
• pgvector: A PostgreSQL extension. The right choice if your data is already in Postgres and scale is moderate. Avoids an additional service entirely.

When to use pgvector

For applications with fewer than a few million vectors and existing PostgreSQL infrastructure, pgvector is often the pragmatic choice. You get HNSW and IVF indexes, metadata filtering via standard SQL WHERE clauses, and transactional consistency for free.

ts
// pgvector query via Prisma raw SQL
const results = await prisma.$queryRaw<{ id: string; content: string }[]>`
  SELECT id, content
  FROM documents
  WHERE category = 'support'
  ORDER BY embedding <=> ${queryVector}::vector
  LIMIT 5
`;

The <=> operator is cosine distance. <#> is inner product (dot product negated), <-> is L2.

Indexing Strategy

How you structure your vector index affects retrieval quality substantially:

• Store the chunk text, document source, section title, and creation date as metadata alongside each vector.
• Use one index per distinct content type (e.g., blog posts and code snippets have different embedding distributions).
• Re-embed when you change embedding models—vectors from different models are not comparable.
• Namespace or partition indexes by tenant in multi-tenant applications to enforce data isolation.

Connecting to a Generation Model

Once retrieval is working, passing the top-k chunks to a generation model is straightforward:

ts
import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic();

async function answerWithContext(
  userQuery: string,
  chunks: string[]
): Promise<string> {
  const context = chunks.join("\n\n");
  const message = await client.messages.create({
    model: "claude-opus-4-8",
    max_tokens: 1024,
    system: "Answer using only the provided context. Be concise.",
    messages: [
      {
        role: "user",
        content: `Context:\n${context}\n\nQuestion: ${userQuery}`,
      },
    ],
  });
  const block = message.content[0];
  if (block.type !== "text") throw new Error("Unexpected content");
  return block.text;
}

For the full picture of building a Claude-powered search feature, see Building Your First AI Feature with the Claude API and Adding AI-Powered Search to Your Blog with Embeddings.

Conclusion

Vector databases bring semantic understanding to storage and retrieval. The core concept—embed once, query by similarity—is simple, but the engineering details (ANN algorithms, metadata filtering, hybrid search, index structure) determine whether a search feature feels precise or frustratingly random. Start with pgvector if you are already on PostgreSQL, graduate to a dedicated vector store as scale demands, and invest in retrieval quality before tuning the generation layer. See the projects page for applications built with these patterns.