Understanding Embeddings and Vector Search

Why keyword search is not enough

Keyword search matches strings. A user searching for "my payment failed" will miss a help article titled "Resolving declined transactions" unless someone manually maintains synonyms. Embedding search matches concepts, so phrasing differences stop being a wall between a question and its answer. The strongest systems run both and merge results, which is called hybrid search.

Chunking decides your quality ceiling

You rarely embed whole documents. You split them into chunks, and that decision matters more than which embedding model you choose. Chunks that are too large dilute meaning across topics; chunks that are too small lose context.

• Split on natural boundaries such as headings and paragraphs, not fixed character counts.
• Keep a chunk focused on one idea; a few hundred tokens is a common sweet spot.
• Store metadata such as source, section title, and date alongside each vector for filtering.

Practical infrastructure notes

You may not need a dedicated vector database on day one. Postgres with the pgvector extension comfortably handles millions of vectors, which keeps your stack simple and your data in one place. Whatever store you choose, evaluate retrieval quality directly: assemble a set of real queries, mark which chunks should come back, and measure how often they do. Retrieval metrics predict end-to-end quality better than any model benchmark.