Memory is what the agent learned; retrieval is how it reaches knowledge it was never trained on — your documents, your database, the live web. Retrieval-augmented generation grounds the model in real, current, citable information, and is the single most effective defense against hallucination. But naïve RAG — embed everything, fetch the nearest few chunks, stuff them in — disappoints often enough that it is worth understanding the pipeline from first principles.

11.1 Why retrieval, and the pipeline that delivers it

A model's parametric knowledge is frozen at training time, lossy, and unattributable. Retrieval fixes all three: it injects fresh, specific, sourceable facts into the context at query time (Ch. 7). The classic pipeline has two phases. Offline (indexing): documents are chunked, each chunk embedded into a vector, and the vectors stored in an index. Online (query): the query is embedded, the most similar chunks retrieved, optionally reranked, and the best passed into the prompt as grounding.

11.2 Where naïve RAG fails — and the fixes

• Chunking — fixed-size splits sever ideas mid-thought. Chunk on semantic/structural boundaries and overlap slightly; size to your embedding model and content.
• Hybrid search — pure vector search misses exact terms (names, codes, error strings). Combine dense vectors with sparse keyword (BM25) and fuse the rankings.
• Reranking — first-stage retrieval favors recall; a cross-encoder reranker re-scores the top candidates for precision, sharply improving what actually lands in context.
• Query transformation — rewrite, expand, or decompose the user's query before retrieval so it matches how the corpus is written.
• Metadata filtering — combine semantic search with structured filters (tenant, date, permission) so retrieval is correct and authorized.

11.3 When vectors aren't enough: GraphRAG

Vector retrieval finds passages similar to a query, but struggles with questions that require connecting facts scattered across many documents, or reasoning over relationships — "how is A connected to C through B?", or "summarize the themes across this whole corpus." GraphRAG addresses this by extracting entities and their relationships into a knowledge graph, then retrieving connected subgraphs (and community summaries) rather than isolated chunks. This gives the model structured, multi-hop context and a global view a flat vector store cannot provide. Tools such as Cognee and Microsoft's GraphRAG build and query these graphs; the cost is a heavier indexing pipeline (entity/relation extraction) and a graph store alongside your vectors. Use it when relationships and cross-document synthesis matter; stay with vector RAG when point lookups suffice.

11.4 Choosing a vector store

The index is the operational heart of retrieval. The pivotal choice is often whether to add a dedicated vector database at all, or to use the vector extension of the database you already run.