Retrieval Pipelines

A retrieval pipeline turns a changing corpus into ranked evidence for a model. In RAG, it spans ingestion, chunking, embedding, lexical indexing, metadata filters, query rewriting, hybrid retrieval, reranking, context packing, and citation validation.

LangChain is often used to assemble these pieces because it provides integrations for loaders, embedding models, vector stores, retrievers, and model-call composition. The retrieval contract still belongs to the application: source versions, permissions, and trace fields must be explicit even when a framework supplies the connectors.

Offline and online contracts

The pipeline has two contracts. The offline contract builds searchable records: source document, chunk boundaries, text hash, permissions, embedding model, index version, and deletion state. The online contract turns a user request into evidence: normalized query, filters, first-stage candidates, scores, reranker output, selected chunks, and the final context handed to the model.

Those logs make failures diagnosable. A bad answer can come from missing ingestion, stale permissions, poor chunk boundaries, a query rewrite that removed the key term, a dense index that missed an exact identifier, a reranker that preferred fluent but irrelevant text, or context packing that dropped the decisive passage.

flowchart TD
  Corpus[Source corpus] --> Chunk[Chunking]
  Chunk --> Embed[Embedding and lexical indexing]
  Embed --> Index[Searchable index with permissions and versions]
  Query[User query] --> Rewrite[Query rewrite and filters]
  Rewrite --> Retrieve[Hybrid retrieval over the index]
  Index --> Retrieve
  Retrieve --> Rerank[Reranking]
  Rerank --> Pack[Context packing and citation validation]
  Pack --> Result[Retrieval result: selected context and citation IDs]
  Result -. handoff .-> Model[Downstream generator]

The solid path ends at the retrieval result: selected chunks, ordering, citation identifiers, and enough metadata to audit why those chunks were chosen. A RAG application usually hands that artifact to a generator, but generation is a downstream consumer of retrieval rather than the retrieval result itself.

A retrieval trace

{
  "trace_id": "ret-2026-07-12-1842",
  "query": "refund approval threshold for enterprise accounts",
  "filters": { "acl": "support", "policy_version": "2026-07" },
  "candidate_sets": {
    "bm25": [{ "chunk_id": "refunds-007", "score": 13.8 }],
    "dense": [{ "chunk_id": "refunds-011", "score": 0.78 }]
  },
  "reranked": [{ "chunk_id": "refunds-007", "rank": 1, "score": 0.92 }],
  "selected_for_context": ["refunds-007"]
}

This is a retrieval artifact, not a generation artifact. It should be evaluated against retrieval labels or hard negatives before judging final answer quality with rag evaluation.

Operational contracts

Contract	Fields to record
Ingestion	source ID, source version, text hash, parser version, deletion state.
Chunking	chunk ID, heading path, token count, neighboring chunks, permissions.
Indexing	embedding model, index version, lexical analyzer, build time.
Querying	original query, rewritten query, filters, user/tenant scope.
Ranking	candidate lists, scores, fusion method, reranker version.
Handoff	selected chunks, dropped high-score chunks, context order, citation IDs.

Without these fields, teams cannot tell whether a bad answer came from the model or from stale retrieval state.

Realistic failure trace

A policy answer cites refunds-007, but the user says the threshold is outdated. The retrieval trace shows policy_version=2025-12 even though the UI selected July 2026. The fix is not a different language model; it is filter propagation and index freshness. Retrieval traces make that kind of root cause visible.

Caveats

Do not tune retrieval only through final answer fluency. A model can answer from prior knowledge even when retrieval failed, or produce a plausible answer from irrelevant chunks. Keep retrieval-specific metrics such as recall@k, nDCG, filter correctness, and citation support separate from answer style.

References

• Lewis et al., 2020, Retrieval-Augmented Generation
• Karpukhin et al., 2020, Dense Passage Retrieval
• Faiss documentation

Nav

Section — Generative AI and Agentic Systems

← Vector Databases Hybrid Retrieval →

Learning path — Generative AI systems

← Foundation Models RAG →