oriolrius.me Production RAGReference guide
Reference guide · AI engineering

Production RAG

From Scratch to Scalable Systems. Retrieval-augmented generation that survives contact with a real corpus, real users, a real bill and a real adversary — stage by stage, with the code and the numbers.

16
parts
379
sections
331
code listings
134k
words

Read it front to back once to build the mental model, then use the table of contents as a lookup index. Every part ends with a checklist you can act on.

The shape of the problem. A RAG system is two pipelines that meet at a vector store. The indexing pipeline runs offline — load, split, embed, store. The retrieval pipeline runs per query — embed the question, search, rank, stuff a prompt, generate. Nearly every production failure traces to a specific stage of one of those two pipelines, so the guide is organised the same way: build the pipelines, then make each stage observable, cheap, fast and safe.

The mental model

Two pipelines that meet at a vector store

Every part of the guide owns one or more boxes of this map. Read it once; then use it as the index to everything else.

OBSERVABILITY(traces · metrics · evals · JSON logs) — wraps everything, changes nothingPart 7PHASE A — INDEXoffline · once per document · amortizedPHASE B — QUERYonline · once per request · recurringLoadPart 2SplitPart 3[Contextualize]Part 14EmbedPart 4Store + IndexParts 4, 8, 10GateP12CacheP9 · P11Embed queryP4RetrieveP4 · P6 · P16Fuse / FilterP6Compress / RerankP6Assemble promptP1 · P5 · P9GenerateP5 · P11Validate outputP11Return + citationsP1 · P6Governed by three budgets, always in tension:QUALITY · LATENCY · MONEY
The master map: both pipeline phases, the Parts covering each step, and observability — from Part 1.
The guide

Sixteen parts, four tracks

Read it front to back once to build the mental model, then use it as a lookup index. Every part ends with a checklist you can act on.

Parts 1–5

Foundations

Build the two pipelines

Parts 6–9

Quality & operations

Make it right, visible and affordable

Parts 10–13

Shipping to production

Host it, serve it, defend it, test it

Parts 14–16

Advanced & frontier

Change what retrieval means

Listen

Audio summaries

Short spoken overviews of the parts that benefit most from being explained out loud. Generated with NotebookLM from the text of each part.

  1. 01Why RAG Breaks in Production, and How the Pieces Fit5:51
  2. 03Chunking: The Single Biggest Quality Lever5:27
  3. 04Embeddings, Vector Stores and Reading Similarity Scores7:02
  4. 05Your First Real RAG System, and Debugging It When It Lies5:21
  5. 06Retrieval Quality: Hybrid Search, Reranking and Citations5:54
  6. 07Observability: Traces, Metrics and Evals6:20
  7. 08Scaling: Index Tuning, HNSW and When to Shard4:33
  8. 09The Cost and Latency Budget6:14
  9. 11The Production API: FastAPI and the Request Path5:40
  10. 12The Security Layer: Prompt Injection and PII6:13
  11. 13Testing the Security Layer, and the Production Checklist5:55
  12. 14Advanced Retrieval: Long Context, Contextual Retrieval and Late Chunking4:36
  13. 15Agentic RAG: Self-Correcting Retrieval with LangGraph6:02
  14. 16The Frontier: GraphRAG, Multimodal RAG and Where This Is Going5:13