3.5 — RAG (Retrieval-Augmented Generation)

1. What is it?

RAG is an architecture pattern where, before generating an answer, the system retrieves relevant information from an external knowledge source (documents, databases) and includes it in the model's prompt, so the model's answer is grounded in that retrieved content rather than relying solely on what it learned during training. It is the single most common pattern in enterprise AI engineering, and it is also the single most over-applied one — reaching for RAG by default, without asking whether it's the right tool, is one of the most common mistakes an AI FDE will see and needs to actively push back against (Part 12).

2. Why does it exist?

Part 2.6 established two structural limitations of an LLM used alone: its knowledge is frozen at training time (a "knowledge cutoff"), and it has no way to distinguish "I actually know this precisely" from "this is a plausible-sounding guess" — both leading directly to hallucination on questions requiring specific, current, or proprietary facts. RAG exists to solve exactly this: instead of asking the model to recall facts from its training (unreliable, ungrounded, unverifiable), you hand it the actual relevant facts as part of the prompt and ask it to answer using those facts — a fundamentally more reliable and verifiable pattern, because you can trace every claim in the answer back to a specific retrieved source.

3. What problem does it solve?

It solves "how do I get an LLM to answer questions accurately using a specific, private, frequently-changing body of knowledge (a company's documents, policies, product data) that the model was never trained on and that changes too often to keep fine-tuning it in." Crucially, RAG also solves this without requiring any model training at all — you can add, update, or remove documents from the retrieval corpus instantly, and the model's effective "knowledge" updates immediately, with zero retraining cost or latency. This is precisely why RAG, not fine-tuning, is the default answer whenever the actual need is "the model should know about our current documents/policies/data" (Part 2.2 established this same distinction from the fine-tuning side).

4. How does it work internally?

The full pipeline, in two phases

RAG has two genuinely distinct phases that happen at very different times, and conflating them is a common source of confusion:

Indexing phase (happens ahead of time, whenever documents are added/updated):

Raw documents → Parsing/extraction → Chunking → Embedding (Part 2.5) → Store in vector DB (Part 3.4)

Query phase (happens live, for every user question):

User query → Embed the query (same model as indexing!) → Retrieve top-k similar chunks
    → Assemble prompt (retrieved chunks + query + instructions) → LLM generates answer

Layout-aware document parsing — the pipeline stage before chunking that most tutorials skip entirely

Before any chunking strategy can be applied at all, raw documents (especially PDFs, scanned forms, and exported reports — the actual majority of real enterprise document corpora) have to be turned into clean, structured text in the first place. Naive text extraction (pulling raw text out of a PDF byte stream in whatever order it's stored) routinely destroys the document's actual structure: a table's cells get flattened into a single run of text with no indication of which value belonged to which row or column, a multi-column layout gets reflowed left-to-right across columns instead of down each column first (interleaving two unrelated columns' sentences into nonsense), and running headers/footers get inserted as noise in the middle of the extracted body text on every single page.

Layout-aware parsing is a distinct pipeline stage, upstream of chunking, that uses the document's actual visual/structural layout — not just its raw text stream — to extract content correctly: detecting table boundaries and preserving row/column relationships (e.g., extracting a table as structured rows or Markdown-table text rather than a flattened word-salad), detecting and stripping repeated headers/footers, and correctly reordering multi-column text into a single coherent reading order before any chunking happens. Concretely, this is one of the most common real enterprise RAG failure modes in practice: a financial report's table has a header row ("Q1 | Q2 | Q3 | Q4") and a data row ("$4.2M | $3.9M | $5.1M | $6.0M") that naive extraction flattens into one undifferentiated text blob — and once that blob is chunked and embedded, there's no reliable way for the model to know that "$5.1M" was specifically the Q3 figure, so it can confidently cite the wrong quarter's number as an answer, with nothing anywhere in the pipeline to flag the mistake. An unstripped header/footer ("Confidential — Internal Use Only — Page 14") repeated on every page can itself get embedded dozens of times, subtly diluting every chunk's embedding with irrelevant boilerplate.

Getting this right typically means using a layout-aware extraction tool (rather than a bare text-extraction library) that outputs structure-preserving representations — Markdown or HTML tables instead of flattened text, explicit reading-order reconstruction for multi-column layouts — before any of the chunking strategies below are applied at all. Chunking quality can only ever be as good as the structural fidelity of what it's chunking; even the best chunking strategy can't recover a fact that layout-unaware extraction already destroyed.

Chunking — the step most tutorials skip, and where most real RAG quality problems live

Documents are rarely embedded whole — a single embedding vector for a 50-page policy manual would be a hopelessly diluted average of dozens of unrelated topics, useless for finding the one relevant paragraph. So documents are split ("chunked") into smaller pieces before embedding, each getting its own vector. This sounds simple; getting it right in practice is genuinely one of the highest-leverage, most underrated skills in RAG engineering.

• Fixed-size chunking (e.g., every 500 tokens) is the simplest approach, but blindly cuts through sentences, paragraphs, and even mid-word, potentially splitting a critical fact across two chunks such that neither chunk alone contains the complete answer.
• Sentence/paragraph-aware chunking respects natural text boundaries, avoiding mid-sentence cuts, but can still separate a heading from the content it introduces, or split a table's header from its rows.
• Semantic chunking uses embedding similarity between adjacent sentences to detect natural topic boundaries and splits there instead of at a fixed size — more sophisticated, more expensive to compute, generally better quality for heterogeneous documents.
• Overlap (e.g., each chunk includes the last 50 tokens of the previous chunk) is a common mitigation for the boundary-splitting problem — a fact split across a chunk boundary is more likely to appear in full within at least one of the two overlapping chunks.

The chunk-size trade-off is real and directly measurable: smaller chunks give more precise retrieval (less irrelevant text bundled in with the relevant fact, better vector representation of one specific idea) but risk losing surrounding context needed to interpret the fact correctly; larger chunks preserve context but dilute the embedding's specificity and waste context-window budget on irrelevant surrounding text. There is no universally correct chunk size — it depends on document structure and query patterns, and should be evaluated (Part 8.1's precision@k/recall@k/MRR/NDCG metrics) against your actual corpus and real user questions, not chosen by convention.

Contextual retrieval — solving chunk-boundary context loss more directly than overlap alone

Overlap (above) mitigates chunk-boundary information loss by duplicating a little text across the boundary — but it doesn't fix a different, equally common problem: a chunk can be complete and still be ambiguous or uninterpretable on its own, because the context that made it meaningful lived earlier in the document, not just at the adjacent boundary overlap would capture. A chunk reading "The fee increases to $50 after the third violation" is retrievable and embeddable, but neither the retriever nor the generation model reading it in isolation has any way to know which policy, which product, or which customer tier this sentence is even about, if that information was established in a heading or an earlier paragraph several chunks away.

Contextual retrieval addresses this at indexing time: before embedding each chunk, an LLM is given the full document (or a relevant surrounding portion) plus that specific chunk, and asked to generate a short (one to two sentence) piece of context situating the chunk within the document — e.g., "This chunk is from the Enterprise Plan section of the 2026 Refund Policy, discussing repeat-violation fees." That generated context is prepended to the chunk before it's embedded (and typically also kept alongside the chunk text used for generation), so both the embedding and the retrieved context carry the disambiguating information the original chunk lacked in isolation.

async def add_contextual_prefix(client, full_document: str, chunk: str) -> str:
    """
    Generates a short LLM-written context blurb situating a chunk within its
    source document, to be prepended before the chunk is embedded.

    Args:
        client: An async LLM client.
        full_document (str): The full source document the chunk was drawn from.
        chunk (str): The specific chunk needing situating context.

    Returns:
        str: The chunk prefixed with a short, LLM-generated contextual blurb.
    """
    response = await client.messages.create(
        model="claude-haiku-4-5",  # small, cheap model — this runs once per chunk at indexing time
        max_tokens=100,
        system=(
            "Given the full document and one chunk from it, write a 1-2 sentence "
            "context blurb situating this chunk within the document (what section/topic "
            "it's from). Output only the blurb."
        ),
        messages=[
            {"role": "user", "content": f"Document:\n{full_document}\n\nChunk:\n{chunk}"}
        ],
    )
    context_blurb = response.content[0].text.strip()
    return f"{context_blurb}\n\n{chunk}"

This meaningfully improves retrieval precision for exactly the boundary-context-loss failure mode overlap only partially addresses — but it has a real, added indexing-time cost: one extra LLM call per chunk, for every document in the corpus, at ingestion time (not per query). For a large corpus this is a real, one-time-per-document expense worth estimating explicitly — a small, cheap model is the appropriate choice here (Part 3.11's routing logic applies directly, since the task itself is simple even though it runs at high volume), and reusing the same full document as context across many chunks from that document in sequence is a good candidate for prompt caching (Part 3.3, section 12).

Why "just retrieve, then generate" isn't the whole story

The description above is the minimal viable RAG pipeline, and it's what most tutorials stop at. Production RAG systems typically add several more stages, each addressing a specific, real failure mode of the minimal pipeline:

• Query rewriting (Part 3.6): a user's literal question is often a poor retrieval query — "what about the thing we discussed last time" retrieves nothing useful without first resolving it using conversation history into something like "what is the cancellation policy for enterprise plans."
• Hybrid search (Part 3.6): pure semantic (embedding) search can miss exact-term matches (a specific product SKU, an exact legal clause number) that a keyword search would catch trivially — combining both catches more of both failure modes.
• Reranking (Part 3.6): the initial retrieval (fast, approximate, via ANN, Part 3.4) over-fetches a larger candidate set, then a slower, more accurate model re-scores and reorders just those candidates before the final top-k is chosen — trading a small amount of extra latency for meaningfully better precision on which chunks actually get passed to generation.

The generation step's own failure modes

Even with perfect retrieval, generation can still go wrong: the model might ignore the retrieved context and answer from its own (ungrounded) training knowledge instead, might partially use the context but blend in fabricated details, or might answer a slightly different question than the one that was actually retrieved-for. This is exactly why RAG systems need explicit faithfulness evaluation (Part 8.2) — checking whether the generated answer is actually, verifiably supported by the retrieved context — as a distinct, measured property, not an assumption that "we did RAG, so it's grounded now."

Agentic RAG — retrieval as a callable tool, not a fixed two-phase pipeline

Everything in this chapter so far describes RAG as a fixed pipeline: retrieve once, then generate — the same fixed sequence every time, exactly the "workflow" pattern from Part 3.7. This is the right default for most cases, but it isn't the only architecture. An alternative, increasingly common pattern — agentic RAG — exposes retrieval as a tool (Part 3.3) the model can call zero, one, or many times, deciding for itself when to retrieve, what to search for, and whether the results it got back are sufficient or warrant a refined follow-up query.

This maps directly onto Part 3.7's workflow-vs-agent decision framework: a fixed retrieve-then-generate pipeline is a workflow (control flow fixed by your code, LLM judgment used only for generation); agentic RAG hands the model the decision of whether and how to retrieve at all — exactly a workflow-to-agent escalation, with exactly the trade-off Part 3.7 established for that escalation in general. The concrete benefit is adaptive, multi-hop retrieval: a question that genuinely requires looking something up, refining the query based on what came back, and looking up something else before it can be answered (e.g., "compare the vendor's Q3 numbers to what the contract's penalty clause requires" — a question needing two logically-sequenced retrievals, not one) is poorly served by a single fixed retrieval pass, but is exactly what a model deciding its own retrieval sequence can do well.

The cost of this benefit is precisely Part 3.7's cost for agents generally: unpredictability. A fixed pipeline retrieves exactly once, with bounded, predictable cost and latency; an agent deciding its own retrieval calls might retrieve zero times (if it judges its own knowledge sufficient — reintroducing exactly the ungrounded-answer risk RAG exists to prevent, section 2), or many times (adding cost and latency unpredictably, and requiring the same bounded-loop safeguard from Part 3.3/3.7's agent sections). The right choice follows Part 3.7's general rule directly: default to the fixed pipeline; escalate to agentic/tool-based retrieval specifically when actual measured query patterns genuinely require adaptive, multi-hop retrieval that a fixed single pass can't serve well — not as a default "more sophisticated" upgrade applied to every RAG system.

5. Simple mental model

RAG is an open-book exam versus a closed-book exam. An LLM without RAG is answering from memory alone (closed-book) — impressive when it genuinely remembers correctly, unreliable and prone to confident guessing when it doesn't. RAG hands the model the specific relevant textbook pages (retrieved chunks) right before it answers, and instructs it to answer from those pages — turning the task from "recall this perfectly from memory" into the much more reliable "read this short excerpt and summarize/apply it," which is a task LLMs are demonstrably much better and more trustworthy at.

6. Real-world example

A healthcare provider's internal AI assistant needs to answer clinician questions about drug interaction policies that change periodically as new guidance is issued. Fine-tuning a model on the current policy documents would require a full retraining cycle every time a policy changes (Part 2.2) — operationally slow, expensive, and risky (catastrophic forgetting). With RAG, updating a policy document in the indexed corpus (re-chunk, re-embed, re-index that one document, Part 3.4) makes the new guidance available to the assistant within minutes, with the answer citing the specific policy document and section — both faster to update and more auditable (a clinician or compliance reviewer can trace exactly which document justified a given answer) than a fine-tuned model's opaque, non-attributable "knowledge."

7. Architecture diagram

Indexing (offline, ahead of time):

Documents

Chunking

EmbeddingPart 2.5

Vector DBPart 3.4

Query (online, per user request):

User query

Query rewritingPart 3.6

Retrieve top-kembed query, hybrid search

Rerank candidatesPart 3.6

Assemble prompt + generate

8. Production considerations

• Chunking strategy needs to be evaluated against real documents and real user questions, not chosen by convention — set up an evaluation set (Part 8.1) early and iterate on chunk size/strategy against measured retrieval quality, not intuition. "Measured retrieval quality" concretely means precision@k, recall@k, MRR, and NDCG (Part 8.1's worked example defines each and when to use which) computed against a labeled query→relevant-chunk set — not a vague impression of whether results "look right."
• Keep the indexing pipeline idempotent and incremental. Document updates should re-chunk/re-embed/re-index only the changed document, not require a full corpus rebuild every time — Part 3.4's blue-green pattern is for embedding-model changes specifically; day-to-day document updates should be a much lighter, targeted operation.
• Instruct the model explicitly to answer only from the provided context, and to say so explicitly when the context doesn't contain the answer (Part 2.6, Part 3.1) — this single prompt-level instruction is a meaningful, cheap mitigation against the model ignoring retrieved context in favor of ungrounded guessing.
• Return citations/sources alongside the answer whenever the use case allows it — this turns "trust the AI" into "verify the AI," which is both more trustworthy for the end user and gives you a debugging trail when an answer turns out to be wrong (was the retrieval bad, or did generation ignore good retrieval?).
• Monitor retrieval quality and generation faithfulness as separate, distinct metrics (Part 8) — a RAG system can have excellent retrieval and poor generation faithfulness, or vice versa, and conflating "is RAG working" into one number hides which half actually needs fixing.

9. Common mistakes

• Reaching for RAG as the default architecture for any "the AI should know about X" requirement, without first asking whether the actual need is structured lookup (Part 1.4 — SQL for precise, exact, frequently-changing structured data), tool calling (Part 3.3 — for real-time or parameterized data), or genuinely just better prompting (Part 3.1 — if the information could just be included directly and isn't that large).
• Choosing a chunk size once, early, and never revisiting it against actual measured retrieval quality.
• Assuming "we implemented RAG" is binary — done or not done — rather than a pipeline with several independently-improvable stages (chunking, retrieval, reranking, generation), each with its own measurable quality and its own way to fail.
• Not evaluating faithfulness — assuming that because the answer used retrieved context, it's automatically fully grounded in it, when models can still blend in fabricated details even with good retrieved context in hand.
• Ignoring document structure (tables, headers, code blocks) during chunking, producing chunks that are technically valid text but have lost critical structural meaning (a table row without its header is often meaningless on its own).

10. Security considerations

• Retrieved documents are untrusted input to the generation step, exactly like user input or tool results (Part 3.3) — if an attacker can get content into your indexed document corpus (a malicious PDF uploaded to a shared knowledge base, for instance), that content becomes part of what the model conditions its answer on, which is the direct mechanism behind indirect prompt injection via RAG (Part 9.1's primary real-world example category).
• Access control must be enforced at the retrieval layer, not assumed from the generation prompt — a document a user isn't authorized to see must never enter the retrieval candidate set for that user's query in the first place (Part 3.4's pre-filtering discussion is exactly this, applied at the security layer).
• Citations that expose document sources can themselves be a data-leakage vector if citation metadata (file paths, internal document IDs) reveals information the user shouldn't have about documents they can't directly access, even if the answer content itself was properly scoped.

11. Performance considerations

• Each additional pipeline stage (query rewriting, reranking) adds latency — a full production RAG pipeline with all stages can meaningfully increase end-to-end response time compared to the minimal "embed and retrieve" version; measure whether each added stage's quality improvement justifies its latency cost for your specific use case.
• Retrieval latency (Part 3.4's ANN search) is typically a small fraction of total pipeline latency compared to the generation (LLM call) step — profile before assuming retrieval is your bottleneck.

12. Cost considerations

• More retrieved chunks passed into the prompt means more input tokens on every single generation call (Part 2.6) — there's a real, direct cost trade-off between retrieving generously (better chance of including the right information) and retrieving tightly (lower cost per call); reranking (Part 3.6) exists partly to let you retrieve a larger initial candidate set cheaply, then pass only the best few to the (more expensive, per-token) generation step.
• Indexing cost (embedding an entire enterprise corpus) is a real, one-time-per-document cost that recurs on every re-embedding migration (Part 3.4) — estimate this explicitly for large corpora before committing to an embedding-model change.
• Large retrieved context reused repeatedly across a session benefits from prompt caching (Part 3.3, section 12). A long-lived conversation that keeps re-sending the same retrieved chunks turn after turn — rather than re-retrieving and re-assembling fresh context every turn — can cache that retrieved-context block: full input-token price once, a fraction of that price on every subsequent turn that reuses it unchanged. This is a direct, practical mitigation for the "more retrieved chunks means more input tokens on every call" cost above, specifically when the same retrieved content is genuinely being reused rather than changing turn to turn.

13. When to use it

Answering questions that require specific, current, or proprietary knowledge not baked into the model's training data, where that knowledge changes often enough that fine-tuning would be operationally impractical, and where source attribution/verifiability matters.

14. When NOT to use it

• Precise, structured, frequently-changing data with an exact lookup key (account balance, order status, ticket status) — this is a tool-calling/SQL problem (Part 1.4, Part 3.3), not a RAG problem; embedding "account #4521 has balance $230.50" into a vector store and hoping semantic search finds it reliably is strictly worse than just querying the database directly.
• Small, static knowledge bases that fit comfortably and cheaply in the prompt directly — if your entire "knowledge base" is a two-page FAQ, just include it in the system prompt every time; the retrieval pipeline adds complexity with no real benefit at that scale.
• Tasks that need reasoning/computation over the retrieved content, not just answer-finding — e.g., "what's the total of all transactions over $500 in Q3" needs aggregation logic (a database query), not similarity-based document retrieval, even if the underlying data happens to live in documents.

15. Alternatives and trade-offs

Approach	Good for	Weak point
RAG	Large, frequently-changing, unstructured knowledge bases needing attribution	Added pipeline complexity; several independent failure points
Fine-tuning (Part 2.2)	Consistent behavior/format/style, narrow domain adaptation	Wrong tool for frequently-changing facts; retraining cost and forgetting risk
Direct prompt inclusion	Small, static knowledge, simplest possible implementation	Doesn't scale past what fits comfortably in context, no retrieval intelligence
Tool calling / SQL (Part 3.3, 1.4)	Precise, structured, exact-match, frequently-changing data	Not suited to unstructured, fuzzy, "find the relevant passage" tasks

16. Practical Python/code example

A minimal but complete RAG pipeline tying together Part 2.5 (embeddings) and Part 3.4 (vector search):

async def answer_with_rag(
    client, vector_store, question: str, tenant_id: str, k: int = 5
) -> dict:
    """
    Answers a question using retrieval-augmented generation, grounded strictly
    in retrieved context, with explicit handling for the no-relevant-context case.

    Args:
        client: An async LLM client.
        vector_store: A vector database client supporting tenant-filtered search (Part 3.4).
        question (str): The user's question.
        tenant_id (str): Tenant scope for retrieval, enforced as a pre-filter.
        k (int): Number of chunks to retrieve.

    Returns:
        dict: The answer text and the list of source chunks used, for citation/audit.
    """
    query_embedding = (await embed_texts(client, [question]))[0]
    results = await vector_store.query(
        vector=query_embedding, filter={"tenant_id": {"$eq": tenant_id}}, top_k=k
    )

    if not results:
        return {"answer": "I don't have relevant information to answer that.", "sources": []}

    context = "\n\n".join(f"[Source {i+1}]: {r['content']}" for i, r in enumerate(results))
    response = await client.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=500,
        system=(
            "Answer the question using ONLY the numbered sources below. "
            "Cite sources by number. If the sources don't contain the answer, say so explicitly."
        ),
        messages=[{"role": "user", "content": f"{context}\n\nQuestion: {question}"}],
    )
    return {"answer": response.content[0].text, "sources": results}

17. Production-quality example

Adding chunking with overlap and structure-awareness (respecting paragraph boundaries), directly addressing the chunking-quality discussion in section 4:

def chunk_document(
    text: str, max_chunk_tokens: int = 400, overlap_tokens: int = 50, encoding=None
) -> list[str]:
    """
    Splits a document into overlapping chunks at paragraph boundaries where possible,
    falling back to a hard split only when a single paragraph exceeds max_chunk_tokens.

    Args:
        text (str): Full document text.
        max_chunk_tokens (int): Approximate maximum tokens per chunk.
        overlap_tokens (int): Number of trailing tokens from the previous chunk to
            repeat at the start of the next, mitigating fact-splitting at boundaries.
        encoding: A tokenizer encoding object (Part 2.3) used for accurate token counting.

    Returns:
        list[str]: The document split into chunks.
    """
    paragraphs = [p.strip() for p in text.split("\n\n") if p.strip()]
    chunks: list[str] = []
    current_chunk = ""

    for paragraph in paragraphs:
        candidate = f"{current_chunk}\n\n{paragraph}".strip()
        if len(encoding.encode(candidate)) > max_chunk_tokens and current_chunk:
            chunks.append(current_chunk)
            overlap_text = encoding.decode(
                encoding.encode(current_chunk)[-overlap_tokens:]
            )
            current_chunk = f"{overlap_text}\n\n{paragraph}".strip()
        else:
            current_chunk = candidate

    if current_chunk:
        chunks.append(current_chunk)
    return chunks

Splitting at paragraph boundaries first (falling back to a hard split only when necessary) directly avoids the mid-sentence-cut problem from section 4, and the overlap mechanism gives a fact near a chunk boundary a second chance to appear whole in the adjacent chunk.

18. Short exercise

A customer's RAG system retrieves the correct document chunk for a policy question, but the generated answer still contains a fabricated detail not present in that chunk. Using the distinction from section 4 (retrieval failure vs. generation faithfulness failure), explain which stage is at fault here, and describe the specific evaluation (Part 8.2) you'd run to confirm your diagnosis before proposing a fix.

19. Interview questions

• Walk through the full RAG pipeline, distinguishing the indexing phase from the query phase, and explain why conflating them causes confusion.
• Why is chunk size a real trade-off rather than a setting with one correct answer?
• Give a concrete example of a use case where RAG is the wrong architecture, and explain what you'd use instead.

20. FDE/customer scenario

Customer: "We want an AI chatbot for our employees that knows everything about our company."

This is the canonical case for practicing the reasoning-before-recommending discipline from Part 12 and prompt.md's explicit teaching rule: before proposing RAG (or anything else), ask what "knows everything" actually means in practice — is it mostly static policy/handbook content (a good RAG fit), precise HR data like PTO balances (a tool-calling/database fit, not RAG), or real-time operational data (also tool-calling, not RAG)? A credible FDE response decomposes "everything" into these different data-shape categories before proposing an architecture, rather than jumping straight to "let's build a RAG chatbot" — because a chatbot that confidently RAG-hallucinates someone's PTO balance from a stale HR PDF is a worse outcome than not building it at all.

---

Key takeaways

• RAG solves the frozen-knowledge and hallucination problem by grounding generation in retrieved, current, attributable content — without requiring model retraining.
• Chunking strategy is one of the highest-leverage, most underrated levers in RAG quality — evaluate it against real data, don't just pick a convention.
• Retrieval quality and generation faithfulness are separate, independently measurable properties — a RAG system can fail at either or both.
• RAG is frequently over-applied; precise structured data belongs in tool calls/SQL, not vector search.

Things you should be able to explain

• The two distinct phases of RAG (indexing vs. query) and what happens in each.
• Why chunk size is a genuine trade-off, and what overlap mitigates.
• When RAG is the wrong tool, with a concrete example.

Things you should be able to build

• A complete, tenant-scoped RAG pipeline from chunking through cited generation.

Common mistakes

• Defaulting to RAG for precise, structured, frequently-changing data.
• Never revisiting chunking strategy against measured quality.
• Assuming grounded retrieval guarantees faithful generation.

Recommended next chapter

06-hybrid-search-reranking-query-rewriting.md