8.2 — Hallucination, Faithfulness, and Relevance

1. What is it?

This chapter defines and distinguishes three related but genuinely different quality properties that get conflated constantly in casual discussion of AI quality: hallucination (generating content not grounded in truth or provided context), faithfulness (whether a RAG system's answer is actually supported by its retrieved context, regardless of whether that context itself is true), and relevance (whether an answer actually addresses what was asked, regardless of its truthfulness or faithfulness) — and gives concrete, measurable techniques for detecting each.

2. Why does it exist?

Part 2.6 introduced hallucination conceptually as a structural property of next-token generation. Part 3.5's RAG chapter introduced faithfulness as a distinct concern from retrieval quality. This chapter exists to make these concepts precise and measurable rather than left as intuitive, loosely-used terms — because "the AI hallucinated" is frequently used to describe three genuinely different failure modes that need different diagnoses and different fixes, and conflating them leads to applying the wrong fix.

3. What problem does it solve?

It solves "how do I precisely diagnose which quality failure occurred, so I fix the actual problem" — a response that's unfaithful to retrieved context needs a different fix (better generation-grounding instructions, Part 3.1) than a response that's faithful to context but irrelevant to the actual question (better retrieval, Part 3.5/3.6) or a response that's relevant and faithful but the underlying retrieved context was itself factually wrong (a data-quality problem in the source corpus, not a model problem at all).

4. How does it work internally?

Precisely distinguishing the three properties

Faithfulness: does the answer's content match/derive from the provided context?
  (independent of whether the context itself is true)

Factual correctness: is the answer actually true, in the real world?
  (independent of whether it came from context or the model's own parametric knowledge)

Relevance: does the answer actually address the question asked?
  (independent of whether it's true or faithful)

A response can be faithful but factually wrong (if the retrieved context itself contained an error — a genuinely faithful summary of bad source data). A response can be factually correct but unfaithful (the model happened to state something true, but not because the provided context supported it — meaning you got lucky, and the underlying grounding mechanism (Part 3.5) that was supposed to ensure reliability wasn't actually the thing that produced this particular correct answer). A response can be both faithful and factually correct but irrelevant (a technically accurate, context-grounded answer to a slightly different question than the one actually asked). Distinguishing these is essential because a RAG system's own evaluation (Part 3.5, Part 8.1) is fundamentally checking faithfulness and relevance — it usually cannot independently verify real-world factual correctness beyond what's in its own retrieved context, which is itself only as good as the underlying document corpus's own accuracy (a distinct, separate data-quality concern, Part 10.2).

Detecting hallucination — techniques

• NLI-based (Natural Language Inference) checking: using a separate model (often a smaller, specialized classifier, or an LLM prompted for this specific task) to check whether a claim in the generated answer is entailed by, contradicted by, or unrelated to the source context — a more precise, structured technique than a vague "does this seem grounded" judgment.
• Claim decomposition: breaking a generated answer into individual, atomic factual claims, then checking each claim's faithfulness to context independently — catches the common case where a mostly-faithful answer contains one specific fabricated detail buried among otherwise well-grounded content, which a single holistic faithfulness score for the whole answer might average away and miss.
• LLM-as-judge for faithfulness (Part 8.3 covers this technique in depth): prompting an LLM specifically to assess "is every claim in this answer supported by this context," often more practical than building/maintaining a dedicated NLI classifier, at the cost of the judge itself potentially being wrong (Part 8.3's own limitations).

Detecting relevance

Relevance evaluation typically checks: does the answer's content actually correspond to the semantic intent of the question (an embedding-similarity check between question and answer, Part 2.5, is a simple, cheap first-pass signal), or, more precisely, does an LLM-as-judge assessment confirm the answer addresses what was actually asked, not a plausible-sounding but subtly different question (Part 2.4's attention/context discussion is relevant here — a model can sometimes drift toward answering a related but not identical question, especially in longer, more complex prompts).

5. Simple mental model

Think of a witness giving courtroom testimony based on a document they were handed: faithfulness asks "did the witness only say things that are actually written in the document they were given" (regardless of whether the document itself is accurate); factual correctness asks "is what the witness said actually true in the real world" (which depends on whether the document was accurate in the first place, a separate question from whether the witness was faithfully reporting it); relevance asks "did the witness actually answer the question they were asked" (a witness can give a perfectly faithful, perfectly true statement that simply doesn't address what was actually asked).

6. Real-world example

A financial services RAG system answers a customer's question about "what's the current interest rate on my savings account" with a faithful, accurate quote from a retrieved document — but the retrieved document turns out to be an outdated rate sheet that hadn't been correctly superseded in the document index (Part 3.4's re-indexing/staleness discussion). The generated answer scores perfectly on faithfulness (it accurately reflects what the retrieved context said) and would even pass a naive factual-correctness check if the evaluator only compared it against that same stale source — but it's still wrong in a way that matters to the customer, because the actual root cause is a data-quality/currency problem in the source corpus (Part 10.2's data-pipeline concerns), not a generation-faithfulness problem at all. Correctly diagnosing this (rather than assuming "the AI hallucinated" and trying to fix the prompt) is exactly the value of distinguishing these properties precisely.

7. Architecture diagram

Retrieved contextis IT accurate/current? — a data-quality/pipeline question (Part 10.2), NOT a generation-quality question

Generated answerfaithful to context? (Part 8.2 core check) · relevant to the question asked?

8. Production considerations

• Evaluate faithfulness and relevance as separate, distinct metrics (Part 8.1's layered-property principle, applied specifically here) — a single blended "quality" score obscures which specific property is failing and therefore which fix actually applies.
• Treat source-corpus data quality/currency as a distinct monitoring concern from generation quality (section 6) — a "hallucination" that's actually a stale-data problem needs a data-pipeline fix (Part 10.2, Part 3.4's re-indexing discipline), not a prompt-engineering fix, and misdiagnosing this wastes engineering effort on the wrong layer.
• Use claim decomposition for high-stakes content where a single fabricated detail buried in an otherwise-good answer is unacceptable (financial figures, medical information, legal claims) — a holistic score can average away exactly the kind of isolated but serious error that matters most in these domains.

9. Common mistakes

• Using "hallucination" as a catch-all term for any quality problem, obscuring whether the actual root cause is ungrounded generation, a genuinely faithful reflection of bad source data, or an irrelevant-but-accurate answer — each needing a different fix.
• Evaluating faithfulness with a single holistic score rather than claim-level decomposition, missing an isolated fabricated detail within an otherwise well-grounded answer.
• Not separately monitoring source-corpus currency/accuracy (a data-pipeline concern) from generation-quality metrics, misattributing a stale-data problem to "the model hallucinating."

10. Security considerations

• A system that appears highly faithful (well-grounded in its retrieved context) is only as trustworthy as that context's own integrity — if an attacker can inject false content into the source corpus (Part 9.1's indirect injection, or a compromised document-ingestion pipeline, Part 10.2), a perfectly faithful generation of that poisoned content is itself the security problem, not a generation-quality failure at all.

11. Performance considerations

• Claim decomposition and NLI-based checking add real latency/cost per evaluation compared to a single holistic score — reserve the more granular technique for evaluation/monitoring contexts (offline gates, sampled online checks, Part 6.2/6.3) rather than running it synchronously in the live user-facing request path.

12. Cost considerations

• Every faithfulness/relevance check implemented via LLM-as-judge (Part 8.3) is an additional LLM call, with the same cost considerations as any other evaluator (Part 6.2, section 12) — claim-level decomposition specifically multiplies this cost by the number of claims checked per answer, a real trade-off between evaluation granularity and cost worth tuning deliberately.

13. When to use it

Faithfulness evaluation: any RAG system, as a core, ongoing quality metric (Part 6.2/6.3). Claim decomposition specifically: high-stakes domains where an isolated fabricated detail is unacceptable even within an otherwise good answer. Relevance evaluation: any system where "technically accurate but doesn't address the actual question" is a real, observed failure mode worth measuring separately from correctness.

14. When NOT to use it

For low-stakes, casual-conversation use cases, the full rigor of claim-level decomposition may be disproportionate (Part 3.12's proportionality principle) — a simpler, holistic faithfulness check may be entirely sufficient.

15. Alternatives and trade-offs

Technique	Good for	Weak point
Holistic faithfulness score	Cheap, fast, simple to implement	Can average away an isolated but serious fabricated detail
Claim decomposition	Catches isolated errors within otherwise-good answers	More LLM calls, more cost, more complex to implement
NLI classifier	Fast, cheap, doesn't require an LLM-as-judge call	Requires training/maintaining a specialized model, or less flexible than LLM-based checking

16. Practical Python/code example

async def check_faithfulness(client, answer: str, context: str) -> dict:
    """
    Checks whether an answer's claims are supported by the provided context,
    using an LLM-as-judge approach (Part 8.3), scoring faithfulness explicitly
    as distinct from factual correctness or relevance.

    Args:
        client: An async LLM client.
        answer (str): The generated answer to check.
        context (str): The retrieved context the answer should be grounded in.

    Returns:
        dict: {"faithful": bool, "unsupported_claims": list[str]}
    """
    response = await client.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=500,
        system=(
            "You check whether an answer's claims are directly supported by the "
            "given context. List any claims NOT supported by the context, even if "
            "they might be true in general — faithfulness means grounded in THIS "
            "context specifically, not general correctness."
        ),
        messages=[{"role": "user", "content": f"Context:\n{context}\n\nAnswer:\n{answer}"}],
    )
    # Parse structured output per Part 3.2 in production — simplified here for clarity
    unsupported = parse_unsupported_claims(response.content[0].text)
    return {"faithful": len(unsupported) == 0, "unsupported_claims": unsupported}

17. Production-quality example

Claim decomposition applied to a high-stakes financial answer, catching an isolated fabricated detail a holistic score might miss:

import logging

logger = logging.getLogger("claim_evaluation")


async def evaluate_answer_claim_by_claim(client, answer: str, context: str) -> dict:
    """
    Decomposes an answer into atomic claims and checks each independently against
    context, for high-stakes domains where a single fabricated detail matters.

    Args:
        client: An async LLM client.
        answer (str): The generated answer.
        context (str): The retrieved context.

    Returns:
        dict: Per-claim faithfulness results and an overall pass/fail requiring
            ALL claims to be supported, not an averaged score.
    """
    decompose_response = await client.messages.create(
        model="claude-sonnet-4-5", max_tokens=300,
        system="List each distinct factual claim in this answer as a separate item.",
        messages=[{"role": "user", "content": answer}],
    )
    claims = parse_claims_list(decompose_response.content[0].text)

    results = []
    for claim in claims:
        check = await check_faithfulness(client, claim, context)
        results.append({"claim": claim, "faithful": check["faithful"]})
        if not check["faithful"]:
            logger.warning("unsupported claim detected: %s", claim)

    all_faithful = all(r["faithful"] for r in results)
    return {"all_claims_faithful": all_faithful, "claim_results": results}

Requiring all() claims to pass, rather than an averaged score, is a deliberate design choice matching section 8's guidance: for high-stakes content, one fabricated detail should fail the whole answer, not be diluted into an acceptable-looking average.

18. Short exercise

A RAG system's answer about a company's return policy is rated "faithful" (it accurately reflects the retrieved document) but a customer complains it's wrong. Using this chapter's distinctions, list the specific follow-up questions you'd ask to determine whether the actual root cause is a stale/inaccurate source document, a relevance mismatch (answering a related but different question), or something else entirely.

19. Interview questions

• Explain the precise distinction between faithfulness and factual correctness, and give an example where a faithful answer is factually wrong.
• Why might claim decomposition catch an error that a holistic faithfulness score misses?
• Why is source-corpus data quality a separate concern from generation faithfulness, even though both can produce a "wrong" answer?

20. FDE/customer scenario

Customer: "Our AI gave a customer wrong information even though it was supposedly grounded in our documents — how is that possible?"

This is precisely the section 6 scenario, and correctly diagnosing it requires exactly this chapter's distinctions: was the answer unfaithful to the retrieved context (a generation problem, fixable with better grounding instructions, Part 3.1), or was it faithful to a stale/inaccurate retrieved document (a data-pipeline problem, fixable by improving document currency/accuracy processes, Part 10.2)? These require entirely different fixes, and conflating them under a vague "the AI hallucinated" diagnosis leads directly to fixing the wrong layer of the system.

---

Key takeaways

• Faithfulness (grounded in context), factual correctness (true in reality), and relevance (addresses the actual question) are three genuinely distinct properties, each needing separate measurement and diagnosis.
• A faithful answer can still be wrong if its source context was itself inaccurate or stale — a data-pipeline problem, not a generation problem.
• Claim decomposition catches isolated fabricated details that a holistic faithfulness score can average away.

Things you should be able to explain

• The precise distinction between faithfulness, factual correctness, and relevance, with an example separating each pair.
• Why "the AI hallucinated" is often an imprecise diagnosis that obscures the actual root cause.

Things you should be able to build

• A claim-decomposition-based faithfulness evaluator requiring all claims to pass, not an averaged score.

Common mistakes

• Conflating faithfulness, correctness, and relevance under one vague "hallucination" label.
• Using a holistic score that averages away an isolated but serious fabricated claim.

Recommended next chapter

03-llm-as-judge-and-human-eval.md