4.5 — Retrievers and Document Processing

1. What is it?

LangChain's Document class, document loaders, text splitters, and BaseRetriever interface are the framework's abstractions for the RAG indexing and retrieval pipeline from Part 3.5 — standardized objects for representing a chunk of content with metadata, loading raw documents from various sources, splitting them into chunks, and retrieving relevant ones, all composable via the same Runnable interface from Part 4.1.

2. Why does it exist?

Part 3.5 established that RAG's indexing phase involves parsing raw documents, chunking, embedding, and storing — and that the query phase involves embedding a query and retrieving relevant chunks. Every one of these steps has provider/format diversity: documents come as PDFs, HTML, plain text, database records; vector stores have different APIs (Part 3.4). LangChain's document-processing abstractions exist to give you one consistent Document object and one consistent BaseRetriever interface regardless of the underlying document source or vector store, the same abstraction strategy as BaseChatModel (Part 4.2) applied to the RAG pipeline specifically.

3. What problem does it solve?

It solves "write RAG pipeline logic once, swap document sources and vector stores without rewriting the pipeline" — directly extending the provider-abstraction value proposition from Part 4.1/4.2 to the retrieval side of the application.

4. How does it work internally?

Document — the standardized chunk representation

from langchain_core.documents import Document

doc = Document(
    page_content="Our refund policy allows returns within 30 days...",
    metadata={"source": "policy.pdf", "tenant_id": "acme_corp", "page": 3},
)

A Document is simply page_content (the text, Part 3.5's chunk) plus a metadata dict — deliberately minimal, because its job is just to be the common currency every loader, splitter, embedder, and retriever in the ecosystem passes around. The metadata dict is exactly where the tenant-scoping/access-control fields from Part 3.4's pre-filtering discussion live in practice — a Document's metadata is what a vector store's filtered search (Part 3.4) actually filters on.

Document loaders

LangChain provides loaders for many source formats (PDF, HTML, CSV, database queries, cloud storage) that each implement a common .load() method returning list[Document] — the diversity of parsing logic for each format (Part 2.3's classical NLP-adjacent parsing concerns) is hidden behind this uniform interface, so your indexing pipeline code doesn't need format-specific branches once documents reach the list[Document] stage.

Text splitters — implementing Part 3.5's chunking strategies as reusable components

from langchain_text_splitters import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=50)
chunks: list[Document] = splitter.split_documents(documents)

RecursiveCharacterTextSplitter implements exactly the paragraph-boundary-aware chunking strategy from Part 3.5's production example — it tries splitting on a prioritized list of separators (paragraph breaks first, then sentence breaks, then words) attempting to respect natural text boundaries before falling back to a harder split, with chunk_overlap implementing the same overlap mitigation Part 3.5 discussed. Other splitter classes exist for more specialized structure-aware splitting (e.g., code-aware splitters that respect function/class boundaries, or Markdown-header-aware splitters) — the right choice, per Part 3.5's core argument, still depends on evaluating chunking quality against your actual document structure and query patterns, not picking a splitter by convention.

BaseRetriever — the standardized retrieval interface

Any vector store integration (Part 3.4's various options) exposes a .as_retriever() method returning a BaseRetriever — itself a Runnable (Part 4.1), meaning it composes into LCEL chains with .invoke() returning list[Document] for a given query, uniformly regardless of which underlying vector store is actually performing the search.

retriever = vector_store.as_retriever(search_kwargs={"k": 5, "filter": {"tenant_id": tenant_id}})

This search_kwargs filter argument is precisely where Part 3.4's pre-filtering discipline gets applied — verify, for your specific vector store integration, that this filter is genuinely applied as a pre-filter at the ANN search level (Part 3.4, section 4's critical distinction) rather than as a post-hoc filter, since this detail is vector-store-specific and the retriever abstraction's convenience shouldn't obscure which behavior you're actually getting.

Implementing Part 3.6's hybrid search and reranking in LangChain

Part 3.6 taught hybrid search (fusing semantic and keyword search via Reciprocal Rank Fusion) and reranking (a cross-encoder rescoring a larger candidate set) conceptually and provider-agnostically. LangChain provides retriever classes implementing both directly on top of the BaseRetriever interface above, so both techniques compose into the same LCEL/retriever pipeline as everything else in this chapter rather than requiring hand-rolled fusion or reranking code.

Hybrid search — an ensemble retriever fusing a keyword retriever with a vector retriever:

from langchain_community.retrievers import BM25Retriever
from langchain.retrievers import EnsembleRetriever

# BM25Retriever implements Part 3.6's keyword-search half directly, over the same
# chunked documents used to build the vector store.
bm25_retriever = BM25Retriever.from_documents(chunks)
bm25_retriever.k = 50  # over-fetch, matching Part 3.6's candidate_k

vector_retriever = vector_store.as_retriever(search_kwargs={"k": 50, "filter": {"tenant_id": tenant_id}})

# EnsembleRetriever fuses the two ranked lists using Reciprocal Rank Fusion —
# mechanically the exact RRF formula from Part 3.6, section 4.
hybrid_retriever = EnsembleRetriever(
    retrievers=[bm25_retriever, vector_retriever], weights=[0.5, 0.5]
)

results = await hybrid_retriever.ainvoke(query)

Reranking — a contextual compression retriever wrapping a base retriever with a cross-encoder:

from langchain_community.cross_encoders import HuggingFaceCrossEncoder
from langchain.retrievers.document_compressors import CrossEncoderReranker
from langchain.retrievers import ContextualCompressionRetriever

cross_encoder = HuggingFaceCrossEncoder(model_name="cross-encoder/ms-marco-MiniLM-L-6-v2")
reranker = CrossEncoderReranker(model=cross_encoder, top_n=5)  # Part 3.6's final_k

# Wraps hybrid_retriever's larger candidate set, applying the cross-encoder
# rescoring pass from Part 3.6, section 4 before returning the final top-n.
reranking_retriever = ContextualCompressionRetriever(
    base_compressor=reranker, base_retriever=hybrid_retriever
)

final_results = await reranking_retriever.ainvoke(query)

EnsembleRetriever is doing exactly the RRF fusion mechanism from Part 3.6, section 4 (combining ranks, not raw incomparable scores) — you don't hand-implement reciprocal_rank_fusion() yourself when using it, though understanding that it's happening underneath (per this book's teaching philosophy) still matters for tuning weights and diagnosing a surprising fused ranking. CrossEncoderReranker wrapped in ContextualCompressionRetriever is doing exactly the bi-encoder-then-cross-encoder two-stage retrieval from Part 3.6 — the "compression" name is a bit of a misnomer worth flagging explicitly: it isn't compressing text, it's filtering/reordering the candidate document list down to the reranked top-n.

Verify against current docs before shipping: exact class names and import paths (EnsembleRetriever, CrossEncoderReranker, ContextualCompressionRetriever, and which package each currently lives in) are exactly the kind of detail Part 4.1 flagged as having moved around in LangChain's 1.0 restructuring — some retriever/compressor classes have historically lived in langchain, others in langchain_community or a legacy langchain-classic path. Confirm current locations against LangChain's documentation rather than assuming these import paths are stable long-term.

5. Simple mental model

Document, loaders, splitters, and BaseRetriever are a standardized shipping and inventory system for RAG: Document is the standard-sized box every item ships in regardless of source; loaders are the receiving docks that convert whatever comes in (a truck of PDFs, a database export) into that standard box format; splitters are the packing stations that break large shipments into standard-box-sized units; BaseRetriever is the warehouse's uniform "find me the relevant boxes" interface, regardless of which specific warehouse (vector store) is actually storing them.

6. Real-world example

A team indexing a customer's mixed document corpus (PDFs, Confluence pages exported as HTML, and database records describing product specs) uses three different LangChain loaders to normalize all three sources into list[Document], applies one consistent RecursiveCharacterTextSplitter chunking strategy across all of them (since the downstream RAG pipeline shouldn't need to know or care which source format a given chunk originally came from), and retrieves via one BaseRetriever interface regardless of source — the format diversity is fully absorbed at the loader stage and never surfaces again in the rest of the pipeline.

7. Architecture diagram

PDF loader

HTML loader

DB loader

list[Document]uniform, regardless of source

RecursiveCharacterTextSplitterPart 3.5 chunking

list[Document]chunked, with metadata

Embed + storein vector store · Part 3.4

vector_store.as_retriever(...)search_kwargs={filter: tenant_id}

BaseRetriever.invoke(query)→ list[Document]

Every loader converges to the same Document shape before splitting/embedding.

8. Production considerations

• Verify pre-filter vs. post-filter behavior for your specific retriever's search_kwargs against Part 3.4's critical distinction — this is vector-store-integration-specific and worth confirming explicitly, not assuming from the uniform-looking interface.
• Preserve source metadata (document ID, page number, tenant) through the entire loader → splitter → embedding pipeline — this is what enables the citation/attribution pattern from Part 3.5's production considerations, and it's easy to accidentally lose if a custom processing step doesn't propagate metadata correctly.
• Test chunking strategy against real documents, per Part 3.5's core argument — LangChain provides many splitter options, but providing options doesn't substitute for the evaluation discipline of picking and validating the right one for your actual corpus.

9. Common mistakes

• Assuming RecursiveCharacterTextSplitter's defaults are appropriate for every document type without evaluating against your actual corpus (Part 3.5's chunking-evaluation argument applies regardless of which splitter class you use).
• Losing metadata (especially tenant/access-control fields) somewhere in a custom loader or processing step, silently breaking the pre-filtering discipline from Part 3.4 downstream.
• Not verifying whether a specific vector store integration's retriever applies filters as pre-filters or post-filters, reintroducing Part 3.4's multi-tenant correctness risk unknowingly.

10. Security considerations

• Metadata fields used for access control (tenant ID, user permissions) must be set correctly and immutably at ingestion time — a document loader or splitter step that doesn't correctly propagate or validate these fields is a direct path to the cross-tenant leakage risk from Part 3.4/9.6.
• Document loaders that fetch from external/network sources (a URL loader, a cloud storage loader) inherit whatever authentication/access considerations that source requires — treat loader configuration (credentials, allowed source scope) with the same care as any other credentialed integration (Part 9.5).

11. Performance considerations

• .as_retriever()'s underlying ANN search performance characteristics (Part 3.4) are inherited directly — the retriever abstraction doesn't change the fundamental speed/recall trade-offs, it just standardizes the calling interface.
• Batch-loading and batch-embedding (Part 3.4's blue-green reindex pattern) should still be used for large corpora — the loader/splitter abstractions don't automatically parallelize or batch for you; verify how a given loader handles large source sets before assuming efficient behavior at scale.

12. Cost considerations

• Identical to Part 3.4/3.5 — the abstraction layer here doesn't change the underlying embedding and storage costs, just the code you write to orchestrate them.

13. When to use it

Whenever building a RAG pipeline in LangChain and you want provider/format-agnostic document processing and retrieval code — essentially the default choice for LangChain-based RAG.

14. When NOT to use it

If your document sources and vector store are fixed and unlikely to change, and you need very fine-grained control over the exact retrieval/filtering behavior that a specific vector store's raw client offers beyond what BaseRetriever's standardized interface exposes, dropping to that vector store's raw client (as in Part 3.4's raw examples) for the retrieval step specifically is a legitimate choice.

15. Alternatives and trade-offs

Approach	Good for	Weak point
LangChain document/retriever abstractions	Format/provider-agnostic pipeline code, ecosystem loader/splitter reuse	May not expose every vector-store-specific feature/parameter
Raw vector store client (Part 3.4)	Full access to store-specific features, explicit control	Provider-specific code, less portable

16. Practical Python/code example

from langchain_community.document_loaders import PyPDFLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter

loader = PyPDFLoader("policy.pdf")
documents = loader.load()

splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=50)
chunks = splitter.split_documents(documents)

for chunk in chunks:
    chunk.metadata["tenant_id"] = "acme_corp"  # explicit, deliberate metadata propagation

17. Production-quality example

A retriever wrapper that explicitly verifies and enforces pre-filtering, addressing section 8's core production consideration directly in code rather than trusting the abstraction implicitly:

import logging

logger = logging.getLogger("verified_retriever")


class TenantVerifiedRetriever:
    """Wraps a LangChain retriever, explicitly asserting every returned document
    matches the requested tenant — a defense-in-depth check in case the underlying
    vector store's filter behaves as a post-filter rather than a pre-filter."""

    def __init__(self, base_retriever):
        """
        Args:
            base_retriever: A LangChain BaseRetriever configured with a tenant filter.
        """
        self._retriever = base_retriever

    async def get_relevant_documents(self, query: str, tenant_id: str) -> list:
        """
        Retrieves documents and asserts tenant scoping is actually correct,
        rather than trusting the underlying retriever's filter silently.

        Args:
            query (str): The retrieval query.
            tenant_id (str): The tenant this retrieval must be scoped to.

        Returns:
            list[Document]: Verified, correctly-scoped documents.

        Raises:
            RuntimeError: If any returned document doesn't match the expected tenant —
                signaling a pre-filter/post-filter misconfiguration worth investigating
                immediately rather than silently serving leaked results.
        """
        results = await self._retriever.ainvoke(query)
        for doc in results:
            if doc.metadata.get("tenant_id") != tenant_id:
                logger.critical(
                    "TENANT ISOLATION VIOLATION: expected tenant_id=%s, got %s",
                    tenant_id,
                    doc.metadata.get("tenant_id"),
                )
                raise RuntimeError("retriever returned document outside requested tenant scope")
        return results

This kind of explicit, defense-in-depth assertion is exactly the pattern Part 1.9's tenant-isolation test discussion argued for — here applied as a runtime guard, not just a test, precisely because Part 3.4/4.5's pre-filter/post-filter distinction is easy to get subtly wrong and expensive to get wrong silently.

18. Short exercise

Using LangChain's documentation for your vector store of choice, verify explicitly whether its .as_retriever(search_kwargs={"filter": ...}) implements pre-filtering or post-filtering (Part 3.4, section 4). Write a short note on what you found and cite where you verified it.

19. Interview questions

• What does the Document object standardize, and why does keeping it minimal (just content and metadata) matter for ecosystem interoperability?
• Why would you add an explicit runtime assertion (like the TenantVerifiedRetriever example) on top of a retriever's built-in filtering, even if you've verified it's a pre-filter?
• What's lost and what's gained by using LangChain's document-processing abstractions versus a vector store's raw client directly?

20. FDE/customer scenario

Customer's security team: "How do you guarantee our documents never appear in another customer's search results?"

Beyond explaining Part 3.4's pre-filtering architecture, being able to show a concrete, defense-in-depth runtime assertion (like this chapter's TenantVerifiedRetriever) — that actively verifies tenant scoping on every retrieval rather than only trusting the vector store's filter configuration — is a genuinely stronger, more auditable answer than "we configured the filter correctly," and it's the kind of layered-defense thinking (never relying on a single control) that a security-conscious enterprise reviewer will specifically be listening for.

---

Key takeaways

• Document, loaders, splitters, and BaseRetriever extend LangChain's provider-abstraction pattern (Part 4.1/4.2) to the RAG pipeline specifically.
• Metadata propagation through the entire pipeline is what enables both citation (Part 3.5) and access control (Part 3.4) — losing it anywhere breaks both.
• Verify pre-filter vs. post-filter behavior explicitly per vector store integration; consider a defense-in-depth runtime assertion rather than trusting the abstraction implicitly for multi-tenant systems.

Things you should be able to explain

• Why Document's minimal shape (content + metadata) is a deliberate ecosystem-interoperability choice.
• Why chunking-strategy evaluation (Part 3.5) still matters regardless of which LangChain splitter you use.

Things you should be able to build

• A loader → splitter → retriever pipeline with a defense-in-depth tenant-verification wrapper.

Common mistakes

• Losing metadata through custom processing steps.
• Assuming a retriever's filter is a pre-filter without verifying.
• Picking a splitter by convention instead of evaluating against real documents.

Recommended next chapter

06-middleware-streaming-async-callbacks.md