4.1 — LangChain Architecture and Internals

1. What is it?

LangChain is a framework for building LLM applications, providing standardized abstractions over models, prompts, tools, and agent orchestration so you don't rebuild the same integration and control-flow plumbing (from Part 3) for every provider and every project. As of LangChain 1.0 (shipped October 22, 2025, with a stability commitment of no breaking changes until 2.0), the framework has been substantially restructured from the pre-1.0 line most tutorials still describe — this chapter explains the current architecture, not the older 0.x patterns you may encounter in older material.

Since you already have practical LangChain experience, this chapter focuses on what's actually happening underneath the abstractions and how the package is currently structured, not basic syntax.

2. Why does it exist?

Every LLM provider has its own SDK, its own message format, its own tool-calling conventions, its own streaming protocol. Part 3 covered these concepts (messages, tools, structured output) at the conceptual level, provider-agnostically. LangChain exists to give you one consistent interface across that provider diversity — write your application logic once against LangChain's abstractions, and swap the underlying model provider with a one-line change, rather than rewriting integration code. It also exists to package the recurring orchestration patterns from Part 3 (agent loops, RAG pipelines, tool-calling loops) as reusable, well-tested components instead of everyone hand-rolling the same bounded-loop code from Part 3.7's examples.

3. What problem does it solve?

It solves two related problems: provider abstraction (one interface across OpenAI, Anthropic, and other providers, so your application code isn't tightly coupled to one vendor's specific API shape) and orchestration reuse (the tool-calling loop, the RAG retrieval pipeline, the agent's middleware hooks — Part 3's concepts — implemented once, correctly, with production concerns like retries and structured tracing already handled, rather than reimplemented per project).

4. How does it work internally?

The current package structure (post-1.0)

LangChain 1.0 split the framework into more clearly separated packages, each with a distinct responsibility:

langchain-core     — base abstractions: Runnable interface, LCEL, message types,
                      the foundational interfaces everything else builds on
langchain          — high-level, batteries-included package: create_agent,
                      middleware system, common patterns
langchain-community — broad, community-maintained integrations (many tools,
                      loaders, less tightly maintained/versioned than core packages)
langchain-openai,
langchain-anthropic,
  etc.             — provider-specific packages, each implementing the core
                      model interface for that specific provider
langchain-classic  — legacy functionality moved out of the main package for
                      backward compatibility: the old AgentExecutor,
                      initialize_agent, and older memory classes like
                      ConversationBufferMemory live here now, not deleted,
                      but no longer the recommended path

This split matters practically: when you see LangChain code (in tutorials, in older internal codebases, in Stack Overflow answers) that imports AgentExecutor or initialize_agent directly from langchain, that's pre-1.0-style code — it likely still runs via langchain-classic for compatibility, but it is not the currently recommended pattern, and building new code against it means building against a path the framework has explicitly moved away from. LangChain 1.0 also requires Python 3.10+ — a hard constraint worth checking against any environment you're deploying into.

The Runnable interface and LCEL — the foundation everything sits on

langchain-core's central abstraction is the Runnable interface: essentially, any component (a prompt template, a model, a parser, a retriever) that can be invoked, batched, streamed, or composed. LCEL (LangChain Expression Language) is the | (pipe) syntax for composing Runnables into a pipeline:

chain = prompt_template | model | output_parser

What's actually happening: each of prompt_template, model, and output_parser implements the same Runnable interface (.invoke(), .batch(), .stream(), .ainvoke() for async), and the | operator composes them into a RunnableSequence that calls each step's .invoke() (or the streaming/async equivalent) in order, passing each step's output as the next step's input. Because every piece implements the same interface, the composed chain also transparently supports .stream() and .ainvoke(), even though you only wrote the sequence once — streaming and async support isn't something you build per-chain, it's inherited from the underlying interface every component already implements. This is the actual architectural reason LCEL is more than sugar over plain function composition: uniform streaming, batching, and async behavior falls out of the interface design itself.

Fuller LCEL composition — parallel retrieval, passthrough, and lambdas in a RAG chain

The prompt | model | output_parser chain above is the minimal case: a strictly linear pipeline where each step's output is the next step's entire input. Real chains — a RAG chain (Part 3.5) being the most common example — usually need to run something in parallel with passing the original input through untouched, then combine both into the next step's input. That's where three more Runnable primitives from langchain-core earn their place:

• RunnableParallel (or the equivalent dict-literal shorthand) runs multiple Runnables concurrently against the same input and returns a dict of their outputs — the LCEL-native way to express Part 1.1's asyncio.gather pattern.
• RunnablePassthrough forwards its input unchanged — used specifically to carry the original question through a step that would otherwise only produce the retrieved context, so the next step still has both.
• RunnableLambda wraps an arbitrary Python function as a Runnable, so a plain function (e.g., formatting retrieved Document objects, Part 4.5, into a single context string) composes into the chain with the same .invoke()/.stream()/.batch() interface as every other step, per this chapter's core Runnable-interface argument.

from langchain_core.runnables import RunnableParallel, RunnablePassthrough, RunnableLambda
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_anthropic import ChatAnthropic


def format_docs(docs: list) -> str:
    """
    Joins retrieved Document chunks (Part 4.5) into a single context string
    for insertion into the generation prompt.

    Args:
        docs (list): Retrieved Document objects.

    Returns:
        str: The chunks' page_content, newline-joined.
    """
    return "\n\n".join(doc.page_content for doc in docs)


prompt = ChatPromptTemplate.from_messages([
    ("system", "Answer using ONLY the provided context. Say so if it doesn't contain the answer."),
    ("user", "Context:\n{context}\n\nQuestion: {question}"),
])
model = ChatAnthropic(model="claude-sonnet-4-5")

# Retrieves context AND passes the original question through, unchanged, in parallel —
# both are needed by the prompt template's next step.
rag_chain = (
    RunnableParallel(
        context=retriever | RunnableLambda(format_docs),
        question=RunnablePassthrough(),
    )
    | prompt
    | model
    | StrOutputParser()
)

answer = await rag_chain.ainvoke("What is the cancellation policy for enterprise plans?")

Reading this the same way the basic chain above was read: RunnableParallel fans the single string input out to two branches — retriever | RunnableLambda(format_docs) (retrieve, then format into a string) and RunnablePassthrough() (return the input unchanged) — runs both, and produces {"context": ..., "question": ...}, which becomes the input dict the prompt template's {context}/{question} placeholders fill from. Every step is still a plain Runnable, so .ainvoke(), .abatch(), and .astream() all work on this composed chain exactly as they did on the three-step chain above — the added structure doesn't cost you any of the uniform-interface benefit.

Provider fallback — the framework's API pattern versus hand-rolled try/except

Part 3.11 and Part 4.2's exercise (section 18) both work through a hand-rolled try/except fallback — genuinely worth building by hand once, since it's what makes explicit exactly what "falling back" mechanically involves (catch an exception, retry against a different model). LangChain also provides this as a first-class Runnable method, .with_fallbacks(), once you understand what it's doing underneath:

primary_model = ChatAnthropic(model="claude-sonnet-4-5")
fallback_model = ChatAnthropic(model="claude-haiku-4-5")  # or a different provider entirely

model_with_fallback = primary_model.with_fallbacks([fallback_model])

# Composes into a chain exactly like any other Runnable
chain = prompt | model_with_fallback | StrOutputParser()
response = await chain.ainvoke({"question": "..."})

.with_fallbacks() accepts a list of alternative Runnables and, on any exception from the primary, tries each fallback in order — mechanically the same try/except-and-retry logic as the hand-rolled version, just packaged as a reusable, composable Runnable method that works uniformly across .invoke(), .batch(), and .stream() without you re-implementing the fallback logic separately for each execution mode. Understanding the hand-rolled version first (Part 3.11, Part 4.2) is still valuable, exactly per this book's teaching philosophy: it's what tells you .with_fallbacks() isn't doing anything conceptually new, just removing the boilerplate of re-implementing the same try/except pattern for every execution mode and every chain. Verify current exact fallback-triggering conditions (which exception types trigger a fallback, whether streaming fallback behavior differs) against current LangChain documentation, since retry/fallback semantics are exactly the kind of detail that can change between versions.

create_agent — the current recommended agent-building path

LangChain 1.0 introduced create_agent (in the langchain package) as the current recommended way to build an agent, replacing the older create_react_agent prebuilt and the AgentExecutor/initialize_agent line entirely. Under the hood, create_agent is built on LangGraph's runtime (Part 5) — meaning what looks like a simple factory function call is, underneath, constructing a graph-based execution engine with LangGraph's state management, checkpointing, and control-flow capabilities available to it, not a separate, simpler agent implementation. This is architecturally important to understand: LangChain's agent layer and LangGraph (Part 5) are not two competing systems — LangChain's high-level agent API is built on top of LangGraph's lower-level graph execution model, which is why capabilities like persistence and human-in-the-loop interrupts (Part 5.3, 5.4) are available to create_agent-built agents, not exclusive to hand-built LangGraph graphs.

The middleware system

A genuinely new architectural piece in LangChain 1.0: a middleware system for the agent loop, passed via a middleware=[...] parameter to create_agent. Middleware are hooks that run at defined points in the agent's execution — before/after the model call, before tool execution starts — supporting retry logic, short-circuiting the loop, and modifying requests/responses as they pass through. Built-in middleware includes human-in-the-loop approval steps, message summarization (directly implementing Part 3.9's short-term memory management as a reusable, drop-in component rather than hand-rolled code), and PII redaction (directly implementing Part 2.3/Part 9.6's redaction discipline as infrastructure). This middleware system is the concrete, current mechanism for adding exactly the production safeguards Part 3.7 argued for (bounded loops, logging, human-in-the-loop checkpoints) without hand-writing the loop yourself — but understanding what each middleware actually does mechanically (per this book's teaching philosophy) matters more than treating it as an unexamined black box.

.content_blocks — provider-agnostic message representation

LangChain 1.0 introduced a standard .content_blocks property on messages — a provider-agnostic representation of reasoning traces, citations, and tool calls, regardless of which underlying provider generated the message. This directly addresses a real pre-1.0 pain point: different providers structure "the model reasoned about X before calling this tool" or "this claim is backed by this citation" differently in their raw API responses, and .content_blocks gives you one consistent way to inspect this content regardless of provider — relevant when you need to display or log a model's reasoning/citations without writing provider-specific parsing code.

5. Simple mental model

Think of langchain-core's Runnable interface as a standard shipping container size: once every component (prompt, model, parser) is packaged in the same-shaped container (implements the same .invoke()/.stream()/.batch() interface), any crane, ship, or truck (the LCEL | composition, .batch(), streaming infrastructure) can handle any combination of them uniformly, without needing custom handling per component type. create_agent being built on LangGraph is like a car's automatic transmission being built on top of the same engine and drivetrain as the manual version — the high-level interface is simpler to operate, but the underlying mechanical capability (the graph execution engine) is the same one available if you popped the hood and worked with it directly.

6. Real-world example

A team maintaining a customer support agent originally built against pre-1.0 AgentExecutor and ConversationBufferMemory finds these still work (via langchain-classic) after upgrading their langchain dependency, but new team members writing new agent code default to create_agent with the middleware system instead — meaning the codebase temporarily has two different agent-construction patterns coexisting. This is a real, common transitional state worth planning for deliberately (a migration plan, not an indefinite split) rather than accumulating both patterns indefinitely as separate, undocumented conventions.

7. Architecture diagram

langchain-coreRunnable interface, LCEL (|), base message types

langchaincreate_agent (on LangGraph runtime, Part 5), middleware system, .content_blocks

langchain-openai

langchain-anthropic

langchain-classiclegacy, backward-compat only: AgentExecutor, initialize_agent, ConversationBufferMemory

langchain-classic is not part of the current stack — shown only as the legacy path to avoid.

8. Production considerations

• Prefer create_agent and the middleware system for new agent code — building new work against langchain-classic patterns means building against a path the framework has explicitly deprecated, with no expectation of future improvement or feature parity with the current recommended path.
• Verify Python 3.10+ across your deployment environments before upgrading to LangChain 1.0 — this is a hard compatibility requirement, not a soft recommendation.
• Pin exact package versions (langchain, langchain-core, provider packages) explicitly in your dependency management (Part 1's uv discussion) — given the recent major restructuring, mismatched versions across these closely-coupled packages can produce confusing import errors.
• Because create_agent sits on top of LangGraph, you get LangGraph's persistence/checkpointing capabilities "for free" — but you still need to explicitly configure a checkpointer (Part 5.3) if you need durability across process restarts; it isn't automatically durable just because the underlying engine supports it.

9. Common mistakes

• Writing new agent code against pre-1.0 tutorials/patterns (AgentExecutor, initialize_agent) found in older blog posts or Stack Overflow answers, unaware that create_agent is now the recommended path.
• Treating create_agent's middleware as a fully opaque black box rather than understanding what each middleware component actually does — Part 3.7's warning against treating "we added an agent framework" as inherently safe applies directly to middleware too; a human-in-the-loop middleware only helps if it's actually configured for the right checkpoints.
• Mixing langchain-classic and current-pattern code within the same codebase indefinitely without a deliberate migration plan.
• Assuming LCEL's | composition is "just Python function composition" and missing that the real value is the uniform .stream()/.batch()/async behavior that comes from the shared Runnable interface underneath.

10. Security considerations

• Every security consideration from Part 3.3/3.7 (tool calling, agents) applies identically here — LangChain's abstractions don't change the underlying fact that your application code, not the model, executes real actions; authorization and validation must still be enforced explicitly in your tool implementations, not assumed from the framework.
• Built-in PII-redaction middleware is a useful, convenient building block, but should be verified against your specific compliance requirements (Part 9.6) rather than assumed to be a complete, certified compliance solution simply because it exists as a framework feature.

11. Performance considerations

• The Runnable interface's uniform .batch() support is a real, concrete performance lever — batching multiple independent LLM calls through .batch() rather than looping over .invoke() calls sequentially can meaningfully improve throughput by taking advantage of concurrent execution under the hood (verify current batching behavior/concurrency limits against the specific model provider integration you're using).

12. Cost considerations

• No LangChain-specific cost beyond the underlying model calls it orchestrates (Part 2.6, Part 3.11's cost framework applies directly) — but middleware like message summarization directly implements Part 3.9's cost-bounding strategy for long conversations, so using it appropriately is a real, concrete cost lever available "for free" as a framework feature rather than custom code you'd otherwise need to write and maintain yourself.

13. When to use it

When you want a standardized, provider-agnostic interface for models/prompts/tools and want to avoid hand-building the orchestration patterns from Part 3 (agent loops, streaming, batching) yourself — which is most production LangChain use cases, given this book's assumption that you're already using it.

14. When NOT to use it

• Extremely simple, single-provider, single-call use cases may not need LangChain's abstraction layer at all — a direct provider SDK call (as used throughout this book's Part 3 code examples) can be simpler and have fewer moving parts for a genuinely minimal use case.
• If your team needs very fine-grained, unusual control over exact request/response handling that the framework's abstractions don't cleanly expose, dropping to the provider SDK directly for that specific piece (while still using LangChain elsewhere) is a legitimate, common pattern rather than fighting the abstraction.

15. Alternatives and trade-offs

Approach	Good for	Weak point
LangChain (current, 1.0+)	Provider abstraction, reusable orchestration patterns, ecosystem integrations	Added abstraction layer; must track a moving, actively-evolving framework
Direct provider SDK (Part 3's examples)	Maximum control, no framework abstraction to learn/track	Reimplement orchestration patterns yourself; provider lock-in without abstraction
LangGraph directly (Part 5), bypassing create_agent	Full control over graph structure for complex/custom agent architectures	More code to write for cases create_agent's middleware system already covers well

16. Practical Python/code example

from langchain.agents import create_agent
from langchain.agents.middleware import SummarizationMiddleware, PIIRedactionMiddleware

# Verify exact current import paths and middleware class names against current
# LangChain documentation before shipping — this is a fast-evolving part of the API.

agent = create_agent(
    model="anthropic:claude-sonnet-4-5",
    tools=[get_order_status_tool],
    middleware=[
        SummarizationMiddleware(max_tokens_before_summary=4000),
        PIIRedactionMiddleware(),
    ],
)

result = await agent.ainvoke({"messages": [{"role": "user", "content": "Where is order #4521?"}]})

This single create_agent call replaces the hand-built bounded loop from Part 3.7's production example — the middleware list is where you attach the production safeguards (context management, PII redaction) that chapter argued for building explicitly.

17. Production-quality example

Composing an LCEL chain that demonstrates the uniform interface benefit from section 4 — one chain definition that supports invoke, batch, and streaming without separate code paths for each:

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_anthropic import ChatAnthropic

prompt = ChatPromptTemplate.from_messages([
    ("system", "Summarize the following support ticket in one sentence."),
    ("user", "{ticket_text}"),
])
model = ChatAnthropic(model="claude-sonnet-4-5")
chain = prompt | model | StrOutputParser()

# Single invocation
summary = await chain.ainvoke({"ticket_text": "Customer reports login failures since yesterday."})

# Batched — runs multiple inputs, taking advantage of concurrent execution underneath
summaries = await chain.abatch([
    {"ticket_text": "Customer reports login failures since yesterday."},
    {"ticket_text": "Customer requests a refund for a duplicate charge."},
])

# Streamed — token-by-token, using the exact same chain definition
async for chunk in chain.astream({"ticket_text": "Customer's account was locked unexpectedly."}):
    print(chunk, end="", flush=True)

The same three-line chain definition supports all three execution modes without any additional code — the concrete, hands-on demonstration of why the Runnable interface's uniformity (section 4) is genuinely valuable, not just architectural elegance for its own sake.

18. Short exercise

Find (via LangChain's current documentation) whether create_agent's middleware system currently includes a rate-limiting or retry-specific middleware built in, or whether that remains something you'd need to implement yourself (potentially adapting Part 3.7/3.3's bounded-loop patterns). Write a short note on what you found and cite where you verified it.

19. Interview questions

• Explain the Runnable interface and why LCEL's | composition provides more than simple function composition.
• What changed architecturally in LangChain 1.0's package structure, and why does it matter which package a given import comes from?
• Why is create_agent built on top of LangGraph rather than being a separate, independent agent implementation?

20. FDE/customer scenario

Customer's engineering team: "We have an existing LangChain-based prototype built a while ago — can we just upgrade the package version to get the new features?"

The honest, correct answer requires checking whether their prototype uses pre-1.0 patterns (AgentExecutor, initialize_agent, ConversationBufferMemory) that now live in langchain-classic — a version bump alone won't migrate them to create_agent and the middleware system automatically, and while the old patterns should continue working via the compatibility package, they won't gain the new middleware capabilities without an active migration. Setting this expectation accurately, rather than promising a painless drop-in upgrade, is a concrete trust-building moment in a real engagement.

---

Key takeaways

• LangChain 1.0 restructured the framework into clearer packages (langchain-core, langchain, provider packages, langchain-classic for legacy patterns) with a stability commitment until 2.0.
• The Runnable interface and LCEL give uniform invoke/batch/stream/async behavior across composed components — the real architectural value, not just syntax sugar.
• create_agent is the current recommended agent-building path, built on top of LangGraph's runtime, with a middleware system replacing hand-rolled production safeguards.

Things you should be able to explain

• Why LCEL composition gives you streaming/batching "for free."
• Why create_agent-built agents have LangGraph's persistence/HITL capabilities available to them.

Things you should be able to build

• An LCEL chain supporting invoke/batch/stream from one definition, and a create_agent-based agent configured with production middleware.

Common mistakes

• Building new code against deprecated langchain-classic patterns from outdated tutorials.
• Treating middleware as an opaque black box without understanding what it actually does.

Recommended next chapter

02-models-messages.md