5.5 — Streaming in LangGraph

1. What is it?

Part 4.6 introduced streaming generally (token-by-token output, .astream()) and flagged that a multi-step graph has multiple distinct things worth streaming: token-level output, and step-level progress updates. This chapter goes into LangGraph's specific streaming modes, which give you fine-grained control over exactly what granularity of information streams out of a running graph.

2. Why does it exist?

A single LLM call has one natural thing to stream: its output tokens (Part 2.6, Part 4.6). A multi-step graph (Part 5.1) has several different, independently useful things a consumer might want streamed: the tokens of whichever LLM call is currently running, the state updates as each node completes, or the full state snapshot after each superstep. LangGraph's distinct streaming modes exist because collapsing all of this into one undifferentiated stream would force every consumer to parse out the granularity they actually want from a mixed firehose — explicit modes let you request exactly the granularity your use case needs.

3. What problem does it solve?

It solves "give a user-facing interface real-time visibility into a multi-step agent/workflow's progress, at whatever level of detail is actually useful for that interface" — a chat UI might want token-level streaming for the final response but step-level status updates ("looking up your order...") for the intermediate tool calls in between, exactly the distinction Part 4.6's astream_events example began to address.

4. How does it work internally?

The stream modes

LangGraph's .astream() accepts a stream_mode parameter with several distinct options (verify the exact current set and names against current documentation, as this is an actively maintained part of the API):

• "values": yields the full state after each superstep — useful when a consumer wants the complete, current picture of the workflow's progress at each step, not just what changed.
• "updates": yields only the partial update each node returned (Part 5.1's partial-update model) after each superstep — more efficient when a consumer only cares about what changed, not the full accumulated state.
• "messages": yields LLM token-level streaming output specifically, for whichever node is currently generating — this is the mode that gives you the token-by-token experience from Part 2.6/4.6, scoped correctly even when it's happening as one step inside a larger multi-step graph.
• "custom": lets a node explicitly emit arbitrary custom events during its own execution (via a callback/writer object passed into the node), for progress reporting that doesn't map cleanly onto state updates or token generation (e.g., "processing document 3 of 10" during a long-running batch step inside one node).

async for chunk in compiled.astream(initial_state, config=config, stream_mode="updates"):
    print(chunk)  # {"classify": {"ticket_category": "billing"}}
                  # {"respond": {"messages": [...]}}

Combining modes for a real UI

A real production interface (Part 4.6's example, made concrete here) typically wants more than one mode simultaneously — status updates for step transitions, plus token streaming for the actual generated text. LangGraph supports streaming multiple modes together (verify the exact current mechanism — a list passed to stream_mode, or the astream_events API from Part 4.6 which unifies multiple event types into one stream), letting a single consuming loop distinguish and handle each type of event appropriately:

async for stream_mode, chunk in compiled.astream(initial_state, config=config, stream_mode=["updates", "messages"]):
    if stream_mode == "updates":
        yield {"type": "status", "detail": summarize_update(chunk)}
    elif stream_mode == "messages":
        yield {"type": "token", "text": chunk.content}

5. Simple mental model

Think of these streaming modes as different camera angles on the same live event: "values" is a wide shot showing the entire stage after every scene change; "updates" is a camera that only cuts to whatever just moved; "messages" is a close-up microphone specifically on whichever actor is currently speaking, capturing their words as they're said rather than waiting for the whole speech; "custom" is a narrator who can cut in with commentary at any point a scene's internal script calls for it. A real broadcast (your production UI) often combines several of these camera feeds into one coherent viewer experience.

6. Real-world example

A customer support UI needs to show: "Checking your order status..." (a step-level status derived from "updates" mode, when the order-lookup node completes) followed by the final response appearing token-by-token as it's generated ("messages" mode, scoped to the final response-generation node) — giving the user continuous, meaningful feedback throughout a multi-second, multi-step agent run, rather than either an opaque single spinner for the whole duration or an overwhelming raw dump of every internal state change.

7. Architecture diagram

Running graph execution

node A (classify)

node B (tool call)

node C (respond)

Consuming UIdistinguishes event types, renders status updates and token stream appropriately

8. Production considerations

• Choose the minimum streaming granularity your UI actually needs — "values" mode streaming full state after every superstep is more bandwidth/processing overhead than "updates" mode for a UI that only needs to know what changed, echoing Part 3.12's "don't include components/granularity beyond what's measured to be needed" discipline applied specifically to streaming verbosity.
• Handle stream interruption/reconnection gracefully — a user's connection dropping mid-stream shouldn't lose the underlying graph's progress (this is exactly what Part 5.3's checkpointing separately guarantees) even if that specific streaming connection needs to be re-established.
• Be deliberate about what state/update content is safe to stream directly to a user-facing client — internal state fields not meant for end-user visibility (e.g., raw fraud-check scores in the loan-review example) shouldn't be indiscriminately streamed via "values"/"updates" mode without filtering, echoing Part 1.2's citation-metadata leakage caution.

9. Common mistakes

• Using "values" mode (full state every superstep) when "updates" mode (just the deltas) would suffice, adding unnecessary bandwidth and client-side processing overhead.
• Streaming raw internal state fields directly to an end-user client without filtering out data not meant for that audience.
• Not handling the case where a client disconnects mid-stream, potentially causing the client-side UI to appear stuck or to lose track of an otherwise-still-progressing (and checkpointed) graph run.

10. Security considerations

• Every field streamed via "values"/"updates" mode is now visible to whatever client is consuming the stream — apply the same data-minimization discipline as any other API response (Part 1.2, Part 9.6): stream only what the specific audience is authorized to see, not the graph's full internal state indiscriminately.

11. Performance considerations

• "messages" mode's token-level streaming is what actually delivers the perceived-latency benefit from Part 4.6/2.6 — using only coarser modes ("values"/"updates") for the final response text would lose this specific benefit even though it correctly shows step-level progress.
• Streaming adds some implementation complexity/overhead on both the producing (server) and consuming (client) side compared to a single non-streamed response — worth the trade-off specifically where perceived latency or step-visibility genuinely matters to the use case (Part 4.6, section 14).

12. Cost considerations

• No direct LLM-token cost difference between streamed and non-streamed generation for the same underlying content (Part 4.6, section 12) — streaming mode choice is a latency/UX/bandwidth trade-off, not a cost-per-token trade-off.

13. When to use it

Any user-facing multi-step agent/workflow where visibility into progress (not just the final result) meaningfully improves the user experience — most interactive, potentially multi-second-or-longer agent interactions benefit from at least step-level status streaming, and any user-facing text generation benefits from token-level streaming.

14. When NOT to use it

Backend, non-interactive batch processing (nothing is waiting on live progress visibility) doesn't need streaming at all — a plain .ainvoke() call is simpler and sufficient (Part 4.6, section 14's guidance applied to the graph level).

15. Alternatives and trade-offs

Stream mode	Good for	Weak point
"values"	Full state visibility at every step	More bandwidth/processing than often needed
"updates"	Efficient, only what changed	Consumer must reconstruct full picture if needed
"messages"	Real token-level output for perceived latency	Only covers LLM-generation steps, not other node types
"custom"	Fine-grained progress from within a single long-running node	Requires explicit instrumentation inside that node

16. Practical Python/code example

async def stream_support_interaction(compiled_graph, initial_state: dict, config: dict):
    """
    Streams both step-level status updates and token-level response generation
    for a support workflow, distinguishing the two for the consuming UI.

    Args:
        compiled_graph: A compiled LangGraph graph.
        initial_state (dict): The initial state to run with.
        config (dict): Graph invocation config, including thread_id.

    Yields:
        dict: {"type": "status", "detail": ...} or {"type": "token", "text": ...}
    """
    async for mode, chunk in compiled_graph.astream(
        initial_state, config=config, stream_mode=["updates", "messages"]
    ):
        if mode == "updates":
            for node_name, update in chunk.items():
                yield {"type": "status", "detail": f"Completed step: {node_name}"}
        elif mode == "messages":
            message_chunk, metadata = chunk
            if message_chunk.content:
                yield {"type": "token", "text": message_chunk.content}

17. Production-quality example

Adding data-minimization filtering before streaming state updates to an end-user client, directly addressing section 8/10's security consideration:

SAFE_TO_STREAM_FIELDS = {"ticket_category", "final_recommendation"}


async def stream_filtered_status(compiled_graph, initial_state: dict, config: dict):
    """
    Streams status updates, filtering out internal fields not meant for
    direct end-user visibility (e.g., raw fraud-check scores).

    Args:
        compiled_graph: A compiled LangGraph graph.
        initial_state (dict): The initial state to run with.
        config (dict): Graph invocation config, including thread_id.

    Yields:
        dict: Filtered status updates safe for end-user display.
    """
    async for chunk in compiled_graph.astream(initial_state, config=config, stream_mode="updates"):
        for node_name, update in chunk.items():
            safe_fields = {k: v for k, v in update.items() if k in SAFE_TO_STREAM_FIELDS}
            if safe_fields:
                yield {"type": "status", "node": node_name, "data": safe_fields}

18. Short exercise

A fraud-investigation graph's internal state includes a fraud_risk_score field that should never be shown directly to the applicant, but a status field ("under review," "approved") that should be. Using the filtering pattern from section 17, write the SAFE_TO_STREAM_FIELDS set you'd use for a customer-facing status page, and explain what would go wrong if fraud_risk_score were accidentally included.

19. Interview questions

• Explain the difference between "values" and "updates" streaming modes and when you'd choose one over the other.
• Why does "messages" mode exist as a separate concept from "updates" mode, given that a node's LLM call result would eventually appear in an "updates" event too?
• What data-minimization risk does streaming raw graph state introduce that a normal, single, filtered API response wouldn't have by default?

20. FDE/customer scenario

Customer: "Our AI assistant's UI just shows a spinner for 8 seconds during a multi-step lookup — customers think it's frozen."

This is a direct, fixable UX problem using this chapter's mechanism: adding "updates"-mode-driven status messages ("Checking your account...", "Looking up your order...") transforms the experience from an opaque, anxiety-inducing spinner into a legible, trust-building progress indicator — a concrete, relatively low-effort improvement with real, measurable impact on perceived quality (Part 15's business-impact framing applies directly: perceived responsiveness is a real, measurable driver of user trust and satisfaction, not just a cosmetic nicety).

---

Key takeaways

• LangGraph's distinct streaming modes (values, updates, messages, custom) let you request exactly the granularity of live visibility your consuming interface needs, rather than parsing a one-size-fits-all firehose.
• Real production UIs typically combine modes (step-level status plus token-level generation) for the best user experience.
• Streamed state/updates must be filtered for data minimization, exactly like any other API response surface.

Things you should be able to explain

• The distinction between values, updates, and messages streaming modes.
• Why combining status-update streaming with token streaming produces a better UX than either alone.

Things you should be able to build

• A combined-mode streaming consumer with explicit field-level filtering for end-user-safe status updates.

Common mistakes

• Streaming full internal state indiscriminately without filtering for the target audience.
• Using a coarser streaming mode than the use case actually needs.

Recommended next chapter

06-subgraphs-multi-agent.md