6.1 — LangSmith Tracing and Observability

1. What is it?

LangSmith is Anthropic-independent, LangChain-affiliated tooling for observability, evaluation, and monitoring of LLM applications. Tracing is its core capability: automatically capturing a structured, hierarchical record of everything that happened during one execution of a LangChain/LangGraph application — every LLM call, every tool call, every retrieval, with inputs, outputs, timing, and token usage — without you writing manual logging code for each of these events.

2. Why does it exist?

Part 4.6 established that callbacks are LangChain's general instrumentation layer, firing at every component's lifecycle events. Part 2.6 and Part 3.7 both established that debugging a non-deterministic, potentially multi-step LLM system requires seeing exactly what happened at each step — which model was called with what exact prompt, what a tool returned, how many tokens were used, where a multi-step agent's reasoning went wrong. LangSmith exists to capture all of this automatically (by registering itself as a callback handler, Part 4.6, section 4) and present it as a navigable, searchable trace, rather than requiring you to build this observability infrastructure yourself from raw application logs.

3. What problem does it solve?

It solves "how do I actually see what a complex, multi-step LLM application did, in enough detail to debug a specific failure or understand a specific behavior" — directly extending Part 3.7's "log every step of an agent's reasoning" production consideration and Part 5.8's observability checklist item into a purpose-built tool rather than hand-rolled logging.

4. How does it work internally?

The trace/run data model

LangSmith's core data model has two levels: a trace is the full, end-to-end record of one complete execution (e.g., one user's chat turn, including every LLM call, tool call, and retrieval that occurred while handling it); a run is one individual unit of work within that trace (a single LLM call, a single tool execution, a single retriever query). A trace is a tree of runs — a top-level run (the overall chain/agent invocation) with nested child runs representing each step, mirroring the actual call structure of your LangChain/LangGraph application.

Trace: "handle_support_message"top-level run

Run: classify_ticketLLM call

Run: get_order_statustool call

Run: generate_responseLLM call · tokens in=245/out=38 · latency 890ms

Run: db_lookup_ordernested function call, if instrumented

A single trace is capped at 25,000 runs — worth being aware of for extremely deep or long-running agent loops (Part 3.7, Part 5.6's multi-agent systems with many nested sub-agent calls), though this is a very high ceiling that the overwhelming majority of applications won't approach; it's more relevant as a signal that an agent loop exceeding it likely has a runaway-loop problem (Part 3.7, section 8's bounded-iteration discipline) worth investigating on its own terms regardless of the trace limit.

How tracing attaches with (close to) zero code changes

Because LangSmith integrates via the callback mechanism (Part 4.6), enabling it for an existing LangChain/LangGraph application is typically a matter of setting environment variables (an API key, a project name, a tracing-enabled flag) rather than instrumenting every component by hand — the same callback events every BaseTool, BaseChatModel, and chain already emits (Part 4.6, section 4) are what LangSmith's registered handler captures into the trace structure automatically. This is the concrete payoff of Part 4.6's callback architecture: observability infrastructure built once, benefiting from any component's instrumentation without that component needing to know LangSmith exists.

What a trace captures, concretely, that raw logs typically don't

• The exact prompt sent to the model at each step, including the fully-resolved ChatPromptTemplate output (Part 4.3) — not just "we called the model," but precisely what it was asked.
• Token usage per call (Part 4.2's response_metadata), aggregated automatically across an entire trace — giving you the exact cost-attribution data Part 3.11/7.10's cost-optimization framework depends on, without hand-building this aggregation.
• Nested timing, showing exactly which step in a multi-step trace consumed the most latency — directly answering the "where's the bottleneck" questions from Part 3.12/5.8's performance considerations, empirically rather than by guessing.
• .content_blocks (Part 4.2) — reasoning traces, citations, and tool calls in their standardized form, viewable directly in the trace UI regardless of provider.

Diagnosing a stuck or looping graph

Section 4's 25,000-run trace cap is an implicit, after-the-fact signal that something looped. In practice, you want to catch a runaway LangGraph agent (Part 5.7's recursion_limit/GraphRecursionError) while looking at the trace itself, not just infer it from a limit having been hit. A looping trace has a distinctive, recognizable shape in the LangSmith UI:

• Repeated sibling runs with an identical or near-identical tool-call pattern — the same node (e.g., a supervisor routing back to the same sub-agent, Part 5.6) appearing many times in a row at the same tree depth, each with a very similar input, rather than the trace tree branching forward into new kinds of work.
• Monotonically increasing latency across those repeated runs — each successive iteration often takes slightly longer (growing context/state being re-processed each pass), producing a visibly worsening latency trend within a single trace rather than a flat, expected-cost step sequence.
• The trace terminating at (or being cut off by) a run-count ceiling — either LangGraph's own recursion_limit (Part 5.7) raising GraphRecursionError, visible in the trace as an error on the top-level run, or, in an extreme, unbounded case, the trace approaching LangSmith's own 25,000-run cap from section 4.

How to find one: filter the traces list for a project by run count (sort descending, or filter for traces above a threshold well beyond your graph's expected legitimate maximum superstep count, Part 5.7 section 4) and by latency outliers (sort descending by total trace duration) — a trace that's both unusually deep in run count and an outlier on total latency is the concrete signature of a loop, as opposed to a trace that's merely slow because of one expensive step. Filtering by error type (specifically GraphRecursionError, if you've let the limit do its job rather than letting the loop run unbounded) narrows this further to traces that actually hit the ceiling, rather than ones still silently accumulating runs below it.

Connecting this to GraphRecursionError (Part 5.7): when a graph's recursion_limit is hit, the exception surfaces as an error on that specific trace's top-level run — clicking into that trace and walking down to where the repeated sibling pattern begins identifies exactly which node's conditional routing decision (Part 5.2) kept sending execution back into the loop, which is the actual fix target (a missing or wrong exit condition), not the recursion_limit value itself. Treat a GraphRecursionError you see in LangSmith as a pointer to a specific routing bug, not as evidence the limit was simply set too low — raising the limit without fixing the underlying routing condition just delays and enlarges the same cost incident.

5. Simple mental model

A LangSmith trace is like an airplane's flight data recorder, automatically capturing every instrument reading throughout a flight — you don't need to have anticipated exactly what might go wrong ahead of time and specifically instrumented for it; the recorder captures a comprehensive record of what actually happened, so that after any anomaly (or just for routine review), you can go back and reconstruct exactly what occurred at every point, in full fidelity, rather than relying on a pilot's imperfect memory (or a hand-written log that happened not to capture the specific detail that turned out to matter).

6. Real-world example

A customer reports that the support agent gave a factually wrong answer about their account status. Without tracing, diagnosing this requires guessing: was retrieval bad (Part 3.5)? Did the model ignore good context (Part 3.5, section 4's faithfulness failure)? Did a tool call return wrong data? With a LangSmith trace for that specific interaction (found by searching for the customer's session/thread ID, Part 5.3's deliberate thread_id design paying off here), the team can see, in order: exactly what the retrieval step returned, exactly what tool calls were made and what they returned, and exactly what prompt the final generation step received — collapsing what might have been hours of speculative debugging into minutes of direct observation.

7. Architecture diagram

Your LangChain/LangGraph applicationevery component (model, tool, retriever, chain) emits callback events during execution (Part 4.6)

LangSmith (hosted service)assembles callback events into a structured trace tree — nested runs, inputs/outputs, tokens, timing

Searchable, navigable trace UIdebugging, cost analysis, and eval dataset curation directly from production traces (Part 6.2)

8. Production considerations

• Enable tracing in production, not just during development — the debugging value (section 6) is highest precisely for production incidents you couldn't have anticipated and specifically logged for in advance; development-only tracing misses exactly the cases you'll most need it for.
• Be deliberate about what's captured in traces given data sensitivity (Part 9.6) — traces capture full inputs/outputs by default, meaning sensitive customer data flows into LangSmith's storage; understand and configure LangSmith's data handling (redaction options, retention settings, self-hosting options if available and required for your compliance posture) explicitly rather than assuming default settings meet your specific requirements — verify current options against current LangSmith documentation.
• Use meaningful metadata/tags on traces (e.g., tenant ID, user ID, feature name) — this is what makes traces searchable and filterable at scale, exactly the way Part 4.6's practical example tagged calls with tenant:{tenant_id}.
• Monitor aggregate cost/latency trends via traces, not just individual-incident debugging — the same data that helps debug one bad interaction also aggregates into the cost/performance dashboards Part 7/8.4 will build on.

9. Common mistakes

• Only enabling tracing in development/staging, leaving production genuinely un-debuggable when a real incident occurs there specifically.
• Not tagging traces with enough metadata (tenant, user, feature) to make them findable later for a specific reported incident — a trace that exists but can't be located when needed provides little practical value.
• Not considering data sensitivity/retention implications before enabling full tracing on a system handling regulated data (Part 9.6, Part 10.5) — tracing infrastructure is itself a data-handling system subject to the same compliance scrutiny as any other.
• Trace volume silently exploding cost: a runaway agent loop (Part 5.7's recursion_limit gap) generates thousands of runs in a single trace, and a high-QPS endpoint with full, untagged tracing enabled on every request generates trace-ingestion volume that scales linearly with traffic with no way to distinguish "worth keeping in full detail" from "routine, low-value" traces — either can spike trace-ingestion cost/volume sharply and unexpectedly, showing up as a billing surprise rather than a caught-early signal. Mitigate with a trace sampling rate (distinct from Part 6.3's online-evaluator sampling, which decides what fraction of already-captured traces get evaluated — this is about what fraction of traffic gets traced/ingested at all), redaction of large payloads (truncating or omitting oversized inputs/outputs rather than ingesting them in full), and deliberate retention tuning (shorter retention for high-volume, low-value trace categories, full retention for the traces most likely to matter later). Verify current sampling/redaction/retention configuration options against current LangSmith documentation.

10. Security considerations

• Trace data is a concentrated record of exactly what your system processed — including, potentially, PII, financial details, or other sensitive content passed through prompts and tool calls (Part 9.6). Treat LangSmith's trace storage with the same access-control and data-handling rigor as any other system holding this data, and verify its specific data-residency/compliance posture against your customer's actual requirements before enabling full tracing on sensitive workloads.
• Traces containing tool arguments/results (Part 3.3) can reveal internal system details (database schemas, internal API shapes) to anyone with trace access — scope who can view traces appropriately within your organization, exactly as you would for application logs containing similar detail.

11. Performance considerations

• Tracing overhead itself is generally small relative to LLM call latency (the bottleneck is almost always the model call, not the instrumentation), but it's worth verifying this assumption under your specific load characteristics rather than assuming it's negligible without measurement.

12. Cost considerations

• LangSmith itself typically has usage-based pricing tied to trace volume (verify current pricing/tiers against current LangSmith documentation) — a real, distinct cost line item from LLM API costs, worth budgeting for explicitly at production scale, though generally a small fraction of total AI system cost given the debugging/reliability value it provides.
• This cost line item is not always small and predictable — section 9's runaway-loop and high-QPS-untagged-tracing scenarios can spike it sharply within a single billing period; a deliberate sampling rate and retention policy (section 9) turns trace-ingestion cost back into a controlled, budgetable line item rather than one that tracks worst-case traffic/failure scenarios by default.

13. When to use it

Any LangChain/LangGraph application beyond a trivial prototype — the debugging and cost-visibility value is high enough, and the setup cost low enough (given the callback-based zero-code-change integration), that this should be close to a default for any production or near-production system.

14. When NOT to use it

A genuinely one-off exploratory script with no expectation of debugging a specific production incident later has little need for persistent tracing infrastructure — though even here, ad hoc local debugging via callbacks (Part 4.6) remains useful without necessarily needing a hosted tracing service.

15. Alternatives and trade-offs

Approach	Good for	Weak point
LangSmith tracing	Purpose-built, automatic, rich structured traces for LangChain/LangGraph specifically	Hosted-service dependency; data-sensitivity considerations to manage explicitly
Generic APM/observability tooling (e.g., general-purpose tracing platforms)	Broader, non-LLM-specific observability across your whole stack	Less LLM-specific structure (prompts, tokens, tool calls) out of the box
Hand-rolled logging (Part 1's structured logging discipline)	Full control, no new dependency	Significant effort to reach parity with purpose-built LLM tracing

16. Practical Python/code example

import os

# Minimal setup — tracing attaches via environment configuration, not code changes,
# because it operates through the existing callback mechanism (Part 4.6).
os.environ["LANGSMITH_TRACING"] = "true"
os.environ["LANGSMITH_API_KEY"] = "your-api-key"  # from a secrets manager, Part 9.5, never hardcoded
os.environ["LANGSMITH_PROJECT"] = "support-agent-production"

# Verify current environment variable names against current LangSmith documentation
# — this naming has evolved (e.g., from a LANGCHAIN_-prefixed scheme) across versions.

# Existing application code needs no changes — every LangChain/LangGraph call
# is automatically traced once these are set.
result = await agent.ainvoke({"messages": [{"role": "user", "content": "Where is my order?"}]})

17. Production-quality example

Adding explicit, meaningful trace metadata for searchability, addressing section 8/9's tagging recommendation directly:

from langsmith import trace  # verify exact current API surface against current docs


async def handle_support_message(user_message: str, user, agent) -> str:
    """
    Handles a support message, tagging the resulting trace with metadata that
    makes it findable during a later debugging session for this specific user/tenant.
    """
    result = await agent.ainvoke(
        {"messages": [{"role": "user", "content": user_message}]},
        config={
            "metadata": {
                "tenant_id": user.tenant_id,
                "user_id": user.id,
                "feature": "support_agent",
            },
            "tags": [f"tenant:{user.tenant_id}"],
        },
    )
    return result["messages"][-1].content

This is the exact same tagging pattern from Part 4.6's production example, now explicitly framed as the mechanism that makes a specific customer's reported incident (section 6) findable in LangSmith's trace search, rather than an anonymous, unsearchable mass of traces.

18. Short exercise

A customer reports a specific bad interaction that occurred "sometime yesterday afternoon." Using the metadata-tagging pattern from section 17, describe exactly what search you'd run in LangSmith to locate the specific trace, and what additional metadata field (not shown in the example) would make this search faster or more precise.

19. Interview questions

• Explain the trace/run data model and why a trace is structured as a tree rather than a flat list.
• What mechanism lets LangSmith capture detailed traces without requiring manual instrumentation of every component?
• What data-sensitivity considerations should you evaluate before enabling full tracing on a production system handling regulated data?
• What does a looping/stuck LangGraph agent's trace look like in the LangSmith UI, and how would you filter traces to find it?
• Why can trace-ingestion cost spike sharply and unexpectedly, and what specific controls would you put in place to prevent it?

20. FDE/customer scenario

Customer: "We can't debug why our AI assistant sometimes gives strange answers — we just see the final response, not what led to it."

This is precisely the observability gap tracing exists to close (section 2, 6). Recommending LangSmith (or equivalent tracing) as a near-immediate, low-effort addition — given the callback-based integration requires minimal code change — is a concrete, high-leverage recommendation, though it should come paired with an explicit conversation about what data sensitivity considerations (section 9/10) apply to their specific customer data before enabling full-detail tracing on their production traffic.

---

Key takeaways

• LangSmith tracing captures a structured tree of runs (LLM calls, tool calls, retrievals) per trace, automatically, via the same callback mechanism every LangChain/LangGraph component already emits.
• This closes the "what actually happened during this specific interaction" debugging gap that's otherwise close to impossible to answer after the fact for a non-deterministic, multi-step system.
• Trace data is sensitive data — apply the same data-handling and access-control rigor as any other system capturing full request/response content.
• A stuck/looping graph has a recognizable trace shape (repeated sibling runs, monotonically increasing latency, hitting a run-count ceiling) that you can filter for directly (by run count and latency outliers), and a GraphRecursionError (Part 5.7) surfaces in the trace as a pointer to the specific looping node's routing bug, not as evidence the limit itself was wrong.
• Untracked trace volume — from a runaway loop or from full, untagged tracing on high-QPS traffic — can spike ingestion cost sharply; sampling rate, payload redaction, and retention tuning are the controls that keep this a budgeted cost rather than a surprise.

Things you should be able to explain

• The trace/run data model and how it's automatically populated via callbacks.
• Why tracing should be enabled in production, not just development.
• What a looping graph's trace looks like and how to filter for it; how trace-ingestion cost can spike and how to control it.

Things you should be able to build

• A tagged, metadata-rich tracing configuration that makes a specific customer's incident findable later.

Common mistakes

• Tracing only in development, leaving production incidents undebuggable.
• Insufficient trace metadata/tagging, making traces hard to locate later.
• Not evaluating data-sensitivity implications before enabling full tracing.
• No trace sampling/redaction/retention strategy, leaving trace-ingestion cost exposed to runaway loops or high-QPS traffic spikes.

Recommended next chapter

02-datasets-and-evaluations.md