3.7 — Agents and Workflows

1. What is it?

An agent is a system where an LLM decides, at runtime, what steps to take (which tools to call, in what order, whether to continue or stop) to accomplish a goal — the control flow is dynamic and determined by the model. A workflow, by contrast, is a system where the sequence of steps is fixed by your code ahead of time, and the LLM is invoked at specific, predetermined points within that fixed sequence. This distinction — who decides the control flow, the model or your code — is the single most important architectural decision in this entire chapter, and conflating the two is the source of a huge amount of unnecessary complexity and unreliability in production AI systems.

2. Why does it exist?

Part 3.3 established that tool calling lets a model request actions and continue reasoning with their results. The natural next question is: what decides the sequence of these tool calls? For many real problems, the sequence is actually fixed and known in advance ("first classify the ticket, then look up the customer's account, then draft a response") — and hardcoding that sequence in your own code, invoking the LLM only for the parts that genuinely need judgment, is more reliable, more debuggable, and cheaper than letting the model figure out the sequence itself every time. For other problems, the sequence genuinely can't be known in advance (open-ended research, multi-step troubleshooting where each step depends unpredictably on the last) — and that's precisely where agents, letting the model drive control flow, earn their added complexity and unpredictability.

3. What problem does it solve?

Agents solve: "the right sequence of actions to take can't be determined ahead of time; it genuinely depends on what's discovered at each step." Workflows solve the more common: "I know the steps, I just need an LLM's judgment at certain specific points" — and workflows solve it far more reliably, because a fixed, code-defined sequence can't go off the rails the way an open-ended, model-driven loop can.

This chapter's central, repeated argument (echoed across most current serious engineering guidance on this topic, including Anthropic's own published guidance on building effective agents) is: default to a workflow; escalate to an agent only when the task's genuine unpredictability requires it. Most production systems marketed as "AI agents" are, on inspection, actually well-structured workflows with one or two points of real LLM judgment — and that's usually the right design, not a lesser one.

4. How does it work internally?

The agent loop, mechanically

An agent is, underneath, the bounded tool-calling loop from Part 3.3, section 17 — the only thing that makes it "agentic" rather than "a single tool call" is that the loop continues, with the model deciding at each iteration whether to call another tool or produce a final answer, until it decides it's done (or your code's iteration cap is hit):

loop:
    model receives: original goal + full history of tool calls and results so far
    model decides: call another tool, OR produce a final answer
    if tool call: your code executes it, appends result to history, loop continues
    if final answer: loop ends

Nothing structurally new is happening here versus Part 3.3 — what changes is that the number and sequence of iterations is fully open-ended and determined by the model's own judgment about whether it has enough information to stop, rather than your code deciding "call tool A, then tool B, then finish."

The ReAct pattern

Most agent implementations use some variant of the ReAct (Reasoning + Acting) pattern: at each step, the model is prompted (explicitly or implicitly, depending on the framework) to reason about what it knows so far and what it needs next, before deciding on an action. This interleaving of explicit reasoning with action — rather than jumping straight to the next tool call — measurably improves the quality of multi-step decision-making, for the same reason chain-of-thought prompting improves single-turn reasoning (Part 3.1): generating intermediate reasoning tokens gives the model more computation to work with before committing to a decision.

Thought: The user wants their order's delivery status, but I don't have the order ID yet.
Action: ask_user_for_order_id()
Observation: order_id = "4521"
Thought: Now I can look up the order.
Action: get_order_status(order_id="4521")
Observation: {"status": "in_transit", "eta": "2026-09-04"}
Thought: I have enough information to answer.
Final answer: Your order #4521 is in transit, expected to arrive 2026-09-04.

Workflow patterns — the underused, more reliable alternative

Several well-established workflow patterns handle a large fraction of real "AI agent" requirements without ever giving the model open-ended control:

• Prompt chaining: a fixed sequence of LLM calls, each step's output feeding the next (e.g., generate a draft → critique the draft → revise based on critique) — control flow is fully fixed by your code; only the content of each step is LLM-generated.
• Routing: an LLM call classifies the input, and your code then dispatches to a specific, predetermined handler based on that classification (e.g., classify a support ticket into billing/technical/account, then route to a category-specific fixed workflow) — the LLM's judgment is used for one decision point, not for driving the entire sequence.
• Parallelization: multiple LLM calls run concurrently on different aspects of the same task, then their results are combined by code (e.g., simultaneously check a document against three separate compliance criteria, then combine the three verdicts) — Part 1.1's asyncio.gather pattern applies directly here.
• Orchestrator-worker: a central LLM call breaks a complex task into subtasks and dispatches them (to other LLM calls or tools), then synthesizes their results — this sits closer to the agent end of the spectrum since the orchestrator's decomposition is itself dynamic, but each worker's task is often bounded and well-defined.
• Evaluator-optimizer: one LLM call generates a candidate solution, another (or the same, in a second pass) evaluates it against explicit criteria and provides feedback, looping until the evaluator is satisfied or an iteration cap is hit — a fixed shape of loop, even though the number of iterations may vary.

Every one of these patterns keeps control flow in your code — the LLM's non-determinism is contained to the content it produces at each fixed step, not the sequence of steps itself, which is exactly why they're more reliable and more debuggable than a fully open-ended agent for tasks whose real structure is actually fixed and knowable in advance.

Why unbounded agents are genuinely riskier

An agent's open-ended loop means each additional step compounds the chance of the whole run going wrong: if each step has even a small independent probability of a mistake (a wrong tool call, a misinterpreted result), the probability that a long multi-step run stays entirely correct end-to-end shrinks multiplicatively with the number of steps. A 10-step agent run where each step is 95% reliable has, roughly, a 0.95^10 ≈ 60% chance of the entire run completing without a single misstep — a sobering, concrete reason why minimizing the number of steps genuinely left to model judgment (by using workflows wherever the sequence is actually fixed) is a real reliability strategy, not just an engineering preference.

5. Simple mental model

A workflow is a recipe with a few "taste and adjust" steps — the sequence of steps (preheat oven, mix ingredients, bake for X minutes) is fixed by the recipe; the cook's judgment is only invoked at specific points ("taste the sauce, add more salt if needed"). An agent is handing the same cook a stocked kitchen and a goal ("make a satisfying dinner for a vegetarian") with no recipe at all — the cook decides everything: what to make, in what order, tasting and adjusting throughout. The agent approach is more flexible and can handle situations no recipe anticipated, but it's also far more likely to go wrong in ways a fixed recipe never would, and far harder to predict how long it'll take or exactly what you'll get.

6. Real-world example

A financial services company initially built a "loan application review agent" that could freely decide, at each step, whether to pull credit data, verify income documents, check for fraud flags, or ask the applicant a clarifying question — in whatever order it judged best. In production, this produced inconsistent review sequences across similar applications (some reviews checked fraud flags first, some last), made auditing individual decisions difficult (the sequence of checks performed differed application to application, complicating a regulator's ability to verify a consistent process was followed), and occasionally skipped a check entirely because the model judged (incorrectly) that it had enough information already. Restructuring this as a workflow — a fixed sequence (credit check → income verification → fraud check → LLM-driven synthesis of findings into a recommendation) — fixed all three problems: the sequence became fully predictable and auditable, every check was guaranteed to run every time, and the LLM's genuine judgment was reserved for the one step (synthesizing findings into a recommendation) that actually needed it, rather than for deciding the entire process.

7. Architecture diagram

Workflow (fixed control flow, LLM judgment at specific points — sequence is always the same, every run):

Step 1fixed, deterministic or LLM

Step 2fixed, LLM call

Step 3fixed, deterministic tool call

LLM: synthesize findingsinto a recommendation (LLM judgment)

Agent (dynamic control flow — the model sees goal + full history and decides which tools to call and when; sequence, and even which steps run at all, can differ every run):

Agent loopbounded by max_iterations

8. Production considerations

• Default to a workflow. Justify an agent, don't default to one. This is the single highest-leverage architectural decision in this chapter — before building an agent, explicitly ask whether the task's actual sequence of steps is knowable in advance (workflow) or genuinely depends on what's discovered along the way (agent justified).
• Bound every agent loop with a hard maximum iteration count (Part 3.3, section 8) — non-negotiable in production, since an unbounded agent loop is an unbounded cost and reliability risk.
• Log every step of an agent's reasoning and actions (this is exactly what LangSmith tracing, Part 6.1, is built to capture) — without this, debugging why a specific agent run produced a specific (wrong) outcome is close to impossible after the fact, since the sequence itself was dynamic and undocumented outside the trace.
• For workflows, keep the fixed steps in version-controlled code (Part 1.7), not implicit in a prompt — the whole value of a workflow is its predictability and auditability, which requires the sequence to be explicit and reviewable.
• Add human-in-the-loop checkpoints (Part 5.4, Part 8.5) at high-stakes decision points, in both workflows and agents — for irreversible or high-cost actions, a pause for human confirmation is often the right design regardless of whether the surrounding system is agentic.

9. Common mistakes

• Building a fully open-ended agent for a task whose steps are actually fixed and well-understood, incurring unnecessary unpredictability, cost, and debugging difficulty for no real benefit (the loan-review example in section 6).
• No iteration cap, leading to runaway cost or, worse, an agent stuck in an unproductive loop repeatedly calling the same tool.
• Treating "we built an agent" as inherently more sophisticated or higher-quality than "we built a workflow" — the FDE-correct framing is that a well-designed workflow that reliably does exactly what's needed is a better engineering outcome than an agent that does the same thing less predictably.
• Not distinguishing, in postmortems, between "the agent's reasoning was sound but the tool result was wrong" and "the agent's reasoning itself was flawed" — these require entirely different fixes (Part 8 evaluation should separate these).

10. Security considerations

• Every security consideration from Part 3.3 (tool calling) applies with amplified stakes to agents, precisely because the sequence and number of tool calls is no longer fixed or predictable — an agent has more opportunities across a longer, less predictable run for a manipulated or confused step to cause harm than a single tool call does.
• Excessive agency (Part 9.2) is specifically an agent-architecture risk: the more autonomy and tool access an agent has, the larger the blast radius of any single bad decision in its loop — this is a direct, load-bearing reason to prefer workflows (bounded, predictable tool usage) over agents wherever the task allows it.
• Multi-step agent runs that incorporate results from external tools/documents at intermediate steps compound the indirect-prompt-injection surface (Part 9.1) — each step is a fresh opportunity for injected content encountered along the way to influence the rest of the run.

11. Performance considerations

• Agent runs have unpredictable latency (an unknown number of steps until completion) compared to workflows (a fixed, predictable number of LLM calls) — this matters directly for setting realistic latency expectations with customers and for capacity planning.
• Parallelizable workflow patterns (the "parallelization" pattern in section 4) can be meaningfully faster than an equivalent agent that would otherwise reason about the same subtasks sequentially, one at a time.

12. Cost considerations

• Agent cost is unbounded until you impose a cap — each additional loop iteration is a full additional LLM call with the accumulated conversation history (Part 3.3, section 12's cost note applies directly and compounds with iteration count).
• Workflows have predictable, budgetable cost (a fixed number of LLM calls per run, known ahead of time) — a real, concrete advantage worth stating plainly to a customer weighing the two approaches, since "hard to predict our AI costs" is a common enterprise pain point (Part 7.10, Part 15).
• Prompt caching (Part 3.3, section 12) is the single highest-leverage mitigation for the agent-loop cost above. Since every loop iteration resends the same system prompt, tool definitions, and accumulated history (section 4), a cache breakpoint on that stable prefix means only the newest tool result and the model's newest response are billed at full, uncached price on each iteration — the round-trip count is unchanged, but the per-round-trip cost drops substantially. This complements, rather than replaces, the hard iteration cap from section 8: caching makes each additional iteration cheaper, it doesn't make an unbounded number of iterations safe.

13. When to use it (agents specifically)

Tasks where the sequence of needed actions genuinely can't be determined ahead of time: open-ended research/investigation, multi-step troubleshooting where each step's findings determine the next action, or tasks where the space of possible needed tools/actions is too large or varied to enumerate as a fixed workflow.

14. When NOT to use it (agents specifically)

Any task whose steps are actually knowable and consistent across runs — which describes a large fraction of real enterprise automation needs. Use a workflow instead, reserving LLM judgment for the specific steps that genuinely need it.

15. Alternatives and trade-offs

Approach	Good for	Weak point
Workflow (fixed control flow)	Predictable, auditable, bounded cost/latency, easier to debug	Can't adapt to situations the fixed sequence didn't anticipate
Agent (dynamic control flow)	Handles genuinely unpredictable, open-ended tasks	Unpredictable cost/latency, harder to debug, larger blast radius per run
Single tool call (Part 3.3, no loop)	Simplest, most predictable, for single-step needs	Can't handle any task requiring multiple dependent steps

16. Practical Python/code example

A routing workflow — one of the most common, underrated production patterns, keeping control flow fixed while using an LLM for the one genuinely judgment-requiring step (classification):

from enum import Enum


class TicketCategory(str, Enum):
    """Fixed set of ticket categories the workflow routes on."""
    BILLING = "billing"
    TECHNICAL = "technical"
    ACCOUNT = "account"


async def classify_ticket(client, ticket_text: str) -> TicketCategory:
    """
    Classifies a support ticket into one of a fixed set of categories.

    Args:
        client: An async LLM client.
        ticket_text (str): The raw ticket text.

    Returns:
        TicketCategory: The classified category.
    """
    response = await client.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=10,
        system=f"Classify into exactly one of: {[c.value for c in TicketCategory]}. Respond with only the category.",
        messages=[{"role": "user", "content": ticket_text}],
    )
    return TicketCategory(response.content[0].text.strip().lower())


async def handle_ticket(client, ticket_text: str) -> str:
    """
    Routes a ticket through a fixed, predictable workflow based on its classification —
    the only LLM judgment call in this workflow is the classification step itself.

    Args:
        client: An async LLM client.
        ticket_text (str): The raw ticket text.

    Returns:
        str: The response generated by the category-specific fixed handler.
    """
    category = await classify_ticket(client, ticket_text)

    if category == TicketCategory.BILLING:
        return await handle_billing_ticket(client, ticket_text)
    elif category == TicketCategory.TECHNICAL:
        return await handle_technical_ticket(client, ticket_text)
    else:
        return await handle_account_ticket(client, ticket_text)

Every possible path through this function is fully known and auditable ahead of time — the only non-determinism is which fixed path is taken, decided by one bounded LLM call, not an open-ended sequence of model-driven decisions.

17. Production-quality example

A bounded agent loop with structured logging of every reasoning/action step, for the genuinely open-ended cases where an agent is justified — building on Part 3.3's tool-calling loop with explicit agent-specific safeguards:

import logging
from dataclasses import dataclass, field

logger = logging.getLogger("agent_loop")


@dataclass
class AgentRunTrace:
    """A structured, appendable trace of one agent run, for debugging and audit."""
    goal: str
    steps: list[dict] = field(default_factory=list)
    final_answer: str | None = None
    stopped_reason: str = "in_progress"


async def run_bounded_agent(
    client, tools: list[dict], tool_implementations: dict, goal: str, max_iterations: int = 8
) -> AgentRunTrace:
    """
    Runs a bounded agent loop for a genuinely open-ended task, logging every step
    for later audit and debugging.

    Args:
        client: An async LLM client.
        tools (list[dict]): Tool definitions available to the agent (Part 3.3).
        tool_implementations (dict): Tool name to actual async implementation function.
        goal (str): The task goal given to the agent.
        max_iterations (int): Hard cap on reasoning/action steps.

    Returns:
        AgentRunTrace: The full trace of the run, including its final answer or
            the reason it was stopped without one.
    """
    trace = AgentRunTrace(goal=goal)
    messages = [{"role": "user", "content": goal}]

    for iteration in range(max_iterations):
        response = await client.messages.create(
            model="claude-sonnet-4-5", max_tokens=1000, tools=tools, messages=messages
        )
        tool_use_blocks = [b for b in response.content if b.type == "tool_use"]

        if not tool_use_blocks:
            trace.final_answer = response.content[0].text
            trace.stopped_reason = "completed"
            logger.info("agent run completed in %d iterations", iteration + 1)
            return trace

        messages.append({"role": "assistant", "content": response.content})
        tool_results = []
        for block in tool_use_blocks:
            step_record = {"iteration": iteration, "tool": block.name, "args": block.input}
            try:
                result = await tool_implementations[block.name](**block.input)
                step_record["result"] = result
            except Exception as exc:
                result = {"error": str(exc)}
                step_record["error"] = str(exc)
            trace.steps.append(step_record)
            tool_results.append(
                {"type": "tool_result", "tool_use_id": block.id, "content": str(result)}
            )
        messages.append({"role": "user", "content": tool_results})

    trace.stopped_reason = "max_iterations_exceeded"
    logger.warning("agent run for goal=%r stopped: exceeded %d iterations", goal, max_iterations)
    return trace

Returning a full AgentRunTrace (rather than just a final string) makes every run auditable after the fact — exactly the debugging capability the loan-review example in section 6 was missing before its redesign.

18. Short exercise

A customer describes a task: "review each incoming vendor invoice, check it against the purchase order, flag discrepancies over $100, and route flagged invoices to the right approver based on department." Decide whether this is better built as a workflow or an agent, and justify your answer using the "is the sequence knowable in advance" test from section 3 — then sketch the fixed steps (if workflow) or the tools and stopping condition (if agent).

19. Interview questions

• What is the precise, technical distinction between a workflow and an agent, in terms of who controls the sequence of steps?
• Why does an agent's reliability degrade multiplicatively with the number of steps in its loop, and what does that imply about when agents are worth the risk?
• Describe a task you'd deliberately build as a workflow instead of an agent, even though an agent could technically do it, and explain the concrete reliability/auditability benefit.

20. FDE/customer scenario

Customer: "We want a fully autonomous AI agent that handles our entire customer onboarding process end-to-end."

Before agreeing to "fully autonomous agent" as the architecture, an FDE should map out the actual onboarding process: how much of it is a known, fixed sequence of steps (collect documents, verify identity, provision account, send welcome email — likely almost all of it) versus genuinely unpredictable judgment calls (handling an edge case that doesn't fit the standard flow)? The right recommendation is very often "a workflow that handles the fixed, well-understood 90% of cases reliably and predictably, with an agent or human escalation path for the genuinely unpredictable 10%" — not a single monolithic agent for the whole process, even though that's the more exciting-sounding pitch. This is one of the clearest, most concrete places an FDE's engineering judgment directly protects a customer from an unreliable, hard-to-audit system they didn't actually need.

---

Key takeaways

• The core distinction is who controls the sequence of steps: fixed code (workflow) or the model itself (agent) — this determines predictability, auditability, cost, and risk.
• Default to workflows; escalate to agents only when the task's genuine unpredictability requires it.
• An agent's end-to-end reliability degrades multiplicatively with its number of steps — fewer, well-bounded steps left to model judgment is a real reliability strategy.

Things you should be able to explain

• The workflow vs. agent distinction precisely, not just "agents use tools."
• Why an agent's reliability compounds negatively with more steps.

Things you should be able to build

• A routing workflow with one bounded classification step.
• A bounded, fully-logged agent loop with a hard iteration cap for genuinely open-ended tasks.

Common mistakes

• Building an agent for a task whose steps are actually fixed and knowable.
• No iteration cap on agent loops.
• Treating "agent" as inherently more sophisticated than "workflow."

Recommended next chapter

08-mcp.md