11.6 — System Design: AI Workflow Automation Platform

Scenario: An HR technology company wants to let their business customers define and automate multi-step approval workflows (e.g., "new hire onboarding: verify documents → run background check → provision accounts → notify manager") where each step may involve an LLM-driven decision, human approval, or a deterministic action — configurable per customer without custom code per workflow.

1. Requirements

Let non-technical customer admins define custom multi-step workflows combining AI-driven steps, deterministic actions, and human approval checkpoints, without requiring the platform vendor to write custom code for each customer's specific workflow variant.

2. Constraints

• Workflows must be customer-configurable (a business requirement directly shaping the technical architecture — this needs to be a genuinely generic workflow engine, not hardcoded logic per customer).
• Workflows can be long-running (a background check might take days) — requiring genuine persistence and resumability (Part 5.3), not just synchronous processing.
• Different customers have very different specific business rules for the same conceptual workflow step (e.g., "approval" criteria vary by company policy).

3. Functional Requirements

• A workflow-definition mechanism customers can configure (a structured schema, not free-form code) mapping to LangGraph's node/edge model (Part 5.1/5.2) underneath.
• Support for AI-driven steps (a classification or extraction task, Part 3.1/3.2), deterministic steps (a fixed API call), and human-approval steps (Part 5.4) within the same workflow.
• Long-running, resumable execution surviving days-long pauses (Part 5.3).

4. Non-Functional Requirements

• Each customer's workflow executions must be fully isolated from other customers' (Part 9.6/10.5's tenant isolation, now specifically applied to workflow execution state, not just data).
• Workflow execution must be auditable — every step's decision and timing traceable (Part 6.1) for compliance-sensitive processes like background checks and hiring decisions.
• The platform must support many customers' many concurrent, long-running workflow instances without cross-contamination.

5. Architecture

Customer-configured workflow DEFINITIONstructured schema, stored per-customer, compiled into a LangGraph StateGraph at execution time (Part 5.1)

LangGraph execution enginePostgresSaver checkpointing (Part 5.3), tenant-scoped thread_id (customer_id + workflow_instance_id)

AI-driven step nodePart 3.1/3.2

Deterministic action nodea plain API call

Human-approval nodeinterrupt() (Part 5.4)

6. Components

• Workflow-definition schema: a customer-facing configuration format (e.g., a JSON/YAML structure defining steps and their type) that the platform compiles into an actual LangGraph StateGraph (Part 5.1/5.2) — the technical implementation detail customers never see directly, but which gives the platform its actual execution engine.
• Tenant-scoped checkpointing: PostgresSaver (Part 5.3) with thread_id explicitly incorporating customer ID, so workflow state isolation follows directly from the persistence layer's own key structure, not just application-layer discipline.
• Pluggable step types: AI-driven, deterministic, and human-approval nodes as a small, well-tested set of reusable node "types" the workflow definition schema can compose, rather than unbounded custom code per customer.

7. Data Flow

1. A customer admin configures a workflow using the platform's structured definition format.
2. The platform compiles this into a LangGraph graph at workflow-instance-creation time.
3. A specific workflow instance (e.g., one new employee's onboarding) begins execution, checkpointed after every step (Part 5.3).
4. AI-driven steps call the configured model with the customer's configured criteria; deterministic steps call the configured API; human-approval steps pause via interrupt() (Part 5.4) until the appropriate approver responds, potentially days later.
5. Execution resumes exactly where it paused once approval arrives, continuing until the workflow completes.

8. Failure Modes

• A days-long-pending approval with no response: an explicit escalation/timeout policy (Part 5.4, section 8) rather than an indefinitely stuck workflow instance.
• A customer's misconfigured workflow definition (e.g., an infinite loop in their custom logic): the platform must enforce a hard iteration/step cap (Part 3.7's bounded-loop principle) regardless of customer configuration, protecting the platform from a single customer's configuration error.
• Checkpoint store outage: given the days-long execution windows, a checkpoint-store outage risks genuinely stuck workflows across potentially many customers simultaneously — the checkpoint store's own availability (Part 5.3/7.4) is a particularly critical dependency for this platform specifically, more so than for a typical short-lived-request system.

9. Security

Tenant isolation at the workflow-execution level (Part 9.6/10.5) is central, enforced structurally via the thread_id scheme rather than relying solely on application-layer checks. Customer-configured AI-driven steps must still respect Part 9.1/9.2's injection and excessive-agency principles — a customer's own configured criteria shouldn't be able to grant a workflow step more system access than the platform's own tool-scoping (Part 3.3/4.4) allows, regardless of what the customer configures.

10. Scalability

Horizontal scaling of the execution engine (Part 7.4/7.5) is straightforward given LangGraph's checkpoint-based resumability — any available worker/instance can pick up any customer's paused workflow, since state lives in the durable checkpoint store, not in any specific process's memory (Part 5.3/7.4's exact interaction).

11. Observability

Per-customer, per-workflow-instance tracing (Part 6.1) is both an operational necessity and, given the compliance-sensitive nature of workflows like background checks, a functional requirement (constraint/non-functional requirement 4's auditability). Pending-approval age monitoring (Part 5.4/8.4) as a specific SLI, given the days-long execution windows and the real business cost of a stuck workflow.

12. Cost

Cost scales with the number of AI-driven steps actually executed (Part 7.10) — since customers configure their own workflows, providing them visibility into the AI-cost implications of their configuration choices (e.g., "this workflow step will cost approximately $X per execution") is both good product design and helps prevent unexpectedly expensive customer configurations.

Worked numeric example: assume the platform serves 200 customer HR teams, each running an average of 15 new-hire onboarding workflow instances/month, each workflow configured with 4 AI-driven steps (e.g., document verification, background-check-summary review, offer-letter customization, manager-notification drafting) → 200 × 15 × 4 = 12,000 AI-driven step executions/month. Each step call, assuming ~800 input tokens (case context) and ~200 output tokens on a mid-tier model at $3/$15 per million: (800/1,000,000 × $3) + (200/1,000,000 × $15) ≈ $0.0024 + $0.003 ≈ $0.0054/step → total generation cost ≈ 12,000 × $0.0054 ≈ $64.80/month platform-wide — small relative to the checkpoint-store storage and availability cost (Part 5.3) that days-long-running, resumable workflows require, which should be modeled as its own line item rather than folded into per-call LLM cost.

Surfacing this per-step cost back to the customer configuring the workflow (e.g., "this step costs approximately $0.0054/execution × your expected monthly volume") directly implements section 12's transparency recommendation with a concrete number rather than a vague estimate.

13. Trade-offs

Chose a constrained, schema-based workflow-definition mechanism over allowing customers to write arbitrary custom code, trading some configuration flexibility for dramatically better security (bounded execution, Part 9.2), auditability, and platform maintainability — appropriate given the compliance-sensitive nature of the workflows this platform targets (hiring, background checks).

14. Alternatives

Allowing customers to write and upload custom workflow logic (arbitrary code) was considered and rejected given the security implications (Part 9.6's sandboxing discussion would apply at maximum severity) and the loss of platform-wide auditability guarantees — the constrained, composable step-type model was chosen specifically to keep the platform's security and compliance guarantees intact regardless of what any individual customer configures.

15. Code Example

The hard step-iteration cap (section 8's mitigation for a customer's misconfigured workflow) — a platform-enforced bound no customer configuration can override:

MAX_WORKFLOW_STEPS_PER_INSTANCE = 200


class WorkflowStepLimitExceeded(Exception):
    """Raised when a workflow instance exceeds the platform-enforced step cap."""


def enforce_step_limit(steps_executed_so_far: int) -> None:
    """
    Enforces a hard, platform-wide cap on steps per workflow instance,
    protecting the platform from any single customer's misconfigured
    workflow (e.g., an accidental loop) regardless of what that
    customer's own configuration specifies.

    Args:
        steps_executed_so_far (int): Count of steps this workflow
            instance has executed, including any repeated steps.

    Raises:
        WorkflowStepLimitExceeded: If the platform-wide cap is reached,
            at which point the instance is halted and flagged for review
            rather than continuing indefinitely.
    """
    if steps_executed_so_far >= MAX_WORKFLOW_STEPS_PER_INSTANCE:
        raise WorkflowStepLimitExceeded(
            f"Workflow instance exceeded the platform-enforced cap of "
            f"{MAX_WORKFLOW_STEPS_PER_INSTANCE} steps — halting and "
            f"flagging for review rather than continuing indefinitely."
        )

16. Interview questions

• Walk through estimating monthly AI-step cost given a customer count, workflows per customer, and steps per workflow, and explain why this number should be surfaced back to the customer configuring the workflow.
• Why must the step-iteration cap be enforced by the platform rather than left to each customer's own workflow configuration?

17. FDE/customer scenario

CUSTOMER: "Our workflow needs more than 200 steps — can you raise the limit just for us?"

The FDE-correct response investigates why a single workflow instance needs that many steps before simply raising the number — per section 8, an unusually high step count is often itself a signal of a modeling problem (e.g., a step that should be a single batched action modeled as many repeated ones) rather than a genuine business need, and the reasoning skeleton (Part 12.3) applies here: understand the actual requirement (problem/evidence) before proposing either a raised limit or a workflow-redesign recommendation (options/trade-offs/recommendation).

---

Key takeaways

• A configurable workflow platform should compile customer configuration into a constrained, well-tested set of composable step types (built on LangGraph, Part 5.1/5.2) rather than allowing arbitrary custom code — trading some flexibility for security, auditability, and platform-wide guarantees.
• Days-long, resumable execution makes checkpoint-store availability (Part 5.3) an unusually critical dependency, more so than for typical short-lived-request systems.
• Tenant isolation at the workflow-execution level should be enforced structurally via the checkpoint key scheme (thread_id), not solely through application-layer discipline.

Things you should be able to explain

• Why a constrained, schema-based workflow definition is safer than allowing arbitrary custom customer code.
• Why checkpoint-store availability is an unusually critical dependency for this specific platform.

Things you should be able to build

• A workflow-definition schema compiling into a LangGraph execution engine with tenant-scoped checkpointing and bounded step execution.

Common mistakes

• Allowing arbitrary customer-uploaded code instead of a constrained, composable step model.
• Underestimating checkpoint-store availability requirements given days-long execution windows.

Recommended next chapter

07-design-agent-platform.md