16.1 — Project: Enterprise Support Agent

This project builds the customer support agent designed in Part 11.3 into a real, runnable system. Read Part 11.3 first — this project doesn't re-derive that architecture, it implements it.

1. Customer Scenario

A mid-market e-commerce company (Part 11.3's scenario) needs an AI agent handling order-status questions, simple returns, and FAQ policy questions, escalating anything complex to a human — targeting a measurable reduction in their human support team's routine-question workload (Part 15's ROI framing).

2. Requirements

Recap of Part 11.3, sections 3-4: real-time order lookup (tool-calling, not RAG), bounded autonomous return processing with human approval above a threshold, FAQ answering via RAG, p95 latency under 3 seconds, zero cross-customer data leakage, graceful handling of 5-10x seasonal traffic spikes.

3. Architecture

Part 11.3's routing-workflow architecture: intent classifier → order-status tool / FAQ RAG / bounded return workflow → escalation path. Built on FastAPI (Part 1.3) + LangGraph (Part 5) for the return-processing workflow specifically (since it needs persistence across a human-approval pause, Part 5.3/5.4) with simpler direct LangChain calls (Part 4) for the stateless order-status and FAQ paths.

4. Technology Selection

• FastAPI (Part 1.3): API layer.
• LangGraph + PostgresSaver (Part 5.1/5.3): the return-processing workflow specifically, given its need for durable, resumable human-in-the-loop pausing.
• pgvector (Part 1.5/3.4): FAQ document embeddings — appropriate at this company's moderate scale (Part 1.5's cost/scale trade-off).
• Redis (Part 1.6): response caching for common FAQ questions and rate limiting.
• Anthropic Claude (small model for intent classification, Part 3.11; mid-tier for generation) via the LangChain provider integration (Part 4.2).

5. Folder Structure

support-agent/
├── src/
│   ├── api/
│   │   ├── main.py                 # FastAPI app, lifespan-managed clients (Part 1.3/4.1)
│   │   ├── dependencies.py         # auth, DB session, rate-limit dependencies (Part 1.3/9.4)
│   │   └── routes/
│   │       └── support.py          # POST /v1/support/message endpoint
│   ├── agent/
│   │   ├── router.py               # intent classification (Part 3.7)
│   │   ├── order_status.py         # tool-calling path (Part 3.3)
│   │   ├── faq_rag.py              # RAG path (Part 3.5)
│   │   └── return_workflow.py      # LangGraph return-processing graph (Part 5)
│   ├── tools/
│   │   └── ecommerce_platform.py   # user-scoped tool construction (Part 3.3/4.4)
│   ├── data/
│   │   ├── models.py               # SQLAlchemy models (Part 1.4)
│   │   └── vector_store.py         # pgvector retriever (Part 3.4/4.5)
│   └── observability/
│       └── tracing.py              # LangSmith config (Part 6.1)
├── tests/
│   ├── unit/                       # parsing, routing logic (Part 1.9)
│   ├── integration/                # DB + mocked LLM calls (Part 1.9)
│   └── eval/
│       └── regression_dataset.py   # Part 6.2's dataset-from-traces pattern
├── infra/
│   ├── Dockerfile                  # multi-stage, non-root (Part 7.1)
│   ├── docker-compose.yml          # local dev: app + Postgres + Redis
│   └── k8s/                        # Deployment, Service, HPA manifests (Part 7.4)
├── .github/workflows/ci-cd.yml     # staged pipeline + eval gate (Part 7.2)
└── requirements.txt

6. Implementation (key excerpt)

The intent router, directly implementing Part 3.7/11.3's routing pattern:

from enum import Enum

class SupportIntent(str, Enum):
    ORDER_STATUS = "order_status"
    FAQ = "faq"
    RETURN_REQUEST = "return_request"
    ESCALATE = "escalate"

async def classify_intent(client, message: str) -> SupportIntent:
    """Classifies a support message using a small, fast model (Part 3.11)."""
    response = await client.messages.create(
        model="claude-haiku-4-5", max_tokens=10,
        system=f"Classify into: {[i.value for i in SupportIntent]}. Respond with only the category.",
        messages=[{"role": "user", "content": message}],
    )
    return SupportIntent(response.content[0].text.strip().lower())

The remaining implementation (order-status tool, FAQ RAG, return workflow) directly reuses the code from Parts 3.3, 3.5, and 5.4's examples — this project's value is in assembling them correctly, not reinventing them.

The .github/workflows/ci-cd.yml staged pipeline named above, abbreviated but syntactically real, directly applying 07-production-ai/02-cicd.md's staged build → test → eval-gate → deploy sequence to this project's own paths and eval dataset:

name: ci-cd

on:
  pull_request:
    branches: [main]

jobs:
  lint-and-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install -r requirements.txt -r requirements-dev.txt
      - run: ruff check .
      - run: pytest tests/unit tests/integration

  evaluation-gate:
    needs: lint-and-test
    runs-on: ubuntu-latest
    if: contains(github.event.pull_request.changed_files, 'src/agent/')
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install -r requirements.txt
      - run: python scripts/run_eval_gate.py --dataset support-agent-regression-v1 --min-pass-rate 0.90
        env:
          LANGSMITH_API_KEY: ${{ secrets.LANGSMITH_API_KEY }}
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}

  build-and-push:
    needs: evaluation-gate
    if: always() && (needs.evaluation-gate.result == 'success' || needs.evaluation-gate.result == 'skipped')
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: |
          docker build -t registry.internal/support-agent:${{ github.sha }} .
          docker push registry.internal/support-agent:${{ github.sha }}

  deploy-staging:
    needs: build-and-push
    runs-on: ubuntu-latest
    steps:
      - run: ./deploy.sh staging ${{ github.sha }}
      - name: smoke test
        run: curl -f https://staging.internal/health || exit 1

  deploy-production:
    needs: deploy-staging
    runs-on: ubuntu-latest
    environment: production  # requires manual approval — Part 7.2's continuous-delivery, not full continuous-deployment, distinction
    steps:
      - run: ./deploy.sh production ${{ github.sha }}

The evaluation-gate job is conditioned on changes under src/agent/ specifically (Part 7.2, section 12's cost-consciousness) — a documentation-only or API-schema-only PR skips the LLM-as-judge evaluation entirely, while any change to the router, RAG path, or return workflow always runs the full regression_dataset.py suite from section 6's folder structure against a required 90% pass rate before the build is even produced, let alone deployed. The environment: production gate on the final job is what turns this into continuous delivery rather than full continuous deployment for this specific system — appropriate given a bad prompt/routing change reaching production directly affects real customer conversations.

7. Testing

Unit tests for intent classification accuracy against a hand-labeled sample; integration tests for the tenant/user-scoping on order_status.py (Part 1.9's tenant-isolation test pattern, applied at the individual-customer level here); a specific test verifying the return workflow correctly pauses via interrupt() above the dollar threshold and resumes correctly after approval (Part 5.4).

8. Evaluation

A LangSmith dataset (Part 6.2) built from real support transcripts, with deterministic evaluators for order-status accuracy (exact match against known order data) and an LLM-as-judge evaluator (Part 8.3, calibrated against human review) for FAQ response helpfulness and faithfulness (Part 8.2).

9. Security

Order-status and return tools constructed per-user via closures (Part 3.3/4.4, section 17's exact pattern), never accepting a customer ID as a model-suppliable argument. Return-processing hard limits enforced in code (Part 9.2's exact mitigation), not the prompt. PII scanning (Part 9.3) on FAQ responses before returning to the user.

10. Observability

LangSmith tracing (Part 6.1) tagged with customer_id for per-customer debugging; task-completion and escalation-rate SLIs (Part 8.1/8.4); cost-per-conversation tracked explicitly (Part 7.10).

11. Deployment

Docker + Kubernetes (Part 7.1/7.4) with HPA scaled on request rate for the stateless paths and on queue depth for return-processing (Part 7.7, if return volume warrants a dedicated queue). PostgresSaver for the return workflow's checkpointing, non-negotiable given Part 7.4's pod-ephemerality warning.

12. Scaling

Autoscaling per Part 7.5, with explicit bottleneck-diagnosis discipline (is the constraint infrastructure or LLM provider rate limit) built into on-call runbooks ahead of the first seasonal peak, not discovered during it.

Concrete capacity numbers behind section 2's "5-10x seasonal spike" requirement — baseline vs. peak (Black Friday/holiday-return-window traffic, this company's actual seasonal pattern), and the HPA configuration (Part 7.4/7.5) sized against them:

Path	Baseline req/s	Seasonal spike target	HPA min replicas	HPA max replicas	Scale-up trigger
/v1/support/message (router + FAQ RAG, stateless)	40 req/s	320 req/s (8x)	3	24	CPU > 65% or p95 latency > 2.5s for 2 consecutive minutes
Order-status tool path	25 req/s	200 req/s (8x)	2	16	CPU > 65%
Return-processing workflow (queue-depth-scaled, Part 7.7)	3 req/s enqueued	18 req/s enqueued (6x)	2	10	Queue depth > 50 pending items

The HPA max of 24 for the stateless message path is set with headroom above the 8x target (not tightly at it), per Part 7.5's guidance to size for the demonstrated peak plus margin rather than the exact historical maximum — a peak 20% above last year's is a realistic, not a worst-case, planning assumption for a growing customer base. The return-workflow path scales far more conservatively (6x, not 8x) because its bottleneck is typically the downstream e-commerce platform API's own rate limit (section 14), not this service's own compute — over-scaling replicas here just produces more requests hitting that external limit faster, not more actual throughput.

13. Cost Considerations

Intent classification on the smallest viable model (Part 3.11) given its high volume and low complexity; FAQ response caching (Part 1.6/7.8) for the highest-frequency questions; per-conversation cost tracked and alerted on (Part 7.10/8.4) to catch a regression before it compounds across a full billing cycle.

14. Failure Scenarios

E-commerce platform API outage → graceful fallback message, never a hallucinated guess (Part 2.6/3.3). LLM provider outage → fallback provider (Part 3.11/7.9). Return-processing tool bug → bounded by hard-coded limits regardless (Part 9.2) — the worst case is capped by design, not by hope.

Customer on-call handoff runbook

Section 12's "on-call runbooks" gets a concrete artifact here — a short excerpt of what actually ships to the customer's own ops team at go-live, so their on-call engineer (not this project's FDE) can handle a routine page without escalating back to the vendor every time:

# Support Agent — On-Call Runbook (Customer Ops)

## Escalation contacts
- P1 (agent fully down, all traffic failing): page vendor on-call via
  the shared PagerDuty service `support-agent-p1` — 15-min response SLA.
- P2 (degraded — elevated latency or error rate, agent still functioning):
  vendor Slack channel #support-agent-oncall, next-business-day if outside
  the vendor's own on-call hours.
- P3 (a single conversation looks wrong, no systemic signal): file a
  ticket with the conversation ID; no page needed.

## Dashboards and logs
- Latency/error-rate/escalation-rate dashboard: Grafana → "Support Agent
  Overview" (Part 8.4's SLIs, customer-facing subset).
- Per-conversation trace lookup (for a specific bad answer): LangSmith,
  filtered by `customer_id` tag (Part 6.1) — ask the vendor for read
  access if not already provisioned.
- Cost-per-conversation trend: same Grafana dashboard, "Cost" tab
  (Part 7.10) — a sustained upward trend outside a known traffic increase
  is a P2, not a P3.

## Common failure modes and first response
| Symptom | Likely cause | First action |
|---|---|---|
| All requests return the fallback "unable to process" message | E-commerce platform API outage (section 14) | Check the platform's own status page first — this is very often not a bug in the agent |
| Escalation rate spikes suddenly | Intent-classification regression, or a genuinely new question pattern (a promotion, an outage on the customer's own side) | Check the LangSmith trace sample for the spike window before assuming a bug |
| One customer reports a wrong order status | Tool-call or platform-data issue, almost never a hallucination given tool-calling (not RAG) is used here (section 2) | Pull that conversation's trace; verify the raw tool response against the platform directly |
| Return gets stuck "pending approval" indefinitely | `interrupt()` resume path issue (Part 5.4) or the approving human simply hasn't acted yet | Check the approval queue UI before treating this as a system bug |

## What NOT to do
- Do not restart the return-processing service to "fix" a stuck approval —
  `PostgresSaver` checkpointing (section 11) means state survives a
  restart, but a restart also won't resolve a pending-human-approval item;
  it just adds noise to the investigation.
- Do not increase the escalation threshold yourself to reduce escalation
  volume — that is a risk-tolerance decision (Part 9.2) for the customer's
  Head of Support (section 1's stakeholder), not an on-call mitigation.

This is deliberately short — a real handoff runbook grows with each incident (per the postmortem "Prevention" step feeding back into it), but even this much means the customer's own ops team can triage a P2/P3 without a vendor page, and knows exactly when a P1 genuinely warrants one.

15. Improvements

Add semantic caching (Part 7.8) for FAQ paraphrase variation once volume justifies the tuning investment. Expand the escalation-context handoff to include a structured summary (not just raw transcript) for the receiving human agent, reducing their ramp-up time on takeover.

16. Business Metrics

Primary: reduction in average human-handled ticket volume for routine categories (Part 15's time-savings ROI pattern), measured via a controlled comparison against a holdout period/segment, netted against the system's full operating cost.

---

Key takeaways

• This project is an assembly exercise, not a from-scratch design — Part 11.3's architecture plus the specific code patterns from Parts 1, 3, 4, 5, 6, 7, and 9 combine directly into a runnable system.
• The return-processing workflow is the one component genuinely requiring LangGraph's persistence (Part 5.3) — the order-status and FAQ paths are simpler, stateless calls that don't need this machinery.
• Every security and cost pattern from earlier parts (user-scoped tool construction, hard-coded limits, small-model routing, caching) appears here as a concrete, load-bearing implementation detail, not an abstract principle.

Recommended next chapter

02-document-intelligence-platform.md