18.24 — Capstone: A Complete Cloud AI Platform Engagement

This capstone mirrors Part 16.8's structure exactly, but for this track: one continuous engagement, narrated stage by stage, citing the specific chapter whose technique is being applied at the moment it's used — so you can trace exactly how 18.1–18.23's individually-deep material composes into one real, deployed, operated production AI platform. This is the culmination the source spec asks for explicitly.

Stage 1: Customer Discovery (Part 12.1, Part 12.2)

First call.

CUSTOMER (VP of Engineering, a mid-size logistics company): "We want an AI platform for our enterprise — something our ops team can ask questions about shipments, and that can eventually take some actions for them."

Per Part 12.1's discipline, the FDE doesn't propose architecture yet:

FDE: "Before we get into approach — can you walk me through what your ops team does today when they need shipment information, and what 'take actions' means concretely? And separately — do you have any constraints on where data can live, or systems you've already standardized on?"

The second question is deliberate and specific to this track: discovery for a cloud/infrastructure-heavy engagement needs to surface infrastructure constraints (18.23's seven scenarios) as early as an AI-architecture discovery conversation surfaces business-process facts (Part 12.2). The answers: shipment data lives in an internal Postgres database and a third-party carrier API (Part 10.3); the company already runs its other services on AWS, in a single account, with no existing Kubernetes usage; there's no formal data-residency requirement, but the security team will review anything before production; "take actions" turns out to mean, concretely, re-routing a shipment or contacting a carrier — real, side-effecting operations (directly relevant to Part 9.2/18.19's excessive-agency reasoning later).

Stage 2: Requirements and Feasibility (Part 12.2, Part 13.1)

Applying Part 12.1's five discovery questions specifically to this engagement's infrastructure shape:

• Data reality: shipment data in Postgres, carrier status via a third-party API with its own, unfamiliar rate limits (18.12's resilience patterns will need to cover this dependency, not just the LLM provider).
• Risk tolerance: a wrong ANSWER (retrieval/Q&A) is low-stakes; a wrong ACTION (re-routing a shipment) is real-money, real-customer- impact — this single distinction will shape the entire architecture's human-in-the-loop design (Part 5.4) and, later, its infrastructure reliability investment (18.12) differently for the two capabilities.
• Scale: ~50 concurrent ops users today, expected to grow with the company — explicitly NOT "10,000 requests/minute" (18.23's Scenario 6) — a genuinely modest scale that should shape compute choice (18.3) away from over-engineering.
• Access control: ops staff should only see shipments for their region — a real tenant/scope-isolation requirement (Part 9.6/18.9), even though this is a single-organization deployment.
• Success metric: "if ops could get shipment status answers without calling three different systems, that's the win" — this becomes the literal metric Stage 8 measures against, unchanged (Part 15's discipline, applied identically to this track's capstone).

Stage 3: Architecture (Part 11, this track's 18.17 synthesis)

Given the risk-tolerance distinction from Stage 2, the FDE proposes two capabilities with genuinely different infrastructure investment: Q&A (RAG over shipment data and carrier documentation, Part 3.5) as the initial, lower-risk capability, and action-taking (re-routing, Part 3.7's agent/tool-calling) explicitly deferred to a second phase, gated on a human-in-the-loop approval step (Part 5.4) — directly reflecting Part 3.7's "don't reach for an agent by default" discipline and Part 13.1's staged-engagement lifecycle, now applied to infrastructure investment specifically: the Q&A phase needs a straightforward, modest-scale deployment (18.17's base architecture, sized down); the action-taking phase, when it arrives, will need 18.12's full resilience stack around the carrier API and re-routing tool specifically, given its real-money stakes.

Given no existing Kubernetes usage and modest scale (Stage 2), the FDE recommends ECS/Fargate (18.3's decision framework) over EKS — explicitly naming this as the simpler, sufficient choice absent a reason to accept Kubernetes's added complexity, a direct application of 18.9's "don't adopt Kubernetes to seem more sophisticated" warning.

Stage 4: Rapid Prototyping (Part 13.2)

The riskiest assumption: can retrieval reliably distinguish between similar shipment records across the company's actual, messy production data. A two-week prototype — a single script, no infrastructure beyond a local Postgres and vector store — tests this against real (access- controlled, anonymized where required, Part 9.6) historical data, following Part 13.2's exact discipline of testing the riskiest assumption cheaply before any infrastructure investment. Results: 89% accuracy overall, with a specific, lower-accuracy sub-case (shipments with recently-changed carrier assignments) — directly informing Stage 5's design (a fallback/escalation path for that specific sub-case) rather than either ignoring the gap or delaying the engagement to close it fully.

Stage 5: Production Implementation

Data and messaging (18.4): shipment data stays in the existing Postgres (RDS-migrated, Multi-AZ, 18.4/18.5); a vector database is added for retrieval (Part 3.4); no async ingestion pipeline is needed yet at this stage (18.15's actual decision criteria correctly says no — Q&A over already-structured data doesn't need the document-ingestion pattern this track's running example used elsewhere).

Networking (18.2/18.5): a standard two-AZ VPC, ALB in public subnets, ECS tasks and RDS in private subnets — 18.5's baseline design, sized for this engagement's modest scale (single NAT Gateway per AZ, not an elaborate multi-region setup this scale doesn't warrant).

Infrastructure as code (18.6): the entire architecture defined in Terraform from the start — not retrofitted later — specifically because this customer's security review (Stage 7) will want to review the actual infrastructure definition, not just a diagram (18.6's exact FDE-relevant value).

Containers (18.7): a cache-optimized, non-root, scanned Dockerfile for the FastAPI + LangGraph application; Trivy in CI (18.8) blocking any high/critical CVE before deploy.

Kubernetes: not used — ECS's simplicity (Stage 3's decision) is carried through consistently, resisting any temptation to introduce Kubernetes mid-engagement without a genuine new requirement forcing it.

CI/CD (18.8): build → test → lint → Trivy scan → sign (cosign) → Terraform plan (reviewed) → deploy to staging → smoke test → canary (5% of ops traffic) → full rollout, with an eval gate (Part 8.3, Part 6.2) specifically covering the recently-changed-carrier-assignment sub-case Stage 4's prototype flagged as weaker.

Reliability (18.12): a circuit breaker + fallback around BOTH the LLM provider AND the third-party carrier API — the carrier API's own, less mature rate-limit behavior (Stage 2's discovery) means this dependency needs the same resilience treatment as the LLM call, not less.

Caching (18.14): Redis for LLM-response caching (Part 7.8) and for rate-limiting outbound carrier-API calls specifically — configured allkeys-lru, correctly reasoned as a pure-cache use case (18.14).

Observability (18.11): CloudWatch for infrastructure health; LangSmith for retrieval/answer-quality traces, correlated via a shared request ID — with an explicit SLO for answer faithfulness (Part 8.2) distinct from and monitored alongside the infrastructure latency SLO.

Security (18.19): least-privilege IAM per component (18.3); region-scoped ops-staff access enforced via a tenant-tagged IAM/RLS pattern (18.19's Scenario-2-style design, applied to regional scoping instead of multi-tenant SaaS); the deferred action-taking phase's tool permissions pre-designed (but not yet enabled) with 18.19's excessive-agency bridge in mind from the start, not bolted on later.

Stage 6: Testing and Evaluation (Part 1.9, Part 6.2, Part 8)

A golden dataset (Part 6.2) built directly from Stage 4's prototype results, weighted to include the recently-changed-carrier-assignment sub-case explicitly; the CI eval gate (18.8) runs three averaged passes against a validated threshold with real margin, exactly per 18.8's flaky-gate mitigation. Load testing (18.13's framework, applied proactively rather than only during an incident) confirms the actual bottleneck at expected volume is the third-party carrier API's own rate limit, not the application tier — informing the caching/rate-limiting design in Stage 5 directly rather than over-provisioning ECS capacity that wouldn't have addressed the real constraint.

Stage 7: Security Review (Part 9, 18.19)

The customer's security team reviews the Terraform configuration directly (18.6's reviewability point, realized concretely) alongside a walk-through of 18.19's bridge table for this specific architecture: IAM scoping for the retrieval tool, region-based access enforcement, carrier-API credential storage (Secrets Manager, 18.5), and network segmentation (18.5). One finding: the carrier API integration's credentials were initially scoped more broadly than needed (a convenience default) — narrowed to the specific carrier-API operations actually used, directly applying 18.3's least-privilege principle as a concrete review outcome, not just a stated ideal.

Stage 8: Deployment (18.5, 18.6, Part 14)

No unusual deployment-environment constraint emerged (unlike Part 16.8's underwriting engagement) — the customer is comfortable with a vendor-hosted deployment model rather than requiring their own account (18.23's Scenario 1/4 didn't apply here), simplifying Stage 8 considerably relative to that other capstone's Stage 5. This is itself worth noting explicitly: not every engagement needs 18.23's most complex scenarios, and correctly recognizing that up front avoided unnecessary architectural complexity.

Stage 9: Production Monitoring and Incident Response (18.11, 18.21)

Two weeks post-launch, a real incident: retrieval latency spikes sharply for roughly twenty minutes (18.21's Incident 10 shape). LangSmith traces isolate the slowdown to the retrieval step specifically; investigation finds a large, one-off historical-data backfill job (run manually by the customer's own data team, uncoordinated with the AI platform team) competing for the same RDS/vector-database resources. Resolved by communicating the resource-contention finding to the customer's data team (18.21's exact incident shape and fix) and adding a monitoring alert (18.11) for future concurrent-load coordination — resolved within the same day specifically because the observability investment from Stage 5 made root-cause diagnosis fast, echoing Part 16.8's Stage 6 almost exactly, now with this track's specific tools.

Stage 10: Cost Review (18.18)

At the three-month mark, a routine cost review (18.18's systematic breakdown, applied proactively rather than triggered by a shock) finds the LLM API cost line is well-optimized (a healthy cache-hit rate from Stage 5's Redis caching), but NAT Gateway cost is higher than expected — traced to the carrier-API integration making far more individual calls per shipment-status check than necessary (18.18's exact "batch the OCR calls" pattern, here applied to carrier-status calls instead) — fixed by batching multiple shipment-status lookups into fewer carrier-API calls, a concrete, quantifiable saving found through 18.18's actual investigation discipline rather than assumed away.

Stage 11: Customer Feedback and Iteration (Part 13.1's lifecycle loop)

Ops staff feedback after three months: Q&A works well; they now want the deferred action-taking capability (shipment re-routing) activated. This triggers a genuine return to Stage 1's discovery discipline for this new scope specifically (Part 13.1's lifecycle loop, echoing Part 16.8's Stage 7 exactly) — now exercising the pre-designed-but-not-yet-enabled tool permissions from Stage 5, the human-in-the-loop approval flow (Part 5.4) requiring PostgresSaver-backed durable checkpointing (Part 5.3, 18.4/18.5's Multi-AZ RDS) since a re-routing approval might reasonably take hours, and 18.12's full resilience stack around the carrier API's actual re-routing operation specifically — including idempotency (18.4/18.12) so a retried re-routing request can never accidentally re-route a shipment twice.

---

Key takeaways

• This capstone and Part 16.8 solve genuinely different engagements (compliance-driven document-heavy underwriting vs. a modest-scale, infrastructure-simple logistics Q&A-then-agent platform) using the SAME underlying disciplines — discovery before architecture, staged risk-appropriate investment, and evidence-based incident/cost review — demonstrating those disciplines are general, not tied to one scenario.
• Correctly recognizing when NOT to reach for more infrastructure complexity (no Kubernetes, no self-hosted model, no customer-VPC deployment) was as much a demonstration of this track's judgment as correctly building the complexity that WAS needed (Multi-AZ RDS, circuit breakers around the carrier API, least-privilege IAM).
• The action-taking capability's human-in-the-loop, idempotent, durably-checkpointed design was planned from Stage 5 even though it wasn't activated until Stage 11 — good infrastructure architecture anticipates a reasonably foreseeable next phase without over-building for it prematurely.

Recommended next chapter

Part 18 complete. This concludes the Cloud, DevOps, and AI Infrastructure track — return to handbook/17-interview-prep/ to integrate this track's material into interview preparation, or revisit handbook/16-projects/08-full-fde-engagement-first-meeting-to-production.md to compare this capstone's infrastructure-first narrative against that project's AI-architecture-first narrative of a real, complete engagement.