18.23 — FDE Cloud Customer Scenarios

Relationship to Part 12: Part 12 taught the discovery and communication discipline generally. Part 12.3 specifically is the general-purpose home of the FDE communication reasoning skeleton (problem → evidence → impact → options → trade-offs → recommendation → next step), taught there as its own named, reusable tool with a fully worked example. This chapter applies that same skeleton narrowly, to seven specific, verbatim customer statements this track's infrastructure vocabulary (18.1–18.22) is what actually lets you respond to well — each reasoned through: clarify → identify constraints → propose architecture → explain trade-offs → identify risks → propose implementation → explain deployment → explain monitoring → explain security, which is this domain's specific instantiation of that same general skeleton.

1. What is it?

Seven realistic, verbatim customer statements an AI FDE hears repeatedly in real engagements, each worked through completely — not just the technical answer, but the reasoning process and the communication of it, per Part 12.3's layered-communication discipline.

2. Why does it exist?

18.1–18.22 built deep technical capability. This chapter exists because a customer never says "please apply the 14-point system design framework" — they say "we already use Kubernetes" or "we cannot use Docker Hub," and translating a plain customer statement into the right technical response, fluently, is a distinct, practiced skill this chapter targets directly.

3. What problem does it solve?

It solves "a customer just said something with real infrastructure implications — what do I actually do with that statement" for the seven most common such statements an AI FDE encounters.

4. How does it work internally?

Scenario 1: "Our security team does not allow data to leave our AWS account."

CLARIFY: does this mean no external LLM API at all (ruling out Anthropic/
  OpenAI's standard API entirely), or specifically no RAW DOCUMENT
  CONTENT leaving (which a provider's standard data-usage/retention
  policy might already satisfy — verify current provider policies rather
  than assuming either way)?
CONSTRAINTS: if genuinely no data leaves the account under any
  circumstance, this rules out external API calls entirely — a hard,
  not negotiable-via-configuration constraint (18.16 flagged the common
  misconception that a "regional endpoint" alone satisfies this).
ARCHITECTURE: self-hosted model (18.16) inside the customer's own VPC
  (18.5), OR a provider's account-scoped/VPC-endpoint offering if one
  exists and genuinely satisfies "never leaves our account" (verify
  current provider offerings precisely — don't assume).
TRADE-OFFS: self-hosting's real capability/cost trade-off (18.16/18.18)
  becomes NON-OPTIONAL here, not a cost-driven choice — communicate this
  distinction to the customer explicitly, since it changes how the
  trade-off should be presented (a required consequence of their own
  constraint, not a vendor limitation).
RISKS: self-hosted model capability gap relative to frontier
  external-API models — set expectations explicitly, early.
IMPLEMENTATION: 18.16's VRAM capacity planning, GPU instance
  provisioning (18.3) inside their VPC.
DEPLOYMENT: Terraform (18.6) into the customer's own account (18.22's
  Design 6, Part 14.1).
MONITORING: their own CloudWatch/observability (18.11), verified to also
  stay within their account/region if that's part of the same requirement.
SECURITY: 18.19's full bridge, with 18.16's model-access-control point
  (which roles can invoke the self-hosted endpoint) as a new, specific
  concern.

FDE COMMUNICATION (BAD vs. BETTER, per the source spec's exact framing):
  BAD: "We'll self-host the model."
  BETTER: "Since no data can leave your AWS account under any
  circumstance, an external API isn't an option — we'd deploy an
  open-weights model inside your own VPC instead. The trade-off is
  real: you'll get somewhat less capability than a frontier external
  model, in exchange for the data guarantee your security team requires.
  Here's what that capacity and cost actually look like for your
  expected volume [18.16/18.18's concrete numbers]."

Scenario 2: "We already use Kubernetes."

CLARIFY: which distribution/version, and is it EKS specifically or
  self-managed/on-prem Kubernetes (relevant to 18.20's multi-cloud
  reasoning if not AWS-hosted at all)?
CONSTRAINTS: this determines compute choice OUTRIGHT (18.3/18.9) — ECS
  is no longer the simpler default once a customer's platform standard
  is already Kubernetes; fighting an established platform standard is
  rarely worth it absent a strong, specific reason.
ARCHITECTURE: 18.9's Deployment/Service/HPA/PDB design, deployed onto
  THEIR cluster (namespace-scoped, 18.9's isolation caveat noted).
TRADE-OFFS: accept their existing operational patterns (Helm usage,
  existing RBAC conventions, 18.9) rather than introducing a
  parallel, unfamiliar-to-them deployment pattern.
RISKS: confirm whether KEDA/Prometheus Adapter (18.9) is already
  installed before promising queue-depth-based autoscaling — a concrete,
  necessary discovery question, not an assumption.
IMPLEMENTATION: Helm chart (18.9) matching their existing conventions,
  if they already use Helm for other services.
DEPLOYMENT: integrate with their existing CI/CD if one exists, rather
  than introducing a separate, parallel pipeline (18.8).
MONITORING: integrate with their existing observability stack (18.11)
  if one exists, rather than mandating a new one.
SECURITY: their existing RBAC/network-policy conventions (18.9/18.19),
  audited for this new workload specifically, not replaced wholesale.

Scenario 3: "We cannot use Docker Hub."

CLARIFY: is this a specific policy against Docker Hub specifically (a
  common stance given Docker Hub's pull-rate limits and past incidents),
  or a broader "no external, unaudited image sources" policy?
CONSTRAINTS: every base image (Part 7.1/18.7's `python:3.12-slim`, for
  instance) must come from an approved registry instead — commonly
  ECR's own public gallery (a mirrored, AWS-hosted alternative) or the
  customer's own private registry/mirror.
ARCHITECTURE: unchanged in shape — only the registry source for base
  images and any third-party images (a Helm chart's own image
  dependencies, 18.9) changes.
TRADE-OFFS: none significant if a suitable alternative registry exists;
  real friction if a specific needed image is ONLY published to Docker
  Hub with no mirror — worth surfacing early as a concrete blocker to
  resolve, not discovered mid-build.
RISKS: an unnoticed transitive dependency on a Docker-Hub-hosted base
  image (e.g., inside a third-party Helm chart) — audit the full image
  dependency tree, not just your own Dockerfiles.
IMPLEMENTATION: update every `FROM` line (18.7) and any Helm chart's
  image references to the approved registry.
DEPLOYMENT: CI/CD pipeline's own pull/push steps (18.8) updated to the
  approved registry, with credentials scoped appropriately (18.3).
MONITORING: no change.
SECURITY: this constraint IS itself a supply-chain security control
  (18.8's image-signing/provenance point) — treat it as reinforcing,
  not merely restricting.

Scenario 4: "Everything must run inside our private VPC."

CLARIFY: does "everything" include the LLM call itself (converging with
  Scenario 1's data-residency question) or specifically the
  application/data infrastructure (a narrower, more common and more
  easily satisfied requirement)?
CONSTRAINTS: 18.5's private-subnet design for app and data tiers is
  already this track's default — the clarification determines whether
  ANYTHING beyond that default is actually needed.
ARCHITECTURE: if it's the narrower reading, 18.5's existing design
  already satisfies it — communicate this clearly rather than assuming
  more work is needed than actually is.
TRADE-OFFS: if the LLM call itself must stay private, this converges
  with Scenario 1's self-hosting requirement and trade-offs.
RISKS: a NAT Gateway's outbound path to an external LLM provider (18.2)
  might itself violate a strict reading of "everything inside the VPC" —
  clarify explicitly rather than assuming NAT-based outbound access is
  automatically acceptable.
IMPLEMENTATION/DEPLOYMENT/MONITORING/SECURITY: as in 18.5/18.17, unless
  the broader reading applies, in which case Scenario 1's answer governs.

Scenario 5: "We need SSO."

CLARIFY: which protocol/provider (SAML, OIDC, a specific identity
  provider like Okta or Entra ID, Part 9.4's SSO mention) — "SSO" alone
  under-specifies the actual integration work.
CONSTRAINTS: this is primarily an APPLICATION-layer authentication
  integration (Part 9.4), not an infrastructure-layer one — but the
  infrastructure implication worth raising: where are session
  tokens/credentials stored and how (Redis, 18.14, with appropriate TTL
  and NOT as a system-of-record for anything unrecoverable), and does
  the identity provider's callback/redirect flow require any specific
  network/ingress configuration (18.5)?
ARCHITECTURE: an OIDC/SAML integration at the application's auth layer,
  session state in Redis (18.14) or a signed, stateless token approach.
TRADE-OFFS: stateless tokens (e.g., JWTs) avoid a Redis dependency for
  session lookup but complicate immediate revocation (Part 9.4's
  JWT-pitfalls point) — a real, worth-stating trade-off.
RISKS: JWT algorithm-confusion and audience/scope validation pitfalls
  (Part 9.4) — an infrastructure-adjacent but genuinely application-
  security concern to flag even though this chapter is infrastructure-
  focused.
IMPLEMENTATION: standard OIDC/SAML library integration (Part 9.4's
  territory primarily).
DEPLOYMENT: ensure the identity provider's callback URL is reachable
  through the ALB (18.5) with correct routing.
MONITORING: authentication failure rate as an explicit SLI (18.11) —
  a spike here can indicate either an integration bug or a real attack.
SECURITY: 18.19's bridge — infrastructure-layer session storage
  security (18.14's encryption-in-transit point) backstops the
  application-layer SSO integration.

Scenario 6: "We need the AI system to handle 10,000 requests per minute."

CLARIFY: ~167 req/sec average — clarify peak-to-average ratio and
  per-request shape (context length, output length, Part 12.1) before
  designing, per 18.22's Design 1 exactly.
CONSTRAINTS: at this volume, 18.13's bottleneck framework should be
  applied BEFORE any specific scaling commitment — the LLM provider's
  own rate limits (Part 7.9/18.12) are very likely the binding
  constraint, not your own infrastructure's capacity.
ARCHITECTURE: 18.22's Design 1, validated specifically against this
  volume's actual bottleneck.
TRADE-OFFS: if the LLM provider's rate limit IS the binding constraint,
  the real conversation is about a quota increase or multi-provider
  routing (Part 7.9), not more application replicas (18.13's central,
  recurring point).
RISKS: promising this throughput without first confirming provider
  quota headroom is a real, credibility-damaging risk if discovered only
  after commitment.
IMPLEMENTATION: load testing against the ACTUAL target volume (not
  assumed) before finalizing any capacity commitment.
DEPLOYMENT: canary rollout (18.8) of any scaling change, monitored
  against the real target load.
MONITORING: request-concurrency and provider-rate-limit-proximity as
  explicit SLIs (18.11), not just latency/error-rate alone.
SECURITY: no change from the base design's posture (18.19).

Scenario 7: "We need audit logs for every AI action."

CLARIFY: 18.5's exact flagged distinction — infrastructure API audit
  (CloudTrail) versus application-level audit of AI/user actions (Part
  9.4/10.5) are genuinely different things; "every AI action" almost
  certainly means the second, not the first — confirm explicitly rather
  than assuming CloudTrail alone satisfies this.
CONSTRAINTS: an application-level audit log needs to capture WHAT
  (which action/tool call), WHO (which user/session), WHEN, and — for an
  AI system specifically — often the REASONING context (which prompt
  version, which retrieved documents informed the response, Part 6.1's
  trace data) for a complete audit trail.
ARCHITECTURE: an application-level audit table (RDS, 18.4) or a
  dedicated audit-log service, populated at every significant AI
  action boundary — distinct from, but complementary to, LangSmith's
  trace data (Part 6.1), which is optimized for debugging, not
  necessarily for the retention/immutability guarantees a compliance
  audit log needs.
TRADE-OFFS: full request/response logging (maximal auditability) versus
  data-minimization concerns (Part 9.3/9.6) — a real, explicit trade-off
  to present, not simply "log everything."
RISKS: audit logs themselves may contain sensitive data (Part 9.3) —
  they need the same access-control/encryption discipline as any other
  sensitive data store (18.4/18.5), not an exemption because "it's just
  logs" (18.11's exact flagged point).
IMPLEMENTATION: an append-only audit table or dedicated log stream,
  with retention policy matching the customer's specific compliance
  requirement (verify what that requirement actually specifies — SOC 2,
  a specific regulation — rather than assuming a generic retention period).
DEPLOYMENT: no unique infrastructure beyond what 18.4/18.5 already
  provides for RDS/log storage.
MONITORING: audit-log write failures should themselves be alerted on
  (18.11) — a silently-failing audit log is worse than an obviously
  missing one.
SECURITY: CloudTrail (infrastructure) AND application-level audit
  logging (this scenario) together, explicitly presented as two
  complementary layers, per 18.5's exact distinction.

5. Simple mental model

Each scenario is a compressed customer requirement that, unpacked, maps onto specific chapters in this track — this chapter's actual skill is the unpacking itself: hearing a plain statement and immediately knowing which of 18.1–18.22's concepts and trade-offs it activates.

6. Real-world example

Scenario 1, combined with Scenario 4, is exactly Part 16.8's underwriting- engagement narrative's Stage 5 deployment constraint — this chapter's value is recognizing that combination quickly and correctly in a live conversation, not deriving the answer from scratch each time.

7. Architecture diagram

(Each scenario's architecture is a specific instance of 18.17's base diagram, per section 4's per-scenario notes — not reproduced separately here.)

8. Production considerations

Always clarify before architecting (section 4's consistent first step, per Part 12.1/17.1) — several of these seven scenarios (2, 4, 5) have genuinely different correct answers depending on a clarification most FDEs are tempted to skip because the statement "sounds" clear enough on its own.

9. Common mistakes

• Answering "everything must run inside our VPC" (Scenario 4) as if it automatically means self-hosting the model, when the narrower, more common reading (app/data tier only) may already be satisfied by this track's default design.
• Treating "we need audit logs" (Scenario 7) as satisfied by CloudTrail alone, missing the application-level audit log a customer almost certainly actually means.
• Promising a specific throughput number (Scenario 6) before confirming the LLM provider's own rate-limit headroom.

10. Security considerations

Scenario 1 and Scenario 7 are this chapter's most security-relevant statements — both directly exercise 18.19's infrastructure-to-AI-security bridge under a concrete, real customer constraint.

11. Performance considerations

Scenario 6 is this chapter's direct performance/scaling scenario, exercising 18.13's bottleneck-identification framework under a specific, quantified target.

12. Cost considerations

Scenario 1's self-hosting requirement carries the most direct, largest cost implication of these seven — worth quantifying explicitly (18.16/ 18.18) rather than leaving as an abstract "it'll cost more."

13. When to use it

Any time a customer states a requirement using their own plain language rather than this track's technical vocabulary — which is, in practice, nearly always, making this chapter's translation skill a constant, everyday part of real FDE work.

14. When NOT to over-apply it

Don't over-interpret an ambiguous customer statement into the most complex possible reading by default (Scenario 4's exact trap) — clarify first, and let the actual requirement (not the most technically interesting one) drive the architecture.

15. Alternatives and trade-offs

Each scenario's own trade-offs subsection (section 4) is this chapter's content — there's no single alternative "approach" beyond consistently clarifying before architecting, across all seven.

16. Practical example — a clarification-question bank

# Quick-Reference Clarifying Questions by Customer Statement

"Data can't leave our account"    → Does this include the LLM call
                                      itself, or just raw documents?
"We already use X"                 → Which specific version/distribution,
                                      and what conventions already exist?
"We can't use Y"                    → Is there an approved alternative,
                                      and does anything transitively
                                      depend on Y unnoticed?
"Everything in our VPC"             → Does "everything" include external
                                      API calls, or just our own tiers?
"We need SSO"                       → Which protocol/provider, and what's
                                      the session-revocation requirement?
"We need N requests/[time]"          → What's peak-to-average ratio, and
                                      per-request shape (tokens)?
"We need audit logs"                 → Infrastructure API audit, AI
                                      action audit, or both — and what's
                                      the actual retention requirement?

17. Production-quality example — an FDE response template

# Response to: "[verbatim customer statement]"

## What I understood, and what I'd like to confirm
[clarifying questions, per section 16]

## What this requires, technically
[the constraint this creates — problem + evidence, per the source
spec's FDE communication structure]

## What I'd propose
[architecture, in plain language first, technical detail second —
Part 12.3's layered communication]

## The trade-off involved
[stated explicitly, never hidden or glossed over]

## Next step
[a concrete, small next action — a validation, a follow-up question,
a prototype, per Part 13's discipline — not a vague "we'll get back to you"]

18. Short exercise

Write out your own full response (using section 17's template) to a customer statement not covered in this chapter — "we need this to work even if our internet connection to AWS is intermittent" — identifying which of 18.1–18.22's concepts it actually activates.

19. Interview questions

• Walk through how you'd respond to each of this chapter's seven customer statements, live, including your clarifying questions.
• Why is "everything must run inside our VPC" genuinely ambiguous, and what's the cost of guessing wrong?
• What's the difference in how you'd respond to "we need audit logs" if the customer already has CloudTrail enabled versus if they don't?
• Why should you avoid committing to a specific requests-per-minute target before checking provider rate-limit headroom?

20. FDE/customer scenario

A customer opens a meeting with three of these seven statements back to back: "we already use Kubernetes, we cannot use Docker Hub, and we need audit logs for every AI action." Practice responding to all three in sequence, live, using this chapter's clarify-first discipline for each, without collapsing them into one generic "we'll handle all your requirements" response that addresses none of them concretely.

---

Key takeaways

• A plain customer statement almost always under-specifies the actual requirement — clarifying first (Part 12.1) is what separates a correct architectural response from a plausible-sounding guess.
• Several common statements ("everything in our VPC," "we need audit logs") have a narrower, more common reading and a broader, more expensive one — confirming which is meant is not optional.
• The FDE communication structure (problem → evidence → impact → options → trade-offs → recommendation → next step) turns a technical answer into one a non-technical stakeholder can actually act on.

Things you should be able to explain

• The clarifying question each of these seven statements actually needs.
• Why "everything must run inside our VPC" and "we need audit logs" are both genuinely ambiguous without further discovery.

Things you should be able to build

• A clarification-question bank for common customer infrastructure statements.
• A structured FDE response template applying problem/evidence/impact/ options/trade-offs/recommendation/next-step to a real customer statement.

Common mistakes

• Architecting from the most complex possible reading of an ambiguous statement instead of clarifying first.
• Treating CloudTrail as satisfying an application-level audit-log requirement.
• Committing to a throughput number before confirming provider rate-limit headroom.

Recommended next chapter

24-capstone-cloud-ai-platform-engagement.md