Appearance
18.17 — GenAI Production Architecture: Assembling the Whole System
1. What is it?
The complete, assembled architecture of the Enterprise AI Assistant — every piece built across 18.1–18.16, drawn as one system, with the specific reasoning for why each component exists and how it connects to the others. This chapter builds nothing new technically; it is the synthesis the source spec explicitly asks for: "show how all the previously learned components connect."
2. Why does it exist?
Eighteen chapters of individually-deep material risk leaving you able to explain any one piece (a VPC, a circuit breaker, an HPA) without being able to draw the whole system fluently, under pressure, the way an FDE actually needs to in a customer architecture review (Part 12.3, 17.1). This chapter exists specifically to close that gap.
3. What problem does it solve?
It solves "can I draw and explain the complete architecture of a production AI system, end to end, connecting every layer" — the single most direct rehearsal this track can offer for both an FDE system-design interview (17.1) and a real customer architecture presentation.
4. How does it work internally?
The complete architecture, layer by layer
Users
Route 53DNS · 18.5
public subnet · 18.5
ALBTLS termination
private app subnet · 18.5
ECS / EKS ServiceFastAPI + LangGraph
private data subnet · 18.5
Rediscache, rate limit · 18.14
RDS Postgresapp data + checkpoints · 18.4
Vector DBretrieval · Part 3.4
Model Routercircuit breaker · 18.12
LLM Provider Aprimary
LLM Provider Bfallback
The async side (18.4 / 18.15):
Upload
S3
SNS
SQSingestion queue · 18.4
Worker poolECS/EKS Jobs · 18.9
Chunk + embedstage-idempotent · 18.15
Cross-cutting, everywhere: IAM least privilege (18.3) · security groups by ID (18.2/18.5) · Secrets Manager (18.5) · encryption at rest/in transit (18.4/18.5) · PDBs + graceful shutdown (18.9) · circuit breakers + bulkheads (18.12).
Observability (18.11): CloudWatch carries infra metrics/logs/alarms (ALB, ECS/EKS, RDS, ElastiCache, SQS); LangSmith (Part 6) carries AI-quality traces — the two are correlated via a shared request ID, not merged into one system.
CI/CD (18.8): git push → test/lint → eval gate (Part 8.3, averaged) → build/scan/sign (18.7) → Terraform (18.6) → canary deploy → SLI-monitored rollout.
Tracing one request through the whole diagram
A user sends a chat message. Route 53 resolves the hostname; the ALB terminates TLS and routes to a healthy ECS/EKS task (health-checked, 18.5/18.9). FastAPI receives the request, generates a correlation ID (18.11), and invokes the compiled LangGraph (Part 5) with the session's thread_id, resuming from PostgresSaver-backed checkpoint state (Part 5.3, stored in RDS, 18.4) if this is a continuing conversation. The graph checks Redis (18.14) for a cached response to a semantically similar prior query (Part 7.8) — on a miss, it retrieves from the vector database (Part 3.4) and calls the LLM through the model router (Part 7.9/18.12), which enforces a timeout, retries with backoff, and would fail over to a secondary provider if the primary's circuit breaker is open. The response streams back (Part 5.5) through the same path; CloudWatch records infrastructure metrics for every hop, and LangSmith (Part 6.1) records the full AI-specific trace, joined to the same correlation ID (18.11) for unified debugging.
Tracing one document through the async side
A document is uploaded, landing in S3 (18.4); an S3 event fires SNS, fanning out to the ingestion SQS queue. A worker (running as part of an ECS/EKS-hosted pool, scaled on queue depth via KEDA, 18.9/18.15) picks up the message, checks RDS for already-completed stages (18.15's resumability), parses, chunks, embeds (Part 3.4), and upserts into the vector database idempotently — updating RDS's ingestion-status table as it goes, queryable by the original uploader (Part 7.6). A failure after repeated attempts routes to a DLQ (18.4), alerting via CloudWatch (18.11) rather than failing silently.
Where each earlier chapter's concern lives in this diagram
18.1 Linux → underlies every container/node in the diagram
18.2 Networking → the public/private subnet boundary throughout
18.3 AWS compute/IAM → ECS/EKS/Lambda choice, every role's permissions
18.4 AWS data/msg → RDS, ElastiCache, S3, SQS/SNS placement
18.5 VPC architecture → the exact network diagram this chapter reuses
18.6 Terraform → how every piece above is actually provisioned
18.7 Docker → what's actually running inside each ECS/EKS task
18.8 CI/CD → how a change reaches this running system safely
18.9 Kubernetes → if EKS is the chosen compute (18.3's decision)
18.10 K8s troubleshoot. → how you diagnose a broken piece of this diagram
18.11 Observability → the two monitoring layers drawn above
18.12 Reliability → the model router's resilience patterns
18.13 Scalability → which piece to scale under load, and how
18.14 Redis/caching → the cache layer's role and failure behavior
18.15 Queues/async → the entire async side of the diagram
18.16 AI infrastructure → what's INSIDE the "LLM Provider" box if self-hosted5. Simple mental model
This diagram is the completed floor plan of a building whose individual rooms (each earlier chapter) you've already toured in detail — this chapter's job is walking the whole building once, start to finish, so you can give someone a coherent tour of the entire structure, not just describe any one room in isolation.
6. Real-world example
This entire chapter's diagram is the real-world example — it is, concretely, the architecture Part 16.8's full FDE engagement narrative would produce as its Stage 4 (Production Engineering) deliverable, now seen fully assembled rather than described stage by stage.
7. Architecture diagram
(Section 4's diagram is this chapter's core artifact — reproduced there in full, deliberately not duplicated here to keep one single, canonical version.)
8. Production considerations
Every production consideration from 18.1–18.16 applies simultaneously to this assembled system — the discipline this chapter adds is checking they're consistent with each other: does the ALB's health-check timeout (18.5) align with the application's actual readiness-probe logic (18.9)? Does the circuit breaker's timeout (18.12) sit comfortably below the overall request timeout a user would experience? Does the async pipeline's worker scaling (18.15) actually use the KEDA/Prometheus Adapter (18.9) the design assumes is installed? Cross-chapter consistency checks like these are exactly what a real architecture review surfaces.
9. Common mistakes
- Being able to explain any individual component deeply while being unable to draw or narrate the complete, connected system fluently — precisely the gap this chapter exists to close.
- Designing each layer's timeouts/thresholds independently without checking they compose sensibly (e.g., a circuit breaker cooldown longer than a user would ever wait, making the fallback path pointless in practice).
- Treating the async (ingestion) and synchronous (chat) sides of the system as unrelated, when they share the same vector database, RDS instance, and observability stack, and can affect each other's performance (18.13's bottleneck-identification framework applies across both sides, not just the request-serving path).
10. Security considerations
The complete system's security posture is the sum of every earlier chapter's individual controls (18.2's network segmentation, 18.3's IAM, 18.4/18.5's encryption, 18.9's RBAC) — 18.19 reviews this holistically as its own dedicated topic, but the practical point here is that a security review examines the whole assembled system's consistency, not each component in isolation.
11. Performance considerations
18.13's bottleneck-identification framework should be applied to this entire assembled diagram under real, representative load — not to any one component's synthetic benchmark in isolation, since the actual constraint under real traffic could be at any one of many points in this now-complete picture.
12. Cost considerations
18.18 quantifies the cost of this exact assembled architecture, piece by piece — this chapter's contribution is making clear that a cost review needs the whole picture (including the async pipeline's NAT/SQS/embedding costs, not just the obviously-visible chat-endpoint compute) to be accurate.
13. When to use it
Any time you need to present, defend, or reason about the complete architecture of a production AI system — a customer architecture review, an internal design review, or an FDE system-design interview (17.1).
14. When NOT to over-apply it
A component-level deep-dive conversation (debugging one specific Kubernetes incident, 18.10) doesn't need the whole-system diagram — zoom in appropriately to the actual scope of the conversation, per 17.1's point about matching depth to how open-ended a given question actually is.
15. Alternatives and trade-offs
This chapter's specific architecture (ECS/EKS + RDS + ElastiCache + a managed vector database + SQS-based async processing) is one coherent, well-reasoned instantiation of this track's principles — not the only valid one; 18.22's senior system-design chapter deliberately varies the requirements to produce genuinely different, equally valid architectures.
16. Practical example — narrating the diagram out loud, timed
Practice narrating section 4's diagram, end to end, from a user's request through to the response, in under three minutes — matching 17.1's system-design-interview time-boxing discipline, since this is precisely the artifact that interview format asks you to produce live.
17. Production-quality example — the architecture as a single Terraform root module
hcl
# environments/production/main.tf — instantiating EVERY module this
# track has built, wired together into the complete section-4 diagram.
module "vpc" { source = "../../modules/vpc" /* 18.5/18.6 */ }
module "rds" { source = "../../modules/rds" /* 18.4 */ }
module "elasticache" { source = "../../modules/elasticache" /* 18.14 */ }
module "messaging" { source = "../../modules/messaging" /* 18.4/18.15 */ }
module "ecs_service" { source = "../../modules/ecs-service" /* 18.3/18.7 */ }
module "worker_pool" { source = "../../modules/worker-pool" /* 18.15 */ }
module "observability" { source = "../../modules/observability"/* 18.11 */ }The point of showing it this way: the entire production architecture this chapter diagrams is, concretely, one reviewable, versioned Terraform root module composed of the exact pieces built individually across 18.3–18.15 — the practical, buildable endpoint of everything this track has taught, not merely a conceptual diagram.
18. Short exercise
Without looking back at section 4, draw the complete architecture from memory, then compare against the actual diagram — specifically noting which components you forgot or misplaced, since that gap is exactly what further review should target before a real interview or customer presentation.
19. Interview questions
- Draw and narrate the complete architecture of a production RAG/agent system, end to end, in under five minutes.
- Where would you add a component to this diagram if the requirement were self-hosted models instead of an external API?
- Which two components in this diagram have the tightest coupling, and what would break if one were unavailable?
- Walk through what changes in this diagram for the async document- ingestion path specifically, versus the synchronous chat path.
20. FDE/customer scenario
A customer's CTO asks, in a single meeting: "Can you show me the whole architecture, from a user typing a message to getting a response, and everything in between?" This chapter's diagram and the request-tracing narrative in section 4 is, directly and concretely, the answer to give — practiced enough in advance (section 16's exercise) to deliver fluently, under real meeting-time pressure, exactly as 17.1 rehearses for an interview.
Key takeaways
- Being able to draw and narrate the COMPLETE architecture, end to end, is a distinct skill from understanding any individual component deeply — and it's the one a real customer meeting or interview actually tests.
- Every earlier chapter in this track has an exact, identifiable place in this one assembled diagram — synthesis, not new material, is this chapter's entire content.
- A real architecture review checks cross-component consistency (do the timeouts, health checks, and scaling triggers actually compose sensibly together), not just each component's individual correctness.
Things you should be able to explain
- The complete request path for both the synchronous chat flow and the asynchronous document-ingestion flow.
- Where every one of 18.1–18.16's concerns lives in the assembled diagram.
Things you should be able to build
- A single, coherent architecture diagram covering the entire system.
- A Terraform root module wiring together every component module built across this track.
Common mistakes
- Deep component knowledge without the ability to narrate the whole system fluently.
- Designing each layer's thresholds/timeouts independently, without checking cross-component consistency.
Recommended next chapter
18-cost-optimization-finops.md