18.14 — Redis and Caching for AI Systems

Relationship to Part 7.8: Part 7.8 already taught caching strategy for AI systems in depth (exact/semantic caching, TTLs, the dead-code bug in one example, the prompt-caching minimum-prefix-length point). This chapter is the infrastructure layer underneath it: how Redis itself is deployed, networked, replicated, and fails in production — not re-teaching cache-aside or TTL strategy, which Part 7.8 already owns.

1. What is it?

Redis's operational characteristics as a piece of production infrastructure: deployment topology (standalone vs. replicated vs. clustered), persistence options, high-availability/failover mechanics, and the specific AWS implementation (ElastiCache, 18.4) — plus the infrastructure-level version of Part 7.8's "when NOT to cache" question.

2. Why does it exist?

Part 7.8 assumed a working Redis instance and taught how to use it well. This chapter exists because "how do I use Redis well" and "how do I keep Redis itself available, sized correctly, and not silently losing critical data" are different questions — and the second one is squarely infrastructure, not application-design.

3. What problem does it solve?

It solves "our Redis instance went down and something we didn't expect to break, broke" and "our Redis is out of memory and evicting keys we needed" — both real, recurring operational incidents distinct from the caching- strategy questions Part 7.8 already answers.

4. How does it work internally?

Redis's execution model and why it matters for capacity planning

Redis is, for most operations, single-threaded for command execution (newer versions offload some I/O to background threads, but the core command-processing loop remains effectively single-threaded) — meaning a single slow command (a very large KEYS * scan, or an expensive computation on a huge value) blocks every other client's requests during that time, not just the slow one's own caller. This is precisely why KEYS is a well-known anti-pattern in production Redis (it scans the entire keyspace, blocking everything else while it runs) and SCAN (which iterates incrementally without blocking) is the correct alternative — a concrete, mechanical reason, not an arbitrary style rule.

Persistence: RDB and AOF

Redis is an in-memory store first, but supports two persistence mechanisms for surviving a restart: RDB (point-in-time snapshots, written periodically) and AOF (Append-Only File, logging every write operation, replayed on restart for a more complete, more durable but slower recovery). For a pure cache use case (the Enterprise AI Assistant's LLM-response cache, Part 7.8) — where every cached value is, by design, always recomputable from the source of truth (a fresh LLM call) — persistence is often deliberately disabled or minimal, since losing the cache on restart is an acceptable performance hit, not a correctness problem. For a Redis use case that holds data with no other source of truth (rate-limiting counters mid-window, a distributed lock's current holder, session data not also persisted elsewhere) losing it on an unplanned restart is a real, different-severity problem — the persistence configuration should match which category a given use of Redis actually falls into, not be a single, one-size-fits-all setting applied without this distinction.

Redis as a cache vs. Redis as a system of record — restated precisely

Part 7.8 already established Redis should be treated as a cache, not a system of record — the infrastructure-level version of this principle: never configure critical business logic to depend on Redis data surviving a restart/failover without an independent source of truth. For the Enterprise AI Assistant: LLM response caching (safe to lose, Part 7.8), rate-limiting counters (usually acceptable to lose — a brief reset of a rate limit after a failover is a minor, tolerable side effect, not a correctness failure), but LangGraph checkpoint state specifically belongs in PostgresSaver (Part 5.3), not a Redis-backed checkpointer for anything requiring durability guarantees — a distinction worth being explicit about since LangGraph does support alternative checkpointer backends, and choosing one without this durability reasoning is a design mistake, not a value-neutral implementation detail.

High availability: replication and Redis Sentinel/Cluster

A Redis replica asynchronously copies data from a primary — useful for read scaling and as a failover target, but asynchronous replication means a small window of recently-written data can be lost if the primary fails before replicating to the replica (relevant precisely to the persistence-category distinction above: acceptable for a cache, a real consideration for anything else). Redis Sentinel monitors a primary/replica set and performs automatic failover (promoting a replica) if the primary becomes unavailable. Redis Cluster shards data across multiple primary nodes for horizontal scaling beyond a single node's memory capacity, each shard with its own replicas for availability. On AWS, ElastiCache (18.4) manages most of this directly: a replication group with Multi-AZ enabled provides automatic failover without you operating Sentinel yourself.

Failure modes, specifically

SYMPTOM: Application errors spike; Redis-dependent code paths fail.

POSSIBLE CAUSES:
  - Redis instance/replication-group failover in progress (a brief,
    normal event, 18.4) — should be a short blip, not a sustained outage
  - Redis out of memory, evicting keys under its configured eviction
    policy (see below) — NOT the same failure mode as Redis being down
  - Network partition between the app and Redis (18.2's territory)
  - A single slow command blocking the single-threaded command loop
    (section 4) — high latency on ALL Redis operations simultaneously,
    not a specific key or client

WHAT TO CHECK / COMMAND:
  redis-cli --latency                 # ongoing command-latency sampling
  redis-cli INFO memory               # used_memory vs maxmemory, eviction stats
  redis-cli --bigkeys                 # find unexpectedly large keys/values
  redis-cli SLOWLOG GET 10            # recent slow commands — directly
                                        # surfaces a section-4 blocking-
                                        # command culprit

FIX: depends on cause — evict-policy tuning or capacity increase for
     memory pressure; query/data-shape fix for a slow-command culprit;
     verify failover completed and DNS/endpoint updated correctly for a
     failover event.

PREVENTION: application code should NEVER treat Redis as required for
            correctness (Part 7.8's principle) — a Redis outage should
            degrade performance, not break functionality, IF this
            principle was actually followed in the application design.

Eviction policies

When Redis reaches its configured maxmemory, an eviction policy determines what happens next: noeviction (reject new writes, safest for non-cache data but breaks a cache workload outright), allkeys-lru (evict the least-recently-used key across the whole keyspace — a sensible default for a pure-cache use case), or several more targeted variants. Choosing noeviction for what's actually a pure LLM-response cache is a common, consequential misconfiguration — it turns "Redis is full" into "writes start failing" rather than "old cache entries are evicted," converting a performance-only concern (Part 7.8's whole premise) into an availability incident instead.

Rate limiting and locks, implemented on Redis

Part 7.8 covered why Redis suits rate limiting (fast, shared, atomic operations) — the concrete mechanism is Redis's atomic increment (INCR) combined with an expiry, forming a simple, correct token/fixed- window counter without a race condition, because the increment and the existence check happen atomically from Redis's single-threaded execution model (section 4) rather than as separate, race-prone steps in application code. A distributed lock (coordinating exclusive access to a resource across multiple application replicas — relevant for, say, ensuring only one worker processes a specific document at a time, Part 7.7/18.15) built on Redis needs care around lock expiry and ownership verification (the Redlock algorithm, or a simpler single-instance SET key value NX EX ttl pattern for less strict requirements) — a real, non-trivial correctness topic beyond this chapter's infrastructure scope, flagged here as something to design deliberately rather than improvise.

5. Simple mental model

Redis is like a very fast, shared whiteboard in an open office — anyone can read or write to it instantly, but if someone starts a very long drawing (a slow command) everyone else has to wait, and nothing written on it is guaranteed to survive if the office (the Redis process) needs to be evacuated and rebuilt (a failover or restart) — which is exactly why you'd never write the one and only copy of something irreplaceable there.

6. Real-world example

An ElastiCache Multi-AZ replication group serving the Enterprise AI Assistant's LLM-response cache (Part 7.8) undergoes an automatic failover during a scheduled maintenance window. Because the application treats every cache miss as "call the LLM fresh" (never as an error) and the eviction policy is correctly set to allkeys-lru, the failover is invisible to users beyond a brief latency uptick as the cache repopulates — directly validating Part 7.8's "never treat cache availability as a correctness dependency" principle at the infrastructure-failure level, not just in theory.

7. Architecture diagram

Application tierECS/EKS, private subnet (18.5) — cache-aside pattern (Part 7.8): check cache → miss → call LLM → write to cache → return

ElastiCache Replication Group · Multi-AZ

Primary (AZ-a)

Replica (AZ-b)

8. Production considerations

• Set eviction policy deliberately based on what a given Redis instance actually holds — allkeys-lru for a pure cache, noeviction only for data where losing a write silently would be worse than rejecting it.
• Enable Multi-AZ/replication for production Redis — a single-node cache is a single point of failure that, per Part 7.8's principle, should degrade rather than break the application, but only if that principle was actually followed everywhere it's used.
• Never run KEYS * against a production Redis instance — use SCAN.
• Monitor SLOWLOG and memory usage proactively (18.11), not only when an incident is already underway.

9. Common mistakes

• Configuring noeviction on what's actually a pure cache, turning a memory-pressure event into a write-failure incident.
• Running KEYS * in application code or an ad-hoc debugging session against a production instance, blocking every other client briefly.
• Storing something that has no other source of truth (a critical, unrecoverable piece of state) in Redis without deliberately reasoning about its persistence/durability configuration first.
• Assuming ElastiCache's Multi-AZ failover is instantaneous and lossless — it's fast and automatic, but asynchronous replication still has a real (usually small) window of potential data loss.

10. Security considerations

Redis should sit in a private subnet with a security group scoped to the application tier only (18.2/18.5's pattern) — an internet-exposed, unauthenticated Redis instance is a well-known, serious, and unfortunately common real-world misconfiguration. Redis AUTH (a password) and, where supported, TLS in transit should be enabled for any Redis instance holding even moderately sensitive cached data (Part 9.3's data-exposure reasoning, applied to cache contents specifically).

11. Performance considerations

Redis's single-threaded execution model (section 4) means overall throughput is bounded by command complexity, not just request count — prefer SCAN over KEYS, avoid storing very large single values, and use pipelining (batching multiple commands into one round trip) for high-throughput access patterns rather than many individual round trips.

12. Cost considerations

ElastiCache bills for provisioned node capacity continuously, whether or not the cache is well-utilized — right-sizing based on actual memory usage and hit rate (rather than provisioning generously "to be safe") is a direct, concrete cost lever (18.18), and a low cache-hit rate specifically means you're paying for capacity that isn't actually reducing LLM-call volume (Part 7.8's actual cost justification for caching in the first place).

13. When to use it

Any production AI system using Redis for caching, rate limiting, or coordination (Part 7.8's use cases) needs this chapter's operational discipline — deployed with replication/failover, monitored, and with a deliberately-chosen eviction policy matching its actual role.

14. When NOT to over-apply it

A local prototype (Part 13.2) can run a single, unreplicated Redis container with default settings — this chapter's HA/persistence rigor is a production-readiness investment, not a prototyping requirement.

15. Alternatives and trade-offs

Memcached is a simpler, purely-caching alternative to Redis (no persistence, no data structures beyond simple key-value, no built-in pub/sub) — appropriate when you genuinely need nothing beyond simple caching and want the operational simplicity that narrower scope brings; Redis's richer feature set (atomic counters, sorted sets, pub/sub, Part 7.8's rate-limiting and locking use cases) is exactly why it's the more common default for AI systems needing more than pure caching.

16. Practical example — diagnosing a slow command with SLOWLOG

redis-cli CONFIG SET slowlog-log-slower-than 10000   # log commands >10ms
redis-cli SLOWLOG GET 10
# 1) (integer) 14
#    (integer) 1699999999
#    (integer) 25000          <- microseconds: 25ms, a real slow command
#    1) "KEYS"
#    2) "*"

Finding KEYS * in the slowlog is a direct, actionable finding — identify what code path issued it and replace it with SCAN, rather than scaling Redis capacity to compensate for an avoidable anti-pattern.

17. Production-quality example — Terraform for a Multi-AZ ElastiCache replication group

As with every specific version/instance-type figure in this handbook, verify current supported ElastiCache engine versions and instance types against AWS documentation before using the values below — engine_version and node_type in particular change as AWS deprecates old engine versions and introduces new instance generations.

resource "aws_elasticache_replication_group" "cache" {
  replication_group_id       = "ai-assistant-cache"
  description                 = "LLM response + rate-limit cache"
  node_type                   = "cache.r6g.large"
  num_cache_clusters           = 2               # primary + 1 replica
  automatic_failover_enabled  = true             # Multi-AZ failover
  multi_az_enabled            = true
  engine                       = "redis"
  engine_version               = "7.1"
  parameter_group_name        = aws_elasticache_parameter_group.cache.name
  subnet_group_name           = aws_elasticache_subnet_group.data_tier.name
  security_group_ids          = [aws_security_group.redis.id]  # app-tier only, 18.5
  at_rest_encryption_enabled  = true
  transit_encryption_enabled  = true
  auth_token                   = var.redis_auth_token           # from Secrets Manager
}

resource "aws_elasticache_parameter_group" "cache" {
  family = "redis7"
  name   = "ai-assistant-cache-params"
  parameter {
    name  = "maxmemory-policy"
    value = "allkeys-lru"   # section 4: correct for a PURE cache use case
  }
}

The maxmemory-policy set explicitly to allkeys-lru (rather than left at whatever the engine default is) is the concrete implementation of section 4's point: this Redis instance is a pure cache, and its configuration should say so deliberately, not by accident.

18. Short exercise

For the Enterprise AI Assistant's three actual Redis use cases (LLM response cache, rate-limiting counters, a hypothetical distributed lock for document-processing coordination), decide the right persistence and eviction-policy configuration for each individually, arguing from section 4's durability-category distinction rather than applying one setting to all three uniformly.

19. Interview questions

• Why is KEYS * dangerous in production Redis, mechanically?
• What's the difference in acceptable data loss between a cache use case and a rate-limiting or locking use case, and how does that affect configuration choices?
• Walk through what happens during an ElastiCache Multi-AZ failover, and why asynchronous replication means it isn't perfectly lossless.
• When would you choose noeviction over allkeys-lru, and why would choosing it for a pure cache be a mistake?

20. FDE/customer scenario

A customer's engineering team reports: "Our Redis cache went down during a failover and part of our AI application broke." A strong response investigates specifically which Redis use case broke — if it was the LLM-response cache itself causing a hard failure rather than merely a recomputation, the actual root cause is an application-level violation of Part 7.8's "cache is never required for correctness" principle, not primarily an infrastructure or failover-mechanics problem — and the fix is at the application layer (treat cache misses/errors as a fallback path, not a failure), not simply "make Redis more available."

---

Key takeaways

• Redis's single-threaded command execution means one slow command (most commonly KEYS *) blocks every other client — use SCAN and monitor SLOWLOG instead.
• Eviction policy should match what a given Redis instance actually holds — allkeys-lru for a pure cache, noeviction only where a rejected write is genuinely safer than a silently evicted one.
• A Redis outage should degrade performance, not break functionality — when it breaks functionality, that's usually a violation of Part 7.8's cache-is-never-required-for-correctness principle at the application layer, not primarily a Redis infrastructure problem.

Things you should be able to explain

• Why KEYS * is dangerous and SCAN is the correct alternative.
• The durability-category distinction between cache data and data with no other source of truth, and why it should drive persistence config.
• What happens, and what can be lost, during a Multi-AZ Redis failover.

Things you should be able to build

• A Multi-AZ ElastiCache replication group with a deliberately-chosen eviction policy and encryption enabled.
• A SLOWLOG-based diagnosis of a Redis performance incident.

Common mistakes

• noeviction configured for what's actually a pure cache use case.
• KEYS * run against production Redis.
• Application code that breaks (rather than degrades) when Redis is unavailable, violating the cache-is-not-a-system-of-record principle.

Recommended next chapter

15-message-queues-and-async-architecture.md