Distributed Caching Strategy — Patterns and Trade-offs

Clarifying questions I'd ask first

• What's the read/write ratio per data type?
• How stale can the cached data be — seconds, minutes, hours?
• Multi-instance fleet — do they need to agree on cached values?
• Critical paths that can never read stale (payments, inventory)?
• Hot key suspects (top product, viral content)?

Layered caching topology

• L1 in-process (Caffeine): ~100ns, hottest keys, eventual consistency across instances
• L2 distributed (Redis cluster): ~1ms, shared state, sharded by key hash
• L3 CDN/edge: ~10-50ms, public HTTP responses with short TTLs

Read/write patterns — pick per use case

• Cache-aside is the safe 95% default — app reads cache, on miss reads DB, populates cache; on write, updates DB then invalidates cache
• Read-through wraps the miss in the cache library (Caffeine LoadingCache)
• Write-through writes to cache and DB synchronously — strong consistency at write-latency cost
• Write-behind batches DB writes asynchronously — fast but risks data loss; reserve for metrics/counters

Eviction

• LRU default; LFU (W-TinyLFU in Caffeine) for stable hot sets
• TTL is the freshness contract; LRU is the size contract — use both
• Jitter TTLs (base ± random) to avoid synchronized expiry stampedes

Stampede mitigation — multiple defenses together

• Singleflight: first miss locks the key, others wait and reuse the result
• Jittered TTL spreads expiries
• Probabilistic early refresh smooths the cliff
• Stale-while-revalidate returns stale immediately, refreshes in background
• Negative caching for missing-key probes

Hot keys

• Local L1 absorbs heat for super-hot keys
• Shadow-key replication (product:42:0, :1, ...) spreads across shards
• Read replicas for read-heavy hot keys
• Pre-aggregate counters in app then flush

Multi-instance L1 coherence

• Versioned keys eliminate read-after-write races
• Pub/sub invalidation bus broadcasts evicts to all app instances; bounded staleness in milliseconds

LLD

• One LayeredCacheManager façade hides tiers from app code
• CacheLayer interface (Caffeine impl, Redis impl, in-memory test impl)
• StampedeGuard.singleflight(key, loader) for the miss path
• TtlPolicy centralizes per-key TTL
• InvalidationBus keeps L1s coherent

Failure modes

• Cache outage = DB overload (hit rate collapse → 100x DB traffic) — needs circuit breaker + L1 fallback
• Split-brain on concurrent writes — versioned keys with monotonic version check
• Stale-after-deploy — schema version in cache key
• Memory pressure — alert on evicted_keys/sec before thrashing

Recommendation

Three-tier topology (L1 Caffeine + L2 Redis + L3 CDN), cache-aside as the default pattern with read-through where the abstraction fits, LRU+TTL with jitter, singleflight for stampedes, local L1 for hot keys, versioned keys with pub/sub invalidation for multi-instance coherence, and a circuit breaker on the source DB so cache outage doesn't cascade. That's the same shape Netflix, Stripe, and modern Spring/Java stacks run.