Cache Stampede (Thundering Herd) Problem and Solutions
Fast recall
Quick revision
Core answer
Awareness of cache failure modes under load and practical mitigation strategies, not just knowing what the problem is called.
Proposing 'just set a longer TTL' as the solution — this reduces frequency but does not eliminate the stampede when the TTL eventually expires.
Explain the probabilistic early expiration algorithm's formula and discuss serving stale-while-revalidate as an HTTP-inspired pattern applicable to Redis.
Interview-ready guide
Interview answer
A cache stampede happens when a popular cache entry expires and many concurrent requests all miss simultaneously, all hit the database, and all try to write the same key back to the cache. At scale, this can take a database from healthy to overloaded in seconds.
Why it happens
- Under normal load, the cache handles thousands of requests per second for a popular entry; when that entry expires, all those requests reach the database at the same moment — which means the database receives the full request rate for that entry directly, rather than a trickle of cache misses
- The database query may be expensive (joining three tables, aggregating millions of rows), so every concurrent thread runs it simultaneously; the database can't handle that load, latency spikes, and the storm persists until the cache is repopulated
Mutex locking — one filler, others wait
- Use Redis's
SET key NX EX 10as a distributed mutex — only the first caller to acquire it queries the database and fills the cache; all other concurrent callers wait briefly and then read from the populated cache - The downside: callers that fail to acquire the lock must wait; if the database query is slow, they wait for the full query duration; implementation must handle the case where the lock holder fails before filling the cache
Probabilistic early expiry
- Rather than expiring all at once, each request has a small probability of re-fetching early based on the remaining TTL — requests start proactively refreshing the cache before it expires, so expiry never causes a synchronized miss
- The algorithm: as TTL decreases, the probability of early refresh increases; by the time the cache actually expires, the value has already been refreshed by a lucky early request
Background refresh — never expose expiry to requests
- A background thread monitors cache TTLs and proactively refreshes entries before they expire — user-facing requests always hit a warm cache because expiry happens in the background, not in the request path
- This is the cleanest solution for high-traffic hot keys: the cache is always warm, and the database query runs once per TTL period regardless of traffic volume
Cache Stampede (Thundering Herd) Problem and Solutions
Direct Answer
A cache stampede occurs when a popular cache entry expires and many concurrent requests all miss simultaneously, each triggering an expensive database or downstream service call at the same time, potentially overwhelming the backend. Solutions include probabilistic early expiration (PER), distributed locking to ensure only one request refreshes the cache while others wait or serve stale data, and background refresh patterns where the cache is populated asynchronously before expiry.
Code Example
@Service
public class PERCacheService {
private final RedisTemplate<String, String> redis;
private final ProductRepository repo;
private static final double BETA = 1.0; // Aggressiveness of early refresh
public Product getProduct(Long id) {
String key = "product:" + id;
String raw = redis.opsForValue().get(key);
if (raw == null) {
return refreshAndCache(id, key);
}
CachedEntry entry = deserialize(raw);
// PER formula: should we proactively refresh?
long ttl = redis.getExpire(key, TimeUnit.MILLISECONDS);
long delta = entry.getComputeTimeMs(); // How long DB query took
double random = -delta * BETA * Math.log(Math.random());
if (random >= ttl) {
// Probabilistically trigger early refresh
return refreshAndCache(id, key);
}
return entry.getProduct();
}
private Product refreshAndCache(Long id, String key) {
long start = System.currentTimeMillis();
Product product = repo.findById(id).orElseThrow();
long computeTime = System.currentTimeMillis() - start;
CachedEntry entry = new CachedEntry(product, computeTime);
redis.opsForValue().set(key, serialize(entry), Duration.ofMinutes(10));
return product;
}
}
How PER works: As TTL approaches zero, the probability of triggering an early refresh increases exponentially. With high concurrency, one request statistically refreshes early — before the actual expiry — so the key never truly expires. The BETA parameter controls aggressiveness: higher values cause earlier refreshes. This approach requires no locking and no coordination between threads.
Common Mistakes & Pitfalls
Proposing 'just set a longer TTL' as the solution — this reduces frequency but does not eliminate the stampede when the TTL eventually expires.
Check your understanding
Answer these without looking back at the article.
- How does the stale-while-revalidate pattern differ from the mutex-based approach?
- What is jitter in TTL and how does it help prevent stampedes on application startup?
- Can @Cacheable in Spring Boot be made stampede-proof without manual locking?
Related interview prep categories
1Looking for a structured roadmap?
Every module on this page is also part of the full Java backend interview track.