You added Redis in front of an expensive query, and it worked beautifully. The dashboard that took 800 ms now returns in 3 ms. The database CPU dropped from 70% to 10%. Everyone is happy.
Then, one afternoon, the database CPU spikes to 100%, latency goes vertical, and the incident channel lights up. The graphs show a sharp, periodic pattern: calm, spike, calm, spike. Nothing deployed. Traffic didn't change.
What happened is a cache stampede (also called the thundering herd or dogpile problem). A hot key expired, and for the next few hundred milliseconds, every request that wanted it missed the cache at the same moment and ran the expensive query itself.
This post explains exactly how that happens and builds up a set of defenses in Node.js with ioredis, from a ten-line fix to a production-grade helper.
Anatomy of a stampede
Here's the classic cache-aside code:
async function getLeaderboard() { const cached = await redis.get('leaderboard'); if (cached) return JSON.parse(cached); const data = await db.query(EXPENSIVE_SQL); // takes 800 ms await redis.set('leaderboard', JSON.stringify(data), 'EX', 60); return data; } Perfectly reasonable. Now watch what happens at the moment the key expires, with 500 requests per second hitting this endpoint:
t=0.000 key expires t=0.001 request 1: cache miss → starts the 800 ms query t=0.003 request 2: cache miss → starts the 800 ms query t=0.004 request 3: cache miss → starts the 800 ms query ... t=0.800 request 1 finishes, writes the cache (by now ~400 requests have missed and started the same query) The query takes 800 ms, so for 800 ms every request is a miss. At 500 req/s that's ~400 identical expensive queries launched against the database in under a second, instead of one. The database slows down under that load, which makes the queries take longer, which widens the miss window, which launches more queries. It's a feedback loop.
Related problems with similar names
People mix these up, but the fixes differ:
| Problem | What happens | Typical cause |
|---|---|---|
| Stampede / thundering herd | Many concurrent misses on one hot key | A popular key expires |
| Avalanche | Many different keys expire together, or the cache is gone | Same TTL set in a batch; Redis restart or failover |
| Penetration | Requests for data that doesn't exist always miss and hit the database | Missing IDs, scanners, bad input |
We'll address all three, since they feed each other.
Defense 1: Single-flight (collapse concurrent misses in one process)
The cheapest fix: if a load for a key is already in progress in this process, wait for it instead of starting another. Store the in-flight promise in a map.
const inflight = new Map<string, Promise<unknown>>(); async function singleFlight<T>(key: string, fn: () => Promise<T>): Promise<T> { const existing = inflight.get(key); if (existing) return existing as Promise<T>; const promise = fn().finally(() => inflight.delete(key)); inflight.set(key, promise); return promise; } async function getLeaderboard() { const cached = await redis.get('leaderboard'); if (cached) return JSON.parse(cached); return singleFlight('leaderboard', async () => { const data = await db.query(EXPENSIVE_SQL); await redis.set('leaderboard', JSON.stringify(data), 'EX', 60); return data; }); } Now 400 concurrent misses in one process become one database query, and all 400 callers share its result. Because Node.js is single-threaded, the check-then-set on the map has no race.
The limitation is in the name: it's per process. With 20 instances you get up to 20 concurrent loads per expiry, not 400, a big improvement but not a guarantee. For that, you need coordination through Redis.
Defense 2: A distributed lock so only one process rebuilds the value
Use Redis itself to elect one loader across the whole fleet. SET key value NX PX ttl sets the key only if it doesn't exist, atomically:
import { randomUUID } from 'node:crypto'; async function tryLock(key: string, ttlMs: number): Promise<string | null> { const token = randomUUID(); const ok = await redis.set(`lock:${key}`, token, 'PX', ttlMs, 'NX'); return ok === 'OK' ? token : null; } // Release only if we still own the lock. Compare-and-delete must be atomic, so use Lua. const RELEASE = ` if redis.call('get', KEYS[1]) == ARGV[1] then return redis.call('del', KEYS[1]) else return 0 end`; async function unlock(key: string, token: string) { await redis.eval(RELEASE, 1, `lock:${key}`, token); } Two details matter:
- The random token + Lua release ensures you never delete someone else's lock. Without it, if your work outlasts the lock's TTL, another process acquires the lock, and your delayed
DELreleases their lock. - The lock TTL is a safety net, not a timer. It must exceed your worst-case load time, so a crashed process can't hold the lock forever.
Then the miss path becomes:
async function getWithLock<T>(key: string, loader: () => Promise<T>, ttlSec: number): Promise<T> { const cached = await redis.get(key); if (cached) return JSON.parse(cached); const token = await tryLock(key, 10_000); if (token) { try { const value = await loader(); await redis.set(key, JSON.stringify(value), 'EX', ttlSec); return value; } finally { await unlock(key, token); } } // Someone else is loading. Wait briefly for their result instead of piling onto the database. for (let i = 0; i < 40; i++) { // ~2s max await sleep(50); const filled = await redis.get(key); if (filled) return JSON.parse(filled); } return loader(); // gave up waiting: load it ourselves } Is this a "correct" distributed lock? Not in the strict sense. A paused process can outlive its lease, and a single Redis node can fail over mid-lock. But for cache filling that doesn't matter: the worst case is that two processes load the same value and write the same result. You're using the lock to reduce duplicate work, not to guard a critical section, so a simple SET NX PX is enough. (If you're protecting something where duplicates are harmful, that's a different problem and needs fencing tokens or a database constraint, not a cache lock.)
The downside: waiters are blocked. Everyone who missed is stuck polling for up to the load time. The next defense removes that wait entirely.
Defense 3: Stale-while-revalidate (never make users wait)
The insight: for most cached data, slightly old data is better than a slow response. So don't delete the value when it "expires". Keep it around, mark it stale, and refresh it in the background while still serving the old copy.
Store two expirations: a soft TTL (when it becomes stale) and a hard TTL (when Redis actually deletes it), longer by a grace window.
interface Entry<T> { v: T; softExp: number; // epoch ms: after this, the value is stale and should be refreshed } async function set<T>(key: string, value: T, softTtlMs: number, graceMs: number) { const entry: Entry<T> = { v: value, softExp: Date.now() + softTtlMs }; await redis.set(key, JSON.stringify(entry), 'PX', softTtlMs + graceMs); } On read:
- Fresh (
now < softExp): return it. - Stale (past
softExpbut still in Redis): return it immediately, and if you win the lock, refresh in the background. - Missing (past the hard TTL): load synchronously. This is the only path where someone waits.
Now the expensive load happens off the request path. Users see 3 ms responses all the time, and the database sees one refresh per interval. As a bonus, if the refresh fails (the database is struggling), you keep serving the stale copy, a "stale-if-error" safety net that turns a database outage into slightly old data rather than errors.
Defense 4: Probabilistic early refresh (no locks needed)
Another approach avoids coordination entirely: have requests randomly refresh the value shortly before it expires, with probability that rises as expiry approaches. By the time the key would expire, someone has almost certainly already refreshed it.
This is the XFetch algorithm from the paper Optimal Probabilistic Cache Stampede Prevention (Vattani et al.). Store how long the value took to compute (delta), and on each read decide:
// Refresh early if: now - delta * beta * ln(random()) >= expiry function shouldRefreshEarly(entry: { softExp: number; delta: number }, beta = 1): boolean { return Date.now() - entry.delta * beta * Math.log(Math.random()) >= entry.softExp; } Math.log(Math.random()) is negative, so the subtraction pushes "now" forward by a random amount. The expression is scaled by delta:
- Expensive values (large
delta) start refreshing earlier, because they need more lead time. - Cheap values refresh almost right at expiry.
-
beta(default 1) tunes aggressiveness. Higher means earlier refreshes and more cache churn.
Used alone, XFetch can still let a few concurrent requests decide to refresh simultaneously, but instead of hundreds it's a handful. Combined with single-flight or a lock, it's very effective.
Defense 5: Prevent synchronized expiry with TTL jitter
If you warm 10,000 keys at deploy time with EX 3600, they all expire in the same second an hour later. That's the avalanche pattern. The fix is one line: add randomness.
const jitter = (ms: number, spread = 0.1) => Math.round(ms * (1 - spread + Math.random() * spread * 2)); // ±10% await redis.set(key, value, 'PX', jitter(60 * 60 * 1000)); Keys written together now expire spread across a window instead of all at once. It's trivial, it costs nothing, and it should be your default for every cached key.
Defense 6: Negative caching (stop penetration)
If a client requests /users/999999999 (an ID that doesn't exist), the cache never has it, so every request goes to the database, forever. Scanners and bugs generate these constantly.
Cache the absence too, with a short TTL:
// Wrap values so that "null" is a legitimate cached result, distinct from "key not in cache" const entry = { v: user ?? null, softExp: Date.now() + (user ? 60_000 : 10_000) }; Use a shorter TTL for "not found" (seconds, not minutes) so a newly created record shows up quickly, or invalidate that key on create. This is why the entry wrapper above stores v inside an object. A raw null would be indistinguishable from a miss.
For very large, hostile key spaces, a Bloom filter in front of the cache can reject IDs that definitely don't exist, but start with negative caching.
Putting it together: a reusable helper
Here's a compact implementation combining single-flight, Redis lock, stale-while-revalidate, XFetch, jitter, and a safe fallback if Redis is unhealthy:
// cache.ts import { randomUUID } from 'node:crypto'; import type Redis from 'ioredis'; interface Entry<T> { v: T; softExp: number; delta: number } export interface CacheOpts { ttlMs: number; // how long the value is considered fresh staleMs?: number; // extra time it may be served stale while refreshing (default: 5x ttlMs) lockMs?: number; // lock safety TTL: must exceed worst-case load time waitMs?: number; // how long a cold-miss waiter polls for another loader's result } const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms)); const jitter = (ms: number) => Math.round(ms * (0.9 + Math.random() * 0.2)); const RELEASE = ` if redis.call('get', KEYS[1]) == ARGV[1] then return redis.call('del', KEYS[1]) else return 0 end`; export function createCache(redis: Redis, log = console) { const inflight = new Map<string, Promise<unknown>>(); async function tryLock(key: string, ms: number) { const token = randomUUID(); const ok = await redis.set(`lock:${key}`, token, 'PX', ms, 'NX'); return ok === 'OK' ? token : null; } const unlock = (key: string, token: string) => redis.eval(RELEASE, 1, `lock:${key}`, token).catch(() => {}); async function load<T>(key: string, loader: () => Promise<T>, o: Required<CacheOpts>): Promise<T> { const started = Date.now(); const v = await loader(); const delta = Date.now() - started; // how long the load took (for XFetch) const softTtl = jitter(o.ttlMs); const entry: Entry<T> = { v, softExp: Date.now() + softTtl, delta }; await redis.set(key, JSON.stringify(entry), 'PX', softTtl + o.staleMs).catch((err) => log.error('cache set failed', err), ); return v; } async function read<T>(key: string, loader: () => Promise<T>, o: Required<CacheOpts>): Promise<T> { let raw: string | null; try { raw = await redis.get(key); } catch (err) { log.error('redis get failed; loading directly', err); return loader(); // fail open (see "When Redis is down") } const entry: Entry<T> | null = raw ? JSON.parse(raw) : null; const now = Date.now(); // 1. Fresh and not selected for early refresh: serve it if (entry && now < entry.softExp && now - entry.delta * Math.log(Math.random()) < entry.softExp) { return entry.v; } // 2. Stale or about to expire: serve the old value, one refresher updates it in the background if (entry) { const token = await tryLock(key, o.lockMs).catch(() => null); if (token) { load(key, loader, o) .catch((err) => log.error('background refresh failed; serving stale', err)) .finally(() => unlock(key, token)); } return entry.v; } // 3. True miss: one caller loads; the others wait briefly for the result const token = await tryLock(key, o.lockMs).catch(() => null); if (token) { try { return await load(key, loader, o); } finally { await unlock(key, token); } } const deadline = Date.now() + o.waitMs; while (Date.now() < deadline) { await sleep(50 + Math.random() * 50); const filled = await redis.get(key).catch(() => null); if (filled) return (JSON.parse(filled) as Entry<T>).v; } return load(key, loader, o); // waited long enough: load it ourselves } return function cached<T>(key: string, loader: () => Promise<T>, opts: CacheOpts): Promise<T> { const o: Required<CacheOpts> = { staleMs: optsSource：https://dev.to/subashthiruppathy_5e0f532/cache-stampede-thundering-herd-in-nodejs-redis-preventing-database-meltdowns-on-cache-misses-693Related posts
Architectural Blueprint: Enterprise AI Application Development Platforms for Deep Legacy Software Integration
Generating WhatsApp Templates With an LLM: Category Rules, Invented Facts and Checks in Code
ValiNum v2.2.0: Phone Number Validation for 10 More African Countries
Playwright Page Object Model in TypeScript (the Right Way)
Taking payment before the user installs your Chrome extension
