Redis Stopped Writing at 3 AM: maxmemory + the Eviction Policy Nobody Set

Our cache stopped accepting writes overnight. Not crashed — just refused. Every SET, HSET, LPUSH came back with the same line:

(error) OOM command not allowed when used memory > 'maxmemory'

Reads still worked. That's the confusing part, and the part that sends most people down the wrong rabbit hole (restarting, dumping the whole DB, praying). The cache wasn't broken. It was doing exactly what it was told: you gave it a hard ceiling and told it to evict nothing, so the moment it hit the ceiling it stopped instead of growing.

Here's the 20-minute drill I run before I touch a single config.

Minute 1: read the memory snapshot 

redis-cli INFO memory

Four lines matter, in order:

Line What I'm looking for
used_memory_human How close to the ceiling we actually are
maxmemory_human Whether the ceiling is even set (0 = unlimited = eventual system OOM kill)
maxmemory_policy The real culprit — noeviction means "refuse writes, don't delete anything"
mem_fragmentation_ratio If it's well over 1.x, the process holds more RSS than logical memory — swap or fragmentation, not a cache-size problem

The trap I keep finding in production: maxmemory_policy is volatile-lru but not a single key has a TTL. volatile-lru only evicts keys that expire. With zero expiring keys it behaves exactly like noeviction — and you get OOM errors despite having a "safe" policy set. This one is invisible unless you check the actual key space.

Minute 2: find what's eating it 

redis-cli --bigkeys

That surfaces the largest key per type. Then confirm the suspect:

redis-cli MEMORY USAGE <key>

Two common offenders:

  • A hash or list that grew to tens of thousands of members because nobody paginated the producer.
  • Client output buffers: CLIENT LIST shows a omem (output buffer) that's huge — a slow consumer is holding responses in memory instead of the cache holding data.

When you delete a big key, use UNLINK, not DEL. DEL is synchronous and blocks the whole instance while it frees — on a multi-hundred-MB hash that can stall the server for seconds. UNLINK frees it in the background.

Minute 3: choose the policy for what you actually are

  • Pure cache → allkeys-lru. Hot-swap it live: 
 redis-cli CONFIG SET maxmemory-policy allkeys-lru redis-cli CONFIG REWRITE
  • Session store (keys already carry TTLs) → volatile-ttl, evict the soonest-expiring first.
  • Need to protect data → noeviction is fine, if you alert on memory at 80% and have headroom. noeviction with no alerting is how you end up here at 3 AM.

The change that would have prevented the page

Set the alert threshold to 80% of maxmemory and make every write path set a TTL by default (SET key value EX 3600 instead of a bare SET). Then OOM stops being a 3 AM surprise and becomes a normal Monday tuning call.

Redis memory problems are usually a key-lifecycle problem, not a disk problem: every key should know when it dies the moment it's written.

Tags: redis, caching, linux, devops, troubleshooting

More field-tested ops notes and server notes: jiunotes · ops notes