A server runs out of memory. One application uses 12 gigabytes, another uses 4 gigabytes, and a small worker suddenly disappears with Killed printed in the terminal. The obvious accusation is that Linux chose the wrong victim.
That conclusion rests on a common misconception: the Linux out-of-memory killer does not simply terminate the process using the most memory.
When the kernel cannot satisfy a memory allocation and reclaiming pages is no longer enough, it may invoke the out-of-memory killer. The kernel assigns eligible processes a “badness” score and selects a victim based on that score. Memory consumption is important, but it is not the only input.
A process exposes its current score through:
/proc/<pid>/oom_score Administrators can influence the calculation through:
/proc/<pid>/oom_score_adj oom_score_adj ranges from -1000 to 1000. Positive values make a process more attractive as a victim. Negative values protect it. At -1000, the process is effectively excluded from normal OOM selection.
This means a moderately sized worker with a high adjustment can be killed before a much larger database with a negative adjustment. Service managers and container runtimes may set these values deliberately so that replaceable workloads die before critical infrastructure.
The location of the memory shortage matters too. Linux can encounter a system-wide OOM condition, but memory control groups can also impose local limits. If a container exceeds its cgroup memory limit, the kernel generally chooses among processes inside that constrained cgroup. A large process elsewhere on the host may be irrelevant because killing it would not resolve the violated cgroup limit.
That distinction explains a confusing operational pattern: the host still appears to have available memory, yet a process inside a container is OOM-killed. The exhausted resource was the container’s permitted memory, not necessarily all physical memory in the machine.
Shared memory, page usage, process relationships, and whether a task can free useful memory may also affect victim selection. The calculation is intended to recover from the immediate failure with an acceptable sacrifice, not to punish whichever process tops a memory-usage chart.
After an unexpected death, checking only ps, top, or a monitoring dashboard can therefore produce the wrong diagnosis. Kernel logs often identify an OOM event and its selected victim. On systems using systemd, relevant messages may be found with:
journalctl -k For a suspected process, inspect its adjustment and score:
cat /proc/<pid>/oom_score cat /proc/<pid>/oom_score_adj In containerized environments, also examine the workload’s cgroup memory limit and memory event counters. An exit status of 137 is consistent with termination by SIGKILL, but it does not prove by itself that the kernel’s OOM killer was responsible; another process or an administrator can send the same signal.
The correct mental model is straightforward: Linux chooses an eligible victim within the scope of the memory failure using a weighted badness calculation. The largest process often scores highly, but size alone does not decide who dies.
