Your Server Died at 3 AM: Reading the Linux OOM Killer's Report (and Stopping It)

It happens to every ops person eventually. The on-call page fires at 03:12: a customer-facing node has stopped answering, yet the process list looks normal and CPU is low. You SSH in, and the first line of dmesg tells you what actually happened:

Out of memory: Killed process 4123 (java) total-vm:31457280kB, anon-rss:28901376kB, file-rss:1024kB

The kernel did not crash. It did not run out of disk. It ran out of memory, and it made a decision for you: it picked one process and killed it, hard. That decision is the OOM killer, and the moment you learn to read its report, half of "mystery" 3 AM incidents stop being mysteries.

This post walks through how the OOM killer chooses its victim, how to read the kernel log fields that matter, and the specific knobs (systemd OOMScoreAdjust, cgroup MemoryMax/MemoryHigh, and a few sysctl values) that let you control who dies and how fast the box degrades. The goal is not to "add more memory." The goal is to make the failure contained, predictable, and easy to explain in the postmortem.

What the OOM killer actually does

The kernel keeps a per-process accounting of memory. When an allocation cannot be satisfied and the reclaimer has genuinely given up, the kernel walks the candidate process tree, computes a "badness" score for each one, and kills the highest.

The score is roughly proportional to resident anonymous memory (RSS), because that is the memory that, once the process dies, is actually returned to the system. Virtual memory (total-vm) mostly does not count: it includes mmap'd files and address-space reservations that may never have touched RAM. That distinction is the most common source of operator confusion, and it is why the OOM killer sometimes appears to have "picked the wrong process."

There are two things worth internalizing:

1. The victim is rarely the culprit. A long-running JVM with a 12 GB heap will almost always outscore a small, misbehaving cron job, even if the cron job is the one that just spiked. The killer rewards the process that is currently resident, not the one that allocated too much.
2. "Just add swap" is not a fix. Swap delays the moment the killer fires, but on a host that is already paging hard it makes everything slow while the OOM still happens. Swap is a cushion for temporary pressure, not a remedy for a leak or an undersized box.

Reading the kernel report

The kernel always leaves a trail. Read both of these; on many systems one of them is empty when you need it.

dmesg -T | grep -iE 'killed process|out of memory|oom'
journalctl -k --since "2 hours ago" | grep -i oom

The line you are looking for looks like this:

Out of memory: Killed process 4123 (java) total-vm:31457280kB, anon-rss:28901376kB, file-rss:1024kB, shmem-rss:0kB, UID:0 pguid:0
oom_score_adj:0

Now, the field-by-field breakdown that actually matters:

Field	What it is	What to do with it
total-vm	Virtual memory size. Mostly meaningless for diagnosis (includes reservations and file-backed maps).	Do not use it to judge "how big this process is."
anon-rss	Resident anonymous (heap/stack) memory. This is what the killer cares about.	Compare against the process's normal baseline.
file-rss / shmem-rss	File-backed and shared-memory resident pages. Reclaimable, lower priority.	Usually not the story in an app OOM.
oom_score_adj	The tuning knob, range -1000 to +1000.	Tells you whether anything had already nudged this process up or down.

The one question the report forces you to answer: was the killed process growing without bound, or did some other process just allocate a huge amount at once and the killer picked the incumbent? Compare the killed process's anon-rss against its normal baseline. If mysqld normally sits at 2 GB and the report shows it at 7.5 GB, MySQL is your problem. If MySQL was at its usual 2 GB and a Java worker was at 16 GB on a 16 GB box, the Java worker is the problem and MySQL was the victim.

You can inspect the live score for a running process, which is useful for deciding who to protect before the next event:

cat /proc/$(pidof myapp)/oom_score      # the computed score right now
cat /proc/$(pidof myapp)/oom_score_adj  # any manual adjustment

One caution: once the process is dead, its PID may be reused by something else. Reading /proc/<old-pid> after the event does not recover the victim's old score.

The two knobs that actually save you

1. OOMScoreAdjust via systemd

If there is a service you genuinely cannot afford to lose (the auth service, the database, sshd itself), tell the kernel that:

# /etc/systemd/system/sshd.service.d/oom.conf
[Service]
OOMScoreAdjust=-900

Conversely, mark a known memory hog as "kill me first" with OOMScoreAdjust=500, so the blast radius stays inside the process you can live without.

Two rules I follow strictly:

• Never set -1000 on more than a handful of processes. -1000 makes a process OOM-immune. If too many critical processes are all immune, the kernel has nothing left to kill and you get a full host panic instead of one contained service restart.
• Set it on the unit, not ad hoc. echo -500 > /proc/$(pidof sshd)/oom_score_adj resets the moment the process restarts. The systemd drop-in survives reboot and restarts.

2. MemoryHigh / MemoryMax — the real lever

This is the single most effective control, and it is the one I reach for first. Instead of deciding who dies after memory is gone, you put a ceiling on each service before it can run the box over:

[Service]
MemoryHigh=3.5G   # soft ceiling: aggressive reclaim + throttling above this
MemoryMax=4G      # hard ceiling: cgroup OOM fires here, kills only this unit

The split is important:

• MemoryHigh is a soft threshold. Above it the kernel applies heavy reclaim pressure and throttles the cgroup. The service slows down instead of dying.
• MemoryMax is the hard ceiling. Crossing it triggers an OOM inside that cgroup only, so a runaway worker takes its own siblings down and the supervisor restarts it cleanly, while the rest of the host is untouched.

A half-killed container (or a half-killed service) is a worse failure mode than a clean full restart. With MemoryMax in place, the box-wide OOM killer essentially stops firing, because each unit is now responsible for its own bad days.

The sysctl knobs, and why to be careful

There are three vm.* values you will see in every blog post about this, and two of them you should probably not touch.

vm.overcommit_memory has three values:

• 0 (default) — heuristic overcommit. The kernel guesses whether an allocation is reasonable. Most workloads are fine here.
• 1 — always overcommit. malloc essentially never returns NULL. Useful for HPC workloads that allocate huge sparse arrays they never touch.
• 2 — strict accounting. The kernel refuses allocations that exceed swap + RAM * overcommit_ratio / 100. This sounds appealing ("fail loudly instead of killing later"), but on a multi-tenant box it is painful: fork() temporarily doubles the parent's virtual size, and many processes that normally work fine will fail to fork. JVMs in particular hate mode 2.

vm.panic_on_oom controls whether an OOM is upgraded to a full kernel panic instead of a kill. Leave it at 0 unless you have a very specific reason; you want a contained service death, not a reboot.

vm.swappiness is the one you can tune, and it only moves the timing of pressure, not the outcome. A lower value (say 10) keeps file cache resident longer and pushes anonymous pages to swap sooner — useful on a cache-heavy box. It does not prevent an OOM; it only changes how the box degrades before one.

The practical check before you walk into any of this:

sysctl vm.panic_on_oom vm.overcommit_memory vm.overcommit_ratio vm.swappiness

Container workloads

For a containerized service the picture inverts, and you want the inversion: you want the whole cgroup to die together rather than a single process inside it, because a half-alive container is not recoverable.

# /etc/systemd/system/worker.service.d/memory.conf
[Service]
MemoryMax=2G
MemoryHigh=1.8G
# cgroup v2: group OOM kills the whole cgroup, not one task

On cgroup v2 with memory.oom.group enabled, a limit breach kills the entire cgroup, which is exactly the clean-restart behavior you want for a worker fleet. On cgroup v1 the per-cgroup OOM still targets one process; the group behavior is newer. Check which one you are on with stat -fc %T /sys/fs/cgroup.

OOM is a symptom, not the cause

Everything above decides who dies. It does nothing about why memory ran out. The fix is the cause:

• A memory leak (RSS grows monotonically under steady load) → profile the process; in a JVM, take a heap dump after the event; in Go, grab a heap profile from pprof and watch what is retaining memory.
• An unbounded cache → set a real bound (L1 cache size, or cap the query result set).
• A query that loads a whole table into memory → limit it, stream it, or add an index.
• A host that is simply undersized for the workload → the honest answer is more RAM, or shed the workload.

Sizing the application and capping the cgroup are the remediation; the score adjustments are only deciding the order of death. If OOM events have recurred more than once and the pattern is still unclear, that is a monitoring-and-support investigation, and the running-host checklist for reading exactly these kernel fields lives in the ops notes on memory monitoring (deep link at the end of this post).

The one-paragraph summary for your postmortem

The box ran out of memory and the kernel's OOM killer chose a victim by highest resident-anonymous score. I read the dmesg report, confirmed whether the killed process was the culprit or an innocent incumbent by comparing its anon-rss to baseline, then applied two controls: MemoryHigh/MemoryMax on the service cgroups so failures stay contained, and a negative OOMScoreAdjust on the handful of processes that must survive. I did not add swap to "fix" it, because swap only delays and slows. The remaining work is the root cause — a leak, an unbounded cache, or an undersized host — which is the only thing that stops the next 3 AM page.

These notes on memory pressure, cgroup limits, and postmortem playbooks are part of my wider field notes on running Linux in production. The running-host checklist that backs this post sits in the travel-and-ops notes at jiunotes.top/en, and the memory-monitoring reference I kept referring to above is at ops.jiunotes.top/en/monitor/linux-memory-monitor.md.