I Blew Past CommitLimit With 24 GiB and Nobody Stopped Me

Back in April I wrote a guide on RAM exhaustion and the OOM Killer. One sentence in it says: "The 2 value is usually preferred for critical, memory-sensitive servers." Rereading my own guide this week, I stopped at that line — because when I wrote it, I had never measured what vm.overcommit_memory=2 does. My assumption went like this: CommitLimit in /proc/meminfo is a ceiling, Committed_AS climbs toward it, and mode 2 cuts the climb off at the ceiling.

One of those three turned out to be true.

The lab is Docker Desktop's linuxkit virtual machine: kernel 6.10.14, aarch64, 14 vCPUs, MemTotal 5,036,592 kB (4.80 GiB), SwapTotal 1,048,572 kB. A privileged container can write to /proc/sys/vm, so that is where I ran the sysctl experiments — and a warning up front: writing to those files affects the entire VM, not the container. Do not try this on a live server; that is exactly why I picked a disposable VM.

I never touched a single one of the pages I mapped. Every test exercises only the accounting gate inside mmap(); no physical memory is spent and MemFree does not move. The whole question is when that gate says no.

Who computes CommitLimit

The first job was verifying the formula. vm_commit_limit() in mm/util.c does exactly one thing: if overcommit_kbytes is non-zero use it, otherwise compute (totalram_pages() - hugetlb_total_pages()) * sysctl_overcommit_ratio / 100, then add total_swap_pages. That is all of it.

The measurement landed on the nose:

Setting	CommitLimit read	Expected from formula	Delta
ratio=50, kbytes=0	3,566,868 kB	3,566,868 kB	0 kB
ratio=80	5,077,844 kB	5,077,845.6 kB	−1.6 kB
kbytes=1048576	2,097,148 kB	2,097,148 kB	0 kB

The 1.6 kB gap in the second row is not an error: the kernel does the arithmetic in pages, not in kB. Eighty percent of 1,259,148 pages is 1,007,318.4, which truncates to 1,007,318 pages — the lost 0.4 of a page is those 1.6 kB. The third row shows that kbytes mode ignores RAM entirely: I wrote 1,048,576 kB and the resulting limit is that number plus swap.

When I reserved hugepages the limit dropped, and by exactly what I expected: with nr_hugepages=256, Hugetlb became 524,288 kB and CommitLimit fell from 3,566,868 to 3,304,724 kB. The drop is 262,144 kB — precisely half the hugetlb pool, because the subtraction happens before the ratio is applied. I covered how far hugetlb accounting sits from the rest of the kernel in the piece where I gave a container 64 MiB and hugepages took 400 MiB; CommitLimit is another face of the same separation.

So far everything matches the documentation. Then I looked at the mode — and the docs are not wrong there either: under CommitLimit, proc.rst says this limit "is only adhered to if strict overcommit accounting is enabled (mode 2)". One line. What I wanted to know was how large a gap that line covers.

Under mode 0, CommitLimit is decoration

vm.overcommit_memory=0, ratio=50, CommitLimit 3,566,868 kB. I ran a loop that maps private, writable, anonymous memory in 128 MiB chunks, and I stopped it at 24 GiB — of my own accord, because the kernel never stopped me:

baslangic: CommitLimit=3566868 kB Committed_AS=2200352 kB chunk=128 MiB tur=priv
HIC HATA YOK: 24576 MiB eslendi; CommitLimit=3566868 kB Committed_AS=27366176 kB fark=23799308 kB

(My script prints in Turkish: baslangic = start, tur=priv = type: private, HIC HATA YOK = no error at all, eslendi = mapped, fark = delta.)

Committed_AS reached 27,366,176 kB. That is 7.67 times CommitLimit. A machine with 4.8 GiB of RAM accepted 26 GiB of commitment without a murmur, and /proc/meminfo went on printing the two numbers side by side as though one bounded the other.

Mode 0 has exactly one rule, three lines of it in the source:

if (sysctl_overcommit_memory == OVERCOMMIT_GUESS) {
        if (pages > totalram_pages() + total_swap_pages)
                goto error;
        return 0;
}

The question it asks is not "has total commitment passed the limit" but "is this single request larger than RAM plus swap". I asked for 4,800 MiB in one call and it went through (Committed_AS climbed to 7,115,552 kB, twice the limit). I asked for 6,200 MiB and got ENOMEM. The difference has nothing to do with the ceiling and everything to do with the size of one request.

I also nailed the threshold down to the page. RAM plus swap on this machine is 6,085,164 kB = 1,521,291 pages:

  --- tam esik (1521291 sayfa):
mmap(1521291 sayfa = 6231207936 bayt) -> OK
  --- esik + 1 sayfa (1521292):
mmap(1521292 sayfa = 6231212032 bayt) -> Cannot allocate memory

(tam esik = exactly the threshold, esik + 1 sayfa = threshold plus one page.) The comparison is >, so the threshold itself passes and one page more does not. One line of code, confirmed at single-page resolution.

It is no accident that this mode is called GUESS internally. It is not a limit enforcer; it is an obvious-nonsense filter.

Mode 1 does not even look at that

In mode 1 (OVERCOMMIT_ALWAYS), __vm_enough_memory() bumps the counter and returns 0 immediately. The measurement agrees: 6,200 MiB in one call went through, and so did 40,960 MiB. Committed_AS became 44,143,392 kB — 42 GiB of commitment on a 4.8 GiB machine, 12.4 times the limit.

I am noting this mode separately, because the first value I read while setting the lab up was this:

vm.overcommit_memory = 1

My Docker Desktop's VM had booted in mode 1, not in the kernel's factory default of 0. One machine, one version, one reading — I am not offering it as a general rule, only as a reason not to assume the factory default when you read a sysctl inside a container. I confirmed this was not left over from an earlier session by checking the VM's /proc/uptime: the counter started from zero when I launched the daemon, so the 1 I read is the boot value. Anyone who runs cat /proc/sys/vm/overcommit_memory inside a container and carries on assuming "it should be 0" is telling the wrong story about the wrong machine. That file is not namespaced, by the way — what you read inside a container is the host kernel's setting.

Under mode 2 the ceiling is real, but not the one you computed

Now for the thing I actually wondered about. I set up mode 2 by adding 256 MiB to whatever Committed_AS was at that moment (overcommit_kbytes=2472108, CommitLimit 3,520,680 kB) and hunted for the ceiling in 1 MiB chunks.

The loop took ENOMEM at 1,241 MiB. At the moment of failure Committed_AS was 3,480,648 kB — 40,032 kB below the limit. The gate turned me away before I ever reached the ceiling.

The reason is the two deductions inside __vm_enough_memory():

if (!cap_sys_admin)
        allowed -= sysctl_admin_reserve_kbytes >> (PAGE_SHIFT - 10);

if (mm) {
        long reserve = sysctl_user_reserve_kbytes >> (PAGE_SHIFT - 10);
        allowed -= min_t(long, mm->total_vm / 32, reserve);
}

The second one hit me. mm->total_vm / 32 — one thirty-second of the process's own address space is subtracted from the limit. At failure I read VmSize as 1,272,980 kB; a thirty-second of that is 39,780 kB. The measured gap was 40,032 kB. The 252 kB in between is the 1 MiB chunk granularity plus the counter slack I will get to shortly.

The ceiling is not fixed; it descends as the process grows. The bigger the process, the bigger its own penalty. I pulled user_reserve_kbytes down to zero and ran the same loop: it got to 1,281 MiB, and this time Committed_AS spilled 928 kB above the limit — I come back to why two sections from here. That forty-megabyte difference is not the value of one sysctl; it is a derivative of the process's own size.

A 128 MiB sidecar and a 4 GiB JVM look at the same CommitLimit and do not see the same ceiling. Since user_reserve_kbytes defaults to 128 MiB, the penalty stays proportional until total_vm passes 4 GiB and is flat after that.

A small aside: the documentation describes this penalty as "3% of current process size", while the code says total_vm / 32 — which is 3.125%. The doc also says it is reserved "of free memory", whereas what is actually reduced is CommitLimit, not free memory. The documentation proves this article's own thesis: the number is right, the place it is attached to is not.

Root and a service user do not see the same ceiling

The first deduction in that code applies to processes without CAP_SYS_ADMIN. Measuring it was easy: inside the same mode 2 window I ran the same binary four times in a row — twice as root, twice as uid=65534, with admin_reserve_kbytes at 8 MiB and at 256 MiB.

Condition	Mapped	Gap (CommitLimit − Committed_AS)	VmSize/32	Expected from formula
root, admin_reserve=8192	1,241 MiB	40,032 kB	39,780 kB	39,780 kB
root, admin_reserve=262144	1,241 MiB	40,032 kB	39,780 kB	39,780 kB
uid=65534, admin_reserve=262144	993 MiB	293,984 kB	31,844 kB	293,988 kB
uid=65534, admin_reserve=8192	1,233 MiB	48,224 kB	39,524 kB	47,716 kB

The first two rows are copies of each other: I grew admin_reserve_kbytes thirty-twofold and root did not notice. In the third row the same binary in the same window found 248 MiB less room, purely because of its uid. Expected gap: 262,144 + 31,844 = 293,988 kB; measured 293,984 kB. One page of deviation.

The residuals across the four rows (+252, +252, −4, +508 kB) all come from one place: __vm_enough_memory() adds the request to the counter with vm_acct_memory(pages) before the comparison and takes it back with vm_unacct_memory() on refusal. So until the loop's last 1 MiB request is refused, the gap I measure swings around VmSize/32 by up to one chunk (1,024 kB) minus whatever has not been folded into the counter yet. Every residual sits inside that 0–1,024 kB band.

The practical consequence: when tuning a server under mode 2, saying "this box has 3.4 GiB of commitment headroom" is not enough. Root sees that headroom. If your services do not run as root — and they should not — their headroom is smaller by admin_reserve_kbytes. The default of 8 MiB keeps this at noise level; the moment someone raises it to 1 GiB "for safety", it comes straight out of the application's ceiling.

The application will only ever see ENOMEM, but the kernel is not entirely silent: the failure path of __vm_enough_memory() emits a pr_warn_ratelimited line to dmesg with the pid, the command name and the number of bytes requested. Start the hunt with dmesg | grep __vm_enough_memory — the "not enough memory for the allocation" line is there (rate limited, so not for every refusal).

How mode 2 actually blows up in production: fork()

Everything so far has been about the mmap() gate. There is a second gate the kernel documentation warns about separately under mode 2: fork().

The kernel documentation says so in a single sentence under user_reserve_kbytes: "Any subsequent attempts to execute a command will result in fork: Cannot allocate memory." The reason is dup_mmap(): while forking, security_vm_enough_memory_mm() is called again for every vma marked VM_ACCOUNT. So at fork time the process's entire accountable address space has to be committed a second time.

That one is measurable too. The ceiling here is set so that about 300 MiB of headroom remains after a 1 GiB mapping:

## mod=2 CommitLimit=3577508 kB MemFree=933696 kB
1024 MiB ozel-yazilabilir eslendi; VmSize=1050772 kB Committed_AS=3269828 kB CommitLimit=3577508 kB
fork() -> BASARISIZ: Cannot allocate memory
  MemFree (hata aninda civari) = 933444 kB

(ozel-yazilabilir eslendi = private writable mapping created, BASARISIZ = failed, hata aninda civari = around the moment of failure.) The process took its 1 GiB without complaint, touched none of it, and the machine had 912 MiB of free RAM — and it could not produce a child. I immediately repeated the same mapping under mode 0: fork() -> OK.

In production this looks like: a 4 GiB JVM cannot call Runtime.exec, a database cannot start a helper process, a health-check script will not run. The error says there is no memory; free shows plenty. Both are true — they are not measuring the same thing. If you are considering mode 2, the question to ask is not "is there enough RAM" but "can my largest process pay for its own address space one more time".

The mappings strict mode never counts

I ran the flag experiments in a separate round with the same setup (CommitLimit was 3,494,516 kB in that one; the window is computed from whatever Committed_AS happens to be, so it moves a few MiB between rounds). I tried single 2,048 MiB requests with different flags. The table is a mirror of accountable_mapping() — on the 6.10 kernel of this lab, that function lives in mm/mmap.c; it moved to mm/vma.c after 6.12:

Mapping	Result	Committed_AS change
MAP_PRIVATE with PROT_READ and PROT_WRITE	ENOMEM	—
MAP_PRIVATE with MAP_NORESERVE	ENOMEM	—
MAP_PRIVATE, PROT_READ only	succeeded	unchanged
MAP_SHARED with MAP_ANONYMOUS	ENOMEM	—
MAP_SHARED on a file	succeeded	unchanged

The documentation states the second row in one sentence: "In mode 2 the MAP_NORESERVE flag is ignored." What that sentence does not say is how it gets ignored, and I wanted to know. MAP_NORESERVE means "do not charge this mapping to accounting"; under mode 2 the kernel does not honour that promise. Two lines in mm/mmap.c:

if (flags & MAP_NORESERVE) {
        /* We honor MAP_NORESERVE if allowed to overcommit */
        if (sysctl_overcommit_memory != OVERCOMMIT_NEVER)
                vma_flags_set(&vma_flags, VMA_NORESERVE_BIT);

The flag is not ignored later; it is never set in the first place. I repeated the same request under mode 0 and it went through. So a library that relies on MAP_NORESERVE silently changes behaviour on a machine moved to mode 2, even though nothing in your code changed.

The third and fifth rows show how narrow the gate is: accountable_mapping() only counts the triple "private and writable and not NORESERVE". A read-only private mapping is free, a shared file mapping is free — the file already lives on disk, no swap needed. The fourth row is a separate gate: MAP_SHARED|MAP_ANONYMOUS cannot get through, because that mapping sets up a shmem file behind the scenes and the accounting happens over there, in shmem_acct_size() via its security_vm_enough_memory_mm() call. Same gate, different handle.

The number the gate reads is not the number you read

Here is an asymmetry I would never have spotted without reading the code. The Committed_AS line in /proc/meminfo comes from vm_memory_committed():

unsigned long vm_memory_committed(void)
{
        return percpu_counter_sum_positive(&vm_committed_as);
}

sum_positive walks every CPU's local accumulation too, giving a much more accurate value (the comment in the source calls it "a more accurate but much slower version of percpu_counter_read_positive()"). The last line of the permission gate, however, is this:

if (percpu_counter_read_positive(&vm_committed_as) < allowed)
        return 0;

read_positive only reads the global counter; it does not see per-CPU deltas that have not been folded in yet. So you are looking at the summed number on screen while the decision is made with the unsummed one.

The budget for that approximation lives in mm/mm_init.c:

if (overcommit_policy == OVERCOMMIT_NEVER)
        memsized_batch = min_t(u64, ram_pages/nr/256, INT_MAX);
else
        memsized_batch = min_t(u64, ram_pages/nr/4, INT_MAX);

The assignment right below that snippet is vm_committed_as_batch = max_t(s32, memsized_batch, max_t(s32, nr*2, 32)) — there is a floor the computed value cannot fall below. I derived the values for this machine rather than measuring them, since I could not read fbc->count from user space: 1,259,148 pages, 14 CPUs. The floor is not binding here, but on a low-RAM, many-core machine the mode 2 batch lands on it and this arithmetic quietly gives the wrong answer. Under mode 2 the batch is 351 pages, i.e. 1.4 MiB per CPU, at most about 19 MiB of total deviation. Under modes 0 and 1 it is 22,484 pages: 87.8 MiB per CPU, roughly 1.2 GiB in total — exactly a quarter of RAM. In the loose modes the counter is entitled to be more than a gigabyte wrong, because nobody is making decisions from that number.

There is more. The sysctl handler for switching into mode 2 does three things in this order: shrink the batch first, then synchronise the counter on every CPU, and only then change the policy. So the strictness of "strict mode" is not merely a comparison; it is also a decision about accounting precision. And my measurement confirms it: in the run with user_reserve_kbytes=0, Committed_AS at the moment of failure had gone 928 kB above the limit. That sits inside the 1,404 kB batch allowance I derived. Even in "never overcommit" mode, the ceiling flexes by one batch.

The trap that actually bit me

While running the second round, CommitLimit suddenly read 1,048,572 kB — the size of swap, to the kilobyte. The RAM share had evaporated. At first I thought I had broken the measurement; the cause was my own cleanup lines.

overcommit_ratio and overcommit_kbytes are mutually exclusive, and they enforce that at write time. overcommit_kbytes_handler() sets sysctl_overcommit_ratio = 0 on every successful write — even when the value you write is zero. So this command, issued with the intent of "let me turn off the kbytes override and go back to the ratio":

sysctl vm.overcommit_kbytes=0

does not go back to the ratio; it deletes the ratio. Measured:

  oncesi:
    ratio=50 kbytes=0 mod=0 CommitLimit=3566868 kB Committed_AS=2200832 kB
  'sysctl vm.overcommit_kbytes=0' sonrasi:
    ratio=0 kbytes=0 mod=0 CommitLimit=1048572 kB Committed_AS=2200832 kB

(oncesi = before, sonrasi = after, mod = mode.)

Under mode 0 nobody notices, because nobody is looking at the limit anyway. Under mode 2 it renders the machine unable to run anything at all: in that state I switched to mode 2 and tried to run my test program, and the shell could not even exec the binary:

/lab/run3.sh: 37: /tmp/alloc: Cannot allocate memory

The correct order is: write overcommit_kbytes first with whatever value it should have, then overcommit_ratio. And after either one, read CommitLimit back and confirm. My cleanup function wrote the ratio before kbytes; swapping two lines was the whole fix.

Checklist

If you are thinking about enabling mode 2 on a server, here is the order:

1. First confirm your largest process can still fork(). Under mode 2 a fork commits the process's accountable address space all over again; a process comfortably below the ceiling may still be unable to spawn a child. It is the one failure mode the kernel documentation names for mode 2.
2. Work out a process's effective ceiling yourself — no /proc field holds it: CommitLimit − (admin_reserve_kbytes if not root) − min(VmSize/32, user_reserve_kbytes).
3. Compare Committed_AS with CommitLimit. If commitment is already above the limit, switching to mode 2 closes the machine to new allocations instantly. My 7.67× was deliberate: with no production load, I mapped those 24 GiB myself. On the same VM at rest the ratio was 0.61 (2,183,552 / 3,566,868). So the number answers nothing about how common that state is; what it answers is that nothing in the loose modes prevents it. Measure your own machine before you switch.
4. Pick overcommit_ratio with the hugepage pool in mind — as the pool grows, the limit shrinks by the ratio's share of it.
5. Mind the ordering: writing any value to overcommit_kbytes zeroes the ratio. Do not call it configured until you have read CommitLimit back.
6. Service users have a lower ceiling. Raising admin_reserve_kbytes feels like security; it is subtracted directly from every non-root process.
7. Large processes have a lower ceiling on their own. The user_reserve_kbytes penalty is one thirty-second of total_vm, which reaches 128 MiB for a single big JVM.
8. Know which dependencies use MAP_NORESERVE. Under mode 2 that flag is dropped; libraries with sparse memory maps behave differently on identical code.
9. Label the ratio correctly in monitoring. Committed_AS / CommitLimit is "fullness" only under mode 2; under modes 0 and 1 it is not an alert threshold, just one number divided by another.

What I did not prove

I derived the batch sizes from source; I found no way to read fbc->count from user space, so 351 and 22,484 pages are not directly measured numbers. The 928 kB overshoot measured earlier is consistent with that derivation, but consistency is not proof.

The code quotes come from master (7.3-rc7), the measurements from 6.10.14. I put five places side by side across both versions — vm_commit_limit, __vm_enough_memory, mm_compute_batch, the MAP_NORESERVE branch and accountable_mapping — and the logic is identical, only the flag API was rewritten. I did not audit other paths between the two versions.

The fork() measurement used a single configuration (1 GiB mapping, ~300 MiB headroom); I did not sweep process sizes to chart which size fails at which ceiling. Everything was measured on one VM: aarch64, 14 vCPUs, with swap. Mode 2 scales quite differently on a swapless machine, and I did not try it. I also never looked at the interaction with cgroup v2 memory limits; every number here belongs to global accounting. And since I never touched a mapped page, this lab never exercised the real memory-exhaustion path — the OOM killer.

The number is right, the meaning is not

The lesson I took from this lab is broader than a sysctl value. Those two adjacent lines in /proc/meminfo — CommitLimit and Committed_AS — create the impression of a gauge: one is capacity, the other is fill. But the wire connecting them lives in a third, invisible file: overcommit_memory. While that file holds 0 or 1, CommitLimit keeps being computed, keeps being plotted on the Grafana panel, and bounds nothing. The number is correct; it is meaningless.

That was my mistake in April too. "Mode 2 is preferred for critical servers" is not wrong, but it is incomplete: when you enable mode 2, the ceiling is neither the number you computed nor the same number for everyone. It becomes a band that shifts with the user, with the size of the process, and with the counter's batch. Saying you have enabled a setting is not the same as saying you have measured it — and I had to disprove my own sentence to understand the difference.

Official Sources

• Linux admin guide — vm sysctl: overcommit_memory, overcommit_ratio, overcommit_kbytes, admin_reserve_kbytes, user_reserve_kbytes
• Overcommit Accounting — the three modes and how mapping types are accounted
• proc filesystem — /proc/meminfo fields (CommitLimit, Committed_AS)
• mm/util.c — vm_commit_limit, __vm_enough_memory, vm_memory_committed, overcommit sysctl handlers
• mm/mm_init.c — mm_compute_batch and vm_committed_as_batch
• mm/vma.c — accountable_mapping and commit charging
• mm/shmem.c — shmem_acct_size and the accounting of shared anonymous mappings
• mm/mmap.c — why MAP_NORESERVE is never set under mode 2; dup_mmap re-charging VM_ACCOUNT vmas at fork
• lib/percpu_counter.c — percpu_counter_add_batch