Mustafa ERBAY

Posted on Sep 21 Originally published at mustafaerbay.com.tr

Who Changed fs.file-max? A Container Did

#linux #systemd #kernel #docker

I wrote /etc/sysctl.d/99-vps3-tuning.conf myself on May 11; its second line is fs.file-max = 2097152. Today I typed sysctl fs.file-max and got 9223372036854775807. Nine quintillion and change; the largest value a signed 64-bit integer can hold. The server rebooted on September 10, systemd-sysctl.service reports a clean exit, the file is in place, and it still says two million. Who closed the gap?

Finding the answer took longer than I expected, and the road was far more crowded than I had guessed. There is no single hand writing this file; in order there is PID 1, a seven-year-old Debian packaging decision, a PAM patch, and finally a lab container I had set up myself five days earlier. This post unwinds that chain; along the way we see the four separate addresses of the "Too many open files" error, why LimitNOFILE=infinity is not infinite, and how Go and Node quietly raise their own limits. Measurements are from VPS3 (Ubuntu 24.04, systemd 255, kernel 6.8.0-139); comparisons come from three other Ubuntu releases in the same fleet.

Four ceilings, two error codes

A process opening a file can hit four different numbers, and two of them produce the same error text. From the bottom up: the RLIMIT_NOFILE soft limit (ulimit -Sn in a shell), the hard ceiling of the same limit (ulimit -Hn), fs.nr_open, which is the highest value the hard ceiling may be raised to, and fs.file-max, which caps the total number of file structures on the whole machine. The first three are per process, the last is system-wide. Exceeding the first two yields EMFILE ("Too many open files"), exceeding the fourth yields ENFILE ("Too many open files in system"); the third never lets you open anything, it returns EPERM at setrlimit time.

I tried the first layer on VPS3 from a non-root shell with Python: opening /dev/null repeatedly under a soft limit of 1024 produced EMFILE after 1021 files (three descriptors were already stdin/stdout/stderr). After setrlimit raised the soft limit to the hard limit (1,048,576), another 5,000 opened without complaint (6,021 in total). Asking for one above the hard limit is EPERM; ulimit -n unlimited is refused even for root, because the kernel does not accept an unlimited RLIMIT_NOFILE. That detail answered a question on systemd's bug tracker in 2018: Lennart Poettering wrote "the kernel does not permit an 'unlimited' RLIMIT_NOFILE" and pointed out that ulimit -n unlimited is refused even in a root shell (systemd#9205).

I did not try the fourth layer on the live server; lowering fs.file-max touches every process on the machine. Instead I used the linuxkit VM under Docker Desktop on my Mac (kernel 6.10.14) with a privileged container. There, file-max was 801,469 (the kernel's memory-derived value: files_maxfiles_init turns ten percent of memory after reserves into a ceiling at roughly 1 KiB per file); file-nr reported 1,088 file structures in use. I pulled the ceiling down to 1,148 and tried to open 300 files as nobody: errno 23 ENFILE after 79. The same script as root opened all 300. The comment in the kernel's alloc_empty_file says it in one line: "Privileged users can go above max_files"; the check is skipped with capable(CAP_SYS_ADMIN). A single line landed in dmesg too: VFS: file-max limit 1148 reached. It is written only when a new high-water mark is seen (the old_max comparison), not on every refusal, which is why logs give the impression of "seen once, then silent".

Layer one: PID 1 writes LONG_MAX

fs.file-max being 9.2 quintillion is systemd's deliberate behaviour since 2018 (v240). The release notes say plainly that two of the four ceilings are being switched off: "from the four limits on file descriptors currently enforced (fs.file-max, fs.nr_open, RLIMIT_NOFILE hard, RLIMIT_NOFILE soft) we turn off the first two, and keep only the latter two". The rationale is that file descriptors are now accounted by memcg like any other memory; limiting them a second time with a separate counter had lost its point. In the source this lives in bump_file_max_and_nr_open() in src/core/main.c: LONG_MAX is written to fs/file-max, then a loop runs for fs/nr_open.

That raises a second question rather than answering the first. PID 1 writes this value at the very start of boot, before any unit starts; systemd-sysctl.service runs afterwards and applies my file. So when boot finished on the evening of September 10, file-max should have been 2,097,152. Within the same boot, something set it back to LONG_MAX. Who?

The fingerprint: 1073741816

The clue came from the neighbouring file. fs.nr_open on VPS3 is 1,073,741,816. The kernel default is 1,048,576 (fs/file.c: sysctl_nr_open = 1024*1024), and none of my sysctl files mention the key. 1,073,741,816 is not a random number; in hex it is 0x3FFFFFF8, that is 2³⁰ minus 8. The loop in systemd 255's source produces exactly that: start at INT_MAX, round down to the pointer size (8), and if the kernel says EINVAL, halve. 2,147,483,647 → 2,147,483,640 (the kernel's ceiling is INT_MAX & -64 = 2,147,483,584, anything above is rejected) → 1,073,741,820 → rounded to 8 → 1,073,741,816. The comment in the source laughs at itself: "Ugly? Yes, absolutely, but kernel APIs are kernel APIs, so what do can we do... 🤯". The kernel side knows the number too: a patch that landed in 6.17 (04a2c4b4511d) cuts off, with EMFILE, a process that calls dup2 near the limit when nr_open is "1073741816, which is what systemd typically sets it to", because that would try to allocate an 8 GB fd table; the value appears verbatim in the comment.

The number carries systemd's signature, but not the signature of the systemd on VPS3. Ubuntu 24.04's systemd 255.4-1ubuntu8.17 is built with Debian's debian/rules, and that file contains -Dbump-proc-sys-fs-nr-open=false. I could see it in the binary as well: strings /usr/lib/systemd/systemd | grep nr_open finds only two function names (read_nr_open, bump_file_max_and_nr_open); write messages such as "Couldn't write fs.nr_open" or "Successfully bumped fs.nr_open" never made it into the build. The file-max bump, on the other hand, is enabled (the text Failed to bump fs.file-max is in the binary). So VPS3's own PID 1 can write file-max but cannot touch nr_open.

I looked across the fleet. On VPS4, same Ubuntu 24.04 + systemd 255, nr_open is 1,048,576, the kernel default, and PID 1's own limit is 1,048,576. On VPS1 and VPS2, Ubuntu 25.04 + systemd 257.4, nr_open is 1,073,741,816 and PID 1 has the same value. On VPS5, Ubuntu 26.04 + systemd 259.5, it is 2,147,483,584: systemd 258 starts the loop from the kernel's constant (NR_OPEN_MAXIMUM) instead of INT_MAX, and the first attempt succeeds (cfba9b9eab). The table is consistent; the one row that does not fit is VPS3: the package of VPS4, the nr_open of VPS1, and a PID 1 limit of 1,048,576, meaning PID 1 had already been set up when nr_open went up, and had taken its ceiling from the kernel's value at that moment.

So what I had was this: nr_open was written after boot, by a systemd with the halving loop of 256 or 257. No such package exists on VPS3. But it did five days earlier: on the night of September 16, for the run0 post, I had installed systemd 257.13 on top of debian:trixie-slim and run it as PID 1 with --privileged on VPS3. In a privileged container /proc/sys is mounted writable. As the container's systemd booted, it wrote file-max = LONG_MAX and nr_open = 1073741816 into the host kernel while I was busy with polkit.

Reproduction

The container from five days ago was gone; I rebuilt the claim as it stood. First I set file-max back to the value in my own file (sysctl -w fs.file-max=2097152; harmless with 11.5 thousand files open), then ran the same image unprivileged and privileged. In the unprivileged container systemd exited before it could even mount /run, and the host did not change; /proc/sys is mounted ro there anyway, and echo 2097152 > /proc/sys/fs/file-max says "Read-only file system". The privileged container (SYSTEMD_LOG_LEVEL=debug, with -t so the console output lands in docker logs) left four lines:

Setting '/proc/sys/fs/file-max' to '9223372036854775807'
Setting '/proc/sys/fs/nr_open' to '2147483640'
Couldn't write fs.nr_open as 2147483640, halving it.
Skipping bump, value is already larger.

A second after the container came up, sysctl fs.file-max on the host read 9223372036854775807 again. The second line is the loop's first step, the third is the kernel's EINVAL, and the fourth is an admission that today 1,073,741,816 is already in place. On September 16 that fourth line must have read "Successfully bumped fs.nr_open to 1073741816"; I have no log of it, but there is no other explanation for the value either.

The general lesson is not confined to a lab. Every systemd running in a --privileged container (a CI runner image, "docker in docker" setups, test containers that use systemd as init) rewrites two of the host's sysctls according to its own build options, and logs that to its own console rather than the host's journal. Your sysctl.d file is right, the service applying it is right, and the value is still something else.

The review round turned up a second trace, and this one is more unsettling. The systemd-sysctl.service inside the container runs as well (its condition is ConditionPathIsReadWrite=/proc/sys/net/, satisfied in a privileged container) and applies the container's own /usr/lib/sysctl.d/ directory to the host kernel. Debian's systemd package ships two files there: 50-pid-max.conf (kernel.pid_max = 4194304, the same value as the host, so no visible change) and 10-coredump-debian.conf (kernel.core_pattern=core). The apport service runs on VPS3 and at boot points core_pattern at its own pipe (|/usr/share/apport/apport -p%p ...); when I measured, I saw kernel.core_pattern = core and core_pipe_limit = 10. Apport's --stop path writes core but sets core_pipe_limit to 0; if it is still 10, apport never stopped and someone else wrote over it. So for five days the container had also silently switched off the host's crash reporting; I restored it with systemctl restart apport. kernel.sysrq being 438 live while the file says 176 remains an open question; I could not find its author.

Debian's 2018 decision and the 2024 reversal

Why was VPS3's nr_open 1,048,576 to begin with, and why is VPS1's 1,073,741,816? The difference is two Debian changelog entries. In December 2018, with systemd 240-2, Michael Biebl disabled the nr_open bump; the reason was #917167 ("systemd: 240 breaks kde"): Debian's pam_limits module reads every limit not written in limits.conf from PID 1; once PID 1 raised its own RLIMIT_NOFILE to nr_open, every user logging in got a hard limit above a billion, and not every piece of software could cope. In May 2024, with 256~rc3-3, Luca Boccassi turned it back on: "Restore open files limit bump on boot. Broken packages ought to have been fixed by now" (#1029152). Ubuntu 24.04 (noble) stayed on 255, so it is off; from 24.10 (oracular, 256.5) onwards the line is gone from the rules file. The three values in the fleet are a snapshot of the transition between those two dates.

The PAM side is measurable too. On VPS3 the sshd process has a limit of 1024/524288 (the default systemd hands to units), yet my SSH session has 1024/1048576. Where did the hard limit go from 524,288 to 1,048,576? In pam_limits: Debian's 027_pam_limits_better_init_allow_explicit_root patch forces the module to read /proc/1/limits on every session (upstream that behaviour is tied to the set_all option; in Debian's patch it is unconditional), and PID 1's hard limit is 1,048,576. The soft limit stays at 1024 thanks to the 2015 pam-limits-nofile-fd-setsize-cap patch, Ubuntu in origin and later carried by Debian too: if the soft value read from PID 1 exceeds FD_SETSIZE, it is clipped to 1024. The patch description points at select() and says "if we want to make such a change to the default distribution soft limit in PAM, we should do it deliberately and carefully, not accidentally". Both numbers in my session are derived from PID 1 without ever looking at limits.conf. Debian dropped this behaviour in January 2025 with pam 1.7.0 ("pam_limits: do not override systemd's limits by default; add the set_all option", #995236); the pam package on Ubuntu 24.04 and 25.04 (1.5.3) still carries the old patch. I measured the consequence on 25.04: on VPS1 and VPS2, sshd runs with 1024/524288 while my SSH session gets 1024/1073741816. That is precisely the KDE complaint from #917167: every Go binary started from that shell pulls itself up to 1,073,741,815, every Node process to 1,073,741,816, and any software that allocates an array sized by the hard limit starts computing in gigabytes.

"infinity" means "as much as PID 1"

I do not know how many unit files say LimitNOFILE=infinity; VPS3 has one (pm2-root.service) and its process runs with 1,048,576/1,048,576. That number is PID 1's own hard limit. The mechanism is setrlimit_closest(): systemd first tries the requested value, and on EPERM clips it to its own hard limit and tries again. Infinity is always EPERM (the kernel does not accept it), so "infinity" effectively means "PID 1's ceiling". This is where the chain described in Debian's changelog closes: on a system with the nr_open bump, PID 1's ceiling is 1,073,741,816, and every unit that says infinity gets that number.

I measured six cases with systemd-run in transient units:

`LimitNOFILE=`	What the process sees (soft / hard)
(not set)	1024 / 524288
`infinity`	1048576 / 1048576
`2000000`	2000000 / 2000000
`2000000000`	1048576 / 1048576
`65535`	65535 / 65535
`1024:524288`	1024 / 524288

Rows three and four are interesting side by side: an explicit 2,000,000 exceeds PID 1's own ceiling (1,048,576) yet is applied, because PID 1 holds CAP_SYS_RESOURCE and can hand out any number up to nr_open. 2,000,000,000 exceeds nr_open, so it is EPERM, the clipping kicks in, and the process silently gets 1,048,576; nothing in the journal. The "write something big, what could go wrong" reflex works exactly backwards here. We also see that a single value pins both soft and hard limits to the same number: the LimitNOFILE=65535 override I gave nginx means 24 nginx processes run with 65535/65535; since I never wrote worker_rlimit_nofile, the workers inherit from the master.

Docker noticed this trap in 2023 and removed it from its unit files. moby's #45534 (August 2023, Docker 25.0) and containerd's #8924 (October 2023, containerd 2.0) deleted the LimitNOFILE=infinity line and left it to systemd's 1024:524288 default; the commit message notes that on post-v240 systems infinity resolves to "2^30 (over 1 billion)" and that this produced serious resource growth in some software. The Docker 29.4.3 and containerd 2.2.3 units on VPS3 have no LimitNOFILE; inside docker run --rm debian:trixie-slim, /proc/self/limits shows 1024/524288. Ask for --ulimit nofile=2000000000:2000000000 and runc is less polite: the container never starts, with "error setting rlimit type 7: operation not permitted" (7 is the number of RLIMIT_NOFILE).

The ones that raise their own limit

I looked at the init processes of the 59 containers running on VPS3 at the time of measurement: 31 at 1024/524288, 17 at 524288/524288, 7 at 524287/524288, and four at their own custom values (10032 and 8161). The two middle groups are the interesting ones, because nobody gave them a --ulimit.

Everything in the 524287/524288 group is a Go binary; the host shows the same: dockerd, containerd, crowdsec, kopru-gateway, all four at 524287. Since Go 1.19 every program importing the os package raises its soft limit to the hard limit at startup (go.dev/issue/46279); today's syscall/rlimit.go deliberately writes Max - 1: "We set Cur to Max - 1 so that we are more likely to detect cases where another process uses prlimit to change our resource limits." So if /proc/<pid>/limits shows a soft limit that is one less than the hard limit, you are probably looking at a Go process. According to the same file, if a program calls Setrlimit(RLIMIT_NOFILE) itself, Go switches off the automation that hands the original limit back to child processes.

The 524288/524288 group is Node. The "Raise the open file descriptor limit" block in src/node.cc sets the soft limit straight to the hard limit when the hard limit is finite; if it is infinite, a binary search up to 2²⁰ finds the highest value that works. The 24 Node processes visible from the host (including those inside containers) are at 524288/524288 for that reason.

There is also one that stays low on purpose: PostgreSQL. The 141 postgres processes visible from the VPS3 host (including those inside containers) run with 1024/524288 and there is no need to raise them; max_files_per_process defaults to 1000 and the documentation says "if the kernel is enforcing a safe per-process limit, you don't need to worry about this setting". Postgres reads its ceiling from its own setting.

What these three behaviours share is the design written in systemd's v240 note: hard limit high, soft limit at 1024, and the raising left to the application that is sure it does not use select(). systemd.exec(5) puts it briefly: applications that can work with descriptors above 1023 should raise the soft limit to the hard limit themselves; the LimitNOFILE= row of that same table opens with "Do not use", says there should be no need to lower the hard limit, and points to MemoryMax= for memory. There is a nod to Java as well: some software allocates arrays with one element per potential descriptor, so a 1G hard limit is a problem on its own; that is why 512K was chosen as the "middle ground". In practice this is what I do: I look at /proc/<pid>/limits rather than ulimit -n, because the shell's limit and the service's limit come from different sources. When I write LimitNOFILE as a single number, I know I am pinning the soft limit to it too. I do not write infinity; that number changes from machine to machine (1,048,576 on VPS4, 1,073,741,816 on VPS1). And I now have a fingerprint for telling whether someone wrote sysctls after boot: if PID 1's hard limit is lower than fs.nr_open, then nr_open changed after PID 1 was set up. If a privileged systemd container has restart: always, this repeats on every boot, after systemd-sysctl.service, and becomes a permanent silent override; I would run such a container only knowing what it is going to write. As for my own fs.file-max = 2097152 line: systemd's argument (memcg already counts, put the ceiling in MemoryMax=) convinces me; the line is not a ceiling but a note from an old habit, and I would delete it.

Conclusion

The 2,097,152 in my sysctl file is still there and will be applied again at the next boot; until someone runs a privileged systemd container on the host. That "someone" was me five days ago, and the container wrote four lines to its own console and moved on; that it took core_pattern with it I only noticed when the reviewer asked. The most useful thing I learned about file-descriptor ceilings is not which one yields which error code; it is that each number has an owner, and the owner is rarely sysctl.d. PID 1 writes file-max, the distribution's build flag decides nr_open, PAM copies the session's hard limit from PID 1, a patch of Ubuntu origin holds the soft limit at 1024, the unit file sets the service's limit, the process itself (Go, Node) has the last word, and a privileged container can write its own distribution's defaults over all of it. By the time this post was finished, one more sysctl on VPS3 was back in place (core_pattern) and one was still open (sysrq).

Versions: VPS3 Ubuntu 24.04, systemd 255.4-1ubuntu8.17, kernel 6.8.0-139, Docker 29.4.3, containerd 2.2.3, runc 1.3.5, libpam-modules 1.5.3-5ubuntu5.7, Node 22.22.2. Lab container debian:trixie-slim + systemd 257.13-1~deb13u1. ENFILE experiment on Docker Desktop 27.4.0, linuxkit 6.10.14. Fleet comparison: VPS4 24.04/255, VPS1-VPS2 25.04/257.4, VPS5 26.04/259.5. Source readings from systemd tags v255 and v257, Linux v6.8 and v7.0 fs/file.c / fs/file_table.c, Go and Node main branches. At the end of the lab the host's file-max (LONG_MAX) and nr_open (1073741816) were left at the values I found them in, and kernel.core_pattern was restored to the apport pipe with systemctl restart apport; on linuxkit file-max was restored to 801469. Ubuntu 25.04 session measurements on VPS1/VPS2, libpam-modules 1.5.3-7ubuntu4.4.

DEV Community