DEV Community

Remdore
Remdore

Posted on AI-assisted

A container is just a process with a few private views. I built one in thirty lines.

Here is a thing that bothered me for years without my quite admitting it: I did not know what a container actually was. I knew how to run one. I could not have told you what the kernel does when you start one, because the honest answer is that there is no such kernel object to point at. A container is just a normal process that has been handed a few private versions of things it would otherwise share, and I only really believed that once I built one myself, by hand, in about thirty lines and with Docker nowhere in sight. The part that surprised me was how little there was to it.

The process I ended up with thought it was PID 1 on a machine called container. It could not see a single one of my other processes. It had its own network, empty, and its own root filesystem, and if it reached for more than 20MB of memory the kernel killed it. That is a container, all of it. And the thirty lines are not doing anything clever, they are just asking the kernel, one request at a time, for each of those private views.

Namespaces are the whole trick

The Linux feature underneath all of this is the namespace. A namespace is a private copy of one kind of global resource. There are several: PID, mount, network, UTS (the hostname), IPC, user. A normal process shares all of them with everything else on the machine. unshare is the command that says: give me a fresh one.

The entire isolation half of a container is one command:

unshare --user --map-root-user --mount --uts --ipc --pid --fork --net \
  python3 enter.py rootfs
Enter fullscreen mode Exit fullscreen mode

Each flag is buying one kind of isolation and no more. --uts hands me a private hostname, which is why I can rename the machine to container and nothing on the host notices. --pid --fork gives a private process table, and because of the fork the child, not the unshare itself, is the one that becomes PID 1. --net drops me into an empty network stack: one loopback interface, down, and nothing else at all. --mount gives a private set of mounts, which is what lets me swap the filesystem later. The odd one out is --user --map-root-user, and it is the flag doing the quiet heavy lifting, because it makes a user namespace and maps my normal login to root inside it. That is what lets the process mount things and call pivot_root, while out on the host I am still nobody special.

Run that and look around inside, and the isolation is already total:

hostname: container
pid: 1
--- processes (own PID ns) ---
PID   USER     TIME  COMMAND
    1 root      0:00 /bin/sh
    4 root      0:00 ps -ef
--- net ---
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN qlen 1000
Enter fullscreen mode Exit fullscreen mode

Two processes visible where the host has eighty-seven. One network interface, down. A hostname I chose. I did not install anything and I was never root on the host. That output is the entire point of the post: isolation is not something you build, it is something you ask the kernel for, one namespace at a time.

Swapping the root filesystem is the fiddly part

The one genuinely tricky step is giving the process a different root directory, because this is where the difference between a toy and a real container lives.

The obvious move is chroot, and it does work, but it is also the bit people warn you off, because a sufficiently privileged process can climb back out of a chroot. The old root is still mounted underneath, only hidden, and there are well-worn tricks for reaching it again. So the real runtimes reach for pivot_root instead. It swaps the root mount out and then lets you unmount the old one completely, and once the old root is unmounted there is simply nowhere left to escape to.

pivot_root is a syscall with no friendly wrapper on my machine, so enter.py calls it directly through libc. The sequence has three requirements that each cost me a run to discover. The new root has to be a mount point, so I bind-mount the rootfs onto itself. The parent mount has to be private, or pivot_root refuses with EINVAL, so I remount / as private first. And after the pivot the old root hangs under a directory I have to detach:

mount(None, "/", None, MS_REC | MS_PRIVATE)
mount(rootfs, rootfs, None, MS_BIND | MS_REC)
os.chdir(rootfs)
sc("pivot_root", b".", b"oldroot")
os.chroot("/")
sc("umount2", b"/oldroot", MNT_DETACH)   # the host filesystem is now gone
Enter fullscreen mode Exit fullscreen mode

After that last line the container cannot see the host filesystem at all. Not hidden, gone. ls /oldroot returns "No such file or directory", and the only mounts left are the new root and a fresh /proc. That is the line that turns a chroot into a container.

The mistake that made ps lie

My first version used chroot and mounted /proc the easy way, with unshare --mount-proc. Inside, ps showed no processes at all, and for a happy minute I thought I had built the most isolated container in history.

I had not. ps reads /proc, and --mount-proc mounts it over /proc in the mount namespace before the chroot happens. After the chroot, the process is looking at rootfs/proc, which is a different, empty directory. ps was not seeing an empty process table, it was seeing an empty folder. The isolation looked perfect because the instrument was unplugged.

The fix is to mount /proc after the root swap, from inside the new root, so it is the container's /proc that reflects the container's PID namespace. Once I did that, ps showed exactly two processes, which is the real answer, and a much better one than zero, because zero was a lie and two is proof the PID namespace works.

Namespaces isolate. Cgroups limit.

Isolation is only half of what people mean by "container". The other half is the ceiling, the guarantee that the thing cannot run off with all the memory on the box, and namespaces do exactly nothing about that. That is a different kernel feature entirely, control groups, and once I had built the two halves separately the division stuck in my head: namespaces govern what a process can see, cgroups govern what it can take.

A cgroup v2 memory limit really is just a number in a file. You put the process into the cgroup, write a byte count into memory.max, and the kernel does the rest. To watch it actually bite I compiled a tiny static binary, dropped it into the rootfs, and had it grab a megabyte at a time, touching each one so the pages were genuinely resident rather than just promised, then set the cap to 20MB and let it run:

cap is 20MB. starting the memory hog as a child...
allocated 5 MB
allocated 10 MB
allocated 15 MB
Killed
hog exited with code 137
Enter fullscreen mode Exit fullscreen mode

That exit code, 137, is 128 plus 9, so signal 9, a SIGKILL, and the sender was the kernel's out-of-memory killer firing the moment the cgroup went over its cap. The container's own shell was fine, still sitting there as PID 1, because the killer took only the greedy child and handed its memory back. So there it is: a real memory ceiling, enforced by the kernel, on a process I had isolated by hand a few minutes earlier.

Getting there cost me two more mistakes. My first memory hog filled a tmpfs instead of allocating anonymous memory, and tmpfs pages are not freed when the writing process dies, so the OOM killer killed the hog, the memory stayed full, and it went round again and killed PID 1, and the whole container collapsed. Anonymous memory that frees on death is the honest way to demonstrate a limit. And I could not attach the process to a cgroup by hand at all at first: on this machine my shell lives in a cgroup called /init.scope that I do not own, outside the tree systemd delegates to my user, so every attempt to move it returned EIO. The way through is systemd-run --user --scope, which starts the process inside the delegated tree where I am allowed to set limits. Which is itself the lesson: namespaces are genuinely unprivileged, cgroups need either root or a slice someone delegated to you.

What it is not

A container shares the host kernel, and once you have built one by hand you can feel exactly how thin the separation is. Inside my container, uname -r prints 6.6.114.1-microsoft-standard-WSL2, the host's kernel, because it is the host's kernel; there is only one, and every container on a machine is running on it. cat /proc/uptime inside reported the same 258,345 seconds as the host, because I did not ask for a time namespace and so the container shares the host's clock and boot time.

This is the whole difference between a container and a virtual machine, and it is not a detail. A VM brings its own kernel and the isolation goes down to the hardware. A container is your process with some of its views swapped out, running on the same kernel as everything else, and a kernel bug is a shared fate. That is the trade: containers are cheap because they are barely anything, and they are barely anything because they are just your process wearing a few borrowed namespaces.

Run it yourself

You need a root filesystem to run in. The easy way to get one is to borrow it from an image you already have:

mkdir rootfs
docker export $(docker create alpine:3.20) | tar -C rootfs -xf -
Enter fullscreen mode Exit fullscreen mode

Then the launcher is the one unshare line above plus the enter.py that does the pivot, twenty-seven lines of it, calling mount, pivot_root and sethostname through libc. Run it and check the three things that prove it worked: hostname is what you set, ps shows one or two processes and not the host's hundred, and ls /oldroot fails because the host filesystem is gone. If ps shows nothing, your /proc is in the wrong place, which is the same mistake I made.

What to take away

The next time Docker feels like magic, remember that the isolating part of it is six flags to unshare and a pivot_root, and the limiting part is a number in a file. Everything else Docker gives you, images, layers, networking, a registry, is real engineering built on top, but the container itself, the thing people imagine as a sealed box, is a process that asked the kernel for a few private views and got them. It is worth building one once, because after you have, "it's just a process" stops being a slogan and starts being something you have watched happen.

Top comments (1)

Some comments may only be visible to logged-in visitors. Sign in to view all comments.