DEV Community: Gancho Stoychev

Hardening Ollama with systemd: limits, sandboxing and two GPUs

Gancho Stoychev — Thu, 15 Oct 2026 00:00:00 +0000

Ollama on my server runs the way its installer left it: a systemd unit with a user, a restart policy and nothing else. No memory limit, no CPU limit, the whole file system writable where Unix permissions allow it, every device in /dev reachable. systemd has dozens of directives to change that, and a tool that grades how much of it a service uses: systemd-analyze security gives every unit an exposure score from 0 (tightly sandboxed) to 10 (nothing applied). Ollama started at 9.2 UNSAFE 😨.

This is the fourth post in my AI homelab series. The first built a container by hand from namespaces and cgroups, the second set up Ansible, and the third put the AI agent I work with into a sandbox. That agent wrote the roles in this lab from inside its sandbox; every command that touched systemd, I ran.

This one is Lab 0b. Ollama and a small service of my own under systemd, with limits taken from measurements, hardening applied in rounds with a regression test after each, and failures caused on purpose to prove the settings do what they say. Ollama ended at 1.4 OK 🙂, the small service at 0.9 SAFE 😀. One memory limit did nothing at all until I found out why.

Guide: every command, step by step
Results: measurements, scores and surprises
Role ollama: drop-ins for the installer's unit
Ollama's hardening: hardening.conf
Role lab0b_demo: a service of my own with a secret
The demo unit: lab0b-demo.service

What "done" meant

Three checks, taken from the roadmap:

Criterion	Result
Ollama and a second service under systemd with `MemoryMax`, `CPUQuota`, `Restart=on-failure`, `ProtectSystem=strict`, `PrivateTmp`, `NoNewPrivileges`	✅ two Ansible roles
Raise the exposure score of `systemd-analyze security`	✅ Ollama 9.2 → 1.4, the demo 8.2 → 0.9
Ad-hoc limits with `systemd-run --scope`	✅ an OOM kill inside a transient scope

The setup: Ubuntu 24.04, kernel 7.0, cgroup v2, systemd 255, Ollama 0.32.14, two RTX 5060 Ti 16 GB, 64 GB of RAM and an 8 GiB swap file. Every directive was read in the manual pages of the installed systemd before it was used, not taken from memory.

Where Ollama started

Ollama's install script writes /etc/systemd/system/ollama.service: User=ollama, Restart=always, RestartSec=3, a PATH. Next to it I had one drop-in of my own:

[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"
ExecStartPre=+/usr/bin/timeout 60 /bin/sh -c 'until /usr/bin/nvidia-smi -L >/dev/null 2>&1; do sleep 1; done'

The listen address is there because other clients on the server use Ollama. The wait is a guard against a race at boot: started too early, Ollama sees fewer cards and runs on the CPU until the next restart. The + runs that command with full privileges, outside the unit's sandboxing. That detail mattered later.

Reading	Value
Exposure score	9.2 UNSAFE 😨
Checks passed	6 of 81
`MemoryMax`, `CPUQuota`	none
`TasksMax`	70833 (systemd's default)
Memory, idle	≈ 550 MiB

What Ollama's memory really is

A memory limit on a model server is only safe if you know what the server's cgroup counts. Models live in VRAM, so I expected the cgroup to stay small. It did not.

Model loaded	Placement	`memory.current`	Page cache (file)	Anonymous (anon)
none	—	550 MiB	—	—
`llama3.1:8b`	100% GPU, one card	5.8 GiB	5.3 GiB	0.5 GiB
`qwen3.8:27b` after unloading the first	100% GPU, both cards	23.1 GiB	22.4 GiB	0.7 GiB

Most of it is page cache: the model files Ollama read on their way into VRAM. The kernel charges page cache to the cgroup that read the file, and unloading a model frees its VRAM, not its cache. After two models the cgroup counted 23 GiB, of which Ollama needed about 1.4 GiB.

The interesting case is a model that does not fit into VRAM. None of mine is larger than both cards together, so for this test I limited Ollama to one card for a while and restored the second afterwards. qwen3.8:27b then ran 25% on the CPU:

Process memory (`llama-server`, the runner)	Context 4096	Context 32768
Mapped from the model file	4.1 GiB	5.4 GiB
Anonymous	0.6 GiB	1.3 GiB
CPU share of the model	25%	35%

The layers on the CPU are mapped from the model file: clean pages the kernel can drop and read again. What grew with the context was anonymous memory, and llama.cpp's own log named it: CPU KV buffer size = 640.00 MiB, the KV cache of the layers on the CPU.

One more case turned up later. With a model entirely on the CPU, Ollama disables mmap (load_mode = none in its log), and everything is anonymous: 8.65 GiB for llama3.1:8b with a 32k context, none of it cache.

Where the model runs	Weights on the CPU	KV cache on the CPU
100% GPU	none	none
Partly on the CPU	mapped from the file, reclaimable	anonymous
Entirely on the CPU	anonymous	anonymous

Limits from measurements

First the mechanism, on a throwaway command. systemd-run --scope puts any command into a transient cgroup with limits:

sudo systemd-run --scope --expand-environment=no -p MemoryMax=100M -p MemorySwapMax=0 sh -c '<cmd>'

A 300 MB allocation was killed inside the scope, and with systemd's default OOMPolicy=stop the whole scope went with it: the wrapping shell printed Terminated before it could report anything. With OOMPolicy=continue only the allocation died.

Then the numbers for Ollama, each from a measurement:

Setting	Value	Measured basis
`MemoryHigh`	16G	Ollama needs 0.6–1.6 GiB; above this the kernel reclaims page cache
`MemoryMax`	32G	half of the host; reached only by memory that cannot be dropped
`CPUQuota`	1600%	one model entirely on the CPU uses 8 threads, about 790%
`TasksMax`	512	21 tasks idle, 76 at peak with one model

The test that matters: load two models in a row, about 22 GiB of files. The cgroup stopped 3.2 MiB below MemoryHigh, the kernel reclaimed cache 916 times, nothing was killed, and the model answered from both GPUs. The CPU quota never throttled a model on the CPU (13.2 tokens/s with it, 13.6 without).

Drop-ins, not the unit

Ollama's install script rewrites ollama.service on every run and never touches ollama.service.d/. So the role manages drop-ins only: my existing one, and hardening.conf for everything in this post.

Three decisions went into the first round:

Setting	Chosen	Why
`Restart=`	`on-failure` (was `always`)	a clean exit is deliberate, a crash is not
`OOMPolicy=`	`continue`	an OOM kill should take the model runner, not the server
Start limit	5 starts in 5 minutes	systemd's default never triggers here

The last one surprised me. systemd stops restarting a unit after 5 starts in 10 seconds, by default. Ollama waits 3 seconds between restarts, so it can never start 5 times in 10 seconds: a permanently broken Ollama would restart forever.

The file system and the GPU

ProtectSystem=strict makes the whole file system read-only for the service, except what ReadWritePaths= lists. Ollama's home is /usr/share/ollama, under /usr, so even the mildest setting would have broken it. Two paths stay writable: ~/.ollama (models, cache, its key) and ~/.nv (NVIDIA's compute cache).

Devices were the risky part, because a GPU service without its devices does not fail: it starts and runs on the CPU. PrivateDevices=yes hides /dev/nvidia*, so it was out. Instead I looked at what Ollama actually opens with a model loaded:

Device	Opened by the runner
`/dev/nvidiactl`, `/dev/nvidia0`, `/dev/nvidia-uvm`	✅
`/dev/nvidia1`	✅ when a model spans both cards
`/dev/dri/`, `/dev/nvidia-caps/`, `/dev/nvidia-modeset`	❌ never

That became DevicePolicy=closed with two device groups, char-nvidia and char-nvidia-uvm. Two details from the manual made it work. Device rules apply to the whole cgroup, so the + of the GPU wait does not bypass them: get them wrong and the wait times out. And a device group that does not exist when the unit starts is silently dropped, so nvidia_uvm is loaded first with Wants=/After=modprobe@nvidia_uvm.service.

A test that only shows "it still works" could also mean "nothing is restricted", so a transient service with the same rules tried four devices as user ollama:

Device	With the rules	Without
`/dev/nvidiactl`	✅ opened	✅ opened
`/dev/nvidia-uvm`	✅ opened	✅ opened
`/dev/dri/renderD128`	❌ Operation not permitted	✅ opened
`/dev/nvidia-caps/nvidia-cap2`	❌ Operation not permitted	✅ opened

Turning the screws, round by round

Each round went through the role, a check run, a real run, systemctl show for what systemd actually loaded, and a test:

Round	Added	Score
Start	nothing	9.2 UNSAFE 😨
1	`NoNewPrivileges`, `PrivateTmp`, restart and OOM policy	8.7 EXPOSED 🙁
2	`ProtectSystem=strict`, `ProtectHome`	8.3 EXPOSED 🙁
3	`DevicePolicy=closed`, `DeviceAllow`	8.2 EXPOSED 🙁
4	16 kernel, process and privilege restrictions	3.4 OK 🙂
5	address families, system call filter, private user namespace, no writable-executable memory	1.5 OK 🙂
6	remove privileged system calls	1.4 OK 🙂

Round 4 did most of the work: empty capability set, protected kernel tunables, modules, logs and control groups, ProtectProc=invisible, no new namespaces, native system calls only.

Round 5 had five directives that can break a GPU service, so each was tried alone in a temporary drop-in under /run before it went into the role:

Directive	Risk	Result
`RestrictAddressFamilies`	blocks a socket type Ollama needs	✅ pass
`SystemCallFilter`	a system call outside the list kills the process	✅ pass
`PrivateUsers`	GPU access through the render and video groups	✅ pass
`MemoryDenyWriteExecute`	CUDA compiling kernels at load time	✅ pass
`ProcSubset=pid`	NVIDIA's files in `/proc`	❌ not used

What went into the role: RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6 AF_NETLINK; SystemCallFilter=@system-service with SystemCallErrorNumber=EPERM, so that a missing call fails instead of killing the process; PrivateUsers=yes, which works because the NVIDIA device nodes are world-accessible; and MemoryDenyWriteExecute=yes, which works because CUDA has ready-made kernels for these cards and compiles nothing at load time.

ProcSubset=pid was the instructive one. The GPU kept working and every test passed, but Ollama logged error looking up system memory … open /proc/meminfo: no such file or directory. It would have kept running, deciding blind whether a model fits into memory.

Round 6 came from reading the score: @system-service, systemd's own system call set for ordinary services, still allows chown, setuid and chroot. A second line, SystemCallFilter=~@privileged, removes them.

Does it still work?

"It answers" is not enough for a service that can fall back to the CPU without an error. After round 4, and again on the final configuration:

Test	Result
Copy and delete a model	✅
Pull a model (network, large files, temporary files)	✅
Embeddings	✅ 384 dimensions
A client in a Docker container	✅
A model on both cards	✅ 100% GPU, same VRAM as before
An image through the model's vision encoder	✅ "Red" for a red square
A model entirely on the CPU	✅ 13.5 tokens/s, as before hardening
A reboot	✅ both cards found 6 s after the start

After the reboot the journal shows the order the unit asks for: nvidia_uvm loaded, then Ollama started, then both cards found through CUDA.

A service of my own

Ollama's unit belongs to its installer. A unit of my own shows what is possible from the start, so the lab has a second service: python3 -m http.server on 127.0.0.1:8765, deliberately trivial, with a secret.

Directive	What it does
`DynamicUser=yes`	a user allocated at start and released at stop, nothing in `/etc/passwd`
`StateDirectory=lab0b-demo`	`/var/lib/lab0b-demo`, created by systemd for that user
`EnvironmentFile=/etc/lab0b-demo.env`	read by systemd as root before the process starts

The secret never enters git: I create the file by hand as root, mode 0600, and the role only asserts that it exists with that owner and mode.

Who tries to read it	Result
`systemctl show`	✅ only the path, never the content
The service process	✅ `DEMO_TOKEN` is set
Another user	❌ Permission denied
A service with a dynamic user, reading the file	❌ Permission denied

The other user tried the process environment, /proc/<pid>/environ; root can read it, nobody else.

With nothing to protect but a socket on loopback, the demo could have what Ollama cannot. It was hardened in one go, so there are two measurements, not six:

Round	Added	Score
Start	`DynamicUser`, `StateDirectory`, `EnvironmentFile`	8.2 EXPOSED 🙁
1	`ProtectHome=yes`, `PrivateDevices=yes`, `PrivateUsers=yes`, `CapabilityBoundingSet=` (empty), `ProtectKernelTunables=yes`, `ProtectKernelModules=yes`, `ProtectKernelLogs=yes`, `ProtectControlGroups=yes`, `ProtectClock=yes`, `ProtectHostname=yes`, `ProtectProc=invisible`, `ProcSubset=pid`, `RestrictNamespaces=yes`, `RestrictRealtime=yes`, `LockPersonality=yes`, `MemoryDenyWriteExecute=yes`, `RestrictAddressFamilies=AF_INET`, `IPAddressDeny=any`, `IPAddressAllow=localhost`, `SystemCallArchitectures=native`, `SystemCallFilter=@system-service`, `SystemCallFilter=~@privileged @resources`, `SystemCallErrorNumber=EPERM`, `UMask=0077`, limits: `MemoryMax=64M`, `MemorySwapMax=0`, `CPUQuota=50%`, `TasksMax=16`, `Restart=on-failure`	0.9 SAFE 😀

DynamicUser=yes alone implies six protections, among them ProtectSystem=strict and NoNewPrivileges, yet the start was 8.2: those weigh little in the score.

The IP filter had one surprise. From inside the service's cgroup, a connection to the internet did not fail with "permission denied". It timed out: systemd's filter drops packets silently.

Breaking it on purpose

The restart and OOM settings had only been read in the documentation. Four tests:

Test	Result
`kill -9` Ollama	✅ back in 4 s, both cards found
Five more kills within a minute	✅ sixth start refused
200 MB inside the demo's 64M limit	⚠️ no OOM at all
6 GiB inside Ollama's cgroup under a temporary 4G limit	✅ only the allocation killed, Ollama kept running

The start limit shows up only in the journal, as Start request repeated too quickly; the unit's Result= still says signal.

The third one was the finding of the lab. The cgroup's counters showed where the memory went:

Counter	Value
`memory.max`	64 MiB
`memory.peak`	64 MiB
`memory.swap.max`	unlimited
`memory.swap.peak`	155 MiB

💡 Key takeaway: MemoryMax= limits RAM. Swap has its own limit, MemorySwapMax=, and it is unlimited by default. On a host with swap, a service above its memory limit does not get killed; it moves to swap and slows down. For Ollama that means a model or its KV cache in swap, answers in minutes instead of an error a client can handle.

Both services got MemorySwapMax=0, and the same test then went as designed: the allocation was killed, and Restart=on-failure brought the demo back within the same second. That also answered the one question the manual had left open: a unit stopped by OOMPolicy=stop counts as failed and is restarted.

What the score does and does not tell you

💡 Note: The manual is explicit: a high score means little sandboxing is applied, not that a service is vulnerable, and a low one says nothing about the program's own code.

What is left is a list of things each service genuinely needs:

Still failed	Ollama	Demo	Why
`PrivateNetwork`	⚠️	⚠️	both serve clients
Internet sockets	⚠️	⚠️	both listen
`IPAddressDeny`	⚠️	partly	Ollama downloads models
`PrivateDevices`	⚠️	✅	the GPU
`DeviceAllow`	partly	partly	device lists
`ProcSubset`	⚠️	✅	Ollama needs `/proc/meminfo`
`@resources` system calls	⚠️	✅	llama.cpp places its threads on cores
`UMask`	⚠️	✅	Ollama's files stay readable for its group
`RootDirectory`	⚠️	⚠️	both run in the host's root

"Partly" means half the exposure of the check: the demo's IP filter allows loopback, and both device lists contain what the tool calls "special devices". Ollama's are the GPU's; the demo's, despite PrivateDevices=yes, is the clock device that ProtectClock=yes adds.

Small things that bite

MemoryMax= without MemorySwapMax=0 limits RAM, not memory.
A service without its GPU devices does not fail; it waits, then runs on the CPU. Check the log for the cards after every change.
Result= does not show the start limit, and the restart counter includes the restart the limit refused. Read the journal.
systemctl revert removes every drop-in of a unit, not just the one you meant. To undo systemctl set-property --runtime, set the old value and remove its file under /run/systemd/system.control/.
sudo cmd < file opens the file as you, not as root.

What surprised me

A memory limit that did nothing: MemoryMax= let a service grow into swap until MemorySwapMax=0 was set.
Ollama's memory is mostly page cache, even of models it has already unloaded; what it needs is a gigabyte or two.
Whether the CPU part of a model can be reclaimed depends on how Ollama loads it: mapped from the file when the model is partly on the GPU, all anonymous memory when it is entirely on the CPU.
A CUDA service took far more hardening than I expected, down to 1.4. What stopped the last points was the device list, the network and /proc/meminfo, not CUDA.
The failures that matter are quiet: a GPU service on the CPU, a server blind to system memory with one warning in the log, a firewall that looks like a slow network.

What's next

The next lab is the firewall: iptables to nftables, and how ufw, Docker's rules and nftables share one kernel. Ollama listens on all interfaces because other clients on the server need it, so what reaches its port is the firewall's decision, and systemd's per-service IP filter is a second layer worth comparing with it. Later, the observability lab gets the alerts systemd cannot send by itself: a unit in failed, a restart counter going up, an oom_kill in a service's memory events.

Everything from this post, in full:

Guide: every command, step by step
Results: measurements, scores and surprises
Role ollama: drop-ins for the installer's unit
Ollama's hardening: hardening.conf
Role lab0b_demo: a service of my own with a secret
The demo unit: lab0b-demo.service

Originally published at gshell.dev.

Sandboxing Claude Code on Ubuntu

Gancho Stoychev — Wed, 14 Oct 2026 00:00:00 +0000

An AI coding agent on my server runs shell commands I approve one at a time. Or, with the sandbox switched on, commands it does not need to ask about at all. That second mode is only as good as the box around it, so before trusting it I wanted to know exactly what that box is: which files a command can read, where it can connect, what it can do as root inside, and what happens when the box itself is missing.

This is the third post in my AI homelab series. The first built a container by hand and ended with a plan to sandbox Claude Code; the second set up Ansible so that the sandbox could arrive as a role instead of another manual tweak.

This one is Lab 0f. Claude Code's built-in sandbox on Ubuntu 24.04, enforced by settings that no repository can loosen, deployed by Ansible, and tested with a matrix of probes run from inside the sandbox by the agent itself. Almost everything held. One thing did not, and it cannot be fixed from outside.

Guide: every command, step by step
Results: the probe matrix, versions and surprises
Role claude_code: packages, AppArmor profile, managed settings
Managed settings: managed-settings.json
AppArmor profile: claude-bwrap
Rules for the agent: CLAUDE.md

What "done" meant

Four checks, taken from the roadmap:

A matrix of probes is blocked: writing outside the repository, reaching the network without the proxy, reading ~/.ssh, connecting to a host that is not allowed.
The session refuses to start without the sandbox.
A repository's own settings cannot switch the sandbox off or widen it.
The whole configuration is an Ansible role, so the next node is one playbook run.

What the sandbox is

Claude Code's sandbox wraps every shell command the agent runs, and everything that command starts, in two independent layers:

Filesystem (bubblewrap on Linux): writes only to the working directory and a per-user temp directory. Reads to almost the whole machine, ~/.ssh included, unless you deny them.
Network: the command gets its own network namespace with no route out. The only exit is a proxy Claude Code runs outside the sandbox, which checks every hostname against an allowlist that starts empty.

It is off by default, and on Linux it needs two packages, bubblewrap and socat. Equally important is what it does not wrap: Claude's own file tools (Read, Edit, Write), web tools, hooks, MCP servers, and anything you type yourself at the ! prompt. Those follow permission rules, not the sandbox.

A pinned Claude Code first

The first surprise came before any sandboxing. Claude Code's native installer lives in ~/.local/share/claude and updates itself in the background. The session I was working in had moved to a release from that same morning. In a repo where every version is pinned, that is not a decision anyone made.

Anthropic publishes a signed apt repository with stable and latest channels, and it keeps older versions. So the role installs Claude Code like any other package:

the signing key, committed to the role after checking its fingerprint against the documentation;
the repository as a deb822 .sources file;
claude-code=2.1.287-1, the stable release, two weeks old at the time;
an apt pin with priority 1001, so a manual apt upgrade does not move it. An upgrade is one changed line in the role.

Before deleting the native installation I looked inside. If you do the same switch, know what you are deleting:

~/.local/share/claude/versions/2.1.292
~/.local/share/claude/versions/2.1.296
~/.local/share/claude/versions/2.1.299   # 739 MB in total
~/.local/bin/claude -> ~/.local/share/claude/versions/2.1.299

Only binaries. Your settings, conversation history, memory and everything else live in ~/.claude/, which the switch does not touch; claude --continue resumes the same conversation on the new binary.

Two small traps along the way. The first --check of the role on a new host always ends with failed=1: check mode reports that it would add the repository but writes nothing, so apt has never heard of claude-code when the install task asks for it. Ansible cannot follow a chain of tasks where a later one depends on what an earlier one would have changed. And after the switch, claude doctor still remembers the old install method and suggests claude install to "repair" it. That would put the self-updating build back in front of the package in PATH. Managed settings now set DISABLE_UPDATES=1, which makes that command refuse.

Letting bwrap through AppArmor

On Ubuntu 24.04 an unprivileged process may create a user namespace, but AppArmor immediately moves it into a profile that denies every capability. bwrap needs a few of them for its own setup:

$ bwrap --ro-bind / / --unshare-all --die-with-parent id
bwrap: loopback: Failed RTM_NEWADDR: Operation not permitted

Audit log:

apparmor="AUDIT" operation="userns_create" ... profile="unconfined" ... target="unprivileged_userns" execpath="/usr/bin/bwrap"
apparmor="DENIED" operation="capable" profile="unprivileged_userns" ... comm="bwrap" capname="net_admin"

The namespace is created; giving it a loopback interface is not allowed. That loopback matters: sandboxed commands reach the proxy through a local port inside their own namespace.

The documentation's fix is an AppArmor profile for /usr/bin/bwrap. That would lift the restriction for every program on the host that runs bwrap. I used the same pattern as unshare in Lab 0a instead: a dedicated copy, /usr/local/bin/claude-bwrap, with its own profile, and Claude Code pointed at it through sandbox.bwrapPath, a setting only managed settings can set. The system bwrap stays restricted, and the aa-exec bypass that Lab 0a closed stays closed for the new profile too.

Settings a repository cannot loosen

Claude Code reads settings from several levels: managed (/etc/claude-code/managed-settings.json), the command line, the repository's .claude/, and your own ~/.claude/settings.json. Managed settings win, and they are root-owned, so neither a cloned repository nor the agent can edit them. The role installs them with these choices:

Fail closed. enabled: true, failIfUnavailable: true (refuse to start without a sandbox, instead of the default: run commands without one), and allowUnsandboxedCommands: false (no "retry this outside the sandbox").
Admin-required. That last setting, set in managed settings, makes Claude Code ignore everything in a repository's settings that would loosen the sandbox: excluded commands, allowed domains, extra writable paths.
Four allowed hosts, locked to managed settings, with strictAllowlist: anything else is refused without even asking me.
Secrets, twice. ~/.ssh, ~/.aws, every .env under my home directory and Claude Code's own OAuth token file are hidden from sandboxed commands (sandbox.credentials) and from the file tools (Read(...) deny rules), because the file tools run outside the sandbox.
Manual permission mode by default, with sandboxed commands approved automatically: the box is the boundary, and I still review every diff before a commit.
allowManagedHooksOnly: true and no managed hooks: hooks run outside the sandbox with my full access, so no cloned repository may bring one.

💡 Important note: One file must stay world-readable: Claude Code silently starts without a managed settings file it cannot read.

The probe matrix

Every probe was run by Claude in its own sandbox. Commands typed at the ! prompt run outside the sandbox and prove nothing.

Probe / Command	Expected / Action	Result	Stopped By
`touch ~/sandbox-probe`	Write to home dir	Read-only file system	✅ Sandbox (bwrap)
`touch .git/hooks/sandbox-probe`	Write inside repo	Read-only file system	✅ Sandbox (Protected path)
`ls ~/.ssh`	Inspect SSH keys	0 entries	✅ Sandbox (`--tmpfs`)
Read `.env` / token file	Read secrets	Permission denied	✅ Sandbox (`sandbox.credentials`)
`Read` tool on `~/.ssh/config`	Read config file	Refused	✅ Permission rule
`curl --noproxy '*' https://example.com`	Bypass HTTP proxy	Could not resolve host	✅ Sandbox (network namespace)
`curl https://example.com`	Unlisted host HTTP	Proxy 403, no prompt	✅ Sandbox proxy
`curl https://pypi.org/simple/`	Allowed host HTTP	200 OK	✅ Allowed (control)
`curl localhost:11434`	Access host Ollama	Connection refused	✅ Sandbox (private loopback)
`ssh -T git@github.com`	SSH outbound	Could not resolve hostname	✅ Sandbox
`nvidia-smi`, CUDA `cuInit`	Access GPU devices	No devices	✅ Sandbox (minimal `/dev`)
Retry with `dangerouslyDisableSandbox`	Override sandbox	Runs sandboxed anyway	✅ Managed settings
Repo settings with `enabled: false`	Loosen security	No effect	✅ Admin-required sandbox
`git push`	Push repository	Refused before run	✅ Permission rule
`unshare --user --map-root-user`	Elevate privileges	Root with CapEff	⚠️ NOT STOPPED (the gap)

Inside, a sandboxed command has no effective capabilities, no_new_privs is set (so sudo cannot raise privileges) and a seccomp filter is active. Then the last line of the table.

And the two checks of the sandbox itself, done with headless claude -p from a terminal:

$ sudo chmod 0644 /usr/local/bin/claude-bwrap; claude -p 'Run: echo sandbox-ok'
Error: sandbox required but unavailable: ... bubblewrap (bwrap) not executable at /usr/local/bin/claude-bwrap ...
  sandbox.failIfUnavailable is set — refusing to start without a working sandbox.

Ansible saw the changed mode and restored it. The second check removed the AppArmor profile from the kernel while leaving its file in place. Claude Code started anyway: its start-up check looks for an executable binary, not for a working sandbox. Every command then failed closed, nothing ran outside the box, and the first version of the role reported ok because the file had not changed. The role now reads /sys/kernel/security/apparmor/profiles and reloads the profile when it is missing, the same "check the running state, not the file" pattern as the sysctl in the previous lab.

Looking inside

From another terminal, while a sandboxed sleep was running, the process tree shows how the box is built:

claude-bwrap --new-session --die-with-parent --unsetenv CLAUDE_CODE_MESSAGING_TOKEN ...
 └─ bash -c socat ...
     ├─ socat TCP-LISTEN:3128 → UNIX-CONNECT:/tmp/claude-http-<id>.sock  # HTTP proxy bridge
     ├─ socat TCP-LISTEN:1080 → UNIX-CONNECT:/tmp/claude-http-<id>.sock  # SOCKS bridge
     └─ 3 (/proc/self/fd/3 ...)                                          # apply-seccomp
         └─ 3
             └─ bash -c ...
                 └─ sleep 300

The two socat processes are the only way out: they listen on ports inside the command's private loopback and forward to a Unix socket that leads to the proxy outside. They start before the seccomp filter, which blocks Unix sockets for the command itself. The process called 3 is Claude Code's seccomp helper, apply-seccomp, executed straight from a file descriptor: it has no path on disk.

lsns on the sleep, compared with my own shell:

Namespace	Created by
net	bwrap
user, mnt	apply-seccomp (nested inside bwrap's)
pid	apply-seccomp's child
time, cgroup, uts, ipc	Shared with the host

The bwrap command line reads like a recipe: --ro-bind / / (the whole host, read-only), then --bind for the writable parts (the repository, the temp directory), then the protected paths mounted read-only again on top (.git/hooks, .git/config, .claude), an empty --tmpfs over ~/.ssh, and /dev/null bound over every denied file. Every protocol it can think of is pointed at the proxy, including git over SSH through a socat ProxyCommand. The proxy URL even carries credentials: its user name is the base64 of the tool call's ID, so every request can be attributed to one command.

The gap

Back to the last line of the matrix:

$ unshare --user --map-root-user sh -c 'id -u; grep CapEff /proc/self/status'
0
CapEff: 000001ffffffffff

A sandboxed command can create a nested user namespace and be root with every capability inside it. This is not an escape on its own: the nested namespace sits inside bwrap's mounts and network, so the home directory stays read-only and the network still ends at the proxy. But it re-opens exactly what Ubuntu's user namespace restriction closes: kernel code that otherwise needs root, the usual route of local privilege escalation bugs.

The cause is the AppArmor exception itself. Programs started by bwrap inherit its profile, claude-bwrap (unconfined), userns permission included. The documentation says the profile "applies only to bwrap itself"; with that profile form it does not, and the documented profile for /usr/bin/bwrap behaves the same way, for every program on the host.

⚠️ Upstream incompatibility: Ubuntu has an answer for bwrap: bwrap-userns-restrict, an experimental profile in apparmor-profiles, enabled by default from 26.04. It lets bwrap use capabilities for its setup and stacks everything bwrap runs with a second profile that has audit deny capability. I adapted it to the copy and tested it. No sandboxed command started:

apply-seccomp: write /proc/self/setgroups (nested userns is capability-restricted; caller must provide CAP_SYS_ADMIN): Permission denied

Claude Code's own seccomp helper creates a nested user namespace and needs CAP_SYS_ADMIN in it. The sandbox relies on precisely the thing Ubuntu's fix takes away, and since the helper has no path on disk, AppArmor cannot make a narrow exception for it. This is known upstream: claude-code#87680 reports the same denial on Ubuntu 26.04, where the sandbox does not start at all, and sandbox-runtime#428 proposes fixes inside the runtime. The workaround in circulation gives the capability back to bwrap's children, which re-opens the same gap for every bwrap user on the system. I kept the simple profile and documented the gap.

Is all this control worth it?

A fair question at that point: if the gap cannot be closed, why manage all of this instead of switching on the built-in sandbox and stopping there?

Because this is the built-in sandbox. Nothing sits on top of it. Around it there are three things, each with a reason:

What	Why	Without it
`bubblewrap`, `socat`, AppArmor exception	Required on Ubuntu 24.04	The sandbox does not start at all
Built-in options in root-owned managed settings	Repo or agent cannot loosen them	Agent can edit `~/.claude/settings.json`, opening the box with one "Yes"
An Ansible role	Repeatable configuration	Manual steps required on every node

The gap is in the built-in sandbox itself. The probes found it; without them it would be there unnoticed.

And there is no single command to block. unshare is a thin wrapper around a system call that python3 -c 'import os; os.unshare(os.CLONE_NEWUSER)' or any small program can make. A permission rule for unshare * stops the literal command and nothing else, so it is a speed bump that looks like protection. AppArmor cannot target a helper without a path. The right tool is a seccomp rule against CLONE_NEWUSER for the command, and that rule belongs in Claude Code's own helper.

Until then the practical defence is boring: a patched kernel. The node had 54 pending updates during the lab, AppArmor among them. Closing the gap for good is part of a later lab that compares four sandboxes for the same agent, two of them with seccomp and gVisor of my own.

What the built-in sandbox does deliver, proven: no writes outside the repository and the temp directory, protected paths read-only even inside it, secrets unreadable, no network except four hosts through the proxy, no way back out of the box, and a refusal to start without it.

Small things that bite

The permission mode is separate from the sandbox. Managed settings that enforce the sandbox do not set the mode; a resumed session kept auto mode until permissions.defaultMode said otherwise.
Auto-approval is not universal. Sandboxed commands still prompt when they name files outside the working directory, assign variables computed at run time, or ask for the escape hatch. curl to a URL does not.
Globs in deny rules are expanded at every command. On Linux, ~/**/.env becomes the list of .env files that exist when a command starts, and deny rules for the file tools are merged into the sandbox too. bwrap accepts at most 9000 arguments, about 3000 mounts, so a glob over /proc is out of the question.
The file tools see what commands cannot. A sandboxed command has its own /proc; the Read tool runs as you, and /proc/<pid>/environ of your processes holds tokens. Without an allow rule it asks first. Do not approve it.
git add . from inside the sandbox would try to stage the empty placeholders it mounts for protected paths (.bashrc, .gitconfig, .mcp.json), which appear as untracked files only inside it. Stage explicit paths.
Ansible in the sandbox fails at start-up because ~/.ansible/tmp is on the read-only home; ANSIBLE_LOCAL_TEMP in the repository's settings fixes it without widening anything.

What surprised me

Seccomp vs AppArmor collision: The sandbox's own seccomp helper needs root capabilities in a nested user namespace, so Ubuntu's fix for bwrap and Claude Code's sandbox cannot run together.
AppArmor profile inheritance: Programs started by bwrap inherit its AppArmor exception, contrary to what the documentation says.
Shallow startup check: failIfUnavailable checks for a binary, not for a working sandbox; a profile missing from the kernel fails closed, but only at the first command.
Auto-update surprises: The agent's update channel had moved my session to a release from the same morning before I had changed anything.
Outer boundary reliance: Half of the protection lives outside the sandbox: file tools, hooks and your own settings file follow permission rules, so managed settings matter as much as bwrap.

What's next

The next lab is systemd as a platform: Ollama and Ghostwheel as hardened services with memory and CPU limits, and systemd-analyze security as the score. The nested user namespace stays on the list for the sandbox comparison later in the series, along with the question of how a sandboxed agent should reach a local model: through an authenticated gateway, not Ollama's open API.

Everything from this post, in full:

Guide: every command, step by step
Results: the probe matrix, versions and surprises
Role claude_code: packages, AppArmor profile, managed settings
Managed settings: managed-settings.json
AppArmor profile: claude-bwrap
Rules for the agent: CLAUDE.md

Originally published at gshell.dev.

From manual tweaks to Ansible roles: first steps

Gancho Stoychev — Tue, 13 Oct 2026 00:00:00 +0000

SSH in. Tweak something. Move on to the next machine. Repeat. Sound familiar? Ansible turns those manual tweaks into configuration you can read, review and run again, on one machine or on ten.

This is the second post in my AI homelab series. The previous ended with a plan to sandbox Claude Code. That lab will set up its sandbox with an Ansible role, and Lab 0a had also left me with two fixes I'd made by hand: an AppArmor exception and a kernel setting. Before moving on, it made sense to step back, set up Ansible properly and learn the basics.

That's what this post is about: installing Ansible, configuring it, and taking the first steps toward managing the homelab as code. The very first run against a server I had configured by hand found exactly one difference: two spaces of indentation.

Even if you're not following the series, you're in the right place: you can follow along and learn the Ansible basics without reading any of the previous posts.

This one is Lab 0g. A laptop becomes the control node, the server is reached over plain SSH, the inventory holds nothing but host names, and the two fixes from Lab 0a become two roles that Ansible checks, applies and keeps in place.

Guide: every command, step by step
Results: versions, runs and surprises
Entry point: site.yml
Role common: the AppArmor sysctl, on disk and in the kernel
Role lab_tools: the unshare copy and its profile from Lab 0a
Configuration: ansible.cfg
Interpreter pin: group_vars/all.yml

What "done" meant

Four checks, decided before the first run:

From the control node over SSH, a dry run shows the drift between the server and the repo, a real run fixes it, and a second run changes nothing.
A change I make by hand on purpose is detected by the dry run and corrected.
The kernel setting from Lab 0a lives in a role, the bypass it closes stays closed, and the AppArmor exception still works.
Every concept used is explained in the guide.

Setup

Two machines with two roles:

Role	Machine	What it needs
Control node	MacBook (Apple M1), macOS 15	Ansible
Managed node	Z790, Ubuntu 24.04	`sshd` and Python 3.9+, nothing else

Ansible has no agent. For every task the control node packages a small Python module with its arguments, copies it to the node over SSH, runs it there with the node's Python, reads back a JSON result and deletes the temporary files. That is why the node needs nothing installed.

Installing Ansible

Ubuntu 24.04's archive ships ansible-core 2.16.3, from early 2024. PyPI had 2.21.5, which needs Python 3.12 or newer on the control node. I took that, with two decisions:

ansible-core, not ansible. The ansible package is a bundle of hundreds of community collections. Everything in this lab uses only ansible.builtin.
pipx with an exact version. pipx installs a Python application into its own virtualenv, so the system Python and its packages are left alone, and ==2.21.5 makes the lab repeatable.

On macOS there was a catch. The system Python is 3.9, too old, and Homebrew's pipx runs on its own Python, 3.15 at the time, which ansible-core 2.21.5 does not list as supported (3.12–3.14). So Ansible's virtualenv gets an explicit interpreter:

brew install python@3.12 pipx
pipx install --python /opt/homebrew/bin/python3.12 'ansible-core==2.21.5'
pipx ensurepath

==2.21.5 pins Ansible, not its dependencies. pipx runpip ansible-core freeze lists the exact versions; I also installed Ansible on the server for checks that need no connection, and both freezes matched package for package (Jinja2 3.1.6, PyYAML 6.0.3, cryptography 50.0.2 and the rest).

Reaching the node

SSH does the connecting

Ansible connects with the control node's OpenSSH and honours ~/.ssh/config. So everything needed to reach the server goes there, and nowhere else:

Host z790
  HostName z790
  User <user>
  Port 22
  IdentityFile ~/.ssh/id_ed25519_lab
  IdentitiesOnly yes

IdentityFile names the key instead of relying on whatever the SSH agent happens to hold, and IdentitiesOnly offers only that key. Before Ansible ever runs, the same connection can be tested the way Ansible will use it, with prompts forbidden:

ssh -o BatchMode=yes -o ConnectTimeout=5 z790 'hostname; python3 --version'

An inventory that holds only names

With the connection details in ~/.ssh/config, the inventory is just a list of hosts. No addresses, users or ports in the repo, and adding a node later is one more line.

I did consider running Ansible on the server itself with a local connection. It would have worked, and it would have hidden exactly the parts that break in real use: SSH authentication, privilege escalation over an SSH session, and Python discovery on another machine.

`ansible.cfg` is read from the current directory

The whole configuration is one setting:

# Read only when Ansible runs from this directory (or via ANSIBLE_CONFIG).
[defaults]
inventory = inventory/hosts.yml

Ansible looks for ansible.cfg in $ANSIBLE_CONFIG, then the current directory, then ~/.ansible.cfg, then /etc/ansible/. A relative path inside it resolves against the file's own directory. So all commands run from ansible/; from anywhere else there is no inventory. ansible-config dump --only-changed shows which settings are in effect and where each one came from.

First contact

`ping` is not ICMP

ansible z790 -m ansible.builtin.ping

Ansible's ping module tests the whole chain: SSH with the configured key, copying a module to the node, finding Python there, running it and getting JSON back. The first one returned pong, and a warning.

Python discovery does not pick the system Python

[WARNING]: Host 'z790' is using the discovered Python interpreter at '/usr/bin/python3.12', but future installation of another Python interpreter could cause a different interpreter to be discovered.

The default discovery walks a fixed list that starts at python3.14 and goes down, and only near the end tries /usr/bin/python3. On Ubuntu 24.04 it found python3.12 first. Today that is the same binary. But a newer python3.13 installed later, from a PPA or as a dependency, would silently become Ansible's interpreter, and modules that need the distribution's Python packages would break: ansible.builtin.apt needs python3-apt, which is built only for the system Python. One line in group_vars/all.yml pins it for every managed host:

ansible_python_interpreter: /usr/bin/python3

After that, ping returned pong with no warning. Privilege escalation got its own test with a harmless command, ansible z790 -m ansible.builtin.command -a id --become, which printed uid=0(root).

Two spaces of drift

The first role, lab_tools, holds the AppArmor exception from Lab 0a: a dedicated copy of unshare and a profile that lets it create user namespaces. I had installed both by hand. The first run was a dry run, --check --diff, which runs every check as root and applies nothing:

TASK [lab_tools : Copy unshare to a dedicated binary for Lab 0a] *****
ok: [z790]

TASK [lab_tools : Install AppArmor profile that allows user namespaces for the copy] *****
--- before: /etc/apparmor.d/lab0a-unshare
+++ after: <repo>/ansible/roles/lab_tools/files/lab0a-unshare
@@ -2,5 +2,5 @@
 include <tunables/global>

 profile lab0a-unshare /usr/local/bin/lab0a-unshare flags=(unconfined) {
- userns,
+userns,
 }

changed: [z790]

RUNNING HANDLER [lab_tools : Reload lab0a-unshare AppArmor profile] *****
skipping: [z790]

PLAY RECAP *****
z790 : ok=3 changed=1 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0

The binary was identical. The profile on the server was indented, the one in the repo was not. The repo is the source of truth, so Ansible reported a change.

The handler line is worth a second look. A handler runs only when a task that notifies it reports changed, so here it started. Its task is a command (apparmor_parser -r), and command cannot know what an arbitrary command would do, so in check mode it does not run it and reports skipped. Ansible does not even evaluate changed_when for a skipped task: I checked that in the ansible-core source before trusting the recap.

The real run replaced the profile and reloaded it (ok=4 changed=2). The file was now byte-for-byte the repo's, 124 bytes instead of 126, and an unprivileged user still got root in a user namespace through the copy, while the system unshare stayed blocked:

uid=0(root) gid=0(root) groups=0(root),65534(nogroup)
unshare: write failed /proc/self/uid_map: Operation not permitted

The second run: ok=3 changed=0, no diff and no handler. That is the whole promise of Ansible: once the machine is in the described state, running it again changes nothing.

💡 Key takeaway: AppArmor does not care about indentation. The policy the kernel loaded was the same before and after; the profile's entries in /sys/kernel/security/apparmor/ still carried the boot time. Ansible compares text, not meaning. A whitespace-only change is a change, and it fires the handler.

Breaking it on purpose

Next I changed something by hand, on purpose:

sudo chmod 0775 /usr/local/bin/lab0a-unshare

Group-write is not a random choice. Git does not store it, which is why every file task in the role sets mode explicitly. And on this binary it is dangerous: AppArmor lets this one file create user namespaces, and if its group can write to it, any member of that group can replace it and the replacement inherits the exception.

The dry run caught it: the binary task changed, ok=3 changed=1, no handler. But there was no diff. The copy module's diff covers file content only; when only the owner, group or mode differ, it reports changed and shows nothing else. A changed without a diff from copy means attributes.

The real run fixed it with the same output, and the next run was changed=0 again. On the server, ctime was new and mtime unchanged: Ansible had reset the mode without rewriting the file, because the checksums matched.

A setting that lives in two places

The second fix from Lab 0a is a kernel setting. With kernel.apparmor_restrict_unprivileged_unconfined = 1, an unprivileged user can no longer use aa-exec to slip into a permissive AppArmor profile and get a user namespace with full capabilities, one of the bypasses Qualys published in 2025. I had put it in /etc/sysctl.d/ by hand.

It became a new role, common, rather than a task in lab_tools: lab_tools is meant to be removed when the labs no longer need it, and this protection must stay.

A file in /etc/sysctl.d/ takes effect at boot. The obvious design, writing the file and reloading it from a handler when the file changes, misses one case: someone runs sysctl -w at runtime. The file stays right, Ansible says ok, and the protection is off until the next reboot. With only ansible.builtin there is no sysctl module, so the role does it in three tasks: install the file, read the running value, and load the file when the value differs.

- name: Read the running value of kernel.apparmor_restrict_unprivileged_unconfined
  ansible.builtin.command: sysctl -n kernel.apparmor_restrict_unprivileged_unconfined
  register: common_restrict_unconfined
  changed_when: false
  check_mode: false

- name: Apply the drop-in when the running value differs
  ansible.builtin.command:
    cmd: sysctl -p /etc/sysctl.d/60-apparmor-restrict-unconfined.conf
    # With removes, check mode reports "would run" as changed instead of skipping.
    removes: /etc/sysctl.d/60-apparmor-restrict-unconfined.conf
  when: common_restrict_unconfined.stdout != "1"

check_mode: false lets the read run during a dry run too; it only reads. The removes line is the interesting one. As the handler showed earlier, command in check mode reports skipped whether it would run or not, so a dry run would never show this drift. With creates or removes set, the module can tell, and reports changed when it would run.

I tested exactly that case. On the server:

sudo sysctl -w kernel.apparmor_restrict_unprivileged_unconfined=0

For a minute, the aa-exec bypass from Lab 0a was open again. The dry run reported the apply task as changed (ok=7 changed=1) and left the kernel at 0; the real run loaded the file and set it back to 1; the next run was changed=0. Then the check that matters:

$ aa-exec -p trinity -- unshare --user --map-root-user id -u
unshare: write failed /proc/self/uid_map: Operation not permitted
$ lab0a-unshare --user --map-root-user sh -c 'id -u; grep CapEff /proc/self/status'
0
CapEff: 000001ffffffffff

The detour through a foreign profile is closed; the exception I granted on purpose still works.

Guard rails

One entry point. site.yml only imports: common.yml (the baseline for every host) and lab0a.yml. ansible-playbook site.yml brings everything to the described state; --limit narrows it to one host.
Lint from day one. ansible-lint 26.9.0, installed with pipx inject into the same virtualenv, so it lints against the same ansible-core that runs the playbooks. It passed at production, its strictest profile: 0 failures, 0 warnings. It also creates a cache directory, .ansible/, at the repo root, which now sits in .gitignore.

Every run, from the first ping to the last:

Run	Recap
`lab_tools`, dry run	ok=3 changed=1 skipped=1
`lab_tools`, real run	ok=4 changed=2
`lab_tools`, again	ok=3 changed=0
after `chmod 0775`: dry run, real run, again	changed=1 → changed=1 → changed=0
`site.yml`, first run	ok=6 changed=0 skipped=1
after `sysctl -w …=0`: dry run, real run, again	changed=1 → changed=1 → changed=0
`ansible-lint`	0 failures, 0 warnings

What surprised me

Python discovery skips the system Python whenever a versioned python3.X is found first; it has to be pinned.
A whitespace-only change is a change, and it fires handlers, even when the result is semantically identical.
copy shows no diff when only the mode changes.
In check mode, command stays silent (skipped) unless creates or removes gives it a way to tell.
Homebrew's pipx runs on a Python that ansible-core does not support yet; the interpreter for Ansible has to be chosen explicitly.

What's next

The next lab is the one Lab 0a promised: sandboxing Claude Code, the AI coding agent I use in this lab. Its sandbox needs an AppArmor profile and settings on the server, and now they will arrive as a role instead of another manual tweak. Later, these roles get automated tests, and a new node in the homelab becomes one more line in the inventory.

Everything from this post, in full:

Guide: every command, step by step
Results: versions, runs and surprises
Entry point: site.yml
Role common: the AppArmor sysctl, on disk and in the kernel
Role lab_tools: the unshare copy and its profile from Lab 0a
Configuration: ansible.cfg
Interpreter pin: group_vars/all.yml

Originally published at gshell.dev.

A container in 81 lines of Bash: what the kernel primitives cost on Ubuntu 24.04

Gancho Stoychev — Mon, 12 Oct 2026 09:35:46 +0000

Early in this lab I had a shell that was PID 1, and from that same shell ps listed systemd as PID 1. Both were right. The kernel had put the shell in a new PID namespace; ps was still reading a /proc mounted for the host. Isolation and what the tools report are two different things, and that gap is where most container debugging happens.

This is the first post in a series about building AI infrastructure on a two-node home lab. I ran Linux servers for years before moving to front-end development, and I am now working my way back to infrastructure, this time for AI workloads: GPU nodes, inference serving, observability and sandboxes for AI agents. Everything happens in the public ai-homelab repository, one lab at a time. Each lab has a step-by-step guide with every command and a results file with the raw numbers, mistakes included; each post tells the story of one lab.

This one is Lab 0a. Every inference server, sandbox and Kubernetes pod I will run later is built from the same few kernel primitives: namespaces, cgroups, a root filesystem and a virtual network cable. So before running any of them, I built a container by hand, one primitive at a time, and wrote down every number and every wrong prediction. The result is minicontainer.sh, 81 lines of Bash.

Guide: every command, step by step
Results: measurements, surprises and mistakes
Script: minicontainer.sh

What "done" meant

Six checks, decided before writing any code:

ps inside shows only the container's processes, and the shell is PID 1.
hostname inside differs from the host, and ls / shows Alpine, not the host.
A process exceeding memory.max is killed, and memory.events shows oom_kill 1.
ping from the container to the host end of a veth pair works.
lsns shows distinct namespace inodes for the container.
The script cleans up after itself: no leftover cgroup, network namespace or veth.

Setup

The Z790 node of my two-node home lab: i7-14700, 64 GB DDR5, two RTX 5060 Ti cards (unused here), Ubuntu 24.04 with the HWE kernel 7.0.0-34-generic.

The container's userland is Alpine minirootfs 3.24.2: a 3.7 MB tarball, 8.7 MB extracted. I read the current version from Alpine's release metadata, then downloaded it by exact path (v3.24/…, not latest-stable/…) so every run uses the same rootfs, and verified it with sha256sum -c. A checksum from the same server proves integrity, not authenticity; Alpine also publishes a GPG signature, which I left for a later lab.

Namespaces, one at a time

UTS: the name of the machine

The UTS namespace isolates only the hostname, which makes it the safest way to see the whole mechanism. Changing the hostname inside a new UTS namespace leaves the host's name untouched, and readlink /proc/self/ns/uts shows why:

inside:  uts:[4026533010]
outside: uts:[4026531838]

The number is an inode, and two processes share a namespace exactly when the numbers match. The host's value is no accident: 4026531838 is 0xEFFFFFFE, a constant. The kernel's initial namespaces have the fixed inode numbers 4026531832–4026531839; new namespaces get dynamic numbers, which are recycled after a namespace dies. Seeing one of those constants means "this is the host's namespace".

PID: why `--fork` is not optional

unshare() moves the calling process into most new namespaces immediately, but not into a new PID namespace. A process's PID is fixed at birth, so the only way in is to be born there: the PID namespace applies only to the caller's future children.

Without --fork, the first child becomes PID 1 of the new namespace. When it exits, the namespace has no init, and the kernel refuses to create any more processes in it:

sh PID: 6000
after 1st child
sh: 1: Cannot fork

I also ran sudo unshare --pid interactively by accident. It printed fork: Cannot allocate memory before the prompt even appeared: .bashrc forks at startup, that child became PID 1 and exited, and every command after that failed to fork. Only the builtin exit worked. The error says "memory", but the real problem is a dead namespace.

`ps` lies until `/proc` is remounted

With --fork, the shell really is PID 1 — and ps still printed 1 systemd, 2 kthreadd and the rest of the host. ps does not ask the kernel which processes it can see; it reads /proc, and a proc mount always shows the PID namespace it was mounted for. Adding --mount-proc (a new mount namespace plus a fresh /proc) left exactly two lines: 1 sh and 2 ps. First check passed.

Mount propagation: a tmpfs that leaked

systemd makes every host mount shared, so a mount in one member of a peer group propagates to the others. A new mount namespace started with --propagation shared stays in the host's peer group: I mounted a tmpfs inside, the namespace ended, and findmnt on the host still showed it. With --propagation private, the same mount never left the namespace. This is why unshare defaults to private, and it is also a precondition for the next step.

`chroot` versus `pivot_root`

chroot into the Alpine rootfs worked: NAME="Alpine Linux", but uname -r still printed the host's 7.0.0-34-generic. A container has its own filesystem and shares the kernel.

I expected /proc/mounts inside the chroot to list dozens of host mounts. It listed one. Same mount namespace, two views:

Root change	Mounts before	Mounts after	The host filesystem is…
`chroot` (outside / inside, same namespace)	26	1	hidden, still attached
`pivot_root` (before / after `umount -l` of the old root)	28	2	detached

/proc/self/mounts only lists mounts reachable from the process's root, so chroot hides the host's mounts without detaching them. pivot_root swaps the root mount and parks the old root at /.oldroot; one lazy umount -l then detaches it with all its submounts. After that, /.oldroot was empty and the process had no path back to the host.

Side quest: a 3.9 MB root filesystem

Any directory can become /. I copied /bin/bash into an empty tiny/ directory and tried to chroot into it:

chroot: failed to run command '/bin/bash': No such file or directory

The file existed. What was missing was the dynamic loader that the kernel starts first; the error names bash, the problem is the loader. After copying the loader and the two libraries ldd listed, bash ran, ls did not exist, and the whole rootfs was 3.9 MB. An image is literally the program plus everything it depends on, which is why statically linked binaries and distroless images exist.

cgroups v2 by hand

How systemd runs a real service

Before writing my own cgroup, I looked at a real one: Ollama, running as a systemd service. It lives in system.slice/ollama.service; idle with no model loaded, memory.current was about 552 MiB (page cache included) and pids.current was 18. There is no separate "cgroup config": the unit file is the configuration, and systemd writes it into the cgroup's files.

One limit nobody wrote: pids.max was 70833. systemd applies a default TasksMax to every service, so a single fork bomb cannot exhaust the machine.

The swap trap and an OOM kill

Creating a cgroup is a mkdir in /sys/fs/cgroup; the kernel fills the directory with 86 control files. I set memory.max to 100M, pids.max to 20 and cpu.max to 50000 100000 (half a core), and memory.swap.max to 0. That last one matters: this machine has an 8 GB swap file, and without it a process at memory.max gets swapped instead of killed.

Then a Python process inside the cgroup allocated 200 MB:

Killed
exit code: 137
Memory cgroup out of memory: Killed process 5462 (python3) total-vm:223152kB, anon-rss:101884kB, file-rss:6404kB

`memory.events`	Value
`max`	18
`oom`	1
`oom_kill`	1
`oom_group_kill`	0

Exit code 137 is 128 + 9, a SIGKILL with no chance to clean up. max 18 is how many times the cgroup hit its ceiling and the kernel tried to reclaim memory before giving up. total-vm is what the process asked for, anon-rss is what was resident when it died, right at the 100 MB limit. That dmesg line is what to look for when a container "just disappears".

`pids.max` versus 30 sleeps

I predicted that starting 30 background sleep processes under pids.max = 20 would produce 11 failed forks. I got one: pids.current was 19, a single sh: 0: Cannot fork, and pids.events showed max 1. A non-interactive dash gives up on the first failed fork. A real fork bomb would keep retrying, and the cgroup would keep stopping it at the ceiling.

`dmem` and NVIDIA: no VRAM limit

The kernel's list of cgroup controllers ends with dmem, the controller for device memory: dmem.min, dmem.low and dmem.max work like their memory counterparts, per memory region of a device. On this machine dmem.capacity exists and is empty.

Drivers have to opt in by registering their memory regions. On this kernel, Intel's xe and AMD's amdgpu do; none of the NVIDIA 580 open kernel modules reference dmem at all. So on this lab, GPU memory cannot be limited through cgroups.

Network: a cable between namespaces

A named network namespace (ip netns add) plus a veth pair, one end moved inside, is a virtual Ethernet cable between two network stacks. Inside, the new namespace had only lo (down) and its end of the cable, shown as @if8: the index of the peer in the other namespace. No LAN interface, no Docker bridges, not even a working loopback.

With addresses on both ends, the ping went through: 0.148, 0.077 and 0.079 ms. The first packet is slower because it includes ARP resolution; all of them stay well under a millisecond because they never leave the kernel. There is no default route, so the internet is unreachable — NAT is a topic for a later lab. Deleting the namespace destroyed its end of the cable, and since a veth pair cannot exist with one end, the kernel removed the host end too. One command cleans up the whole thing.

User namespaces and AppArmor on Ubuntu 24.04

Everything so far ran with sudo. A user namespace is the way to do it without: root inside, an ordinary user outside. As an unprivileged user it failed with write failed /proc/self/uid_map: Operation not permitted, and the kernel audit log explained why in two lines:

apparmor="AUDIT" operation="userns_create" … info="Userns create - transitioning profile" profile="unconfined" … target="unprivileged_userns"
apparmor="DENIED" operation="capable" profile="unprivileged_userns" … capability=21 capname="sys_admin"

Creating the namespace is allowed, but the process is immediately moved into the restrictive unprivileged_userns profile, which denies CAP_SYS_ADMIN inside. User namespaces give an unprivileged user "root" over large parts of the kernel, and that path has carried many privilege-escalation bugs. Ubuntu 24.04 allows them only for programs with an explicit AppArmor profile.

I did not turn the protection off globally. Instead, a root-owned copy of unshare gets its own profile, whose only addition is userns, so AppArmor no longer moves it into the restrictive profile. The profile is part of the repo as an Ansible role. The trade-off is written down rather than hidden: the copy does not receive util-linux security updates, so the role must be re-run after upgrades, or the copy removed.

With the copy, the rootless test worked, and showed what "root" in a user namespace means:

uid=0(root) gid=0(root) groups=0(root),65534(nogroup)
CapEff: 000001ffffffffff
touch: cannot touch '/etc/lab0a-test': Permission denied

All capabilities, and still no write access to /etc: the kernel checks host resources against the UID outside the namespace.

💡 Key takeaway: In March 2025, Qualys published three bypasses of this restriction. Canonical treats it as a defense-in-depth measure. The restriction is a tripwire that leaves audit records, not a wall; the real defense against kernel exploits is a patched kernel.

On my host, the first one worked: kernel.apparmor_restrict_unprivileged_unconfined was 0, and aa-exec -p trinity gave an unprivileged user a user namespace with all capabilities, through the profile of a program that is not even installed. One sysctl closes it: with kernel.apparmor_restrict_unprivileged_unconfined = 1, the same command fails with Operation not permitted, and the dedicated unshare copy still works.

The 81-line script

minicontainer.sh puts all of this together in four blocks, after some pre-checks:

cgroup: create it and write the limits, swap off.
network: namespace, veth pair, the container end renamed to eth0 as Docker does.
inside, as PID 1: hostname, bind mount, pivot_root, a fresh /proc, umount -l of the old root, then exec Alpine's shell with a clean environment.
launch: a subshell moves itself into the cgroup first, so everything it starts inherits it, then enters the network namespace and creates the rest with unshare --uts --pid --mount --ipc --cgroup --fork.

A trap on EXIT deletes the network namespace (and with it both veth ends) and removes the cgroup with a short retry loop, since a cgroup can only go once its last process has exited. After exit, the cgroup, the namespace, the veth pair and the mounts were all gone. All six checks passed.

This is the process tree while the container runs, with the host PIDs from one run:

subshell → exec ip netns exec → exec unshare      host PID 16242
  joins the lab0a network namespace and cgroup,
  unshare() moves it into new uts, mnt, ipc and cgroup namespaces
  └── fork → /bin/sh                                host PID 16244, PID 1 inside
        the first process in the new pid namespace

Seen from the host: mine versus Docker

lsns shows all eight namespace types of a process. Here is my container next to docker run alpine:3.24 on Docker 29.8.0:

Namespace	`minicontainer.sh`	Docker
`mnt`, `uts`, `ipc`, `cgroup`, `net`	new	new
`pid`	new	new
`time`	shared with the host	new
`user`	shared with the host	shared with the host

Same core model: new mnt, uts, ipc, pid, cgroup and net namespaces, and no user namespace. Root in my container is root on the host, and the same is true of a default Docker container.

Two differences. Docker also created a time namespace, which I had predicted it would not. It is part of the OCI spec that Docker 29.8.0 generates with its default configuration, with zero offsets: a new namespace that changes nothing visible. And Docker's process model is cleaner: every namespace starts at the container's own process, because runc creates it directly in all of them, while my script leaves the unshare parent (16242 above) inside five of the six new namespaces. That is exactly the PID-namespace rule from earlier, made visible.

Start-up: 124 ms versus 518 ms

Five runs of each, with true as the command, so only create and destroy are measured:

Implementation	5 runs (real)	Per run	User + Sys (5 runs)
`minicontainer.sh`	0.619 s	~124 ms	0.022 + 0.047 s
`docker run --rm alpine:3.24`	2.589 s	~518 ms	0.025 + 0.042 s

About four times faster, and still slower than my guess of "tens of milliseconds". CPU time is about 14 ms per run, so most of the rest is waiting: on namespace, veth and mount setup and teardown in the kernel. Profiling exactly where it goes is a job for a later lab.

💡 Key takeaway: Docker's user and sys are just as small, and that is misleading. time only counts the client and its children; the real work happens in dockerd, containerd and runc, which are not children of the shell. On a client-server system, time measures latency, not cost.

None of this makes Docker slow. It pays for layered images shared between containers, overlayfs, bridge networking with NAT, logging, restart policies and an API. My 124 ms is the cost of the kernel primitives alone.

What I got wrong

Mistakes were the most useful part of this lab, so here they are:

I expected dozens of mounts in /proc/mounts after chroot. It showed one.
I expected 11 failed forks against pids.max. dash stopped after the first.
I used ip -br addr inside the container. -br is iproute2; Alpine's ip is a BusyBox applet. Inside the container, the tools are Alpine's.
I expected start-up in tens of milliseconds. It was ~124 ms.
I expected Docker to share the time namespace. It created a new one.
I ran an interactive sudo unshare --pid by accident, and nothing could execute in it.

Gaps and what's next

⚠️ Not for untrusted code: The script is deliberately incomplete. It has no /dev (not even /dev/null), no /tmp or /sys, no capability dropping, no seccomp and no user namespace, and, like default Docker, root inside is root on the host. That is fine for learning what each primitive does; it is not something to run untrusted code in.

The next lab puts these primitives to work on the same server: sandboxing Claude Code, the AI coding agent I use in this lab. On Linux its sandbox is built on bubblewrap, which creates the same namespaces this post built by hand, and on Ubuntu 24.04 it meets the same AppArmor wall. Further down the road, the gaps above become a hardened-sandbox comparison: Claude Code's built-in sandbox, NVIDIA OpenShell, gVisor and a hardened version of this script.

Everything from this post, in full:

Guide: every command, step by step
Results: measurements, surprises and mistakes
Script: minicontainer.sh

Originally published at gshell.dev.

DEV Community: Gancho Stoychev

Hardening Ollama with systemd: limits, sandboxing and two GPUs

What "done" meant

Where Ollama started

What Ollama's memory really is

Limits from measurements

Drop-ins, not the unit

The file system and the GPU

Turning the screws, round by round

Does it still work?

A service of my own

Breaking it on purpose

What the score does and does not tell you

Small things that bite

What surprised me

What's next

Sandboxing Claude Code on Ubuntu

What "done" meant

What the sandbox is

A pinned Claude Code first

Letting bwrap through AppArmor

Settings a repository cannot loosen

The probe matrix

Looking inside

The gap

Is all this control worth it?

Small things that bite

What surprised me

What's next

From manual tweaks to Ansible roles: first steps

What "done" meant

Setup

Installing Ansible

Reaching the node

SSH does the connecting

An inventory that holds only names

ansible.cfg is read from the current directory

First contact

ping is not ICMP

Python discovery does not pick the system Python

Two spaces of drift

Breaking it on purpose

A setting that lives in two places

Guard rails

What surprised me

What's next

A container in 81 lines of Bash: what the kernel primitives cost on Ubuntu 24.04

What "done" meant

Setup

Namespaces, one at a time

UTS: the name of the machine

PID: why --fork is not optional

ps lies until /proc is remounted

Mount propagation: a tmpfs that leaked

chroot versus pivot_root

Side quest: a 3.9 MB root filesystem

cgroups v2 by hand

How systemd runs a real service

The swap trap and an OOM kill

pids.max versus 30 sleeps

dmem and NVIDIA: no VRAM limit

Network: a cable between namespaces

User namespaces and AppArmor on Ubuntu 24.04

The 81-line script

Seen from the host: mine versus Docker

Start-up: 124 ms versus 518 ms

What I got wrong

Gaps and what's next

`ansible.cfg` is read from the current directory

`ping` is not ICMP

PID: why `--fork` is not optional

`ps` lies until `/proc` is remounted

`chroot` versus `pivot_root`

`pids.max` versus 30 sleeps

`dmem` and NVIDIA: no VRAM limit