DEV Community: Piotrek Karasinski

cgroups v2 as a Native Runtime Isolation Primitive — No Docker Required

Piotrek Karasinski — Wed, 01 Jul 2026 06:00:00 +0000

The Overhead Tax of Container Abstraction

Every time a team reaches for Docker to isolate a background service—a scheduled job, a data pipeline, a monitoring daemon—it pays an overhead tax. The container runtime (containerd, runc) must initialize a namespaced environment, mount overlay filesystems, and manage network virtualization, even when the only requirement is CPU throttling and memory capping for a single process.

cgroups v2 (Control Groups version 2), unified in Linux 4.5 and enabled by default on most modern distributions including Arch Linux since kernel 5.8+, provides that isolation primitive natively—with zero daemon overhead, no image layers, no OCI spec compliance ceremony.

This article dissects the operational architecture of the cgroups v2 unified hierarchy, its integration with systemd's transient unit model, and programmatic control via Python—delivering the isolation contract without the abstraction tax.

cgroups v2: Architectural Shift from v1

The fundamental flaw of cgroups v1 was its fragmented, per-controller hierarchy. CPU limits lived in /sys/fs/cgroup/cpu/, memory limits in /sys/fs/cgroup/memory/—independent trees that could produce conflicting resource assignments and made atomic process migration across controllers operationally fragile.

cgroups v2 enforces a single, unified hierarchy rooted at /sys/fs/cgroup/. All controllers—cpu, memory, io, pids—operate within one coherent tree. The key architectural invariant: a process can belong to exactly one cgroup.

Unified Hierarchy Verification

# Verify cgroups v2 is the active mode
$ mount | grep cgroup
cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot)

# Inspect available controllers on root cgroup
$ cat /sys/fs/cgroup/cgroup.controllers
cpuset cpu io memory hugetlb pids rdma misc

If the output shows cgroup2 as the filesystem type and a unified set of controllers, the system is fully migrated. On hybrid systems, the cgroup_no_v1=all kernel parameter forces exclusive v2 mode.

Direct Cgroup Lifecycle Management

Cgroup management at its core is filesystem manipulation. Creating a cgroup means creating a directory; assigning a process means writing its PID to cgroup.procs; constraining resources means writing values to controller-specific interface files.

Creating and Configuring a Cgroup

# Create a cgroup for an isolated workload
$ mkdir /sys/fs/cgroup/devmindset-worker

# Enable CPU and memory controllers on this cgroup
$ echo "+cpu +memory" > /sys/fs/cgroup/devmindset-worker/cgroup.subtree_control

# Cap CPU utilization: 20% of one core (quota/period in microseconds)
$ echo "20000 100000" > /sys/fs/cgroup/devmindset-worker/cpu.max

# Hard memory limit: 256 MiB
$ echo $((256 * 1024 * 1024)) > /sys/fs/cgroup/devmindset-worker/memory.max

# Assign current shell process to the cgroup
$ echo $$ > /sys/fs/cgroup/devmindset-worker/cgroup.procs

The cpu.max interface accepts $QUOTA $PERIOD pairs, directly mapping to CFS (Completely Fair Scheduler) bandwidth throttling. Any process inside this cgroup consuming more than 20ms per 100ms window will be throttled at the scheduler level—no userspace daemon required.

Systemd Integration: Transient Units as Isolation Primitive

For production workloads on systemd-based systems, systemd-run is the idiomatic abstraction over cgroups v2. It spawns a process inside a transient systemd scope or service unit, inheriting the systemd cgroup delegation model.

# Launch an isolated process with resource constraints via transient unit
$ systemd-run \
    --scope \
    --unit=devmindset-worker \
    --property=CPUQuota=20% \
    --property=MemoryMax=256M \
    --property=IOWeight=10 \
    /usr/bin/python3 /opt/workers/pipeline_runner.py

# Inspect live resource accounting
$ systemctl status devmindset-worker.scope
$ cat /sys/fs/cgroup/system.slice/devmindset-worker.scope/cpu.stat

The --scope flag creates a transient unit scoped to the calling session, while --service creates a persistent transient service with full systemd lifecycle management. The IOWeight property maps to the CFQ/BFQ I/O scheduler weight, providing proportional I/O bandwidth allocation.

Programmatic Control via Python

For dynamic workload management—spinning up isolated workers at runtime, adjusting quotas based on telemetry, implementing backpressure mechanisms—Python's pathlib provides clean, idiomatic access to the cgroupfs interface.

from __future__ import annotations

import os
import subprocess
from pathlib import Path
from typing import Final

# Micro-Rationale: Using pathlib.Path for type-safe filesystem operations
# over raw string concatenation. O(1) path construction, zero external deps.
CGROUP_ROOT: Final[Path] = Path("/sys/fs/cgroup")

class CgroupV2Controller:
    """Manages a single cgroups v2 hierarchy for process isolation.

    Enforces CPU quota and memory hard limits on a named cgroup.
    Implements context manager protocol for deterministic cleanup.
    """

    def __init__(
        self,
        name: str,
        cpu_quota_percent: int = 25,
        memory_max_mib: int = 256,
    ) -> None:
        if not 1 <= cpu_quota_percent <= 100:
            raise ValueError(f"cpu_quota_percent must be in [1, 100], got {cpu_quota_percent}")
        if memory_max_mib < 16:
            raise ValueError(f"memory_max_mib must be >= 16 MiB, got {memory_max_mib}")

        self.name = name
        self._cpu_quota_percent = cpu_quota_percent
        self._memory_max_bytes = memory_max_mib * 1024 * 1024
        self._cgroup_path = CGROUP_ROOT / name

    def _write(self, interface: str, value: str) -> None:
        """Writes a value to a cgroup interface file."""
        target = self._cgroup_path / interface
        try:
            target.write_text(value, encoding="utf-8")
        except PermissionError as exc:
            raise PermissionError(
                f"Insufficient privileges to write to {target}. "
                "Run as root or with CAP_SYS_ADMIN."
            ) from exc
        except OSError as exc:
            raise OSError(f"Failed to write '{value}' to {target}: {exc}") from exc

    def create(self) -> "CgroupV2Controller":
        """Creates the cgroup directory and configures resource limits."""
        try:
            self._cgroup_path.mkdir(parents=False, exist_ok=False)
        except FileExistsError:
            raise FileExistsError(f"Cgroup '{self.name}' already exists at {self._cgroup_path}")

        self._write("cgroup.subtree_control", "+cpu +memory")

        # CPU quota: $QUOTA $PERIOD in microseconds (CFS bandwidth throttling)
        period_us = 100_000
        quota_us = int(period_us * self._cpu_quota_percent / 100)
        self._write("cpu.max", f"{quota_us} {period_us}")

        # Memory hard limit — OOM killer fires at this threshold
        self._write("memory.max", str(self._memory_max_bytes))
        return self

    def assign_pid(self, pid: int) -> None:
        """Moves a process into this cgroup by writing to cgroup.procs."""
        try:
            os.kill(pid, 0)
        except ProcessLookupError:
            raise ProcessLookupError(f"No process with PID {pid} found.")
        self._write("cgroup.procs", str(pid))

    def assign_current_process(self) -> None:
        """Moves the calling process into this cgroup."""
        self.assign_pid(os.getpid())

    def destroy(self) -> None:
        """Removes the cgroup directory. All processes must be migrated first."""
        procs = (self._cgroup_path / "cgroup.procs").read_text().strip()
        if procs:
            raise OSError(
                f"Cannot destroy cgroup '{self.name}': "
                f"PIDs still assigned: {procs.splitlines()}"
            )
        self._cgroup_path.rmdir()

    def __enter__(self) -> "CgroupV2Controller":
        return self.create()

    def __exit__(self, *_: object) -> None:
        try:
            self.destroy()
        except OSError:
            pass  # Best-effort cleanup; log in production

# --- Usage ---
if __name__ == "__main__":
    worker_proc = subprocess.Popen(["/usr/bin/python3", "/opt/workers/cpu_intensive_task.py"])

    with CgroupV2Controller(
        name="devmindset-worker",
        cpu_quota_percent=20,
        memory_max_mib=256,
    ) as cgroup:
        cgroup.assign_pid(worker_proc.pid)
        worker_proc.wait()

Overhead Comparison: Docker vs. Direct cgroups v2

Dimension	Docker (`runc`)	Direct cgroups v2
Runtime daemon	`containerd` + `dockerd`	None
Process start latency	~80–150ms (OCI init)	< 1ms (mkdir + write)
Filesystem isolation	OverlayFS mount required	Not applicable
Network namespace	Virtual NIC + iptables rules	Not applicable
Memory overhead	~15–30 MiB per container	Zero
Kernel interface	Indirect (runc → seccomp → cgroup)	Direct
Appropriate use case	Full application isolation, portability	Single-process resource capping

Conclusion: Choosing the Right Isolation Primitive

cgroups v2 is not a Docker alternative—it is the mechanism Docker is built on. Reaching for it directly means operating at the right abstraction level for the problem at hand. On a hardened Arch Linux system running a set of long-lived workers, background scrapers, or ML inference daemons, a 150-line Python controller and three systemd-run flags deliver full runtime isolation without the operational surface of a container orchestration stack.

The engineers who understand the kernel primitives their tools are built on will always debug faster, profile more accurately, and architect more efficiently than those who interact exclusively through abstraction layers.

Originally published on devmindset.dev — Linux internals, systems programming, and the self-taught developer mindset.

Related deep-dives:

strace as a Debugging Primitive — When the Stack Trace Lies

Piotrek Karasinski — Tue, 30 Jun 2026 18:29:04 +0000

Your logs tell you what your code thinks it did. Your stack trace tells you where the code believed it was. strace tells you what the kernel actually did — every openat(), every read(), every connect() that returned -1 while your application swallowed the error and kept lying to you. When the gap between intent and reality is the bug, re-reading the source will not close it. You have to observe the boundary where userspace ends and the kernel begins.

Most engineers know strace -p <pid> as a panic button. Few actually read the output, and fewer reach for -f, -c, or -k when it counts. That gap is the difference between staring at a wall of syscalls and extracting a root cause in ninety seconds. Treating strace as a first-class instrument is a direct extension of hypothesis-driven debugging methodology: you form a precise claim about what the process is doing and let the evidence confirm or kill it.

What strace actually observes

strace is built on ptrace(2), the same kernel facility debuggers use. It attaches to a target and stops it on every transition across the syscall boundary — once on entry, once on exit — reading the registers to decode the syscall number, its arguments, and its return value. That is the whole model, and it explains the cost: every syscall now pays for two extra context switches into the tracer. On a syscall-heavy workload the target can slow by an order of magnitude. This is not a footnote — it is the reason strace belongs in diagnosis, never in a hot path or a latency-sensitive production service.

Reading a trace line — the part nobody teaches

Every line has the same shape: syscall(arguments) = return_value. The return value is where the truth lives. A successful call returns a file descriptor, a byte count, or 0; a failure returns -1 followed by the symbolic errno and its description. That errno is the signal you came for.

openat(AT_FDCWD, "/etc/app/config.yaml", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/app/config.yaml", O_RDONLY) = 3
read(3, "host: 127.0.0.1\nport: 6379\n", 4096) = 27
connect(4, {sa_family=AF_INET, sin_port=htons(6379), sin_addr=inet_addr("127.0.0.1")}, 16) = -1 ECONNREFUSED (Connection refused)

Four lines, two findings, zero guesswork: the app probed a config path that does not exist before falling back to the real one, and the Redis connection was refused outright. No log statement told you either fact — the syscalls did. The failure worth memorising is EFAULT: it means a syscall received a pointer to memory the process does not own. That is the same class of bad-address access which, followed to its conclusion by the MMU and the kernel, produces a segmentation fault.

The flags that separate signal from noise

-f — follow forks and threads. The moment a process calls fork() or spawns a worker pool, an unadorned trace goes blind to the children. Almost every real service needs this flag.
-e trace=… — filter by class instead of drowning. -e trace=network, -e trace=%file, -e trace=memory cut the stream to the syscalls you care about.
-c — aggregate instead of stream. Produces a summary table of time, call count, and errors per syscall. This is your profiler the moment you suspect a process is syscall-bound.
-T / -tt — time spent inside each syscall / absolute timestamps. Together they turn "it's slow" into "it spends 600 ms in fsync".
-y / -yy — decode descriptors, so read(3, …) becomes read(3</usr/share/app/config.yaml>, …) and sockets show their endpoints.
-k — attach a userspace stack trace to each syscall, bridging "which syscall" to "which line of code".
-s N — raise the printed string length (the default truncates payloads).

The summary mode also answers "why is this slow?" without a dedicated profiler. Run any command under strace -f -c and the table ranks syscalls by cumulative time — the same technique that turns a sluggish terminal into a fixable list of offending stat() and openat() calls when you profile shell startup and .bashrc performance.

From raw trace to an actionable summary

Reading -c output by eye is fine once. When you want it in CI, in a regression gate, or aggregated across runs, parse it. The wrapper below runs a command under strace -f -c, captures the summary from stderr, parses it into typed records, and returns the syscalls that dominate time. It handles the obvious failure modes — strace missing, the target hanging, a malformed table — instead of assuming the happy path.

from __future__ import annotations

import shutil
import subprocess
from dataclasses import dataclass
from typing import Final

_HEADER_TOKENS: Final[tuple[str, ...]] = ("seconds", "calls", "syscall")

@dataclass(frozen=True, slots=True)
class SyscallStat:
    name: str
    seconds: float
    calls: int
    errors: int

    @property
    def usec_per_call(self) -> float:
        return (self.seconds / self.calls) * 1_000_000 if self.calls else 0.0

class StraceError(RuntimeError):
    """Raised when strace cannot be run or its summary cannot be parsed."""

def profile_syscalls(cmd: list[str], *, timeout: float = 30.0) -> list[SyscallStat]:
    """Run *cmd* under ``strace -f -c`` and return per-syscall stats, busiest first.

    Raises:
        StraceError: strace is missing, the command times out, or the summary
            table cannot be located in strace's stderr.
    """
    if not cmd:
        raise StraceError("empty command")
    if shutil.which("strace") is None:
        raise StraceError("strace is not installed or not on PATH")

    try:
        proc = subprocess.run(
            ["strace", "-f", "-c", "--", *cmd],
            capture_output=True,
            text=True,
            timeout=timeout,
            check=False,  # the traced command may legitimately exit non-zero
        )
    except subprocess.TimeoutExpired as exc:
        raise StraceError(f"traced command exceeded {timeout:.0f}s") from exc

    return _parse_summary(proc.stderr)

def _parse_summary(stderr: str) -> list[SyscallStat]:
    lines = stderr.splitlines()
    header = next(
        (i for i, ln in enumerate(lines) if all(t in ln for t in _HEADER_TOKENS)),
        None,
    )
    if header is None:
        raise StraceError("no summary table found; was -c passed to strace?")

    stats: list[SyscallStat] = []
    for line in lines[header + 1:]:
        if not line.strip() or line.lstrip().startswith("-"):
            continue
        if "total" in line:
            break  # the total row terminates the table
        cols = line.split()
        if len(cols) = 6 and cols[4].isdigit() else 0
        except (ValueError, IndexError):
            continue  # skip anything that is not a data row
        stats.append(SyscallStat(name=cols[-1], seconds=seconds, calls=calls, errors=errors))

    if not stats:
        raise StraceError("summary table present but no rows could be parsed")
    return sorted(stats, key=lambda s: s.seconds, reverse=True)

if __name__ == "__main__":
    import sys

    if len(sys.argv) < 2:
        raise SystemExit("usage: profile.py COMMAND [ARGS...]")

    for stat in profile_syscalls(sys.argv[1:]):
        marker = "  (has errors)" if stat.errors else ""
        name = stat.name.ljust(18)
        calls = str(stat.calls).rjust(8)
        print(f"{name}{stat.seconds:9.4f}s {calls} calls  {stat.usec_per_call:8.1f} us/call{marker}")

It is deliberately defensive: a missing binary, a timeout, and an unparseable table each raise a typed StraceError rather than returning silently wrong data. The check=False is intentional — the program you are tracing is allowed to exit non-zero; that is frequently the entire reason you are tracing it.

strace vs ltrace vs gdb vs perf — pick the right lens

strace is one lens, not the only one. Each tool observes a different layer at a different cost, and reaching for the wrong one wastes time or perturbs the very behaviour you are chasing.

Tool	Observes	Overhead	Best question	Production-safe?
strace	Syscalls (kernel boundary)	High (2 ctx switches/call)	What is the process asking the kernel to do?	No — diagnosis only
ltrace	Library / function calls (userspace)	High	Which library calls fire, with what arguments?	No
gdb	Full program state, memory, breakpoints	Very high (stops the world)	What is the exact state at this line?	No (interactive)
perf trace	Syscalls via perf_events	Low	How do syscalls behave under real load?	Yes
bpftrace / eBPF	Arbitrary kernel + user probes	Very low	Custom, low-overhead production tracing	Yes

When not to reach for strace

The overhead is not academic, and it is not the only constraint. On hardened systems, /proc/sys/kernel/yama/ptrace_scope can forbid attaching to a process you do not own — you will get EPERM no matter how root you feel. Inside containers, a restrictive seccomp profile may block ptrace entirely. And on a service handling live traffic, doubling the cost of every syscall is not a debugging session, it is an incident.

For anything that must run against production load, reach for perf trace or an eBPF tool such as bpftrace, which observe the same syscall boundary at a fraction of the cost. But on a box you control, with a process you can afford to slow down, strace remains the fastest way to answer the only question that matters when intent and reality diverge: what is this process actually doing? Once you can read its output fluently, that question stops being a mystery and becomes a lookup.

Originally published on devmindset.dev — Linux internals, systems programming, and the self-taught developer mindset.

Related deep-dives: