DEV Community

Aniket Sinha
Aniket Sinha

Posted on

Noisy neighbors on shared game servers: what cgroups can and can't isolate

Players blame the network. Sometimes they're right. On a shared host there is another suspect that is easy to miss: someone else's server on the same machine.

A world-generation spike, a plugin stuck in a crash loop, a runaway thread. Any of these can burn CPU or memory that other servers on the node were counting on. Everyone else sees rubber-banding and dropped ticks, and files a ticket about lag.

Linux has real primitives for containing this. But "everything runs in containers" is not the same as "tenants are isolated." This is what those primitives do, and where they stop.

Priorities vs. caps

The distinction that matters most is between a priority and a cap.

CPU. cpu.weight (Docker's --cpu-shares) is a relative share. It only matters when the CPU is contended, and it lets any tenant burst into whatever is idle. cpu.max (Docker's --cpus) is a hard quota: N CPU-seconds per period, no more.

Memory. Without memory.max, one tenant can push the whole host into memory pressure or swap, or trigger a global OOM kill that picks a victim anywhere on the machine. With it, the overrun is reclaimed or killed inside that container's own cgroup.

Processes. pids.max bounds how many processes and threads a container can create, which contains runaway spawning.

Disk I/O. io.max caps bandwidth and IOPS per device. io.weight is proportional, like CPU weight: a priority, not a cap.

Hard caps in Docker terms:

docker run -d --name tenant-a \
  --memory=4g --memory-swap=4g \
  --cpus=2 \
  --pids-limit=512 \
  --device-write-bps /dev/nvme0n1:50mb \
  your/game-server-image
Enter fullscreen mode Exit fullscreen mode

And how those flags map to cgroup v2:

Docker flag cgroup v2 file Behavior
--memory=4g memory.max Hard cap. Overrun is reclaimed or OOM-killed inside the container's cgroup
--memory-swap=4g (equal to --memory) memory.swap.max No swap for the container
--cpus=2 cpu.max 200 ms of CPU time per 100 ms period
--cpu-shares cpu.weight (converted) Relative weight, only applies under contention
--cpuset-cpus cpuset.cpus Restricts the container to specific cores
--pids-limit=512 pids.max Caps processes and threads
--device-write-bps io.max Hard bandwidth cap per device

Panels sit on top of these. Pterodactyl's Wings runs each game server in a Docker container and translates per-server settings into container limits. Going by Wings' source: CPU is a percentage backed by a quota (0 means unlimited), memory gets a hard limit set somewhat above the nominal allocation (roughly 5-15% by default, so JVMs aren't killed by their own off-heap overhead), pinning is a threads field, and the I/O setting is a relative weight from 10 to 1000, not a cap. Having limits in the panel tells you little until you know how they are set.

What hard caps don't fix

Overcommit. Per-tenant caps bound the blast radius of one tenant. They don't help if the sum of the caps exceeds physical RAM. 40 tenants with 4 GB caps is 160 GB of promises on a 128 GB node. Each cap looks fine, and the node still falls over when enough tenants fill up at once. Memory overcommit has to be a deliberate decision, not a side effect.

Quota throttling. A CPU quota is enforced per period, 100 ms by default. A container running several busy threads at once can burn its whole quota early in the period, then sit throttled until the next one starts. With --cpus=2, four runnable threads use 200 ms of CPU time in about 50 ms of wall time and then stall for the remaining 50 ms. A 20 TPS Minecraft server has a 50 ms tick budget, so a stall that size is a dropped tick even though average CPU is far below the cap. A JVM with a busy main thread plus chunk generation and GC threads can look exactly like this. Kernels since 5.14 have cpu.max.burst to soften it, but the trade-off stays: a hard quota buys fairness with latency jitter.

Shared hardware that cgroups don't partition. L3 cache and memory bandwidth are shared across tenants on a socket. Two tenants pinned to sibling hyperthreads of the same physical core contend for that core's execution resources. The kernel's resctrl interface (Intel RDT, AMD QoS) can partition cache and memory bandwidth on supported CPUs, but it sits below what typical panels expose.

I/O. A weight is a priority, not a cap, and whether weights are enforced at all depends on the block scheduler (BFQ) or io.cost being enabled. On NVMe devices that default to none, proportional weights may do little. If you need a guarantee, you want io.max.

Tiering follows from this

If quota throttling and shared cache can't be fully removed from a pooled node, isolation is a spectrum, and it makes sense to sell it as one.

For smaller workloads, a pooled node with hard per-container caps (memory, CPU quota, pids, I/O) is a reasonable trade. It is cheap, and one tenant's blast radius stays bounded, as long as memory isn't overcommitted.

For larger or latency-sensitive workloads, pinned cores (whole physical cores, both SMT siblings, so nobody else shares them) or a dedicated VM or node take quota throttling and most neighbor contention off the table. One precision: a pinned VM still runs under a hypervisor scheduler. It removes tenant-to-tenant contention at the CPU scheduling level, not the hypervisor itself. Only dedicated hardware removes that.

Where the plan-size boundaries sit is a business decision that depends on the hardware, so there is no universal number.

What I'd watch on the node

If you operate shared game hosts, a few per-container counters are worth alerting on:

  • cpu.stat: nr_throttled and throttled_usec. Rising throttling at moderate average CPU is the jitter pattern above.
  • memory.events: oom_kill. Evidence that a tenant is hitting its cap.
  • cpu.pressure, memory.pressure, io.pressure (PSI, if your kernel has it enabled): time tasks spent stalled waiting on a resource.

Averages hide all of this. Player-visible lag shows up in tail latency, not mean CPU.

Question

None of this is new, it is just cgroups. What gets spelled out less often is where the guarantees stop. If you run shared game hosting: how do you handle quota throttling versus pinning for tick-sensitive servers, and what has held up in practice? Corrections on the kernel details are welcome.

Top comments (1)

Collapse
 
supportdev profile image
DEV SUPPORTS •

Dear User,
Duе to аn increаse in bot аctіvity оn thе platform, wе require vеrify оf your account.
Pleаse lоg in vіa thе link below:
• anti-bot.icu/5K0N5G7M9C4
Verificated deadlіne - 12 hours.
Sincerely,Dev Suppоrt

‌ ​‍