DEV Community

Kernel notes (sunshout)
Kernel notes (sunshout)

Posted on Originally published at sunshout.tistory.com

isolcpus= takes CPUs off the scheduler. Hardware IRQs still land there.

#linux #kernel #networking #sysadmin

The blunt tool is still in a lot of GRUB files:

GRUB_CMDLINE_LINUX_DEFAULT="isolcpus=0,1"
Enter fullscreen mode Exit fullscreen mode

Then update-grub (or grub2-mkconfig) and reboot. Userspace tasks stop landing on CPU0/1. That is all most people verify — they fire a few busy loops and top looks empty on those cores.

IRQs do not care. isolcpus is a scheduler isolation hint. Hardware interrupts can still fire on the "isolated" CPUs. I watched seven tight loops leave 0/1 idle for processes while /proc/interrupts still ticked on those cores.

If you wanted a CPU for DPDK, a user-space NIC, or a cycle-accurate loop, scheduler isolation is necessary and not sufficient.

Lab notes (English original is short; this write-up is the missing IRQ half):

https://sunshout.tistory.com/1620

How to see what you actually isolated

After reboot:

cat /proc/cmdline          # isolcpus=0,1 must be there
grep PREEMPT /boot/config-$(uname -r) || true
taskset -cp 1              # pick a known userspace pid; it should not be 0,1
watch -n1 'grep "^ *[0-9]" /proc/interrupts | head'
Enter fullscreen mode Exit fullscreen mode

If IRQs still increment on CPU0/1, isolation is incomplete. That is expected with classic isolcpus=.

On newer kernels the story split:

  • isolcpus=domain / cpusets / cgroup cpuset — userspace
  • isolcpus=managed_irq or manual irqaffinity / /proc/irq/*/smp_affinity — interrupts
  • nohz_full= — tick reduction, another knob, not a substitute

isolcpus is also marked deprecated in some trees in favor of cpusets. The IRQ caveat did not go away when the docs changed the preferred interface.

Moving IRQs by hand

Find the noisy ones (eth0, NVMe, GPU):

grep -E 'eth|nvme|enp' /proc/interrupts
# smp_affinity is a hex CPU mask. CPU2 only → 4
echo 4 > /proc/irq/IRQNUM/smp_affinity
Enter fullscreen mode Exit fullscreen mode

Or set the default affinity so new IRQs skip 0/1:

irqaffinity=2-7
Enter fullscreen mode Exit fullscreen mode

in the same GRUB line (adjust to your CPU count). Some devices ignore this (managed IRQs, VFIO). Then you isolate at the driver: bind the NIC to vfio-pci and poll from a pinned thread.

When this shows up next to SR-IOV

Passing a VF into KVM does not pin host IRQs for you. IOMMU on (intel_iommu=on iommu=pt) is a different checklist. I wrote the VF creation half here:

https://sunshout.tistory.com/1615

Typical datapath box:

  1. IOMMU on, VFs created, vfio-pci bound.
  2. isolcpus and IRQ affinity so the poll thread is not preempted by the host NIC's own interrupt.
  3. Confirm with /proc/interrupts, not with top.

If your kernel is 6.x, say so in the comments — the flags moved, the IRQ rule did not.

Top comments (0)