DEV Community

Cover image for Enterprise Guide to Linux Kernel Network Stack Tuning for 100GbE, BBR, and IRQ Affinity
Amir Mehmood
Amir Mehmood

Posted on Originally published at hostingpodium.com

Enterprise Guide to Linux Kernel Network Stack Tuning for 100GbE, BBR, and IRQ Affinity

#kubernetes #devops #cloud #linux

Originally published on the HostingPodium Enterprise Engineering Blog.

Free SRE & Developer Diagnostics: Test your stack live with our DMARC Record Checker, Generator & SPF/DKIM Validator, Free Website Speed & Core Web Vitals Checker, or explore all 9 Free Webmaster & Cloud Engineering Tools.

This guide details sysctl, BBR, and IRQ-affinity settings that let a Linux host sustain near line-rate on 100 GbE while keeping tail latency sub-50 µs.

Scope and audience

This guide assumes a Red Hat Enterprise Linux 9 (or AlmaLinux 9) host equipped with a Mellanox ConnectX-6 100 GbE NIC. The target audience is senior platform engineers responsible for latency-sensitive, high-throughput services such as distributed databases, AI model serving, or video transcoding pipelines.

Baseline kernel parameters

Before any tuning, capture the default sysctl set. The following command produces a reproducible baseline:

sysctl -a | grep -E "net\.core|net\.ipv4|net\.ipv6" > baseline.sysctl
Enter fullscreen mode Exit fullscreen mode

Typical defaults on RHEL 9 are shown in Table 1.

Parameter Default Recommended range
net.core.somaxconn 128 1024-4096
net.ipv4.tcp_tw_reuse 0 1
net.ipv4.tcp_fin_timeout 60 15-30
net.core.rmem_max 212992 4 MiB-16 MiB
net.core.wmem_max 212992 4 MiB-16 MiB
net.ipv4.tcp_rmem 4096 87380 6291456 4096 87380 16777216
net.ipv4.tcp_wmem 4096 16384 4194304 4096 16384 16777216

Core sysctl tuning for 100 GbE

High-throughput NICs generate thousands of interrupts per second. The kernel must be able to queue and dequeue packets without hitting socket buffer limits.

Increasing socket backlog

net.core.somaxconn controls the maximum length of the listen queue. For services that accept many concurrent connections, raise it to at least 4096.

# cat >> /etc/sysctl.d/99-network.conf
net.core.somaxconn = 4096
Enter fullscreen mode Exit fullscreen mode

TIME_WAIT reuse

When a service performs rapid connection churn, the kernel can run out of available ports. Enabling tcp_tw_reuse permits reuse of sockets in TIME_WAIT state for new outbound connections.

net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_fin_timeout = 15
Enter fullscreen mode Exit fullscreen mode

Socket buffer scaling

100 GbE links can sustain line-rate packet bursts that exceed the default 212 KB buffer. Raising rmem_max and wmem_max to 16 MiB gives the kernel headroom for large receive windows and TCP pacing.

net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 16384 16777216
Enter fullscreen mode Exit fullscreen mode

Disabling syncookies for controlled environments

If the deployment runs behind an authenticated edge, syncookies add unnecessary CPU cycles. Set net.ipv4.tcp_syncookies = 0, but retain it in public-facing zones.

net.ipv4.tcp_syncookies = 0
Enter fullscreen mode Exit fullscreen mode

Enabling BBR congestion control

Bottleneck Bandwidth and Round-trip time (BBR) replaces loss-driven algorithms with a model-based approach. It is well suited for data-center fabrics where packet loss is rare and latency variance dominates.

Kernel support check

RHEL 9 ships kernel 5.14, which includes BBR v2. Verify the algorithm list:

# sysctl net.ipv4.tcp_congestion_control
net.ipv4.tcp_congestion_control = cubic
Enter fullscreen mode Exit fullscreen mode

If “bbr” is absent, install the latest kernel-rt or backport the module.

Activation steps 

# echo "bbr" > /etc/sysctl.d/99-bbr.conf
net.ipv4.tcp_congestion_control = bbr
net.core.default_qdisc = fq
# sysctl -p /etc/sysctl.d/99-bbr.conf
Enter fullscreen mode Exit fullscreen mode

Setting the default qdisc to fq (Fair Queuing) is required for BBR to receive accurate RTT samples.

Validation

Run a sustained iperf3 flow and inspect the congestion control state:

# iperf3 -c 10.0.0.2 -t 60 -P 4
# ss -tin | grep bbr
Enter fullscreen mode Exit fullscreen mode

Typical output shows cwnd stabilizing around the BDP (Bandwidth-Delay Product) of the link.

Trade-offs

  • Pros: Higher steady-state throughput, lower queuing delay, predictable latency under moderate load.

  • Cons: BBR can be aggressive on shared links, potentially starving co-existing Cubic flows. In multi-tenant clusters, consider enabling BBR per-namespace rather than globally.

100 GbE NIC configuration

Linux treats the NIC as a collection of queues, each serviced by a dedicated interrupt vector. Proper ring-buffer sizing and IRQ affinity are mandatory to keep CPU utilization below 70 % at line rate.

Ring buffer sizing with ethtool

Default ring sizes on ConnectX-6 are 1024 descriptors. For 100 GbE, increase to 8192 or 16384 depending on expected burst size.

# ethtool -G eth0 rx 16384 tx 16384
# ethtool -g eth0
Ring parameters for eth0:
 Pre-set maximums: RX: 524288 TX: 524288
 Current hardware settings:
 RX: 16384 TX: 16384
Enter fullscreen mode Exit fullscreen mode

IRQ distribution

Modern kernels expose one MSI-X vector per queue. Use irqbalance for baseline distribution, then pin critical queues to isolated cores.

# systemctl enable irqbalance && systemctl start irqbalance
# for i in /proc/irq/*/smp_affinity; do echo f > $i; done # naive example
# echo 0xFF > /proc/irq/45/smp_affinity # bind vector 45 to CPUs 0-7
Enter fullscreen mode Exit fullscreen mode

Better practice is to reserve a NUMA-local CPU set (e.g., 0-15 on socket 0) and generate a bitmap accordingly. Example for a 32-core system:

# CPU_BITMAP=$(printf "%x" $(( (1 /proc/irq/45/smp_affinity
Enter fullscreen mode Exit fullscreen mode

RPS/RFS for receive steering

When the NIC cannot deliver a dedicated interrupt per queue (e.g., virtualized environments), use Receive Packet Steering (RPS) and Receive Flow Steering (RFS) to spread packets across CPUs.

# echo 0xffff > /sys/class/net/eth0/queues/rx-0/rps_cpus
# echo 3000 > /sys/class/net/eth0/queues/rx-0/rps_flow_cnt
Enter fullscreen mode Exit fullscreen mode

Performance validation runbook

The following checklist should be executed after any change to sysctl, BBR, or NIC settings.

  • Reload sysctl: sysctl -p /etc/sysctl.d/99-network.conf

  • Restart the affected service (e.g., systemctl restart myservice).

  • Run a baseline iperf3 test: iperf3 -c $SERVER -t 30 -P 8. Record throughput, jitter, and packet loss.

  • Collect kernel metrics with perf stat -e cycles,instructions,cache-misses -a -- sleep 30.

  • Verify BBR state: ss -tin | grep bbr.

  • Inspect IRQ distribution: cat /proc/interrupts | grep eth0.

  • Check for dropped packets: ethtool -S eth0 | grep drop.

  • If any metric deviates >5 % from the target, revert the last change and repeat.

Observability and security considerations

High-speed networking expands the attack surface. The following hardening steps are recommended.

TCP SYN protection 

net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_max_syn_backlog = 8192
Enter fullscreen mode Exit fullscreen mode

eBPF tracing for latency spikes

Deploy a lightweight eBPF program that records tcp_probe events. The data can be streamed to Prometheus.

# cat /usr/share/bcc/tools/tcpconnect
# ./tcpconnect -p 80 -d 10
Enter fullscreen mode Exit fullscreen mode

Monitoring BBR metrics

Export net_tcp_bbr_bw and net_tcp_bbr_min_rtt from the kernel via node_exporter or collectd to spot congestion mis-behaviour.

Comparison with alternative approaches

Table 2 contrasts the chosen stack with three common alternatives.

Aspect BBR + IRQ-affinity Cubic + default IRQ DCTCP + hardware offload
Throughput @ 100 GbE ≈99 % line rate ≈92 % line rate ≈95 % line rate (requires loss-aware switch)
Average RTT (µs) 30-45 55-80 40-60
CPU utilization ≈55 % ≈70 % ≈65 % (offload saves ~10 %)
Complexity Medium (sysctl + irq-pinning) Low High (switch config, DCTCP kernel patches)
Tenant fairness Potentially aggressive Balanced Balanced if ECN enabled

For isolated workloads where latency determinism outweighs raw bandwidth, BBR with dedicated IRQ affinity remains the most predictable configuration. In multi-tenant environments, Cubic or DCTCP may provide better fairness at the cost of higher tail latency.

Rollback procedure

All changes are stored in separate .conf files under /etc/sysctl.d. To revert:

# rm /etc/sysctl.d/99-network.conf /etc/sysctl.d/99-bbr.conf
# sysctl -p /etc/sysctl.d/99-default.conf # a minimal baseline file
# systemctl restart irqbalance
# ethtool -G eth0 rx 1024 tx 1024
Enter fullscreen mode Exit fullscreen mode

After rollback, repeat the validation runbook to confirm that metrics have returned to pre-tuning values.

Summary

By raising socket buffers, enabling BBR, and pinning NIC queues to isolated CPUs, a Linux host can sustain near-line-rate traffic on a 100 GbE fabric while keeping 99th-percentile latency below 50 µs. The guide provides concrete configuration artifacts, a repeatable validation workflow, and a security checklist suitable for production deployment.

Free Webmaster, SRE & Cloud Diagnostic Tools (No Signup Required)

Verify and benchmark your production infrastructure directly from your browser using the HostingPodium Free Webmaster & Developer Suite:

Free Engineering Utility Primary Diagnostic Use Case
Global DNS Propagation & SSL Checker Verify live A, AAAA, CNAME, MX, TXT, NS & CAA records across 21+ global resolvers + TLS certificate audits
Website Speed Test & Core Web Vitals Auditor Measure server TTFB, LCP, INP, CLS, and HTTP/2 compression latency in real time
Rich Snippets & Schema.org JSON-LD Generator Generate & validate structured JSON-LD markup for TechArticles, FAQs, Products & Organizations
HTTP Redirect Chain & Status Code Tracer Trace 301/302 redirect hops, inspect response headers, and verify load balancer health checks
DMARC, SPF & DKIM Email Security Validator Audit domain email authentication policies and generate compliant DMARC records
Google SERP Simulator & Meta Tag Previewer Preview desktop/mobile Google snippets, OpenGraph cards, and pixel-width truncation
Free WebP Image Converter & Optimizer Client-side PNG/JPG to WebP compression for faster Core Web Vitals LCP

Related Enterprise Architecture & Benchmarks

Top comments (0)