DEV Community

Cover image for Stop Single-NIC Outages: Practical Network Bonding with systemd-networkd on Linux
Lyra
Lyra

Posted on

Stop Single-NIC Outages: Practical Network Bonding with systemd-networkd on Linux

#linux #networking #systemd #devops

Stop Single-NIC Outages: Practical Network Bonding with systemd-networkd on Linux

One unplugged cable should not take a server offline.

If a host has two NICs into the same switch (or two uplinks into a redundant pair), Linux can present them as one logical interface with the bonding driver. Fail one path and traffic keeps moving. Do it declaratively with systemd-networkd, and you get a config you can review, version, and reload without shell archaeology.

This is link aggregation / NIC failover, not storage multipath and not a VPN. Different layer, different failure domain.

What you get (and what you do not)

Bonding aggregates multiple physical Ethernet ports into one logical bond device.

Common goals:

  • Failover — keep a single IP alive when one cable, SFP, or NIC dies
  • Throughput — spread flows across links when the switch and mode allow it (especially 802.3ad / LACP)
  • Clean ops — one address, one default route, fewer “which NIC is primary?” surprises

This article does not cover:

  • Device Mapper Multipath (multipathd) for storage LUNs
  • Software bridges for VMs/containers (though a bond can be a bridge port)
  • WireGuard / VPN tunnels
  • AQM / bufferbloat (fq_codel / CAKE)
  • Teamd (older alternative; bonding + networkd is the path here)

Modes that matter in real racks

The bonding driver supports several modes. For most homelab and production Linux hosts, start with one of these two:

Mode Switch help needed? Typical use
active-backup No special config (two independent ports is enough) Simple HA: one NIC active, one standby
802.3ad (LACP) Yes — LACP EtherChannel / LAG on the switch Aggregated bandwidth + redundancy with a standards-based partner

Other modes (balance-rr, balance-xor, broadcast, balance-tlb, balance-alb) exist and are documented in the kernel bonding HOWTO. They are easier to misconfigure against a switch. Prefer active-backup when the switch is dumb or unmanaged; prefer 802.3ad when you control LACP on both ends.

Critical kernel guidance: enable link monitoring. Without miimon (MII monitoring) or ARP monitoring, the bond can keep a dead slave “up” and black-hole traffic. The kernel bonding docs call this out explicitly — very few devices lack MII support, so MIIMonitorSec= should almost always be set.

Prerequisites

  • Two (or more) Ethernet interfaces you can dedicate to the bond
  • systemd-networkd managing those interfaces (not NetworkManager / ifupdown on the same NICs)
  • Console or out-of-band access the first time you cut over production networking
  • For LACP: switch ports configured as an LACP lag/port-channel with a matching hash policy preference

Identify interfaces first:

networkctl
ip -br link
Enter fullscreen mode Exit fullscreen mode

Use stable names (enp1s0, enp2s0, or names you set with .link files). Do not build production bonds on temporary USB NIC names if you can avoid it.

Disable conflicting managers on those NICs only. One DHCP client / network manager per interface.

Layout: three small files

networkd splits the job cleanly:

  1. .netdev — create bond0
  2. .network (slaves) — enslave physical NICs, no IP on the members
  3. .network (bond) — address, routes, DNS on bond0

Put them in /etc/systemd/network/ with numeric prefixes so ordering stays obvious.

1) Create the bond device

Active-backup (works without switch LACP):

/etc/systemd/network/10-bond0.netdev

[NetDev]
Name=bond0
Kind=bond
# Optional but useful for predictable MAC across reboots:
# MACAddress=aa:bb:cc:dd:ee:ff

[Bond]
Mode=active-backup
MIIMonitorSec=100ms
UpDelaySec=200ms
DownDelaySec=200ms
# After failover, remind peers who owns the IP:
GratuitousARP=2
# Primary reselect when the preferred NIC recovers:
PrimaryReselectPolicy=always
Enter fullscreen mode Exit fullscreen mode

Notes from systemd.netdev(5) / kernel docs:

  • Mode= accepts active-backup, 802.3ad, balance-rr, balance-xor, broadcast, balance-tlb, balance-alb
  • MIIMonitorSec= is the MII poll interval; 0 disables monitoring (do not leave it disabled in production)
  • UpDelaySec= / DownDelaySec= are rounded down to multiples of the MII interval — they reduce flapping on flaky PHYs
  • GratuitousARP= (active-backup) controls how many peer notifications go out after failover

LACP / 802.3ad (switch must participate):

[NetDev]
Name=bond0
Kind=bond

[Bond]
Mode=802.3ad
TransmitHashPolicy=layer3+4
LACPTransmitRate=fast
MIIMonitorSec=100ms
MinLinks=1
AdSelect=stable
Enter fullscreen mode Exit fullscreen mode

Why these knobs:

  • TransmitHashPolicy=layer3+4 spreads TCP/UDP flows better than pure MAC hashing for many east-west workloads (valid for 802.3ad / balance-xor / balance-tlb per networkd)
  • LACPTransmitRate=fast asks the partner for 1s LACPDUs instead of 30s (slow) — faster detection when the switch agrees
  • MinLinks= controls how many member links must be up before the bond asserts carrier (802.3ad)

2) Enslave the physical NICs (no addresses)

/etc/systemd/network/20-bond0-members.network

[Match]
Name=enp1s0 enp2s0

[Network]
Bond=bond0

[Link]
# Members should not block boot if one cable is missing
RequiredForOnline=no
Enter fullscreen mode Exit fullscreen mode

For active-backup, mark the preferred NIC as primary (only valid for active-backup, balance-tlb, balance-alb):

/etc/systemd/network/20-bond0-primary.network

[Match]
Name=enp1s0

[Network]
Bond=bond0
PrimarySlave=true

[Link]
RequiredForOnline=no
Enter fullscreen mode Exit fullscreen mode

/etc/systemd/network/21-bond0-backup.network

[Match]
Name=enp2s0

[Network]
Bond=bond0

[Link]
RequiredForOnline=no
Enter fullscreen mode Exit fullscreen mode

PrimarySlave=true means: while enp1s0 is healthy, it stays active. The backup is used only when the primary is offline — exactly what you want for “10G preferred, 1G spare” or “NIC on the main switch preferred.”

3) Put the IP on the bond, not the members

/etc/systemd/network/30-bond0.network

[Match]
Name=bond0

[Link]
RequiredForOnline=routable

[Network]
DHCP=yes
# Or static:
# Address=192.0.2.10/24
# Gateway=192.0.2.1
# DNS=192.0.2.53
Enter fullscreen mode Exit fullscreen mode

Using RequiredForOnline=routable on bond0 (and no on members) keeps systemd-networkd-wait-online honest: boot waits for a working bond address, not for every physical port to show carrier.

Apply safely

On a live host, prefer console/IPMI the first time.

# Review files
networkctl cat bond0 || true
ls -l /etc/systemd/network/

# Make sure networkd owns the stack
systemctl enable --now systemd-networkd.service

# Load new netdev/network definitions
networkctl reload
networkctl reconfigure enp1s0 enp2s0 bond0
# If a netdev kind/setting cannot hot-update, restart once:
# systemctl restart systemd-networkd.service
Enter fullscreen mode Exit fullscreen mode

If an existing bond0 was created with a different kind or immutable setting, networkd may keep the old device. Remove it and reload (console recommended):

ip link delete bond0 || true
systemctl restart systemd-networkd.service
Enter fullscreen mode Exit fullscreen mode

Verify the bond is real 

networkctl status bond0
networkctl
cat /proc/net/bonding/bond0
ip -br addr show bond0
ip route
Enter fullscreen mode Exit fullscreen mode

Healthy signals:

  • bond0 operational state is degraded or routable with at least one slave up (degraded with one of two slaves is normal and still useful)
  • Members show as enslaved
  • /proc/net/bonding/bond0 lists mode, MII status, active slave (active-backup), or aggregator/partner info (802.3ad)
  • Addresses and default route sit on bond0, not on enp1s0/enp2s0

Example fields you want in /proc/net/bonding/bond0:

Bonding Mode: fault-tolerance (active-backup)
Primary Slave: enp1s0 (primary_reselect always)
Currently Active Slave: enp1s0
MII Status: up
MII Polling Interval (ms): 100
Slave Interface: enp1s0
MII Status: up
Slave Interface: enp2s0
MII Status: up
Enter fullscreen mode Exit fullscreen mode

For LACP, confirm the switch also shows both ports bundled and in distributing state. Linux looking “up” while the switch still has independent access ports is a classic misconfig.

Controlled failover test

Do this during a maintenance window if the host is production.

  1. Start a continuous ping from another machine to the bond IP.
  2. Note the active slave: grep -E 'Currently Active Slave|Slave Interface|MII Status' /proc/net/bonding/bond0
  3. Pull the active cable (or ip link set enp1s0 down from console).
  4. Watch ping loss — a handful of drops can be normal; multi-second black holes are not.
  5. Confirm the bond failed over: active slave changed, bond0 still has carrier/address.
  6. Restore the link and confirm recovery policy (PrimaryReselectPolicy=always should return traffic to the primary when it is healthy again).

Optional one-liner while testing:

watch -n1 'networkctl; echo; grep -E "Currently Active|MII Status|Slave Interface|802.3ad|Aggregator" /proc/net/bonding/bond0'
Enter fullscreen mode Exit fullscreen mode

Optional: VLAN on top of the bond

If the uplink is a trunk, create VLAN devices on bond0, not on the physical NICs:

/etc/systemd/network/40-bond0.10.netdev

[NetDev]
Name=bond0.10
Kind=vlan

[VLAN]
Id=10
Enter fullscreen mode Exit fullscreen mode

/etc/systemd/network/40-bond0.vlan.network

[Match]
Name=bond0

[Network]
VLAN=bond0.10
Enter fullscreen mode Exit fullscreen mode

/etc/systemd/network/45-bond0.10.network

[Match]
Name=bond0.10

[Network]
DHCP=yes
Enter fullscreen mode Exit fullscreen mode

Keep L3 configuration on the VLAN interfaces when the bond itself is only a tagged pipe.

Operational pitfalls

No monitoring configured.

Default MIIMonitorSec=0 disables MII monitoring in networkd’s Bond section. Set it. Kernel docs also allow ARP monitoring (ARPIntervalSec= + ARPIPTargets=) when MII is insufficient; do not run with neither.

IP still on a member NIC.

Leftover NetworkManager profiles, netplan, or old .network files that match enp* will fight the bond. networkctl should show members without global addresses.

LACP without a partner.

Mode=802.3ad against non-LACP switch ports will not give you a happy aggregator. Use active-backup until the switch lag exists.

wait-online hangs at boot.

If every member is RequiredForOnline=yes, a single unplugged cable can stall boot. Mark members RequiredForOnline=no and require bond0 (often routable).

Hash policy mismatch expectations.

LACP does not stripe a single TCP flow across NICs like a RAID-0 for packets. Flow distribution depends on the transmit hash and partner behavior. Measure with multiple flows, not one iperf stream, before declaring “2× bandwidth.”

MAC surprises.

By default networkd can generate a bond MAC from name + machine-id. Pin MACAddress= if your DHCP reservations, switch port security, or license managers key off MAC stability.

Minimal health check you can schedule

A tiny oneshot is enough to catch “bond has no active slaves” before users do:

/usr/local/sbin/check-bond0

#!/bin/bash
set -euo pipefail
BOND=${1:-bond0}
proc=/proc/net/bonding/$BOND
[[ -r $proc ]] || { echo "missing $proc"; exit 2; }
if ! grep -q 'MII Status: up' "$proc"; then
  echo "$BOND has no MII up state"
  exit 1
fi
# active-backup: ensure an active slave exists
if grep -q 'Currently Active Slave: None' "$proc"; then
  echo "$BOND has no active slave"
  exit 1
fi
echo "$BOND ok"
Enter fullscreen mode Exit fullscreen mode 
chmod 755 /usr/local/sbin/check-bond0
Enter fullscreen mode Exit fullscreen mode

Wire it to a timer if you already run node health checks elsewhere. Keep alerts on “no active slave” / “only one slave in a 2-link LACP bundle” — those are the pages that prevent silent single-path operation.

Quick mode chooser

  • Unmanaged switch / two random wall ports: active-backup + PrimarySlave= + MIIMonitorSec=100ms
  • Proper ToR with LACP: 802.3ad + TransmitHashPolicy=layer3+4 + LACPTransmitRate=fast + matching switch lag
  • Need VLANs: bond first, VLAN subinterfaces second
  • Need VM bridge: bond (or bond.VLAN) as the bridge port; do not put the host IP on a member NIC

References

  • systemd.netdev(5) — [Bond] options (Mode=, MIIMonitorSec=, LACPTransmitRate=, TransmitHashPolicy=, …)
  • systemd.network(5) — Bond=, PrimarySlave=, RequiredForOnline=, VLAN=
  • networkctl(1) — link operational states (enslaved, degraded, routable, …)
  • Linux kernel docs: Ethernet Bonding Driver HOWTO — modes, miimon, LACP, failover behavior
  • Debian manpage mirror: systemd.netdev(5)
  • ArchWiki: systemd-networkd — wait-online, file layout, bridge patterns that compose with bonds

Wrap-up

Bonding is one of the highest-leverage “two cables” upgrades you can give a Linux host. Keep the recipe boring:

  1. Create bond0 with an explicit mode and MII monitoring
  2. Enslave NICs with no IPs and RequiredForOnline=no
  3. Put addressing only on bond0 (or VLAN children)
  4. Verify with networkctl + /proc/net/bonding/bond0
  5. Pull a cable on purpose once before you trust it

Do that, and a single NIC or cable fault stops being an outage and becomes a short blip you already tested.

Top comments (0)