Imagine you have two 10 Gbps links bundled together using LACP.
The port-channel provides 20 Gbps of aggregate capacity. Both physical links are up, and the configuration looks correct.
Yet during peak traffic, one link is congested while the other is barely utilized.
Applications slow down. Packet drops increase. Meanwhile, your monitoring dashboard still shows spare bandwidth across the bundle.
What is going on?
The answer is that aggregate bandwidth does not guarantee balanced traffic distribution.
Let's examine a practical example and identify what network engineers should check before upgrading their uplinks.
1. Understanding the Topology
Consider this simple design:
SWITCH A
|
+------+------+
| |
10 Gbps 10 Gbps
| |
+------+------+
|
SWITCH B
LACP Bundle: 20 Gbps
Both interfaces are active members of the same LACP bundle.
The total theoretical bandwidth is:
2 × 10 Gbps = 20 Gbps
It is tempting to assume that traffic will automatically be distributed evenly between the two links.
In reality, traffic distribution depends on the platform's forwarding algorithm, traffic characteristics and hash inputs.
2. Why LACP Does Not Guarantee Equal Utilization
LACP coordinates link aggregation. It does not, by itself, guarantee a 50/50 traffic split.
Switches commonly use a hash based on some combination of:
- Source and destination MAC addresses
- Source and destination IP addresses
- TCP or UDP port numbers
- Other platform-specific packet fields
Under conventional per-flow hashing, packets from the same flow generally follow the same physical member to help preserve packet ordering.
This means that a large flow may consume most of one 10 Gbps member while the other member has substantial unused capacity.
For example, a single flow cannot normally use the full 20 Gbps aggregate simply because two 10 Gbps links are bundled together.
The exact behavior depends on the hardware and forwarding implementation.
3. A Practical Congestion Scenario
Suppose four independent flows generate the following offered traffic:
| Flow | Offered demand |
|---|---|
| Flow A | 6 Gbps |
| Flow B | 5 Gbps |
| Flow C | 2 Gbps |
| Flow D | 1 Gbps |
| Total | 14 Gbps |
The aggregate demand utilization is:
14 / 20 × 100 = 70%
At the bundle level, 70% utilization may appear acceptable.
However, imagine the hash distributes the flows like this:
LACP BUNDLE
20 Gbps
|
+--------+--------+
| |
LINK A LINK B
10 Gbps 10 Gbps
| |
Flow A: 6G Flow C: 2G
Flow B: 5G Flow D: 1G
| |
Demand: 11G Demand: 3G
Capacity: 10G Capacity: 10G
| |
CONGESTION HEADROOM
Link A receives 11 Gbps of offered demand but can transmit no more than its 10 Gbps physical capacity.
The excess demand may result in queue buildup, packet drops or reduced application throughput.
Meanwhile, Link B has 7 Gbps of nominal unused capacity.
The important distinction is that 11 Gbps represents offered demand, not actual transmitted traffic on the 10 Gbps interface.
This is why monitoring only aggregate port-channel utilization can hide a real bottleneck.
4. Five Troubleshooting Checks
Before changing the configuration, collect evidence.
Check 1: Verify LACP Member State
Confirm that both physical interfaces are actively forwarding in the intended aggregation group.
Check:
- LACP member state
- Negotiated link speed
- Partner identity
- Port-channel membership
- Suspended or standby interfaces
A physically up interface is not necessarily an active forwarding member.
Check 2: Compare Physical Interface Counters
Measure each member independently over the same observation interval.
Review:
- Transmit and receive utilization
- Output drops
- Interface errors
- Queue congestion
- Peak utilization
- Short traffic bursts
Do not rely exclusively on five-minute averages. Brief congestion can affect applications even when long-term averages look healthy.
Check 3: Identify the Largest Traffic Flows
Use available flow telemetry or traffic monitoring to determine which endpoints and applications generate the most traffic.
Look for:
- Large backup transfers
- Storage replication
- Video traffic
- Application synchronization
- High-throughput server connections
A small number of high-volume flows can create uneven utilization even when many smaller flows are distributed reasonably well.
Check 4: Review the Hashing Policy
Inspect the platform's documented load-distribution algorithm.
A hash using only source and destination IP addresses may distribute traffic differently from one that also includes transport-layer ports.
Changing the hash policy can improve distribution for some workloads, but it does not guarantee perfect balancing.
Test proposed changes with representative traffic before applying them in production.
Check 5: Compare Single-Flow and Multi-Flow Tests
A single-stream throughput test may use only one physical member.
A multi-stream test can reveal how additional independent flows are distributed across the bundle.
Run controlled tests while monitoring individual interface counters.
Use an approved test environment or maintenance window for production-sensitive testing.
5. Should You Add More Links or Upgrade to 25G?
Not necessarily.
Adding links increases aggregate capacity, but it does not guarantee that an individual flow can use multiple physical members.
Before upgrading, answer three questions:
- Is the bottleneck caused by uneven flow distribution?
- Is a single flow approaching the physical member's bandwidth limit?
- Can the network handle the expected demand after one member fails?
Consider the same 14 Gbps of offered demand.
With two active 10 Gbps links:
- Normal aggregate capacity: 20 Gbps
- Offered demand: 14 Gbps
- Nominal aggregate headroom: 6 Gbps
Now suppose one member fails.
Remaining aggregate capacity:
1 × 10 Gbps = 10 Gbps
Capacity deficit:
14 - 10 = 4 Gbps
Even with perfectly balanced traffic under normal conditions, the remaining link cannot carry the full offered demand.
This is an N-1 capacity problem, not simply a load-balancing problem.
For a more systematic approach, see the Switch Oversubscription and N-1 Capacity Guide.
6. Calculate Your Uplink Requirements
Before purchasing additional interfaces or optical modules, compare different designs:
- 2 × 10G uplinks
- 4 × 10G uplinks
- 2 × 25G uplinks
Evaluate aggregate capacity, expected demand, failure-state capacity and the impact of traffic distribution.
The free Switch Uplink Bandwidth and Oversubscription Calculator can help estimate bandwidth requirements and capacity headroom.
Remember that a capacity calculator cannot predict the exact LACP hash assignment or guarantee the absence of packet drops. Those require real device counters and representative traffic measurements.
7. A Practical Troubleshooting Checklist
Before closing a congestion incident, verify the following:
- [ ] Both LACP members are actively forwarding.
- [ ] Physical member utilization has been measured over matching intervals.
- [ ] Interface errors, output drops and queue congestion have been checked.
- [ ] The largest traffic flows have been identified.
- [ ] The platform's hashing behavior has been reviewed.
- [ ] Proposed changes have been tested against representative workloads.
- [ ] Remaining capacity has been evaluated for an N-1 failure.
- [ ] Upgrade decisions are supported by actual measurements.
Final Thoughts
A 20 Gbps LACP bundle does not guarantee two perfectly balanced 10 Gbps links.
It also does not guarantee that a single connection can transmit at 20 Gbps.
Reliable network design requires engineers to evaluate three separate factors:
Aggregate capacity, physical member utilization and failure-state capacity.
Ignoring any one of these factors can lead to unexpected congestion and unnecessary infrastructure upgrades.
For more practical network engineering tutorials and free engineering tools, visit NetEngineerLab.
Question for network engineers: Have you encountered an LACP bundle where one member was heavily congested while another had significant spare capacity? What did you find was the root cause?
Top comments (0)