DEV Community: NetEngineerLab

Fiber Latency Calculation Explained

NetEngineerLab — Thu, 05 Nov 2026 06:53:22 +0000

Fiber latency at this level is propagation time plus a few delays you can name. The propagation term is the route length divided by the speed of the modulated pulse in the glass. Round-trip time, in the model used here, is twice the one-way total. Queuing, forward error correction, and a protocol's acknowledgements are separate and are left out.

The equations match the Fiber Latency Calculator: distance scaled by a route factor, group index, a flat equipment delay, and optional serialization.

Propagation uses the group index

Distance is converted to kilometres first. One international mile is 1.609344 km. One metre is 0.001 km. A route factor of at least 1 then stretches that length:

L_km = d_km × f_route

A factor of 1 means the distance you typed is already the glass length you want charged. A factor above 1 is how you grow a shorter map distance into a routed length. Factors below 1 are rejected.

Group index n_g sets the delay of a modulated pulse. A phase index describes a single-frequency wavefront and will mis-state that delay if you drop it into this formula. The vacuum speed in the engine is the defined constant c = 299792.458 km/s. One-way propagation in microseconds is:

t_prop_µs = L_km × n_g × 1e6 / c

The default group index is 1.468. The accepted range is 1 to 1.6. Outside that range the engine rejects the input. Per kilometre at the default index:

t_per_km = 1.468 × 1e6 / 299792.458 = 4.8967209… µs/km

The speed implied by that index is c / 1.468 ≈ 204218 km/s. The selector on the calculator page also offers 1.4677 at 1310 nm and 1.4682 at 1550 nm as published G.652-style presets. The formula notes still mark a specific recommendation edition for those two presets as pending, so the worked example below stays on the engine default, 1.468.

Vacuum delay on the same kilometre is 1e6 / c = 3.335641 µs. The fiber figure is that vacuum delay multiplied by n_g.

Equipment, serialization, and round trip

Equipment delay is a constant per device, multiplied by a count. It is not a lookup by platform:

t_eq_µs = N_equipment × t_device_µs

Serialization is the time to clock a frame onto a link, repeated for a hop count. S is the frame size in bytes and R is the rate in Gbit/s:

t_ser_µs = N_hops × (S × 8) / (R × 1e9) × 1e6

One hop of 1500 bytes at 1 Gbit/s is 12 µs. Two hops are 24 µs. The engine applies that same hop delay N_hops times. It does not build a store-and-forward sum that changes packet size at each node.

One-way delay adds the three terms. Round-trip time doubles the sum. The return path is the same total, not a second route with its own length:

t_one_way = t_prop + t_eq + t_ser
t_rtt = 2 × t_one_way

Displayed values follow the engine's rounding. Under 1000 µs the figure is rounded to two decimal places and kept in microseconds. At 1000 µs and above it is converted to milliseconds and rounded to three decimal places. Rounding is half-up on the scaled value, using the same Math.round((value + Number.EPSILON) × 10^n) step as the page.

The engine also prints comparison delays that stay out of t_one_way: vacuum, microwave at 1.0003 times vacuum, hollow core at 1.003 times vacuum, and copper at velocity factors 0.65 and 0.70. Those rows are labels next to the fiber result.

Worked example: 100 km, default group index

Inputs: 100 km, route factor 1, n_g = 1.468, equipment count 0, packet size 0, link rate 10 Gbit/s, serialization hops 0.

Fiber length is 100 km.
t_prop = 100 × 1.468 × 1e6 / 299792.458 = 489.67209175… µs.
Equipment and serialization are 0, so one-way delay equals propagation.
Round-trip time is 979.3441835… µs.

Display rounding reports 489.67 µs one way and 979.34 µs round trip. The per-kilometre figure is 4.8967209… µs/km. A 5 µs/km rule of thumb on this span would say 500 µs one way, about 10.3 µs high, a little over 2 percent.

The same 100 km at the top of the allowed group index, 1.6, is 533.70255… µs one way, which displays as 533.70 µs. Distance must be greater than 0.

The 24 µs serialization check is a different input set, used only to confirm that term: 1500 bytes, 1 Gbit/s, 2 hops, and the other delays left at zero. It does not belong in the 100 km total above, where the hop count and the packet size are both zero.

Common mistakes

Charging 5 µs/km and presenting it as this model. At group index 1.468 the engine's own kilometre is 4.8967 µs. The 5 µs figure is a rough allowance, and on 100 km it moves the one-way result by about 10 µs.
Inserting a phase index, or a generic "fiber is two-thirds of c", in place of the group index. Two-thirds of c is a group index of 1.5, which is inside the allowed range and is simply a different input: 1.5 / 1.468 ≈ 1.022, about 2.2 percent more delay than the default.
Folding the vacuum, microwave, hollow-core, or copper comparison rows into the fiber total. They are printed beside it. Copper at velocity factor 0.65 is 3.335641 / 0.65 ≈ 5.13 µs/km of comparison delay, and it is not part of t_one_way.
Building round-trip time as twice the fiber and then adding equipment once. Equipment sits inside the one-way sum, and the round trip doubles that whole sum. With 10 µs of equipment, one-way delay grows by 10 µs and round-trip time grows by 20 µs.
Adding serialization on top of a 100 km run that already used zero hops. Serialization appears only for the hop count and frame size you entered.
Reading this round trip as a ping. The sum has no queue, no FEC, no ACK timeout, and no asymmetric return path. A ping above t_rtt still has room for those delays; the calculator never received them.

Limits that stay outside the number

Equipment delay is the constant you typed. Mile inputs use the international mile, 1.609344 km. The comparison media are labels beside the fiber result. They are not measured routes on the same trench.

The 100 km case can be rerun in the Fiber Latency Calculator. Constants, the rounding rule, and this numerical check are in NetEngineerLab/fiber-propagation-latency.md.

Understanding Optical Loss: dB, dBm and Link Margin

NetEngineerLab — Thu, 05 Nov 2026 06:52:49 +0000

Optical loss and optical power share a logarithmic scale and then part ways. A reading in dBm is a power. A reading in dB is a ratio between two powers. Link margin is what remains after that ratio has been spent against a budget or against a receiver threshold. Mixing the units is how a span that looks fine on a spreadsheet fails on a power meter.

The arithmetic below is the single-span model in the Fiber Loss Calculator. It is a loss sum plus a comparison to numbers you type. It is a different calculation from a power budget built out of transmitter minimums and receiver overload.

dBm is a power, dB is a ratio

dBm is power referred to 1 mW:

P_dBm = 10 log10(P_mW / 1 mW)

Power	Level
1 mW	0 dBm
10 mW	10 dBm
1 µW (0.001 mW)	−30 dBm

A change of 10 dB moves the power by ten times. A change of 3 dB moves it by about two times: 10^(0.3) ≈ 1.995. So 3 dBm is about 2.00 mW, and −3 dBm is about 0.50 mW.

Loss in dB is the same logarithm applied to a ratio, which is why it also equals the difference of the two levels:

L_dB = 10 log10(P_in / P_out) = P_in_dBm − P_out_dBm

Subtract dB from dBm and the result is still dBm. Add two dBm figures and the result is a sum of logarithms, which is the product of the two powers, scaled by the reference. That product is rarely the quantity a link budget wants. Passive losses along one span add in dB because each element multiplies the power by a factor less than one, and logarithms turn those factors into a sum.

What this span model adds

For one attenuation over the whole length:

L_fiber = d_km × α_dB_per_km
L_splice = N_splice × ℓ_splice
L_conn = N_connector × ℓ_connector
L_split_other = L_splitter1 + L_splitter2 + L_other
L_physical = L_fiber + L_splice + L_conn + L_split_other
L_design = L_physical + M

α is an input in dB/km. The engine does not look up a wavelength and pick a coefficient. Splitter fields are insertion losses in dB, already inclusive of whatever excess loss you assigned. The engine does not evaluate 10 log10(N) for an N-way split.

Two comparisons then use that loss. They answer different questions:

B_remaining = B_available − L_design
P_rx = P_tx − L_physical
M_rx = P_rx − P_threshold

B_available is a budget in dB that you supply. P_tx and P_threshold are levels in dBm. Design loss includes the engineering margin, so remaining budget does too. Receive power uses physical loss alone, so the margin you reserved does not move the estimated meter reading.

The link is failed when remaining budget or receive margin is negative. It is a warning when either is under 3 dB and both are still at least zero. Otherwise it is healthy. The 3 dB band is a planning rule in this calculator. Absolute values at or below 10⁻⁹ dB are treated as zero.

Because the available budget is its own input, it can differ from P_tx − P_threshold. The gap between the two margins is fixed by that choice:

M_rx − B_remaining = (P_tx − P_threshold) − B_available + M

When you set the available budget equal to transmit level minus receive threshold, the two margins differ by exactly the engineering margin. When you set a different budget, they differ by the engineering margin plus that offset.

Worked example: 10 km PON feeder

These inputs match the calculator's 10 km PON feeder loss case, together with the budget and power fields the engine requires.

Input	Value
Distance / attenuation	10 km / 0.25 dB/km
Splices	6 × 0.10 dB
Connectors	4 × 0.30 dB
Splitter 1 / splitter 2 / other	21.0 dB / 0 dB / 0 dB
Engineering margin	3 dB
Available budget	32 dB
Transmit power	3 dBm
Receive threshold	−27 dBm

Fiber: 10 × 0.25 = 2.50 dB.
Splices 0.60 dB. Connectors 1.20 dB. Splitter and other 21.00 dB.
Physical loss: 2.50 + 0.60 + 1.20 + 21.00 = 25.30 dB.
Design loss: 25.30 + 3.00 = 28.30 dB.
Remaining budget: 32 − 28.30 = 3.70 dB.
Receive level: 3 − 25.30 = −22.30 dBm. Receive margin: −22.30 − (−27) = 4.70 dB.

Physical loss is 25.30 dB, design loss is 28.30 dB, and the status is healthy. Both margins clear the 3 dB warning line, the remaining budget by 0.70 dB and the receive margin by 1.70 dB.

Check the units on the receive level. 3 dBm is 10^(0.3) ≈ 1.995 mW. −22.30 dBm is 10^(−2.230) ≈ 0.00589 mW, about 5.89 µW. The ratio of those powers is 25.30 dB, which is the physical loss, and 3.00 − 25.30 = −22.30 brings the level back in dBm.

Here P_tx − P_threshold = 3 − (−27) = 30 dB, while the available budget was entered as 32 dB. The identity above gives 4.70 − 3.70 = 30 − 32 + 3 = 1 dB. The receive margin is 1 dB larger than the remaining budget because the typed budget is 2 dB more generous than the raw window and the engineering margin then takes 3 dB back.

Drop only the available budget to 28 dB. Remaining budget becomes 28 − 28.30 = −0.30 dB and the status is failed. Receive margin is unchanged at 4.70 dB, because the plant and the threshold did not move. A zero-length span drops only the fiber term and leaves physical loss at 25.30 − 2.50 = 22.80 dB.

Common mistakes

Adding dBm to dBm and calling the sum a loss. Loss is a difference of levels, or 10 log10 of a power ratio.
Subtracting design loss from transmit power when estimating what a meter should read. On this feeder that would give 3 − 28.30 = −25.30 dBm. The model estimates −22.30 dBm, because the 3 dB engineering margin was never a loss in the glass.
Assuming the available-budget box is filled in from the optics. In the example it is 32 dB and P_tx − P_threshold is 30 dB. Both numbers are inputs, and the status uses both.
Entering 32, or 1:32, in a splitter field that expects decibels. The 21.0 dB in the table is already an insertion loss. An ideal 1:32 split is 10 log10(32) ≈ 15.05 dB before excess loss. The extra 21.0 − 15.05 ≈ 5.95 dB in this example is excess and other allocated loss sitting inside that one field.
Using one margin to declare the link passed when the other has gone negative. The 28 dB budget case fails on remaining budget while the receive margin is still 4.70 dB.
Letting the connector count mean two different things in two spreadsheets. The engine multiplies count by loss per count. Agree whether one count is a mated pair or one connector end, then keep that rule for the 0.30 dB figure.
Treating a healthy result as an acceptance test. The 3 dB warning is local to the calculator. Reflection, PMD, and dispersion penalties are outside this sum.

Where the same numbers are evaluated

One attenuation covers the whole route. There is no splice map, no bidirectional average, and no second wavelength. Splitter loss is whatever decibel value you entered.

Run the table in the Fiber Loss Calculator. The matching derivation is NetEngineerLab/fiber-link-loss.md. The loss page links ITU-T G.652, G.984.2, and G.671, IEEE 802.3, and IEC 61280-4-2. A healthy flag is the comparison defined above. It is a planning result from the numbers you entered.

How to Calculate Fiber Optic Power Budget

NetEngineerLab — Thu, 05 Nov 2026 06:52:48 +0000

A fiber power budget is the decibel window between the weakest launch a transmitter is allowed to produce and the weakest optical power a receiver can still accept. Fiber, splices, connectors, and splitters spend that window. The sums below are the ones implemented by the Optical Power Budget Calculator, with the same 20 km check recorded in the formula notes.

Two launch powers, two receiver limits

Sensitivity is judged at minimum transmit power. Overload is judged at maximum transmit power. Both ends of the transmitter specification belong in the budget, because a short jumper can deliver more power than the receiver's overload point while a long span on the same optics still clears sensitivity.

Receiver sensitivity and overload are absolute powers in dBm. The gap between minimum launch and sensitivity is already in dB, since subtracting two dBm values cancels the 1 mW reference:

B_raw = P_tx,min − P_sens
B_std = B_raw − P_penalty

P_penalty is one number you enter, in dB. It stands for the path penalties you have chosen to reserve. The calculator does not compute it from chromatic dispersion or from a wavelength table.

Plant loss splits into fiber, which scales with length, and fixed losses, which do not:

L_fiber = d_km × α_dB_per_km
L_splice = N_splice × ℓ_splice
L_conn = N_connector × ℓ_connector
L_split = L1 + L2 + L3
L_fixed = L_splice + L_conn + L_split + L_other
L_physical = L_fiber + L_fixed
L_design = L_physical + M

Each splitter field is an insertion loss in dB. A split ratio such as 1:32 has to be converted to an insertion loss before it is typed. The three fields are added. Attenuation α must be greater than 0. With α at 0 the power-limited reach is undefined, and the engine rejects that input.

Remaining budget subtracts design loss, so the engineering margin is included:

B_remaining = B_std − L_design

Estimated receive power subtracts physical loss only. The margin is an allowance for the unknowns you listed, and it is kept out of the power you expect the meter to show:

P_rx = P_tx,min − L_physical
P_rx,max = P_tx,max − L_physical
M_sensitivity = P_rx − P_sens
M_overload = P_overload − P_rx,max

Power-limited distance reuses the same fixed loss, the same margin, and the same attenuation at every length:

d_max = max(0, (B_std − M − L_fixed) / α)

That length is the point where design loss equals the standard budget. It is a power reach. Dispersion and polarization-mode dispersion can limit a faster client before this distance is used up.

The result is failed when remaining budget, sensitivity margin, or overload margin is negative. It is a warning when any of those three is under 3 dB and still at least zero. Otherwise it is healthy. The 3 dB line is internal to this calculator. Margins within 10⁻⁹ dB of zero are treated as zero. When the standard budget is positive, usage is 100 × L_design / B_std.

Worked example: 20 km at 0.35 dB/km

Input	Value
Minimum / maximum transmit power	2 dBm / 5 dBm
Sensitivity / overload	−28 dBm / +2 dBm
Path penalty	1 dB
Length / attenuation	20 km / 0.35 dB/km
Splices	4 × 0.10 dB
Connectors	2 × 0.50 dB
Splitters and other loss	0 dB
Engineering margin	3 dB

Raw budget: 2 − (−28) = 30 dB. After the 1 dB penalty, B_std = 29 dB.
Fiber: 20 × 0.35 = 7.00 dB. Splices 0.40 dB. Connectors 1.00 dB. Fixed loss 1.40 dB.
Physical loss 8.40 dB. Design loss 8.40 + 3.00 = 11.40 dB.
Remaining budget: 29 − 11.40 = 17.60 dB.
Receive level at minimum launch: 2 − 8.40 = −6.40 dBm. Sensitivity margin: −6.40 − (−28) = 21.60 dB.
Receive level at maximum launch: 5 − 8.40 = −3.40 dBm. Overload margin: 2 − (−3.40) = 5.40 dB.
Power-limited reach: (29 − 3 − 1.40) / 0.35 = 70.2857 km.

Design loss is 11.40 dB, remaining budget is 17.60 dB, and the status is healthy. Usage of the standard budget is 100 × 11.40 / 29 = 39.31%.

On these same losses, sensitivity margin sits 4.00 dB above remaining budget. That gap equals the path penalty plus the engineering margin, because both margins are built from the same minimum launch, the same sensitivity, and the same physical loss. Change any one of those and the 4 dB relationship changes with them.

The overload limit is doing separate work. Set it to −8 dBm and leave every other input alone. Overload margin becomes −8 − (−3.40) = −4.60 dB, and the status is failed, while the sensitivity margin is still positive. A link can have spare sensitivity and still be rejected because the strong launch is too strong.

Common mistakes

Running the sensitivity check from the maximum launch. The weak end of the transmitter range is the one that has to clear sensitivity. On this example that would replace −6.40 dBm with −3.40 dBm and would overstate the sensitivity margin by 3 dB.
Subtracting the engineering margin a second time when estimating receive power. Physical loss already excludes it. Putting the 3 dB margin into P_rx would report −9.40 dBm at minimum launch.
Quoting remaining budget and sensitivity margin as if they were the same reading. On the table above they are 17.60 dB and 21.60 dB.
Typing a split ratio into a splitter field. The field adds decibels. An ideal 1:32 split is 10 log10(32) ≈ 15.05 dB before excess loss. The calculator adds the decibel value you type.
Treating a module class letter as an output. B+ or C+ shows up in the result only to the extent you typed that class's sensitivity, overload, and penalty into the inputs.
Publishing d_max as a guaranteed span length. At 70.2857 km this model has used up the standard budget only while attenuation stays 0.35 dB/km and fixed loss stays 1.40 dB. A higher line rate can become dispersion-limited earlier.
Double-counting connector ends. Loss per connector is whatever you decided one count represents. Count mated pairs if 0.50 dB is your pair allowance. Count ends if the allowance is per ferrule. The engine multiplies the count by the loss you entered.

What a healthy flag leaves open

The flag compares the inputs. It does not replace a power-meter measurement or an OTDR acceptance test. One attenuation covers the whole length. There is no per-wavelength loss table and no automatic penalty from dispersion. Splice and connector counts are integers you supply.

The same 20 km inputs can be rerun in the Optical Power Budget Calculator. The equation list and this numerical check are in NetEngineerLab/optical-power-budget.md.

VLAN Trunk Troubleshooting: 7 Checks Before You Blame the Switch

NetEngineerLab — Wed, 28 Oct 2026 09:14:38 +0000

A VLAN trunk can look perfectly healthy while users experience intermittent packet loss, DHCP failures, or connectivity problems between network segments.

The interface is up. The link light is green. The trunk appears to be forwarding traffic.

Yet something is wrong.

In real-world network operations, these problems are often caused by configuration inconsistencies, VLAN mismatches, spanning-tree behavior, or MTU issues rather than a failed switch.

Here is a practical, layer-by-layer checklist I use to narrow down VLAN trunk problems without making unnecessary configuration changes.

1. Confirm That the Physical Link Is Actually Healthy

Start with the basics.

Check both ends of the trunk for:

Interface status and recent link flaps
CRC and input errors
Output drops and queue congestion
Speed and duplex inconsistencies
Optical power levels, if the link uses fiber

A trunk can remain operational while experiencing intermittent physical-layer errors.

Useful commands vary by vendor. On Cisco IOS, for example:

show interfaces
show interfaces counters errors
show logging

Look for counters that increase during the problem, not just counters that have accumulated over months.

If errors are increasing, investigate the physical link before changing VLAN configuration.

2. Verify the Allowed VLAN List on Both Ends

One of the easiest mistakes to overlook is an inconsistent allowed VLAN list.

For example:

Switch A: VLANs 10,20,30,40
Switch B: VLANs 10,20,30

VLAN 40 may work on one part of the network but fail to cross this trunk.

Check the effective allowed VLAN list on both sides. Also verify that the VLANs exist and are active where required.

On Cisco IOS:

show interfaces trunk
show vlan brief

Do not assume that configuring a VLAN globally means it is automatically permitted across every trunk.

3. Check Native VLAN or Untagged Traffic Behavior

Native VLAN behavior can create confusing symptoms, especially when devices on opposite ends of a trunk have different expectations about untagged frames.

For example:

Switch A: Native VLAN 10
Switch B: Native VLAN 20

Depending on the platform and configuration, this mismatch can result in traffic being classified into different VLANs or trigger a warning.

Check:

Native VLAN configuration on both ends
Whether untagged traffic is expected
Whether the platform uses native VLAN tagging
Whether the trunk configuration matches the network design

On Cisco IOS:

show interfaces trunk
show logging

Avoid changing the native VLAN simply to silence a warning. First establish which VLAN should carry untagged traffic, if any.

4. Investigate Spanning Tree

A trunk may be physically up but not forwarding traffic for a particular VLAN.

Spanning Tree Protocol can place a port into a blocking or alternate state to prevent Layer 2 loops.

Check the spanning-tree state for the affected VLAN rather than looking only at the physical interface status.

On Cisco IOS:

show spanning-tree vlan 10
show spanning-tree inconsistentports

Replace VLAN 10 with the affected VLAN.

Look for:

A port in blocking or alternate state
Unexpected topology changes
Root bridge changes
Inconsistent port states
A recently introduced redundant link

Do not disable spanning tree as a troubleshooting shortcut. That can turn a localized connectivity problem into a Layer 2 loop.

5. Check MTU and Frame-Size Differences

MTU mismatches are particularly difficult to diagnose because small packets may work while larger packets fail.

Typical symptoms include:

Applications connecting but transferring data unreliably
Large pings failing while small pings succeed
Problems appearing only on paths that cross specific links
Encapsulated traffic failing unexpectedly

Check the configured MTU on the relevant interfaces and the maximum frame size supported along the entire path.

For IPv4, a basic ping test can help compare small and large packets.

Linux:

ping -M do -s 1472 <destination-ip>

Windows:

ping -f -l 1472 <destination-ip>

These examples test an IPv4 packet with a 1500-byte IP MTU when the 28 bytes of IPv4 and ICMP headers are included.

However, a failed test does not automatically prove an MTU mismatch. Firewalls may block ICMP, and some paths handle fragmentation or filtering differently.

Test progressively smaller payloads and compare results across the suspected path.

6. Verify MAC Learning and the Actual Forwarding Path

Sometimes the trunk is configured correctly, but traffic is not reaching the expected destination because of MAC learning or forwarding behavior.

Check:

Whether the source MAC is learned in the correct VLAN
Whether the destination MAC is present
Whether the MAC address moves between ports unexpectedly
Whether port security or other Layer 2 controls are dropping traffic

On Cisco IOS:

show mac address-table dynamic
show mac address-table dynamic vlan 10

If the MAC address appears on an unexpected interface, investigate the topology and forwarding path before clearing tables or restarting interfaces.

Clearing the MAC table may temporarily change symptoms without resolving the underlying problem.

7. Compare Both Ends and Capture Traffic if Necessary

If the previous checks do not explain the issue, compare the effective configuration on both ends of the trunk.

Pay particular attention to:

Trunk mode and negotiation
Allowed VLANs
Native VLAN settings
MTU
Link aggregation membership, if applicable
Spanning-tree state
Interface counters

Then capture traffic at an appropriate observation point.

A packet capture can help answer three important questions:

Does the frame arrive at the sending switch?
Does it leave the trunk with the expected VLAN tagging?
Does it arrive at the receiving switch?

A SPAN or mirror session can help, but remember that the capture point and switch platform affect what tags and frames you see.

If the frame leaves one switch but never appears at the next observation point, you have narrowed the problem considerably.rom the simplest and least disruptive checks toward deeper packet-level investigation.

Order	Check	What It Helps Identify
1	Physical interface and counters	Link errors and instability
2	Allowed VLANs	Missing VLANs across the trunk
3	Native VLAN behavior	Untagged traffic inconsistencies
4	Spanning tree	VLAN-specific forwarding states
5	MTU and frame size	Large-packet failures
6	MAC address learning	Unexpected Layer 2 forwarding
7	Packet capture	Where frames disappear or change

This order is a starting point, not a universal rule. If the symptoms clearly point to one layer, investigate that layer first.

Final Thoughts

A green interface does not guarantee a working VLAN trunk.

The most effective troubleshooting approach is to verify the physical link, compare both ends of the trunk, inspect forwarding state, and then follow the actual traffic path.

Most importantly, change one variable at a time and record the result. That makes the diagnosis repeatable and reduces the risk of introducing a second problem while trying to fix the first.

For more practical network engineering tools and troubleshooting resources, visit NetEngineerLab.

What is the most difficult VLAN trunk issue you have encountered in production?

networking #devops #cisco #networkengineering

Why Is One LACP Link at 100% While the Other Is Almost Idle?

NetEngineerLab — Sat, 24 Oct 2026 08:57:40 +0000

Imagine you have two 10 Gbps links bundled together using LACP.

The port-channel provides 20 Gbps of aggregate capacity. Both physical links are up, and the configuration looks correct.

Yet during peak traffic, one link is congested while the other is barely utilized.

Applications slow down. Packet drops increase. Meanwhile, your monitoring dashboard still shows spare bandwidth across the bundle.

What is going on?

The answer is that aggregate bandwidth does not guarantee balanced traffic distribution.

Let's examine a practical example and identify what network engineers should check before upgrading their uplinks.

1. Understanding the Topology

Consider this simple design:

           SWITCH A
               |
        +------+------+
        |             |
      10 Gbps       10 Gbps
        |             |
        +------+------+
               |
           SWITCH B

      LACP Bundle: 20 Gbps

Both interfaces are active members of the same LACP bundle.

The total theoretical bandwidth is:

2 × 10 Gbps = 20 Gbps

It is tempting to assume that traffic will automatically be distributed evenly between the two links.

In reality, traffic distribution depends on the platform's forwarding algorithm, traffic characteristics and hash inputs.

2. Why LACP Does Not Guarantee Equal Utilization

LACP coordinates link aggregation. It does not, by itself, guarantee a 50/50 traffic split.

Switches commonly use a hash based on some combination of:

Source and destination MAC addresses
Source and destination IP addresses
TCP or UDP port numbers
Other platform-specific packet fields

Under conventional per-flow hashing, packets from the same flow generally follow the same physical member to help preserve packet ordering.

This means that a large flow may consume most of one 10 Gbps member while the other member has substantial unused capacity.

For example, a single flow cannot normally use the full 20 Gbps aggregate simply because two 10 Gbps links are bundled together.

The exact behavior depends on the hardware and forwarding implementation.

3. A Practical Congestion Scenario

Suppose four independent flows generate the following offered traffic:

Flow	Offered demand
Flow A	6 Gbps
Flow B	5 Gbps
Flow C	2 Gbps
Flow D	1 Gbps
Total	14 Gbps

The aggregate demand utilization is:

14 / 20 × 100 = 70%

At the bundle level, 70% utilization may appear acceptable.

However, imagine the hash distributes the flows like this:

             LACP BUNDLE
                20 Gbps
                   |
          +--------+--------+
          |                 |
        LINK A            LINK B
         10 Gbps           10 Gbps
          |                 |
       Flow A: 6G        Flow C: 2G
       Flow B: 5G        Flow D: 1G
          |                 |
       Demand: 11G       Demand: 3G
       Capacity: 10G     Capacity: 10G
          |                 |
       CONGESTION         HEADROOM

Link A receives 11 Gbps of offered demand but can transmit no more than its 10 Gbps physical capacity.

The excess demand may result in queue buildup, packet drops or reduced application throughput.

Meanwhile, Link B has 7 Gbps of nominal unused capacity.

The important distinction is that 11 Gbps represents offered demand, not actual transmitted traffic on the 10 Gbps interface.

This is why monitoring only aggregate port-channel utilization can hide a real bottleneck.

4. Five Troubleshooting Checks

Before changing the configuration, collect evidence.

Check 1: Verify LACP Member State

Confirm that both physical interfaces are actively forwarding in the intended aggregation group.

Check:

LACP member state
Negotiated link speed
Partner identity
Port-channel membership
Suspended or standby interfaces

A physically up interface is not necessarily an active forwarding member.

Check 2: Compare Physical Interface Counters

Measure each member independently over the same observation interval.

Review:

Transmit and receive utilization
Output drops
Interface errors
Queue congestion
Peak utilization
Short traffic bursts

Do not rely exclusively on five-minute averages. Brief congestion can affect applications even when long-term averages look healthy.

Check 3: Identify the Largest Traffic Flows

Use available flow telemetry or traffic monitoring to determine which endpoints and applications generate the most traffic.

Look for:

Large backup transfers
Storage replication
Video traffic
Application synchronization
High-throughput server connections

A small number of high-volume flows can create uneven utilization even when many smaller flows are distributed reasonably well.

Check 4: Review the Hashing Policy

Inspect the platform's documented load-distribution algorithm.

A hash using only source and destination IP addresses may distribute traffic differently from one that also includes transport-layer ports.

Changing the hash policy can improve distribution for some workloads, but it does not guarantee perfect balancing.

Test proposed changes with representative traffic before applying them in production.

Check 5: Compare Single-Flow and Multi-Flow Tests

A single-stream throughput test may use only one physical member.

A multi-stream test can reveal how additional independent flows are distributed across the bundle.

Run controlled tests while monitoring individual interface counters.

Use an approved test environment or maintenance window for production-sensitive testing.

5. Should You Add More Links or Upgrade to 25G?

Not necessarily.

Adding links increases aggregate capacity, but it does not guarantee that an individual flow can use multiple physical members.

Before upgrading, answer three questions:

Is the bottleneck caused by uneven flow distribution?
Is a single flow approaching the physical member's bandwidth limit?
Can the network handle the expected demand after one member fails?

Consider the same 14 Gbps of offered demand.

With two active 10 Gbps links:

Normal aggregate capacity: 20 Gbps
Offered demand: 14 Gbps
Nominal aggregate headroom: 6 Gbps

Now suppose one member fails.

Remaining aggregate capacity:

1 × 10 Gbps = 10 Gbps

Capacity deficit:

14 - 10 = 4 Gbps

Even with perfectly balanced traffic under normal conditions, the remaining link cannot carry the full offered demand.

This is an N-1 capacity problem, not simply a load-balancing problem.

For a more systematic approach, see the Switch Oversubscription and N-1 Capacity Guide.

6. Calculate Your Uplink Requirements

Before purchasing additional interfaces or optical modules, compare different designs:

2 × 10G uplinks
4 × 10G uplinks
2 × 25G uplinks

Evaluate aggregate capacity, expected demand, failure-state capacity and the impact of traffic distribution.

The free Switch Uplink Bandwidth and Oversubscription Calculator can help estimate bandwidth requirements and capacity headroom.

Remember that a capacity calculator cannot predict the exact LACP hash assignment or guarantee the absence of packet drops. Those require real device counters and representative traffic measurements.

7. A Practical Troubleshooting Checklist

Before closing a congestion incident, verify the following:

[ ] Both LACP members are actively forwarding.
[ ] Physical member utilization has been measured over matching intervals.
[ ] Interface errors, output drops and queue congestion have been checked.
[ ] The largest traffic flows have been identified.
[ ] The platform's hashing behavior has been reviewed.
[ ] Proposed changes have been tested against representative workloads.
[ ] Remaining capacity has been evaluated for an N-1 failure.
[ ] Upgrade decisions are supported by actual measurements.

Final Thoughts

A 20 Gbps LACP bundle does not guarantee two perfectly balanced 10 Gbps links.

It also does not guarantee that a single connection can transmit at 20 Gbps.

Reliable network design requires engineers to evaluate three separate factors:

Aggregate capacity, physical member utilization and failure-state capacity.

Ignoring any one of these factors can lead to unexpected congestion and unnecessary infrastructure upgrades.

For more practical network engineering tutorials and free engineering tools, visit NetEngineerLab.

Question for network engineers: Have you encountered an LACP bundle where one member was heavily congested while another had significant spare capacity? What did you find was the root cause?

How Many Uplinks Does a 4-Switch Stack Need? A Practical Network Design Guide

NetEngineerLab — Mon, 12 Oct 2026 15:42:34 +0000

The Real-World Design Question

Imagine you're designing a campus network with the following topology:

4 access switches in a single stack
48 × 1 Gbps access ports per switch
2 distribution switches
10 Gbps or 25 Gbps uplink options

You have 192 access ports in total.

Should you connect the access stack to the distribution layer using two, four, or eight uplinks?

The answer depends on traffic demand, redundancy requirements, stack architecture, and failure scenarios — not simply the number of switches in the stack.

1. Start With Two Redundant Uplinks

For many enterprise access networks, two appropriately sized uplinks are a reasonable starting point.

A simplified topology looks like this:

       Distribution Layer
       +-------+  +-------+
       | Dist A|  | Dist B|
       +---+---+  +---+---+
           |          |
         10/25G     10/25G
           |          |
     +-----+----------+-----+
     |     Access Stack     |
     | SW1--SW2--SW3--SW4  |
     +----------------------+

Ideally, the uplinks terminate on different stack members to reduce the impact of a single member failure.

However, connecting to two distribution switches does not automatically create a working port channel.

A cross-device LAG requires a supported technology such as stacking, MLAG, or an equivalent vendor-specific solution.

Otherwise, consider a supported Layer 3 design or another appropriate redundancy mechanism.

2. Calculate the Oversubscription Ratio

Suppose each access switch has 48 × 1 Gbps ports.

Four switches provide:

192 × 1 Gbps = 192 Gbps

With two active 10 Gbps uplinks:

192 / 20 = 9.6:1

With two active 25 Gbps uplinks:

192 / 50 = 3.84:1

Uplink configuration	Aggregate nominal capacity	Oversubscription
2 × 10G	20 Gbps	9.6:1
2 × 25G	50 Gbps	3.84:1
4 × 10G	40 Gbps	4.8:1
4 × 25G	100 Gbps	1.92:1

These figures assume all listed uplinks are active and usable for the traffic being considered.

They describe nominal port-capacity ratios, not measured network utilization.

3. Estimate Real Traffic Demand

Not every connected device transmits at full line rate simultaneously.

Suppose:

192 connected access ports
30% average busy-hour demand per port
1 Gbps access port speed

Estimated demand:

192 × 1 × 0.30 = 57.6 Gbps

In this scenario, two 10G uplinks would not provide sufficient capacity if all 57.6 Gbps actually needed to cross the uplinks simultaneously in the same direction.

But this calculation is only a simplified planning assumption.

Real demand depends on endpoint activity, application behavior, traffic locality, and directional traffic patterns.

For a more accurate design, measure or estimate traffic that actually needs to traverse the distribution uplinks.

4. Don't Ignore N-1 Redundancy

One of the most important questions is:

What happens when an uplink or distribution switch fails?

Consider two active 10G uplinks.

Normal aggregate nominal capacity:

2 × 10 = 20 Gbps

After losing one uplink:

1 × 10 = 10 Gbps

If your required peak traffic is 15 Gbps, the remaining link cannot carry that demand without congestion.

This is why redundancy should be evaluated separately from normal operating capacity.

For critical environments, size the network so the remaining path can support the required traffic after a single failure.

5. Consider Stack Bandwidth

If uplinks are connected only to SW1 and SW2, traffic originating from SW3 and SW4 may need to traverse the stacking fabric.

That isn't automatically a problem.

Modern stacking systems may provide substantial internal bandwidth, but the actual performance depends on:

Stack topology
Stack interconnect capacity
Traffic forwarding architecture
Stack member failure behavior
Vendor-specific limitations

A ring topology may provide better resilience than a simple chain, depending on the platform.

Always verify the manufacturer's stacking specifications.

6. Why More Uplinks Aren't Always Better

Adding uplinks increases potential aggregate capacity, but it can also introduce additional cost and operational complexity.

7. A Practical Design Checklist

Before deciding how many uplinks to deploy, answer these questions:

How much traffic actually crosses the distribution layer during peak periods?
Are 10G or 25G interfaces available on both sides?
Do the distribution switches support MLAG, stacking, or another suitable redundancy design?
Can the remaining uplink handle critical traffic after a failure?
Is the stacking fabric capable of carrying traffic from members without direct uplinks?
Are interface discards, queue congestion, and peak utilization being monitored?
Can additional uplinks be added later without major redesign?

8. Use a Calculator to Validate the Design

You can explore switch port capacity, uplink bandwidth, and oversubscription scenarios using the free engineering tools at:

Switch Uplink Bandwidth & Oversubscription Calculator

A calculator can help compare different uplink configurations, but final designs should always consider actual traffic measurements and equipment capabilities.

Final Recommendation

For a four-member enterprise access stack, I would generally begin by evaluating two appropriately sized, redundant uplinks.

I would not automatically deploy eight uplinks simply because the stack contains four switches.

The right decision depends on actual traffic demand, acceptable oversubscription, distribution architecture, and N-1 failure requirements.

Design for real traffic and failure conditions, not just the number of ports.

Written by NetEngineerLab — practical tools and technical guides for network engineers.