DEV Community

sujan khadka
sujan khadka

Posted on

MAC ADDRESS FLAPPING: A Troubleshooting Guide: Follow the MAC, Find the Root Cause

#cisco #networking #debugging #networkengineering

1. The Incident: Recognizing the Symptom

When a network begins to degrade, the first indicator is often a rapid-fire sequence of syslog messages. As a Senior TAC Engineer, I can tell you: ignore these at your own peril.

-- HIGH VELOCITY MOVEMENT DETECTED --

*Apr 26 12:27:55: %SW_MATM-4-MACFLAP_NOTIF:
Host [MAC: AA:AA:AA:AA:AA:AA] in vlan 10 is flapping
between port [Gi1/0/10] and port [Gi1/0/20]

*Apr 26 12:27:56: %SW_MATM-4-MACFLAP_NOTIF:
Host [MAC: AA:AA:AA:AA:AA:AA] in vlan 10 is flapping
between port [Gi1/0/20] and port [Gi1/0/10]
Enter fullscreen mode Exit fullscreen mode

Is This Automatically a Layer-2 Loop?

CRITICAL WARNING: MAC movement ≠ automatically an L2 loop.

The Cisco syslog message %SW_MATM-4-MACFLAP_NOTIF triggers when a switch learns the same source MAC in the same VLAN on two different interfaces within a short interval.

It is a notification of instability in the Forwarding Database (FDB). That instability may be caused by a loop, but it can also result from other topology or configuration conditions.

2. Fundamentals: How MAC Learning Works

Catalyst switches operate as transparent bridges. They dynamically build the MAC address table by inspecting the Source MAC of incoming Ethernet frames.

The Learning Process

  1. Host A sends a frame.
  2. The switch inspects the Source MAC.
  3. The Source MAC is used for reachability and MAC learning.
  4. The Destination MAC is used for forwarding decisions, not learning.
  5. The switch maps the Source MAC to the receiving interface in the FDB.

Basic MAC Learning 

        [ Host A ]
        MAC: aaaa
             |
             v
      [ Gi1/0/1 ]
             |
       +-------------+
       |   Switch    |
       |             |
       | FDB:        |
       | aaaa ->     |
       | Gi1/0/1     |
       +-------------+
Enter fullscreen mode Exit fullscreen mode

Technical Rule

A MAC address is normally associated with one interface at a time within a VLAN.

If the same MAC appears on Port B while it is currently associated with Port A, the switch updates the entry and associates the MAC with Port B.

MAC Table Update: Before vs. After

State Source MAC Interface VLAN Logic
Initial aaaa.aaaa.aaaa Gi1/0/1 10 Standard association.
Update aaaa.aaaa.aaaa Gi1/0/2 10 MAC moves: previous association is replaced.

3. The Spectrum: Learning vs. Movement vs. Flapping

You cannot troubleshoot a flap if you cannot distinguish normal client movement from a genuine forwarding problem.

1. MAC Learning

The standard, healthy association of a source MAC with a physical or logical interface.

2. MAC Movement

A learned MAC is subsequently observed on another interface.

This can be completely normal in environments such as wireless networks, where a client moves between access points.

3. MAC Flapping

Repeated, high-frequency movement of the same MAC between interfaces.

When this occurs multiple times per second, it warrants investigation for conditions such as Layer-2 loops, duplicate MAC addresses, or configuration problems.

4. Link Flapping

A physical Layer-1 or logical Layer-2 interface repeatedly transitions between Up and Down states.

A flapping link can cause MAC-table changes, but link flapping and MAC flapping are different symptoms.

The Golden Rule: MAC flapping is a symptom, not the root cause.
The log tells you where the MAC is moving; the topology explains why.

4. Root Cause Analysis: The Classic Layer-2 Loop

A Layer-2 loop occurs when redundant paths exist without an effective mechanism to block one of those paths.

Unlike Layer-3 packets, Ethernet frames do not have a TTL field. A broadcast or unknown-unicast frame can therefore circulate indefinitely when a physical loop exists.

The Redundant Triangle Topology 

                 [ Switch A ]
                 (Root Bridge)
                  /         \
                 /           \
        [ Switch B ] ------- [ Switch C ]
           (Path 1)             (Path 2)
Enter fullscreen mode Exit fullscreen mode

Mechanics of the Loop and TCNs

  1. Frame Entry: Host A sends a broadcast frame.
  2. Re-entry: Because of the loop, a switch can receive traffic that has circulated back through another path.
  3. Topology Change: Spanning Tree Protocol (STP) detects topology changes and generates Topology Change Notifications (TCNs) where applicable.
  4. MAC Aging: A topology change can cause MAC entries to age or be flushed more aggressively, depending on the STP implementation and mode.
  5. Instability: Frequent MAC-table changes cause additional flooding and relearning, increasing traffic and potentially driving CPU or backplane utilization higher.

The important point is that STP-related table changes can amplify the symptoms of an existing topology problem. They are not necessarily the original cause.

5. Beyond the Loop: Alternative Root Causes

In the TAC, we see non-loop MAC flaps regularly. Always broaden the investigation beyond the obvious.

Wireless Roaming

A client can legitimately move between access points, causing its MAC address to appear on different switch interfaces.

If the movement occurs seconds or minutes apart, roaming may be perfectly normal. If it occurs multiple times per second, investigate further.

Duplicate Virtual MACs

In active/standby environments such as HSRP or VRRP, a failure or split-brain condition can cause two devices to claim the same virtual MAC.

The switch may then observe that MAC alternately arriving from two different physical ports.

EtherChannel Misconfiguration

If one side of a link is operating as an EtherChannel while the other side fails to negotiate or form the bundle correctly, individual links can remain operational independently.

The switch may then observe the same MAC through different physical interfaces that were intended to operate as a single logical bundle.

Unmanaged Switch Loops

An unmanaged switch can introduce a loop into the network without providing the expected STP controls.

A common example is an unmanaged switch being connected to the network through multiple ports, creating a physical loop that must be isolated or corrected.

Faulty Hardware

Failing transceivers, damaged fiber, or unidirectional links can create unexpected STP behavior.

For example, if a switch can transmit but cannot properly receive BPDUs, a port may transition into a forwarding state when it should remain blocked.

6. The Tactical Troubleshooting Workflow

If you are in the middle of a MAC-flap storm, do not reboot the switch immediately.

A reboot may temporarily clear the symptom while destroying valuable evidence needed to identify the root cause.

Follow this search-and-destroy sequence.

1. Identify the MAC

Locate the flapping MAC address and VLAN in the logging output.

show logging
Enter fullscreen mode Exit fullscreen mode

Record:

  • MAC address
  • VLAN
  • Interfaces involved
  • Frequency of the messages
  • Timestamp

2. Analyze Traffic Rates — Your Primary Metric

During a high-velocity loop, the MAC table can become unreliable.

Inspect interface traffic rates:

show interfaces | include is up|input rate
Enter fullscreen mode Exit fullscreen mode

Look for interfaces receiving unusually high packet rates.

A port receiving hundreds of thousands or millions of packets at an unexpected rate deserves immediate attention.

3. Trace the MAC

Follow the MAC through the network:

show mac address-table address aaaa.aaaa.aaaa
Enter fullscreen mode Exit fullscreen mode

Repeat the process on each relevant switch to determine where the MAC is being learned.

4. Map Your Neighbors

Use CDP to identify the device connected to a suspicious interface:

show cdp neighbors detail
Enter fullscreen mode Exit fullscreen mode

For environments using LLDP, also consider:

show lldp neighbors detail
Enter fullscreen mode Exit fullscreen mode

5. Check for Topology Changes

Inspect STP topology-change information:

show spanning-tree detail | include ieee|occur|from|is
Enter fullscreen mode Exit fullscreen mode

Frequent topology changes can help confirm that the network is experiencing repeated STP events.

Senior TAC Tip: Input packet rate is your "north star."
If an interface has no known legitimate neighbor and is receiving an abnormally high packet rate, investigate or isolate it. If the MAC-flap messages stop after isolating that interface, you have a strong lead toward the source.

7. Network Hardening and Prevention

Feature Technical Purpose
STP PortFast Allows immediate forwarding for end-host ports and prevents unnecessary STP transitions during normal host connectivity changes.
BPDU Guard Places a PortFast-enabled interface into an error-disabled state when an unexpected BPDU is received, helping detect unauthorized switches.
Root Guard Prevents an unauthorized downstream switch from becoming the STP Root Bridge.
UDLD Aggressive Detects certain unidirectional link failures and can place affected ports into an error-disabled state.
Storm Control Suppresses excessive broadcast, multicast, or unknown-unicast traffic during traffic storms.

These features do not replace proper topology design. They provide additional protection when correctly deployed.

8. Summary Checklist

  • [ ] Check the rate: Is the MAC moving multiple times per second, or minutes apart?
  • [ ] Follow the PPS: Run show interfaces | include input rate and identify interfaces with unusually high packet rates.
  • [ ] Trace the MAC: Use show mac address-table to follow the MAC hop-by-hop.
  • [ ] Audit the channel: Verify show etherchannel summary and confirm that intended bundles have formed correctly.
  • [ ] Check STP: Verify the expected Root Bridge and investigate frequent topology changes.
  • [ ] Inspect suspicious endpoints: Look for unmanaged switches, duplicate devices, cabling loops, and unexpected bridges.
  • [ ] Verify hardening: Confirm that PortFast, BPDU Guard, Root Guard, UDLD, and Storm Control are appropriately deployed.

Final Reminder

A MAC-flap message tells you where the MAC is moving.

Your job is to determine why it is moving.

Do not simply clear the MAC table and wait for the logs to disappear. A cleared table only resets the symptom temporarily.

Follow the MAC. Map the topology. Find the root cause. Fix the tree.

Top comments (0)