DEV Community

Aviral Srivastava
Aviral Srivastava

Posted on

Network Troubleshooting Tools (traceroute, mtr, etc.)

#networking #cli #performance #tooling

Network Troubleshooting Tools: A Deep Dive

Introduction

Network troubleshooting is an indispensable skill for network administrators, system engineers, and anyone responsible for maintaining a reliable and performant network infrastructure. When connectivity issues arise, or performance degrades, the ability to quickly identify and diagnose the root cause is crucial. A range of powerful command-line tools exists to aid in this process, providing valuable insights into network paths, latency, packet loss, and potential bottlenecks. This article will delve into the operation and application of some of the most commonly used and effective network troubleshooting tools, specifically focusing on traceroute, mtr (My Traceroute), and related techniques.

Prerequisites

Before diving into the specifics of these tools, it's essential to have a fundamental understanding of networking concepts. This includes:

  • TCP/IP Model: Familiarity with the layers of the TCP/IP model (Application, Transport, Network, Data Link, and Physical) is crucial for understanding how data flows through the network.
  • IP Addressing and Subnetting: Understanding IP addresses (IPv4 and IPv6) and subnet masks is fundamental to identifying devices on the network and their respective networks.
  • Routing Principles: Knowledge of routing protocols (e.g., RIP, OSPF, BGP) and how routers determine the best path for data packets is highly beneficial.
  • Basic Command-Line Interface (CLI) Skills: Familiarity with navigating the command line is essential for executing these tools and interpreting their output.
  • ICMP (Internet Control Message Protocol): Understanding ICMP's role in error reporting and diagnostic messaging (used heavily by tools like ping and traceroute).

Traceroute: Mapping the Network Path

traceroute (or tracert on Windows) is a utility that traces the route a packet takes from your computer to a destination. It achieves this by sending packets with increasing Time-To-Live (TTL) values. The TTL is a field in the IP header that limits the number of hops a packet can take before being discarded.

How Traceroute Works:

  1. traceroute sends a series of packets (typically UDP or ICMP) to the destination, starting with a TTL of 1.
  2. The first router in the path receives the packet, decrements the TTL by 1, and discards the packet because the TTL is now 0.
  3. The router then sends an ICMP "Time Exceeded" message back to the sender (the computer running traceroute). This message contains the IP address of the router.
  4. traceroute repeats this process, increasing the TTL value by 1 for each subsequent set of packets.
  5. Each router along the path sends an ICMP "Time Exceeded" message when the TTL reaches 0.
  6. Eventually, a packet reaches the destination. If using UDP packets, the destination will send an ICMP "Port Unreachable" message. If using ICMP packets (some implementations do this), the destination will respond with a standard ICMP Echo Reply (the same as ping).
  7. traceroute records the IP address and round-trip time (RTT) for each hop.

Advantages of Traceroute:

  • Path Discovery: Reveals the exact path a packet takes to reach its destination, identifying intermediate routers.
  • Bottleneck Identification: High latency or packet loss at a particular hop can indicate a network bottleneck.
  • Routing Loop Detection: If the trace returns to a previously visited router, it can indicate a routing loop.
  • Verification of Routing Configuration: Confirms whether traffic is being routed as expected.

Disadvantages of Traceroute:

  • ICMP Filtering: Routers can be configured to block or rate-limit ICMP traffic, potentially preventing traceroute from completing or providing inaccurate results.
  • Load Balancing: Multiple paths may exist to the same destination, leading to varying routes for different packets.
  • Privacy Concerns: Some organizations may consider traceroute as a form of network reconnaissance and block it.
  • Asymmetric Routing: The path taken from source to destination may differ from the path taken from destination to source.
  • Inaccurate RTT measurements: The ICMP Time Exceeded messages can be rate limited or have lower priority on network devices leading to false high latency values.

Traceroute Usage:

Linux/macOS:

traceroute example.com
Enter fullscreen mode Exit fullscreen mode

Windows:

tracert example.com
Enter fullscreen mode Exit fullscreen mode

Interpreting Traceroute Output:

The output typically shows the hop number, hostname or IP address of the router, and the round-trip time (RTT) in milliseconds for three probes sent to each hop. An asterisk (*) indicates that no response was received from that hop within the timeout period.

Example output:

traceroute to example.com (93.184.216.34), 30 hops max, 60 byte packets
 1  192.168.1.1 (192.168.1.1)  1.234 ms  1.345 ms  1.456 ms
 2  10.0.0.1 (10.0.0.1)  5.678 ms  5.789 ms  5.890 ms
 3  203.0.113.1 (203.0.113.1)  15.234 ms  15.345 ms  15.456 ms
 4  93.184.216.34 (93.184.216.34)  25.567 ms  25.678 ms  25.789 ms
Enter fullscreen mode Exit fullscreen mode

In this example:

  • Hop 1 is your local gateway (router).
  • Hop 4 is the destination server.
  • The RTT increases with each hop, reflecting the added latency of each network segment.

MTR (My Traceroute): Combining Traceroute and Ping

mtr combines the functionality of traceroute and ping into a single, continuously updated display. It provides a dynamic view of the network path and packet loss statistics for each hop. It repeatedly sends packets to each hop along the path, providing real-time information about latency and packet loss. This makes it a powerful tool for identifying intermittent network issues.

Advantages of MTR:

  • Continuous Monitoring: Provides a continuous stream of data, making it easier to detect intermittent issues.
  • Packet Loss Statistics: Tracks packet loss at each hop, indicating potential problem areas.
  • Dynamic Path Visualization: Shows changes in the network path over time.
  • Easy-to-Read Output: Presents data in a clear and concise format.

Disadvantages of MTR:

  • Requires Installation: Unlike traceroute, mtr is not always pre-installed on systems.
  • Can be Resource Intensive: Continuously sending packets can consume network bandwidth and CPU resources.
  • Interpreting Packet Loss: Careful interpretation is required, as packet loss at a single hop does not always indicate a problem with that hop itself. It could indicate an issue on the return path.

MTR Usage:

Linux/macOS:

mtr example.com
Enter fullscreen mode Exit fullscreen mode

You can switch to report mode using the -r flag:

mtr -r example.com
Enter fullscreen mode Exit fullscreen mode

This will output a report after the program has run for a specified amount of time (by default 10 seconds).

Interpreting MTR Output:

The output shows the hop number, hostname/IP address, packet loss percentage, and statistics for each hop (Last, Avg, Best, Worst, StDev).

Example output (terminal UI):

HOST: myhost                      Loss%   Snt   Last   Avg  Best  Wrst StDev
  1. gateway                      0.0%    10   0.5  0.6   0.5   0.7   0.1
  2. 10.0.0.1                    0.0%    10   6.2  6.3   6.1   6.4   0.1
  3. 203.0.113.1                  0.0%    10  15.8 16.0  15.7  16.2   0.2
  4. example.com                0.0%    10  26.1 26.3  26.0  26.5   0.2
Enter fullscreen mode Exit fullscreen mode

Features

Both traceroute and mtr have several configurable options that enhance their utility:

  • -I (ICMP): Use ICMP Echo Request packets instead of UDP. Useful when UDP is blocked. (Linux/macOS traceroute and mtr)
  • -T (TCP): Use TCP SYN packets. This is useful for testing connectivity on specific ports. (Linux/macOS traceroute)
  • -n: Do not resolve hostnames. This can speed up the process. (Linux/macOS traceroute and mtr)
  • -q NUM: Sets the number of queries per hop. The default is usually 3. (Linux/macOS traceroute)
  • -w TIMEOUT: Sets the timeout in seconds for waiting for a response. (Linux/macOS traceroute)
  • -j (Windows tracert): Specifies loose source route along host-list.

Conclusion

traceroute and mtr are invaluable tools for diagnosing network connectivity issues. By understanding how these tools work and how to interpret their output, network administrators can quickly pinpoint problem areas and take corrective action. While these tools have limitations (such as ICMP filtering and the potential for asymmetric routing), they remain essential for network troubleshooting and performance analysis. Mastering these tools is a crucial step in becoming a proficient network troubleshooter. In addition, by combining the use of these tools and other network tools, such as ping, nslookup and tcpdump, administrators can gain a greater picture of the current state of their networks.

Top comments (1)

Collapse
 
hashbyt profile image
Hashbyt •

Great deep dive! I love how you highlight MTR's continuous monitoring—it's absolutely invaluable for spotting those intermittent network issues that plain traceroute misses.