LinkedIn Hook

You ran ping google.com and it replied. Congrats — you know someone is home.

But you have no idea which roads you took to get there, which intersection caused the 200ms delay, or which city the traffic jam was in.

That's the difference between ping and traceroute.

Traceroute doesn't just ask "are you there?" — it maps every single router hop between you and the destination, with the round-trip time for each one.

It uses a clever trick: the TTL field. Every packet has a Time To Live — a number that decrements by 1 at each router. When TTL hits 0, the router drops the packet and sends back an ICMP "Time Exceeded" message with its own IP address. By sending packets with TTL=1, then TTL=2, then TTL=3... you force every router on the path to reveal itself.

Reading traceroute output is a skill. Here's what to look for:

Hop 1 is always your local gateway (your home router)

Private IPs (192.168.x, 10.x) are inside your LAN or your ISP's internal network

A latency jump that stays high = real bottleneck, investigate it

A latency spike that drops back down = ICMP deprioritization, not a real problem

* * * does NOT mean the network is broken — it means a router is blocking ICMP responses

And if you want continuous monitoring instead of a one-shot snapshot? Run mtr — it combines ping and traceroute into a live updating display that shows packet loss per hop over time.

Interviewers love traceroute. It tests whether you understand IP, ICMP, TTL, routing, and debugging methodology all at once.

Read the full lesson →

#Networking #SysAdmin #DevOps #CloudEngineering #InterviewPrep #Traceroute #TCP #NetworkDiagnostics

Traceroute thumbnail

What You'll Learn

Why traceroute exists and what problem it solves that ping cannot
The incrementing TTL trick — the exact mechanism that forces routers to reveal themselves
How to read every field of real traceroute output, including * * *
The difference between traceroute (Unix/Linux/Mac) and tracert (Windows)
How to identify a real network bottleneck vs a false alarm
What mtr is and why it is more powerful than traceroute for intermittent issues
The most useful traceroute flags for interviews and real diagnostic work

The Analogy: Every Street, Every Intersection

Ping tells you whether someone is home. You knock, they answer, you measure how long it took.

Traceroute tells you everything that happened between the knock and the answer — every street you drove down, every intersection you passed through, every city you crossed, and how long each stretch of road took.

If your ping round-trip time is 300ms but it should be 30ms, ping cannot tell you where the problem is. Traceroute can. It identifies the exact hop in the network path where latency is introduced.

This makes it the first tool a network engineer reaches for when debugging slow connections, partial outages, or routing problems.

What Traceroute Does

Traceroute maps the network path between your machine and a destination by discovering every router (hop) along the way. For each hop it reports:

The router's IP address (and hostname if DNS resolves it)
The round-trip time (RTT) for three probe packets sent to that hop
Whether the hop responded at all

By the time traceroute finishes, you have a complete picture of the route your packets travel, and the latency contributed by each segment of that route.

How Traceroute Works — The Incrementing TTL Trick

This is the mechanism interviewers ask about most often. Understand it completely.

The TTL Field

Every IP packet has a field called Time To Live (TTL). It is an integer (typically starting at 64, 128, or 255 depending on the OS). Every router that forwards the packet decrements TTL by 1. When TTL reaches 0, the router:

Drops the packet
Sends an ICMP "Time Exceeded" message back to the source with its own IP address

This is an error-reporting mechanism built into IP to prevent packets from looping forever. Traceroute exploits it deliberately.

The Incrementing TTL Strategy

Your Machine          Router 1          Router 2          Router 3     Destination
     |                    |                 |                 |              |
     |-- TTL=1 ---------->|                 |                 |              |
     |   (TTL hits 0)     |                 |                 |              |
     |<-- ICMP Time Exceeded (Router 1 IP) -|                 |              |
     |                    |                 |                 |              |
     |-- TTL=2 ---------->|-- TTL=1 ------->|                 |              |
     |                    |   (TTL hits 0)  |                 |              |
     |<-------- ICMP Time Exceeded (Router 2 IP) -------------|              |
     |                    |                 |                 |              |
     |-- TTL=3 ---------->|-- TTL=2 ------->|-- TTL=1 ------->|              |
     |                    |                 |   (TTL hits 0)  |              |
     |<---------------- ICMP Time Exceeded (Router 3 IP) -----|              |
     |                    |                 |                 |              |
     |-- TTL=4 ---------->|-- TTL=3 ------->|-- TTL=2 ------->|-- TTL=1 --->|
     |                    |                 |                 |             |
     |<----------------------- ICMP Echo Reply / Port Unreachable -----------|
     |                    |                 |                 |              |

Step by step:

TTL=1: First packet is sent with TTL=1. The very first router in the path decrements it to 0, drops the packet, and sends back an ICMP Time Exceeded message. Traceroute records that router's IP and the time taken.
TTL=2: Second packet is sent with TTL=2. The first router decrements to 1 and forwards it. The second router decrements to 0, drops it, and sends back ICMP Time Exceeded. Traceroute records the second router.
TTL=3, 4, 5...: The pattern repeats. Each iteration discovers one more hop.
Destination reached: When the packet finally arrives at the destination with TTL still > 0, the destination responds with an ICMP Echo Reply (if using ICMP mode) or an ICMP Port Unreachable (if using UDP mode, because no service is listening on those high ports). Traceroute recognizes this as the final hop.

This is clever because routers that would otherwise silently forward packets are forced to reveal themselves. The only way to stay hidden is to not send the ICMP Time Exceeded reply — which is what firewalls do, producing the * * * output.

Three Probes Per Hop

Traceroute sends 3 probe packets per hop by default. This gives you three RTT measurements for each router, which lets you:

Average out minor fluctuations
Detect variance (jitter) at that hop — if the three values differ significantly, that hop is inconsistent
Spot if one probe was lost (a * replaces it)

Unix/Linux/Mac vs Windows

Feature	`traceroute` (Unix/Linux/Mac)	`tracert` (Windows)
Default protocol	UDP (port 33434+)	ICMP Echo Request
ICMP mode	`-I` flag	Always ICMP
Output format	Similar, minor differences	Similar, minor differences
Max hops default	30	30
DNS resolution	Yes (disable with `-n`)	Yes (disable with `-d`)

Why does the default protocol matter?

On Unix, traceroute sends UDP packets to high, unused port numbers. The destination sends back ICMP Port Unreachable (because nothing is listening) to signal arrival. Some firewalls block UDP differently than ICMP — so sometimes traceroute -I (ICMP mode) gets through where UDP does not, or vice versa. Windows always uses ICMP, matching the behavior of traceroute -I.

Reading Traceroute Output — Line by Line

Here is real traceroute output. Read every field carefully.

traceroute to google.com (142.250.80.46), 30 hops max, 60 byte packets
 1  192.168.1.1 (192.168.1.1)        1.234 ms   1.198 ms   1.102 ms
 2  10.0.0.1 (10.0.0.1)              8.432 ms   8.201 ms   8.367 ms
 3  * * *
 4  72.14.215.165 (72.14.215.165)   14.123 ms  13.987 ms  14.234 ms
 5  142.250.80.46 (142.250.80.46)   12.456 ms  12.321 ms  12.567 ms

Header Line

traceroute to google.com (142.250.80.46), 30 hops max, 60 byte packets

google.com (142.250.80.46) — destination hostname and its resolved IP
30 hops max — traceroute will give up after 30 hops if destination is not reached
60 byte packets — size of each probe packet

Hop 1 — Your Local Gateway

 1  192.168.1.1 (192.168.1.1)  1.234 ms  1.198 ms  1.102 ms

1 — hop number
192.168.1.1 — the router's hostname (same as IP here, no PTR record)
(192.168.1.1) — the router's IP address
1.234 ms 1.198 ms 1.102 ms — RTT for the three probe packets

This is your home router or default gateway. Sub-2ms is normal for a LAN hop. The three measurements are very consistent here, indicating a stable local connection.

Hop 2 — ISP Internal Network

 2  10.0.0.1 (10.0.0.1)  8.432 ms  8.201 ms  8.367 ms

A private IP in the 10.x range. This is inside your ISP's internal network infrastructure — not publicly routable. The jump from ~1ms to ~8ms represents the physical distance from your home to your ISP's nearest point of presence (PoP), traversed over a DSL, cable, or fiber link.

Hop 3 — Silent Hop (Firewall Blocking ICMP)

 3  * * *

All three probes timed out. This router is either:

Configured to not send ICMP Time Exceeded responses (common on ISP backbone routers for security or performance reasons)
Firewalled off from sending ICMP outbound
Very slow to respond and exceeded the wait timeout

Critically: this does NOT mean the network is broken here. The packet still passed through this router — it just did not announce itself. The evidence is that Hop 4 responded normally. If the packet had truly been dropped at Hop 3, Hop 4 would never have received it.

Hop 4 — Public Internet Backbone

 4  72.14.215.165 (72.14.215.165)  14.123 ms  13.987 ms  14.234 ms

A public IP now — you are in the public internet backbone (72.14.x.x is a Google infrastructure range). Latency has increased to ~14ms, reflecting the geographical distance covered.

Notice something interesting: Hop 4 has higher latency than Hop 5. This can happen because ICMP Time Exceeded replies are generated by the router's control plane (lower priority), while regular forwarding happens in the data plane. The latency you see at each hop is "from your machine to that hop and back" — not the per-segment cost. More on this below.

Hop 5 — Destination Reached

 5  142.250.80.46 (142.250.80.46)  12.456 ms  12.321 ms  12.567 ms

The destination IP. Latency is slightly lower than Hop 4 — this is normal (ICMP deprioritization at Hop 4 made that measurement artificially higher). Traceroute stops here because it received an ICMP Echo Reply (ICMP mode) or ICMP Port Unreachable (UDP mode) from the destination itself.

Interpreting the Output — Finding Real Problems

Hop 1 is Always Your Router

If Hop 1 has high latency (>20ms for a wired connection), the problem is inside your LAN — check the cable, WiFi signal, or router performance. Nothing on the internet can fix that.

Private IPs = Local or ISP Internal

Hops showing 192.168.x.x, 10.x.x.x, or 172.16.x.x to 172.31.x.x are inside private address space — either your LAN or your ISP's internal network. You cannot directly contact these from the public internet.

Identifying a Real Bottleneck

Hop  IP                  RTT 1     RTT 2     RTT 3
 1   192.168.1.1          1ms       1ms       1ms    <- LAN, fine
 2   10.0.0.1             8ms       8ms       8ms    <- ISP, fine
 3   * * *                                           <- silent, skip
 4   72.14.215.165       14ms      14ms      14ms    <- backbone, fine
 5   203.0.113.1        280ms     278ms     281ms    <- SPIKE! stays high
 6   198.51.100.5       281ms     279ms     282ms    <- still high
 7   destination        280ms     281ms     280ms    <- destination

Latency jumped from ~14ms at Hop 4 to ~280ms at Hop 5, and it stayed high for all subsequent hops. This is a real bottleneck: something at or between Hop 4 and Hop 5 is introducing ~265ms of latency. This is the hop to investigate — it could be a congested link, a misconfigured route, or a transoceanic cable.

Distinguishing a False Alarm (ICMP Deprioritization)

Hop  IP                  RTT 1     RTT 2     RTT 3
 4   72.14.215.165       14ms      14ms      14ms    <- fine
 5   203.0.113.1        180ms     181ms     179ms    <- SPIKE!
 6   198.51.100.5        15ms      14ms      15ms    <- back to normal
 7   destination         15ms      15ms      14ms    <- destination

Latency spiked at Hop 5 but dropped back to normal at Hop 6 and destination. This is not a bottleneck. What is happening: the router at Hop 5 is deprioritizing ICMP packets in its control plane — it is handling your regular traffic at full speed but adding extra delay to ICMP responses because they are less important than real traffic. The end-to-end latency (measured at the destination) is fine.

Rule: Only trust the destination's latency. A single-hop spike that resolves itself is always ICMP deprioritization.

Asterisks That Continue to the End

 5  * * *
 6  * * *
 7  * * *
...
30  * * *

If asterisks run all the way to the max hop count and never resolve, the destination is either unreachable or actively blocking all ICMP/UDP probes. Compare with a successful ping to the same destination — if ping also fails, the host is down or unreachable. If ping succeeds but traceroute shows all stars, the host is allowing ICMP Echo Reply but blocking Time Exceeded (unusual, but it happens).

mtr — My Traceroute (the Better Tool)

traceroute gives you a snapshot — one measurement in time. For intermittent issues (packet loss that comes and goes, jitter that appears only under load), a snapshot is often useless.

mtr (My Traceroute) solves this by combining traceroute and ping into a continuously updating real-time display. It keeps sending probes to every hop and updates the statistics live.

Installing mtr

# Debian/Ubuntu
sudo apt install mtr

# macOS
brew install mtr

# CentOS/RHEL
sudo yum install mtr

Running mtr

mtr google.com

Sample mtr Output

                             My traceroute  [v0.95]
hostname (192.168.1.50)                          2024-01-15T10:30:00+0000
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
                                         Packets               Pings
 Host                                Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. 192.168.1.1                       0.0%    47   1.2    1.1   0.9   1.8   0.2
 2. 10.0.0.1                          0.0%    47   8.3    8.4   7.9  11.2   0.5
 3. ???                                                                         
 4. 72.14.215.165                      2.1%    47  14.1   14.3  13.8  18.9   0.8
 5. 142.250.80.46                      0.0%    47  12.4   12.5  12.0  15.1   0.4

Reading mtr Columns

Column	Meaning
`Loss%`	Percentage of probes lost at this hop — 0% is normal
`Snt`	Total probes sent so far
`Last`	RTT of the most recent probe (ms)
`Avg`	Average RTT across all probes
`Best`	Minimum RTT seen
`Wrst`	Maximum RTT seen — reveals the worst-case spikes
`StDev`	Standard deviation — high value means inconsistent (jitter)
`???`	Same as `* * *` — hop is not responding

Why mtr is More Powerful

In the output above, Hop 4 shows 2.1% packet loss. A single traceroute snapshot would likely not have caught this — the three probes might have all succeeded by chance. But over 47 probes, mtr reveals that one in every ~47 packets is being dropped at that hop. Intermittent packet loss like this causes TCP retransmits, stuttering in video calls, and performance degradation that is very hard to diagnose without mtr.

Use traceroute for a quick check. Use mtr for anything intermittent.

Useful Flags

traceroute (Linux/Mac)

# Basic usage
traceroute google.com

# Show IPs only — skip DNS reverse lookups (much faster)
traceroute -n google.com

# Use ICMP instead of UDP (useful when UDP is blocked)
traceroute -I google.com

# Set max hops (default 30)
traceroute -m 20 google.com

# Change number of probes per hop (default 3)
traceroute -q 5 google.com

# Set wait time per probe in seconds (default 5)
traceroute -w 2 google.com

# Combine flags: ICMP, no DNS, max 20 hops
traceroute -I -n -m 20 google.com

tracert (Windows)

:: Basic usage
tracert google.com

:: Skip DNS resolution (equivalent to -n)
tracert -d google.com

:: Set max hops
tracert -h 20 google.com

:: Set timeout per reply in milliseconds
tracert -w 2000 google.com

mtr

# Basic (interactive, live updating)
mtr google.com

# Report mode (non-interactive, prints once and exits — good for logging)
mtr --report google.com

# No DNS lookups
mtr -n google.com

# Send N packets and generate report
mtr --report-cycles 100 google.com

# Use TCP SYN instead of ICMP (bypasses many firewalls)
mtr --tcp --port 443 google.com

Complete Traceroute Workflow for Debugging

When a user reports "the website is slow" or "I can't reach the server," here is the diagnostic approach using traceroute:

Step 1: Run ping first
   ping -c 4 destination.com
   → Is the destination reachable at all? What is the baseline RTT?

Step 2: Run traceroute
   traceroute -n destination.com
   → Where does the path go? Is it a normal route?

Step 3: Identify the pattern
   → All asterisks at the end?     → Host may be unreachable
   → Spike that stays high?        → Real bottleneck at that hop
   → Spike that drops back?        → ICMP deprioritization, ignore it
   → Hop 1 is already high?        → Local LAN problem

Step 4: For intermittent issues, run mtr
   mtr -n destination.com
   → Watch Loss% per hop
   → Watch StDev for jitter
   → Run for at least 50-100 cycles

Step 5: Determine scope
   → Problem in your LAN (Hop 1)?        → Your equipment
   → Problem in ISP (Hop 2-3)?           → Call your ISP
   → Problem in backbone (Hop 4+)?       → Out of your control, route around it
   → Problem at destination?             → Contact the destination's ops team

Common Mistakes

Thinking * * * means the network is broken at that hop. It almost never does. A * * * hop simply means the router is not sending ICMP Time Exceeded responses — usually by design or firewall policy. If subsequent hops respond normally, traffic is flowing fine through that router. The packet is not being dropped; only the ICMP reply is suppressed. Always check whether hops after the asterisks respond before concluding there is a problem.
Treating a single-hop latency spike as a real bottleneck when latency drops back to normal at the next hop. If Hop 5 shows 200ms but Hop 6 shows 15ms, Hop 5 is not the bottleneck — it is just deprioritizing ICMP replies in its control plane. The data plane is forwarding your real traffic at full speed. Only trust latency that persists from the spike point all the way to the destination.
Using traceroute alone when mtr would give better data. Traceroute is a snapshot — three probes per hop, one measurement in time. For intermittent packet loss, jitter, or issues that appear only under sustained load, mtr is far more useful. It sends continuous probes to every hop and builds statistics over time, revealing the 2% packet loss or 50ms jitter spikes that a single traceroute run would almost certainly miss.

Interview Questions

Q1: How does traceroute discover each hop in the path?

Traceroute exploits the TTL (Time To Live) field in IP packets. It sends a packet with TTL=1 — the first router decrements TTL to 0, drops the packet, and sends back an ICMP Time Exceeded message with its own IP address. Traceroute records that IP and the RTT, then sends TTL=2 to discover the second hop, TTL=3 for the third, and so on. By incrementing the TTL by 1 each round, it forces every router on the path to identify itself through ICMP replies.

Q2: What does * * * mean in traceroute output?

It means all three probe packets for that hop timed out — no ICMP Time Exceeded response was received. This typically means the router at that hop is configured not to send ICMP Time Exceeded messages, either by firewall policy or administrative choice. It does NOT necessarily mean the network is broken at that point. If subsequent hops respond normally, the packet is passing through that router just fine; only the ICMP reply is being suppressed. Only if * * * continues to the maximum hop count does it suggest an unreachable destination.

Q3: What is the difference between traceroute and tracert?

traceroute is the Unix/Linux/Mac command; tracert is the Windows equivalent. The key technical difference is the default protocol: traceroute uses UDP packets sent to high port numbers (33434+) by default — the destination signals arrival with ICMP Port Unreachable. tracert uses ICMP Echo Request packets by default, the same as ping. You can make traceroute behave like tracert using the -I flag (ICMP mode). The output format is functionally similar between the two.

Q4: How would you use traceroute to identify a network bottleneck?

Run traceroute -n destination.com and examine the RTT values at each hop. A real bottleneck appears as a hop where latency suddenly increases by more than 50ms and remains elevated at all subsequent hops including the destination. That is the segment where the congestion or problem is occurring. By contrast, if latency spikes at one hop but drops back to normal at the next, it is ICMP deprioritization on that router's control plane — not a real bottleneck. For intermittent issues, use mtr instead of traceroute: it sends continuous probes and shows packet loss percentage per hop over time, which reveals problems that a single traceroute snapshot would miss.

Quick Reference — Cheat Sheet

Traceroute Output Field Breakdown

 4  72.14.215.165 (72.14.215.165)  14.123 ms  13.987 ms  14.234 ms
 ^  ^             ^                 ^           ^           ^
 |  |             |                 |           |           |
 |  Hostname      Resolved IP       Probe 1     Probe 2     Probe 3
 Hop number                         RTT (ms)    RTT (ms)    RTT (ms)

`* * *` Interpretation Guide

Scenario	Meaning	Action
Single `* * *` hop, subsequent hops respond	Router blocking ICMP Time Exceeded	Ignore — not a problem
Multiple consecutive `* * *` hops, then resumes	Multiple ICMP-blocking routers	Ignore — check final RTT
`* * *` all the way to max hops	Destination unreachable or blocking all probes	Check ping; contact destination
`* * *` only on hop 1	Your gateway not responding	Check local network
Increasing `* * *` with some responses	Overloaded or flapping link	Use mtr to monitor over time

traceroute vs tracert vs mtr Comparison

Feature	`traceroute` (Unix/Mac/Linux)	`tracert` (Windows)	`mtr` (Unix/Mac/Linux)
Default protocol	UDP	ICMP	ICMP
ICMP mode	`-I` flag	Default	Default
One-shot or continuous	One-shot	One-shot	Continuous
Shows packet loss over time	No	No	Yes
Shows jitter/StDev	No	No	Yes
Best for	Quick path check	Quick path check (Windows)	Intermittent issues
Non-interactive report	N/A	N/A	`--report` flag
DNS resolution	Yes (skip with `-n`)	Yes (skip with `-d`)	Yes (skip with `-n`)

Key Flags Reference

Flag	traceroute	tracert	mtr	Effect
No DNS lookups	`-n`	`-d`	`-n`	Show IPs only, faster output
Max hops	`-m N`	`-h N`	`-m N`	Stop after N hops (default: 30)
Probes per hop	`-q N`	—	`--report-cycles N`	Number of probe packets
Wait timeout	`-w N`	`-w N` (ms)	—	Wait N seconds per probe
Use ICMP	`-I`	Always	Default	Use ICMP instead of UDP
Use TCP	—	—	`--tcp --port N`	Use TCP SYN (bypasses firewalls)
Report mode	—	—	`--report`	Non-interactive, print and exit

Latency Interpretation at a Glance

Pattern	Meaning
Hop N: low latency → Hop N+1: high, stays high	Real bottleneck between Hop N and N+1
Hop N: low → Hop N+1: spike → Hop N+2: low	ICMP deprioritization at Hop N+1, not a problem
Hop 1: high latency (>20ms wired)	Local LAN or router problem
All hops fine, destination slow	Problem at the server itself
`* * *` then resumes	Router suppressing ICMP — not broken
`* * *` to end	Unreachable or all probes blocked

Previous: Lesson 7.1 — ping Next: Lesson 7.3 — Other Diagnostic Tools

This is Lesson 7.2 of the Networking Interview Prep Course — 8 chapters, 32 lessons.

Traceroute