Wi-Fi Was Dead for 5 Hours While Chrome Kept Working: 4,960 DNS Failures and the 5-Minute Probe That Catches Them

Last time I wrote about how many jobs you should re-run the moment your quota comes back. This post goes back a few days earlier, to a quieter and far more annoying problem: the morning my home Wi-Fi was effectively dead for five hours — while the browser kept loading pages as if nothing were wrong.

The problem: the browser works, but only the jobs die

On the morning of 2026-09-12, four unattended jobs failed one after another.

Job	Time
com.lily.es-daily-rows	05:01
com.lily.paid-note-pin-guard	07:20
com.lily.line-column-gen	08:40
com.lily.line-pdca	08:41

The errors were Node fetch failed and python getaddrinfo failures. Yet during that same window, browsing in Chrome worked perfectly. This is the worst kind of failure: "the network is up, but some processes die anyway."

Digging into the unified log, airportd had been emitting SlowWiFiDnsFailure at an abnormal rate. In the last 24 hours there were 4,960 of them. They ramped up around 05:43, ran at 1,000–1,700+ per hour through the 06:00–11:00 window, peaked at 1,720 in the 10 o'clock hour, and stopped cold after 12:38. The human action that ended it came at 13:09: I manually switched from home Wi-Fi to my phone's tethering. After the switch, 37 probe runs showed failed: 0, with lookups under 60ms and connects under 100ms on both IPv4 and IPv6.

The culprit was that only the DNS path was dead. Chrome has its own DNS resolution (DoH plus a cache), so it sails on even when the home router's DNS is rotten. Node and Python go straight to the system resolver — the home router's DNS — and get stuck. The intuition that "the browser works, so the network is fine" was itself the diagnostic trap.

Pitfalls I hit (diagnosis edition)

• I almost misread a symlink's mtime as "we've been on tethering for three weeks" → /etc/resolv.conf is a symlink, and its mtime still said Aug 15. Checking the real file, /var/run/resolv.conf, its mtime matched the switchover time (13:09:39) exactly, so the first reading was wrong. You always have to know whether you're looking at the symlink's mtime or the target's.
• Unified log retention windows differ per facility → configd only had entries from 05:54 onward, while airportd went back to 19:00 the previous day. I thought I was comparing two facilities on the same timeline, but one of them had already dropped data.
• The name SlowWiFiDnsFailure is misleading → It's logged as "slow," not "dead," so even grepping for anomalies requires a threshold decision.
• Node fetch failed appeared 343 times in that same 24 hours. The apparent gap between the job failure times (05:01–08:41) and the start of the airportd burst (05:43 onward) is also explained by the difference in facility retention.

Note
On that day I was able to dig through the unified log afterward and pin down "roughly 5,000 events between 06:00 and 12:38." But that only worked because the incident happened to be large enough to stand out. If the same thing happens next time for a shorter period or at lower frequency, it may be past the unified log's retention window and leave nothing behind. What I needed was a way to trace root causes that doesn't depend on "digging through the log after the fact" — and that's the real subject of this post.

Never chasing it after the fact again: net-probe.sh

What I built is a script that does one thing: every 5 minutes it appends a one-line JSONL "snapshot of the network state." It uses neither Chrome nor claude, and the target is to finish in under 20 seconds.

#!/bin/bash
# 5分毎の軽量ネットワークプローブ（Chrome/claude 不使用・目標20秒以内)。
# gateway / SSID / nameserver / dig 3ホスト / curl -4,-6 2ホスト / python getaddrinfo を
# 1行JSONで ~/.claude/logs/net-probe.jsonl へ追記する。
# 目的: Node fetch failed / python getaddrinfo 失敗が「どの回線・どのDNSで」起きたかを後から突合する。

The key is to fire everything in parallel. Fetching the SSID via system_profiler alone takes 5 seconds, so it gets pushed into the background alongside the other collection.

# --- SSID は system_profiler が5秒かかるので並列 ---
( system_profiler SPAirPortDataType 2>/dev/null | awk '/Current Network Information:/{getline; gsub(/^ +| *:$/,""); print; exit}' > "$TMP/ssid" ) &

# --- DNS: システム順序で3ホスト + 各NSで oauth2 (すべて並列, 2秒×1回) ---
for h in oauth2.googleapis.com discord.com note.com; do
  ( dig +time=2 +tries=1 "$h" A > "$TMP/dig.$h" 2>&1 ) &
done
i=0
for ns in ${NS//,/ }; do
  ( dig +time=2 +tries=1 @"$ns" oauth2.googleapis.com A > "$TMP/digns.$i" 2>&1; echo "$ns" > "$TMP/digns.$i.ns" ) &
  i=$((i+1))
done

dig uses +time=2 +tries=1 so it always gives up at 2 seconds, curl uses --max-time 6, and Python's getaddrinfo runs in a thread with join(5) to force a 5-second cutoff. Every timeout has to be explicit, because when DNS is truly dead, the probe itself would hang and miss the next 5-minute slot.

def f():
    t = time.time()
    try:
        r = socket.getaddrinfo('oauth2.googleapis.com', 443)
        res.update(ok=True, n=len(r), ms=round((time.time()-t)*1000))
    except Exception as e:
        res.update(ok=False, err=type(e).__name__+': '+str(e), ms=round((time.time()-t)*1000))
th = threading.Thread(target=f, daemon=True); th.start(); th.join(5)
if not res: res.update(ok=False, err='timeout>5s', ms=5000)

Once wait confirms all the fragments are in, the one-line JSON is assembled in Python rather than bash. Hand-building JSON via string concatenation breaks the moment an SSID contains a space or a special character, so this one part is left to json.dumps.

gw = os.environ['GW']
fail = sum(1 for d in dns.values() if d['status'] != 'NOERROR') + sum(1 for h in http.values() if h['http'] == 0) + (0 if py.get('ok') else 1)
rec = {
  'ts': os.environ['TS'], 'gateway': gw, 'gateway6': os.environ['GW6'], 'iface': os.environ['IFACE'],
  'ip4': os.environ['IP4'], 'ssid': rd(f'{T}/ssid').strip(), 'security': os.environ['SEC'],
  'isIphoneHotspot': gw == '172.20.10.1', 'resolvers': [x for x in os.environ['NS'].split(',') if x],
  'utun': int(os.environ['UTUN'] or 0), 'dns': dns, 'dnsPerResolver': per_ns, 'http': http,
  'pyGetaddrinfo': py, 'failures': fail,
}

isIphoneHotspot is determined by whether the gateway IP is 172.20.10.1 (the default address for iPhone tethering). Instead of eyeballing the gateway IP every time, this single field tells you at a glance which connection you're on. failures is the sum of dns / http / pyGetaddrinfo failures, so pulling anomalous lines out of the JSONL is just grep '"failures":[1-9]'.

launchd: run it quietly every 5 minutes

StartInterval gives a fixed-interval launch, and the priority is lowered so it stays strictly in the background.

<key>StartInterval</key>
<integer>300</integer>
<key>LowPriorityIO</key>
<true/>
<key>Nice</key>
<integer>10</integer>
<key>ProcessType</key>
<string>Background</string>

Nice 10 plus LowPriorityIO ensures it never steals CPU or IO from the other automation jobs. A job whose purpose is to watch whether the network is alive should not itself become a source of resource contention.

Four days of real measurements

From deployment until today, it has accumulated 662 lines.

$ wc -l ~/.claude/logs/net-probe.jsonl
662

Broken down by gateway: home Wi-Fi (192.168.3.1) accounts for 594 entries, iPhone tethering (172.20.10.1) for 68. Lines matching "failures":[1-9] number 50 — about 7.6% of 662. Some days had zero, while the hour 2026-09-13T09 alone had 5 failures clustered together — unevenness that would have been flattened out without 5-minute granularity.

A single line looks like this (excerpt from the actual log).

{"ts":"2026-09-16T20:28:45+0900","gateway":"192.168.3.1","iface":"en0",
 "security":"WPA2_PSK","isIphoneHotspot":false,
 "dns":{"oauth2.googleapis.com":{"status":"NOERROR","ms":17,"answers":5,"err":null},
        "discord.com":{"status":"NOERROR","ms":17,"answers":11,"err":null},
        "note.com":{"status":"NOERROR","ms":15,"answers":8,"err":null}},
 "http":{"v4:discord.com":{"http":200,"lookup":0.004285,"connect":0.013422,"total":0.115357}, ...},
 "pyGetaddrinfo":{"ok":true,"n":4,"ms":107},"failures":0}

Cross-referencing: reconstructing when, which connection, and which DNS after the fact

The network-flap-probe-2026-09-12.json I put together right after the incident still holds verdict: "dns-resolver-flap" along with the explanation of the cause, exactly as written that day (in Japanese).

"verdictDetail": "症状の発生源は自宅Wi-Fiの DNS 経路。airportd が 06:00-12:38 に
SlowWiFiDnsFailure を約5,000件（毎時1,000-1,700件）記録しており、failing 4 ジョブと
fetch failed 343件はこの回線上で発生。13:09に人間が iPhone テザリングへ手動切替した後は
fault 0・全プローブ成功。Chrome 経由ジョブが通っていたのは Chrome が独自 DNS
(DoH/非同期リゾルバ+キャッシュ)を使い、Node/pythonはシステムリゾルバ
（自宅ルータ DNS）を直撃するため"

In short, it says: the source of the symptoms was the home Wi-Fi's DNS path; airportd recorded roughly 5,000 SlowWiFiDnsFailure events (1,000–1,700 per hour) between 06:00 and 12:38; the 4 failing jobs and the 343 fetch failures all occurred on that connection; after the manual switch to iPhone tethering at 13:09, faults dropped to 0 and every probe succeeded; and the Chrome-based jobs kept working because Chrome uses its own DNS (DoH / async resolver + cache) while Node and Python hit the system resolver (the home router's DNS) directly.

That day I dug all the way there by hand. With net-probe.jsonl, the same conclusion can be reached mechanically, at 5-minute granularity. The gateway field records the connection, and the dnsPerResolver field records how each individual DNS server responded to its query, so "when, on which connection, against which DNS server" can be reconstructed just by filtering fields. Whether a manual reproduction step (rebooting the router, pinning DNS to 1.1.1.1/8.8.8.8) is even needed can be decided case by case from this record.

Summary

• Wi-Fi can be "connected" while only the DNS path is dead. Chrome carrying on as normal is because it has its own DNS cache — it is not evidence of health.
• If root-cause tracing depends on "digging through the unified log after the incident," you run into traps like per-facility retention differences and misreading a symlink's mtime.
• net-probe.sh is a thin mechanism that fires gateway / SSID / DNS / HTTP (v4, v6) / getaddrinfo all in parallel with timeouts and appends one JSONL line in under 20 seconds.
• With launchd's StartInterval=300 + Nice 10 + LowPriorityIO, it runs quietly every 5 minutes without competing with other jobs.
• 662 lines in 4 days, 50 of them with failures>0 (7.6%). With this accumulated, the next time the same thing happens, reaching the cause means "filter the fields," not "dig after the fact."

Using this JSONL, I'd like to grow it to the point where a detected anomaly automatically notifies Discord and, if needed, switches connections automatically. That's for another post.

Have you ever had a job fail while the browser insisted everything was fine — and how long did it take you to find out DNS was the reason?

---

Written by **Lily* — I ship iOS apps and automate my content stack with Claude Code.
Follow along: Portfolio · X · GitHub*