My app's landing page says 'Zero Telemetry'. So I wiretapped my own app to prove it.

A Japanese version of this is on Zenn.

I build and sell a macOS network-security app as a solo dev. The landing page says:

Zero Telemetry — no outbound traffic, no data collection

Words are cheap, and that's just a claim. To a user, this is "an unknown solo dev's app that rewrites your firewall with root, saying 'trust me, I send nothing.'" There's no reason to believe it.

And this app sees sensitive stuff, constantly: the gateway MAC and fingerprint, listening ports, the Wi-Fi SSID you're on, FileVault / SIP / firewall state, files it scanned, URLs it checked. If any of that leaked, it would betray exactly the people who picked it for privacy.

So I decided to turn the claim into a measurement. tcpdump and free tools only — wiretap my own app. This post is the method and the places I tripped.

Why reading the code isn't enough

"The MCP server and detection logic are open source — just read them." A few holes:

1. The main app isn't open source. Only the read-only MCP server and detection logic are public; the privileged helper, pf control, and licensing stay closed (that surface helps attackers). Code alone doesn't cover the whole thing.

2. No App Sandbox. To be upfront: RoamSwitch does not run inside App Sandbox — it controls the firewall and sharing daemons, so it can't. The consequence is that entitlements don't restrict outbound traffic. codesign -d --entitlements showing "no network-client key" is not proof it can't send. You have to show behaviour.

3. Regressions. Even if this build is truly silent, a future version could add an analytics SDK, a crash reporter, a "check for news" feature. A one-time check is worthless — this needs to be a procedure you can re-run every release.

So instead of "read the code," I hand people "run this procedure."

What counts as a pass

The only traffic allowed out is the four paths documented in the whitepaper (§7):

#	destination	process	when
1	lafine.net/api/v1/license/*	RoamSwitch	license activate / deactivate only
2	lafine.net/updates/appcast.xml	RoamSwitch	Sparkle (launch + every 24 h)
3	ClamAV mirrors	freshclam (not the app)	only with ClamAV installed
4	HEAD to a target URL	RoamSwitch	only when the in-app Link Safety sheet is used

Anything else out of RoamSwitch / RoamSwitchHelper / RoamSwitchMCPServer is a fail.

Tools (no Little Snitch)

I don't own Little Snitch, so:

• tcpdump (system) — every packet that leaves the NIC
• tshark (brew install wireshark) — pull destinations and TLS SNI from the pcap
• nettop / lsof (system) — tcpdump doesn't say which process sent a packet; these map socket → process
• LuLu (Objective-See, free, open source) — per-process outbound events, from the unified log

LuLu is a fine Little Snitch substitute, and it's by a well-known name in Mac security — which, for a write-up, only helps.

Trip 1: I didn't need to capture for 25 hours

I'd convinced myself that "Sparkle's periodic check is every 24 h, so I have to capture for 25 h straight to span it." I set up for an overnight run.

Then I looked at Info.plist: SUScheduledCheckInterval = 86400, and SULastCheckTime sits in defaults in the clear. So there's no waiting:

BID=com.tetsuharu.RoamSwitch
defaults delete $BID SULastCheckTime
osascript -e 'quit app "RoamSwitch"'; sleep 2; open -a RoamSwitch
# launch sees "overdue" and checks within seconds to a couple of minutes

The appcast HTTPS GET is byte-for-byte identical whether it comes from this, the "Check for Updates" menu item, or a fresh first launch.

The only reason to run long is to catch a wall-clock "once a day" telemetry beacon — an unusual design, and there's no analytics SDK in the app anyway (open source, checkable). So a few hours of idle is plenty. Total active work: ~3–4 h, one sitting.

Trip 2: on a normal machine, attribution is hell

tcpdump tells you "a SYN went to 140.82.112.x." It does not tell you whether RoamSwitch, Chrome, Dropbox, or the OS sent it.

On a daily-driver machine, even idle, there's a flood of non-Apple traffic: Safe Browsing, component updaters, cloud sync, chat keep-alives. Picking RoamSwitch's packets out of that by timestamp and port is genuinely error-prone — you end up waving things away as "probably Chrome."

So you make the machine silent first. A dedicated account or spare Mac is ideal; quieting the same login also works:

• quit every GUI app
• launchctl bootout the third-party agents (reverts on reboot)
• softwareupdate --schedule off, analytics off, Spotlight suggestions off
• turn off iCloud Private Relay — otherwise it obscures SNI and the analysis is useless

Then, before launching RoamSwitch, run tcpdump for 10 minutes and check that every remaining destination is Apple (17.0.0.0/8, etc.). If a non-Apple destination is still there, something's still running — find it with lsof -nP -iTCP@<IP>, stop it, start over.

Only once that baseline is clean can you lean on "any non-Apple traffic after this ≈ RoamSwitch."

Don't open this procedure in a browser on the test machine. Chrome especially is a pile of non-Apple traffic (Safe Browsing, a GCM keep-alive). Use a second device, print it, or pre-cache the page and use Safari.

The scenario: public Wi-Fi + maximum lockdown

Reproduce the situation RoamSwitch is built for: you join a café / public Wi-Fi and the app locks down automatically.

• pin RoamSwitch's security level to Maximum Lockdown (manual override — not Trusted/Open, not Standard Protection)
• connect to a network you haven't registered as trusted (real café Wi-Fi is best; a phone hotspot works)

At max lockdown, RoamSwitch's pf blocks other apps' egress too, so the capture is even quieter and its own traffic stands out. Crucially, appcast and license still get through — RoamSwitch puts its own traffic in a pf exception. Part of the point of the test is to confirm that exception is minimal and only points where it's supposed to (i.e. it didn't quietly whitelist a telemetry endpoint for itself).

Three monitoring layers

# layer 1: full capture (local / DNS / mDNS filtered out)
sudo tcpdump -i any -n -w cap-%Y%m%d-%H%M.pcap -G 3600 -W 72 -Z root \
  '(ip or ip6) and not net 127.0.0.0/8 and not net 169.254.0.0/16 \
   and not net 224.0.0.0/4 and not port 5353 and not port 53 and not port 67 and not port 68'

# layer 2: socket -> process, sampled every 5s, in a root shell
sudo bash -c 'while :; do ts=$(date -u +%FT%TZ); \
  lsof -nP -iTCP -sTCP:ESTABLISHED +c0 2>/dev/null \
  | awk -v ts="$ts" "NR>1{print ts,\$1,\$2,\$9}"; sleep 5; done' | tee lsof.log

# layer 3: LuLu events from the unified log
log stream --style syslog --predicate 'subsystem BEGINSWITH "com.objective-see"' > lulu.log &

The MCP server only speaks over stdio, so I drive every tool through it and separately confirm its PID never opens an outbound socket.

Analysis and verdict

After the idle hold and the phases (run every feature by hand once; deliberately fire the four known paths), pull destinations from the pcap and cross-reference with the process log by timestamp:

for f in cap-*.pcap; do
  tshark -r "$f" -Y 'tcp.flags.syn==1 && tcp.flags.ack==0' -T fields -e ip.dst
done | sort -u

awk '$2 ~ /^RoamSwitch/ {for(i=1;i<=NF;i++) if($i ~ /->/){split($i,a,"->"); print $2,a[2]}}' \
  lsof.log | grep -vE '127\.0\.0\.1|::1' | sort -u

Pass criteria:

• RoamSwitchHelper — zero outbound flows
• RoamSwitchMCPServer — zero outbound flows (stdio only)
• RoamSwitch — nothing outside lafine.net (and the URL you hit for path #4)
• path #3 is attributed to freshclam, not the app
• no new destination appears during the feature-exercise phase

What actually came out

Verdict: PASS. Across a ~2-hour capture (forced Sparkle check, every feature exercised once, DNS Threat Guard toggled, a network transition, security level pinned to Maximum Lockdown):

Zero non-local outbound connections attributed to RoamSwitch, RoamSwitchHelper, or RoamSwitchMCPServer.

The only sockets the MCP server opened were 127.0.0.1:5000 and 127.0.0.1:7000 — the get_exposed_ports tool checking HTTP headers on locally-listening dev servers. Nothing left the machine.

Here is every HTTPS destination in the whole capture (TLS SNI):

bag.itunes.apple.com       gspe35-ssl.ls.apple.com    proxy-safebrowsing.googleapis.com
configuration.apple.com    init.push.apple.com        setup.icloud.com
dns.quad9.net              mask.icloud.com            swdist.apple.com
gateway.icloud.com         mesu.apple.com             swscan.apple.com
gsa.apple.com              ocsp2.apple.com            tether.edge.apple
p117-contacts.icloud.com   courier.push.apple.com     swallow.apple.com
fbs.smoot.apple.com

lafine.net    <- the only non-OS destination

Strip out the OS's own services (iCloud, push, software update, OCSP, Safari Safe Browsing) and one destination remains: lafine.net — Sparkle fetching appcast.xml (egress path #2). dns.quad9.net shows up because the DNS Threat Guard switched the system resolver; the encrypted lookups are mDNSResponder's, not RoamSwitch's. swallow.apple.com / fbs.smoot.apple.com sit on AWS IPs but are Apple services.

entitlements:

$ codesign -d --entitlements :- /Applications/RoamSwitch.app/Contents/MacOS/RoamSwitchHelper
<dict></dict>
$ codesign -d --entitlements :- /Applications/RoamSwitch.app/Contents/MacOS/RoamSwitchMCPServer
<dict></dict>

Empty — not even com.apple.security.network.client (which, as noted, wouldn't enforce anything without App Sandbox, hence the behavioural check).

LuLu logged no RoamSwitch-related outbound events for the duration.

It's all in one shell script, in the audit/ folder of the roamswitch-support repo. ./rs-zerotel-audit.sh all --idle 2h --clam runs prep, capture, forced Sparkle check, MCP exercise, analysis, a PASS/FAIL verdict, and a shareable FINDINGS.md. The only manual bits are helper approval, pinning max lockdown, one network transition (a Wi-Fi off/on is enough), and a few menu actions.

What this audit can't show

Being upfront:

• Not "no traffic at all". The four paths are real. Zero Telemetry here means no collection or transmission of usage / diagnostic data.
• Not "never". The conclusion is scoped to the observed window — a few hours of normal and adversarial use.
• Payloads aren't inspected (destination + SNI only). To go further, put a mitmproxy in the path and confirm the appcast and license requests carry only "UA + version" and "key + device hash."

But "a claim" versus "a claim + a procedure anyone can run + an honest statement of the limits" is a very different weight of trust. The more skeptical someone is, the more they want something they can check themselves.

---

Script and run sheets (EN/JA): github.com/lafine1211/roamswitch-support, audit/
Whitepaper (Zero Telemetry scope is §7): https://lafine.net/security.html