~/.ssh/config's Permission Check: When OpenSSH Enforces It, and What Changes Between ssh -F and Include

A draft I was reviewing claimed that OpenSSH's client doesn't enforce permissions on ~/.ssh/config. The evidence was a chmod sweep — 600, 644, 664, 666 — with every run exiting 0. The commands were real. They ran. The output was genuine. The conclusion was backwards: permissions are enforced.

Re-running doesn't catch this one. Run the same commands again and you get exit=0 every time, forever. What was broken wasn't the output — it was the path that produced it. Point -F at a config file directly and OpenSSH's permission check never runs at all. Four runs down a path that skips the check got read as four pieces of evidence that the check doesn't exist.

Even when a command and its output are genuine and reproducible, that still does not show that the test exercised the path it claims to — and that gap survives a re-run. What follows is four checks that caught both failures above, roughly cheapest first: a contradiction check, an existence check, a positive control, and an ablation test. All four run inside ssh and a terminal. This isn't a complete taxonomy of measurement errors — it's the cheap heuristics that happened to work on these two cases; what it doesn't cover comes at the end.

A note on how this was checked. Every command and output below was re-run in a disposable scratch directory immediately before publishing and matched verbatim, with one substitution: the scratch directory's absolute path was replaced with the placeholder /tmp/ssh-demo for readability. The numbers, output, and line order are otherwise the run's own. None of the real ~/.ssh on this machine was touched.

$ command -v ssh
/usr/bin/ssh

$ /usr/bin/ssh -V
OpenSSH_10.2p1, LibreSSL 3.3.6

$ codesign -dv /usr/bin/ssh 2>&1 | grep Identifier
Identifier=com.apple.ssh

Every command below runs this same /usr/bin/ssh. How the default config path — ~/.ssh/config itself — was safely tested, and the limits that testing ran into, are covered in a later section.

Check 1 — ssh -G myhost Returns port 2200: Two Explanations for the Same Result

The cheapest check needs no terminal at all. Read a single passage for one observed result that gets explained two incompatible ways.

Take this config:

Host myhost
    HostName 127.0.0.1
    User specific-user
    Port 2200

Match host myhost user specific-user
    Port 2300

$ /usr/bin/ssh -F config_match_test -G specific-user@myhost | grep '^port '
port 2200

Match's Port 2300 doesn't win here. A draft I reviewed offered two different reasons for this, in the same section. First: Port 2200 locks in before Match is evaluated, so first-value-wins discards 2300. Second: Match user is checked against the OS login account, and since neither that account nor the Host block's User specific-user has taken effect yet, Match never fires in the first place.

Those can't both be true. First-value-wins is a rule about what happens once Match's condition is already satisfied. If the second explanation is right, the condition was never satisfied, and first-value-wins never got a turn to do anything. If the first explanation is right, Match must have fired — which is exactly what the second explanation denies.

Knowing they can't both be true doesn't say which one is. The single port 2200 above is consistent with either story: Match firing and losing to first-value-wins looks identical, from the outside, to Match never firing at all. Telling them apart needs a config built so the two predictions come apart.

A narrower config decides which half is wrong:

Host myhost
    HostName 127.0.0.1
    User specific-user

Match user specific-user
    Port 2300

$ /usr/bin/ssh -F config_match_user -G myhost | grep -E '^(user|port) '
user specific-user
port 2300

Match user specific-user succeeds, and the shell running this command is logged in under a different OS account entirely, not specific-user. The only way Match fires here is by reading specific-user back out of the Host block's User line, not the OS account. The second explanation doesn't survive this — for this config.

Calling it flatly wrong would overreach, though. Match user reads a resolved remote-user value, and when nothing has resolved one yet, that value falls back to the OS login name by default. A config with no explicit User line shows the fallback:

Host myhost
    HostName 127.0.0.1

Match user deploy
    Port 2400

$ /usr/bin/ssh -F config_match_localdefault -G deploy@myhost | grep -E '^(user|port) '
user deploy
port 2400

$ /usr/bin/ssh -F config_match_localdefault -G myhost | grep '^port '
port 22

Match fires only when the command line supplies deploy@. Called as plain myhost, the unresolved user value falls back to whoever's logged in locally — not deploy — so Match doesn't fire. The second explanation describes a real fallback path; it's just misapplied to the original config, where the Host block already resolves User to specific-user before Match is evaluated, leaving no unresolved value for a local login name to fall back into.

There's a keyword for exactly the mechanism the second explanation was reaching for: Match localuser matches the OS login account directly, unconditionally. Match user doesn't — it matches a resolved remote-user value that only falls back to the OS login name once nothing else, like a Host block's User line, has already set it. readconf.c's match_cfg_line() recomputes that value fresh every time it starts evaluating a Match line: ruser = options->user == NULL ? pw->pw_name : options->user; — whatever a prior Host block already put in options->user wins; the OS login name is only what's left when nothing did. The second explanation wasn't describing a mechanism that doesn't exist; it was describing localuser's behavior and attaching it to user.

That leaves the first explanation standing for the original config — Check 1 can only show that the draft contradicted itself, not settle which half, if either, was actually right in general. Check 4 comes back to close that out.

Nothing about this is specific to ssh_config. The same shape shows up wherever a write-up explains one failure two ways: a CI failure blamed on both "the code regressed" and "the test environment never came up" in the same paragraph, a support ticket blamed on both a caching bug and a race condition. In each case the two stories can't both be the reason, and the write-up that offers both hasn't actually settled which one is.

This check catches a draft that states two competing explanations for the same result in the same place; it does nothing for a draft that's wrong consistently. It costs zero commands, which is exactly why it's the first pass on anything that lands.

Check 2 — chmod 600/644/664/666 All Exit 0 on ssh -F: Looking for Bad owner or permissions

A different draft's permissions chapter made a clean claim: SSH's client doesn't enforce permissions on ~/.ssh/config — 600, 644, 664, 666 all pass.

$ for m in 600 644 664 666; do chmod $m config && stat -f '%Lp' config && \
    /usr/bin/ssh -F config -G myhost >/dev/null 2>&1; echo "exit=$?"; done
600
exit=0
644
exit=0
664
exit=0
666
exit=0

(stat runs right after chmod here, rather than chaining chmod && ssh — that way a chmod that silently failed can't produce a false pass.)

The commands are real, they ran, the exit codes are genuine. "Not enforced" is still backwards. What catches this one: before accepting a negative claim — "not enforced," "not required," "not checked" — confirm the code that would do the enforcing exists in the binary at all.

$ strings /usr/bin/ssh | grep "Bad owner"
Bad owner or permissions on %s

It's there. Finding nothing wouldn't have proven the check is absent — it could live in a shared library, or surface as a different message — but finding this string means "not enforced" needs more than the sweep above to stand. This costs one strings call, cheap enough to run before accepting any negative conclusion: absent code kills the claim right here; present code is the cue for Check 3.

Check 3 — ssh -F and Include Give Different Results: Adding the chmod 664 Case Into the Measurement

strings only shows that the check exists somewhere in the binary; it can't say which path reaches it. That needs the source, and the caveat from the note at the top applies here too: this is a static reading of openssh-portable upstream, not a trace of what the local binary actually executes.

From ssh.c, process_config_files():

if (config != NULL) {
    if (strcasecmp(config, "none") != 0 &&
        !read_config_file(config, pw, host, host_name, cmd,
        &options,
        SSHCONF_USERCONF | (final_pass ? SSHCONF_FINAL : 0),
        want_final_pass))
        fatal("Can't open user config file %.100s: "
            "%.100s", config, strerror(errno));
} else {
    r = snprintf(buf, sizeof buf, "%s/%s", pw->pw_dir,
        _PATH_SSH_USER_CONFFILE);
    if (r > 0 && (size_t)r < sizeof(buf))
        (void)read_config_file(buf, pw, host, host_name, cmd,
            &options, SSHCONF_CHECKPERM | SSHCONF_USERCONF |
            (final_pass ? SSHCONF_FINAL : 0), want_final_pass);

    /* Read systemwide configuration file after user config. */
    (void)read_config_file(_PATH_HOST_CONFIG_FILE, pw,
        host, host_name, cmd, &options,
        final_pass ? SSHCONF_FINAL : 0, want_final_pass);
}

A file handed to ssh via -F only ever carries SSHCONF_USERCONF. SSHCONF_CHECKPERM — the flag that turns permission checking on — is set only on one call in the else branch, the one that builds and reads the default per-user ~/.ssh/config path; the systemwide config read right below it doesn't carry the flag either. The four-pattern sweep in Check 2 ran entirely on the -F branch, which never sets SSHCONF_CHECKPERM at all; it never had a chance to fail, independent of what the actual permissions were.

One path that does set the flag is Include. From readconf.c, Include handling inside process_config_line_depth():

r = read_config_file_depth(gl.gl_pathv[i],
    pw, host, original_host, remote_command,
    options, flags | SSHCONF_CHECKPERM |
    (oactive ? 0 : SSHCONF_NEVERMATCH),
    activep, want_final_pass, depth + 1);

Every file pulled in through Include gets SSHCONF_CHECKPERM unconditionally. What the flag actually does, further down the same function, read_config_file_depth():

if (flags & SSHCONF_CHECKPERM) {
    struct stat sb;

    if (fstat(fileno(f), &sb) == -1)
        fatal("fstat %s: %s", filename, strerror(errno));
    if (((sb.st_uid != 0 && sb.st_uid != getuid()) ||
        (sb.st_mode & 022) != 0))
        fatal("Bad owner or permissions on %s", filename);
}

Re-run the same permission sweep, but through a path that actually sets the flag: an outer file, held at 600, that Includes an inner file whose permissions vary. Both ssh calls below use -T (it suppresses an unrelated pty warning and doesn't affect the port or permission-check results these checks read) and the same absolute paths for -F and Include. stat runs immediately before each call, so the permission and path actually observed are the ones under test, not assumed:

# outer.conf, permissions 600, fixed
Include /tmp/ssh-demo/inner.conf

# inner.conf, permissions varied 600 -> 644 -> 664 -> 666
Host myhost
    HostName 127.0.0.1
    User specific-user

$ cat perm.sh
set -euo pipefail
D=/tmp/ssh-demo
for m in 600 644 664 666; do
  chmod "$m" "$D/inner.conf"
  stat -f '%d %i %u %Lp %N' "$D/inner.conf"
  /usr/bin/ssh -T -F "$D/inner.conf" -G myhost >/dev/null && s=0 || s=$?
  echo "  direct -F      exit=$s"
  stat -f '%d %i %u %Lp %N' "$D/inner.conf"
  /usr/bin/ssh -T -F "$D/outer.conf" -G myhost >/dev/null && s=0 || s=$?
  echo "  via Include    exit=$s"
done

$ bash perm.sh
16777230 296585647 503 600 /tmp/ssh-demo/inner.conf
  direct -F      exit=0
16777230 296585647 503 600 /tmp/ssh-demo/inner.conf
  via Include    exit=0
16777230 296585647 503 644 /tmp/ssh-demo/inner.conf
  direct -F      exit=0
16777230 296585647 503 644 /tmp/ssh-demo/inner.conf
  via Include    exit=0
16777230 296585647 503 664 /tmp/ssh-demo/inner.conf
  direct -F      exit=0
16777230 296585647 503 664 /tmp/ssh-demo/inner.conf
Bad owner or permissions on /tmp/ssh-demo/inner.conf
  via Include    exit=255
16777230 296585647 503 666 /tmp/ssh-demo/inner.conf
  direct -F      exit=0
16777230 296585647 503 666 /tmp/ssh-demo/inner.conf
Bad owner or permissions on /tmp/ssh-demo/inner.conf
  via Include    exit=255

stat's columns, in order: device, inode, owner uid, permissions, path. Across the 8 stat calls immediately preceding each ssh run, device and inode stayed constant and the owner uid matched this account's. That's as far as the observation goes — whether the file could have changed between stat and ssh's own open(), or whether both paths' open file descriptors actually resolved to that same device/inode, wasn't checked.

The only edit made to this transcript for publication was substituting the working directory's absolute path with /tmp/ssh-demo, as noted at the top. The numbers, output, and order are the run's own.

Same file, same permissions, only the path differs. -F direct passes all four permission levels; Include fails at 664 and 666 with the exact string Check 2 found sitting in the binary. Because 664 and 666 disagree between the two paths on files that are otherwise identical, path is what's gating the check — and once a path does trigger it, group- or other-writable is what fails it.

I stopped there. The default path itself, ~/.ssh/config, was never touched — the real ~/.ssh on this machine stayed out of scope for this kind of audit. Redirecting $HOME to a scratch directory looks like a shortcut around that, but it doesn't work: the else branch in ssh.c builds the default path from pw->pw_dir, the account's actual home directory from getpwuid(), and never reads $HOME. That same branch always carries SSHCONF_CHECKPERM regardless of which directory it ends up reading, so the source gives no reason to expect the default path at 664/666 to behave any differently from inner.conf above — but that's a prediction from the source, not a run I have to show for it.

The same pattern shows up outside SSH. In CI, plant a known break in the exact path a test claims to cover and check whether the intended assertion fails for the intended reason; adding one assert false to force red only proves the test runner executes, not that the target path was ever reached. For a health check, take the monitored service down in an isolated environment and check whether the contract's failure signal — non-2xx, connection refused, timeout, whichever applies — actually fires; if it stays green, the thing being watched is a different process or a different environment. Mock-heavy tests invert this: with the real implementation swapped for a mock, breaking that implementation should leave the test green, so the thing to break instead is the mock's return value or the contract behind it, or route through an integration test that exercises the real implementation. Same goal every time: plant a known failure and check it gets caught. If it doesn't, the measurement may never have reached the path it claims to test. A sweep with no case that has to fail can't tell a silent mechanism from an absent one — adding 664/666 through a path that actually sets the flag is what turns "everything passed" into a result that means something.

Redirecting $HOME Doesn't Move the Default Path — Measuring the Boundary Through Include

The previous section closed by leaving the default path as inference. Here's how far that can actually be pushed.

The obvious next move looked like redirecting $HOME to point the default path at an isolated environment:

$ mkdir -p /tmp/ssh-demo/isolated-home/.ssh
$ cat /tmp/ssh-demo/isolated-home/.ssh/config
Host myhost
    HostName 127.0.0.1
    Port 2200

$ chmod 700 /tmp/ssh-demo/isolated-home/.ssh
$ chmod 600 /tmp/ssh-demo/isolated-home/.ssh/config
$ HOME=/tmp/ssh-demo/isolated-home /usr/bin/ssh -G myhost | grep -E '^(hostname|port) '
hostname myhost
port 22

Neither HostName 127.0.0.1 nor Port 2200 took effect. The file itself isn't broken — pointing -F straight at it reads it fine:

$ HOME=/tmp/ssh-demo/isolated-home /usr/bin/ssh -F /tmp/ssh-demo/isolated-home/.ssh/config -G myhost | grep -E '^(hostname|port) '
hostname 127.0.0.1
port 2200

Drop -F and keep only the $HOME redirect, and it's back to hostname myhost (unsubstituted) and port 22 (the compiled-in default). -vvv shows which file actually got opened:

$ HOME=/tmp/ssh-demo/isolated-home /usr/bin/ssh -vvv -G myhost 2>&1 | grep "Reading configuration data"
debug1: Reading configuration data $REAL_HOME/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config

$REAL_HOME stands in for this machine's actual home directory, substituted for publication; the rest of the line is the run's own. The default path is built from pw_dir, not from $HOME, so redirecting the environment variable never moves it. The source explains why: the default path is assembled from pw->pw_dir, and that pw comes from getpwuid(getuid()), not from getenv("HOME").

/* ssh.c, line 729 */
pw = getpwuid(getuid());

/* ssh.c, lines 589-590, the `config == NULL` else branch */
r = snprintf(buf, sizeof buf, "%s/%s", pw->pw_dir,
    _PATH_SSH_USER_CONFFILE);

Since the default path's construction never reads $HOME, giving it a pw_dir different from the real one takes a different UID — which takes root, which this working environment doesn't have. Touching the real ~/.ssh on this machine was never on the table either. So the literal thing — set the default path to 664 and watch it fail — wasn't measured here.

What the source does confirm directly: reading readconf.c, the default-path branch (the else in ssh.c) and the Include branch both end up in the same read_config_file(), hitting the same check (if (flags & SSHCONF_CHECKPERM) { ... }, testing sb.st_mode & 022). The only difference between the two callers is whether SSHCONF_CHECKPERM gets set before getting there; once it's set, there's exactly one check to pass. The Include measurement above exercises that shared check directly. It isn't a measurement of the default path itself, but it is a measurement of the exact code the default path relies on — a proxy, and one worth keeping distinct from a direct measurement.

Three paths, side by side:

Path	SSHCONF_CHECKPERM	How it was tested	600	644	664	666
-F direct	not set	measured directly, isolated environment	exit 0	exit 0	exit 0	exit 0
via Include	set	measured directly, isolated environment	exit 0	exit 0	Bad owner... (255)	Bad owner... (255)
default path (no -F)	set (confirmed from source — same branch as Include)	can't be measured directly in this environment; Include's result stands in as a proxy	—	—	—	—

That 664 and 666 both fail is already established from the measurement above. What isn't yet established is which bit does it. Starting from 600 and adding one bit at a time — read bits (040, 004) kept separate from write bits (020, 002) — settles that:

$ cat /tmp/ssh-demo/boundary.sh
D=/tmp/ssh-demo
for m in 600 604 620 640 660 662 664 666; do
  chmod "$m" "$D/boundary_inner.conf"
  /usr/bin/ssh -T -F "$D/boundary_outer.conf" -G myhost >/dev/null 2>&1
  echo "mode=$m exit=$?"
done

$ bash /tmp/ssh-demo/boundary.sh
mode=600 exit=0
mode=604 exit=0
mode=620 exit=255
mode=640 exit=0
mode=660 exit=255
mode=662 exit=255
mode=664 exit=255
mode=666 exit=255

boundary_outer.conf stays fixed at 600 and contains a single line, Include /tmp/ssh-demo/boundary_inner.conf. Only boundary_inner.conf's own permissions moved:

Mode	owner	group	other	group write	other write	Result
600	rw-	---	---	no	no	exit 0
604	rw-	---	r--	no	no	exit 0
620	rw-	-w-	---	yes	no	255
640	rw-	r--	---	no	no	exit 0
660	rw-	rw-	---	yes	no	255
662	rw-	rw-	-w-	yes	yes	255
664	rw-	rw-	r--	yes	no	255
666	rw-	rw-	rw-	yes	yes	255

Whether a read bit is set (604, 640) makes no difference to the result. One write bit on group or other, and it fails (620 onward) — matching readconf.c's condition, (sb.st_mode & 022) != 0, exactly. 022 in octal is group-write (020) OR'd with other-write (002); the read bits (044) never enter into it.

The earlier Include measurement — outer file pinned at 600, only the inner file's permissions varied — already satisfied this condition. A 600 parent doesn't shield an included file sitting at 664 or 666. The parent's own permissions play no role in the check applied to whatever it Includes.

Bad owner or permissions: One Message for Two Different Causes

Bad owner or permissions comes out of a single fatal() call in readconf.c:

if (((sb.st_uid != 0 && sb.st_uid != getuid()) ||
    (sb.st_mode & 022) != 0))
    fatal("Bad owner or permissions on %s", filename);

The two conditions joined by || produce the same one-line message regardless of which one fired. A file owned by neither root nor the current user (st_uid mismatch) and a file with a group- or other-write bit set (st_mode & 022) are indistinguishable from the message alone. Telling them apart takes a separate stat:

$ stat -f '%u %Lp' /tmp/ssh-demo/boundary_inner.conf
503 664

503 matches this account's own id -u, so ownership checks out — the failure here traces to permissions (664's group-write bit), not to a mismatched owner. The owner-mismatch branch — a file that ends up owned by someone else after being copied over shared storage, say — wasn't exercised here, since setting up a second user account was out of reach; that side stays at what the upstream source's condition says, not something measured directly.

The path in the message deserves attention too. filename names whichever specific file was open at the time. In the Include measurement above, the parent outer.conf passes the check at 600, so the error only ever names the included inner.conf. Once a config is split across files, check which file's path the error actually names before touching anything — fixing the parent's permissions does nothing if the real cause sits in the included file.

One path that actually lands on this boundary, confirmed here: with the shell's umask set to 002, a new file created by redirection comes out at 664.

$ umask
022
$ bash -c "umask 002; printf 'Host x\n    Port 22\n' > /tmp/ssh-demo/umask_test_config"
$ stat -f '%Lp' /tmp/ssh-demo/umask_test_config
664

This environment's default umask is 022, which tops out at 644 on its own — 002 is a value someone sets deliberately for group sharing, not something a default-umask environment ever produces by itself. cp behaves differently:

$ bash -c "umask 002; cp /tmp/ssh-demo/src_600.conf /tmp/ssh-demo/umask_test_cp600.conf"
$ stat -f '%Lp' /tmp/ssh-demo/umask_test_cp600.conf
600

cp on this machine duplicates the source file's own permissions and doesn't go through umask at all. What trips the boundary isn't copying a config into place — it's generating one fresh: a shell redirect, an editor save, a template rendering. Running the umask-created file through Include fails for the same reason as the boundary sweep above:

$ /usr/bin/ssh -T -F /tmp/ssh-demo/umask_outer.conf -G x >/dev/null 2>&1; echo exit=$?
exit=255

Check 4 — Match host vs originalhost: Removing Port 2200 to See Which One Fired

The last check: delete the value you suspect matters, and check whether the result flips the way the theory predicts — rather than taking one matching run as confirmation.

Back to the contradiction Check 1 left open. What settles it is in the man page:

The criteria for the host keyword are matched against the target hostname, after any substitution by the Hostname or CanonicalizeHostname options. The originalhost keyword matches against the hostname as it was specified on the command-line.

— ssh_config(5), OpenBSD manual pages

host is compared after HostName substitution. The config sets HostName 127.0.0.1, so Match host myhost compares myhost against 127.0.0.1 and never matches. originalhost compares against what was literally typed on the command line, so Match originalhost myhost does match — but matching alone still doesn't hand Port to 2300 while Host's Port 2200 is set first:

Host myhost
    HostName 127.0.0.1
    User specific-user
    Port 2200          # present in the first run, removed in the second

Match originalhost myhost
    Port 2300

$ /usr/bin/ssh -F config_match_originalhost_only -G myhost | grep -E '^(user|port) '
user specific-user
port 2200

$ /usr/bin/ssh -F config_match_fixed -G myhost | grep -E '^(user|port) '
user specific-user
port 2300

Deleting Port 2200 flips the result exactly as predicted. That's still one data point, though, on a config where Match was already confirmed to fire.

The actual ablation test applies the same move to the config that started this section — Match host myhost user specific-user, the one where Match was never established to fire at all. Remove Host's Port 2200, so first-value-wins has nothing left to defend, and leave Match's Port 2300 in place. If Match had ever fired here, this is exactly the config where 2300 should now win by default:

Host myhost
    HostName 127.0.0.1
    User specific-user

Match host myhost user specific-user
    Port 2300

$ /usr/bin/ssh -F config_match_noport -G myhost | grep '^port '
port 22

$ /usr/bin/ssh -F config_match_noport -G specific-user@myhost | grep '^port '
port 22

Both calls land on 22, SSH's compiled-in default — not 2200 (nothing left for first-value-wins to defend), and not 2300 either. Match host myhost never fired, under either way of calling the host, exactly as the man page's substitution rule predicts. Neither of Check 1's explanations was actually the reason: there was no Port conflict to resolve, because Match never matched in the first place.

One grep detail worth flagging:

$ /usr/bin/ssh -F config_match_noport -G myhost | grep port
port 22
gatewayports no

grep port without an anchor also catches gatewayports no — every command above uses grep '^port ' for exactly this reason.

To pin down when Match host does and doesn't fire, and separate that cleanly from the Port conflict, run all four combinations of condition keyword × Port 2200 present/absent, holding everything else fixed — same user specific-user clause, same -G myhost call in all four cells:

$ cat cell.sh
set -euo pipefail
D=/tmp/ssh-demo
for cond in "host myhost user specific-user" "originalhost myhost user specific-user"; do
  for port in 2200 none; do
    { printf 'Host myhost\n    HostName 127.0.0.1\n    User specific-user\n'
      [ "$port" != none ] && printf '    Port %s\n' "$port"
      printf '\nMatch %s\n    Port 2300\n' "$cond"
    } > "$D/cell.conf"
    chmod 600 "$D/cell.conf"
    result=$(/usr/bin/ssh -T -F "$D/cell.conf" -G myhost | grep '^port ')
    printf 'Match %-44s Port=%-4s %s\n' "$cond" "$port" "$result"
  done
done

$ bash cell.sh
Match host myhost user specific-user               Port=2200 port 2200
Match host myhost user specific-user               Port=none port 22
Match originalhost myhost user specific-user       Port=2200 port 2200
Match originalhost myhost user specific-user       Port=none port 2300

	Port 2200 present	Port 2200 removed
Match host ...	port 2200	port 22
Match originalhost ...	port 2200	port 2300

With Port 2200 in place, both conditions read port 2200 — from this row alone, there's no telling whether Match failed to fire or fired and lost to first-value-wins. Removing it splits the two apart: originalhost moves to 2300 (Match fired), host falls to the compiled-in default 22 (Match never fired at all).

Deleting a value, flipping a flag, dropping a cache layer, removing a dependency — same move each time, and it's what turns "a setting that looks like it's taking effect" into either confirmed or exposed as dead weight. This check earns its place last on the list because a single passing run can't show the value you're testing caused the result; deleting it and watching the comparison move is the closest thing on this list to a controlled experiment.

Check 1 and Check 4 are both things a reviewer can redo just by rerunning the same commands after a draft lands. Check 2 and Check 3 aren't — they require deciding whether the enforcing code plausibly exists and which path reaches it, which means opening the source before deciding what to even measure. The "survives a re-run" failure from the opening is exactly the case where the last two are necessary.

What These Four Checks Don't Cover — Apple's ssh Binary and the Upstream Source

These four worked on the two failures above; they aren't a complete system for catching path-of-measurement errors, and this piece leaves gaps of its own.

First, target identity was never confirmed. What ran here is Apple's build of ssh; what got read was the upstream source. There's no proof they're the same build — which is exactly why the default-path claim above stays a prediction rather than a measurement. But "does the binary I ran correspond to the source I'm reading" is a question that should get settled before reasoning about paths at all, not folded into a single disclaimer.

Second, the expected result isn't independently grounded. The "664 should fail" prediction is derived from reading the same source used to test it. A misread source and a matching misread test would agree with each other and look confirmed.

Third, there's no direct observation of the path itself. Include failing is evidence the check ran, but nothing here traced or logged the actual branch taken. How far ssh -G's resolved config represents real-connection behavior also wasn't checked in this piece.

And structurally: Check 1 isn't a measurement at all — it only works when a draft happens to state two competing explanations in the same place; a consistently wrong draft sails through it. Check 3 and Check 4 are really the same underlying move (change an input, watch the causal effect) rather than two clean categories. Read this as four heuristics ordered cheapest-first, not a taxonomy.

The Reviewer's Own Accuracy — 10 of 36 Adversarial Review Findings Were Later Overturned

None of this makes the checking process infallible. In a separate pre-publication audit of a different draft, the same style of adversarial, disprove-it-first review flagged 36 issues. Of those, 10 were later overturned — the finding itself judged wrong or overreaching, not the underlying draft. That's an overturn rate of just under 3 in 10, and it's the only honest name for it — "error rate" would claim more than is warranted, because whether those 10 overturn decisions were themselves correct hasn't been independently checked. What the number actually describes is that the second-pass review produced a 10/36 overturn outcome; nothing stronger than that.

Running more automated checks isn't the fix by itself, either. What did the work here wasn't the count of checks — it was two of the four refusing to accept a negative claim without evidence: Check 2 asking whether the enforcing code exists, Check 3 insisting on a case that has to fail.

Four Questions for Checking a Measurement You've Been Handed

The four checks above are concrete moves against one small system. The questions below are the level above them — what each check is actually standing in for. When a write-up lands, these are worth asking in order.

1. Am I measuring the same target the source describes? Does the binary that ran correspond to the source or docs being treated as ground truth?
2. Is the expected result independently grounded? Is the "this should fail" prediction riding on the same misreading as the thing it's supposed to test?
3. Is there evidence the intended path actually fired — or is a nearby path succeeding being read as evidence for the target path?
4. Does the result causally depend on the variable that changed? Delete the value, flip the condition, and check whether the result actually moves.

"Everything passed" or "every variant gave the same result" isn't evidence of success on its own — read it as a prompt to check question 3 before crediting the thing under test. If four differently-labeled cases all come back identical, the first thing worth doubting isn't the target's robustness. It's whether those four cases ever took four different paths.

"I verified it" guarantees a command ran and the output is genuine. It does not guarantee the path that produced that output passed through the mechanism the claim is actually about. And unlike a faked output, this kind of failure survives a re-run — the same input in the same environment comes back identical no matter how many times it's replayed. What's left for whoever's on the receiving end isn't re-reading the output. It's reconstructing the path itself and checking it independently.

References

• ssh_config(5), openssh-portable at V_10_2_P1
• ssh.c, openssh-portable at V_10_2_P1
• readconf.c, openssh-portable at V_10_2_P1
• ssh_config(5), OpenBSD manual pages