Yesterday I threw away an article before publishing it. The real reason was duplication:
I had already written the same subject a month earlier (Unicode normalization, two methods
giving two different counts in the same ~/Downloads folder), and my first scan had missed
it because the list was cut short with head. But that run produced a second lesson, and
that is the one that actually stung.
The article's surprise was this: a filename was found by grep -F and not by grep -E. I
ran the same measurement with /usr/bin/grep — it found the file in all three modes. The
thing I was presenting as a surprise was not the filename's behaviour. It was my shell's.
I wrote one line in my working notes: "Run every measurement about text or file behaviour
with an absolute path (/usr/bin/grep, /usr/bin/find, /usr/bin/tar) and print its
version in the article." Correct lesson, incomplete one. Because the real question is
bigger: I type the name of a command — who decides which program runs? Today I
counted. The answer sits in four authorities: aliases, the shell's function table, the
order of PATH, and the first line of the script. I set up all four, each at a different time.
The experiment: same command, same flags, three files against one
I wanted to see the mechanism in the smallest possible rig first. An empty directory, a
.gitignore, and three files carrying the same string: one ordinary, one listed in
.gitignore, one under node_modules/. No repository needed — I did not run git init;
the presence of the ignore file is enough.
$ mkdir -p deney/node_modules && cd deney
$ printf 'gizli.txt\nnode_modules/\n' > .gitignore
$ echo 'ARANAN_DIZGE burada' > acik.txt
$ echo 'ARANAN_DIZGE burada' > gizli.txt
$ echo 'ARANAN_DIZGE burada' > node_modules/paket.txt
$ /usr/bin/grep --version
grep (BSD grep, GNU compatible) 2.6.0-FreeBSD
$ /usr/bin/grep -rl ARANAN_DIZGE .
./acik.txt
./node_modules/paket.txt
./gizli.txt
$ grep -rl ARANAN_DIZGE .
acik.txt
Same flags, same directory, same minute. Three files when I call it by absolute path, one
file when I type its name. Two thirds of the matches never appeared — and no warning was
printed. In a counting exercise, that is more than enough silence to write a "clean" report.
type tells you why:
$ type grep
grep is a shell function from /Users/mustafaerbay/.claude/shell-snapshots/snapshot-zsh-...sh
In this shell grep is not a binary, it is a function. Its body hands the command to a
different executable under the identity ugrep and inserts flags such as
--ignore-files --hidden --exclude-dir=.git along the way. --ignore-files reads
.gitignore and skips whatever is listed there. So the function is not misbehaving; it is
doing exactly the job it was designed for. My expectation was wrong: that is the behaviour
I want while moving around a repository, not while counting an inventory.
The most irritating part is this: the function has an escape hatch. When it sees certain
flags it steps aside and calls the real grep.
$ grep -rlZ ARANAN_DIZGE . | tr '\0' '\n'
./acik.txt
./node_modules/paket.txt
./gizli.txt
I added one letter and the result went back to three. Same name, same directory, two
different programs depending on a flag. There is no way to spot that by eye inside a
measurement script.
The order is documented, and it is not in my favour
This is not an accident, it is zsh's documented behaviour. When the shell resolves a command
word it goes: "If there exists a shell function by that name, the function is invoked", then
"If there exists a shell builtin by that name, the builtin is invoked", and only at the end
"Otherwise, the shell searches each element of $path". PATH is the last place it looks.
If a function exists, PATH is never opened at all.
There is one more layer above functions: aliases. An alias is expanded while the line is
being parsed, before resolution starts. A two-line experiment is enough:
$ deney() { echo FONKSIYON; }
$ alias deney='echo ALIAS'
$ deney
ALIAS
$ unalias deney; deney
FONKSIYON
My own terminal has five aliases and none of them shares a name with a command on PATH — so
this layer is empty on my machine. It is not empty for someone who writes
alias grep='grep --color=auto', and most "useful dotfiles" repositories suggest exactly that.
There are two ways out, both in the documentation — but they do not do the same thing.
command grep ... runs "an external command instead of a function or builtin"; that is, it
skips the function, not the ordering. For someone whose PATH has /opt/homebrew/bin/grep
in front, command grep still means GNU grep, not BSD grep. An absolute path cuts through
both. And whence -p grep does "a path search for name even if it is an alias, reserved
word, shell function or builtin". My reflex for years has been to type which; in this
shell that prints the function body — it is answering correctly, but I read it as "a path"
and move on.
The shell also has a memory. The first time zsh runs a command it notes its location
(HASH_CMDS: "Note the location of each command the first time it is executed") and does not
search PATH again on later calls. Add a directory to PATH mid-session and you keep getting
the old binary; rehash (that is, hash -r) empties the table. If you changed PATH while
measuring, this is the most common reason the winner disagrees with your arithmetic.
This shell has 131 functions defined. Three of them share a name with a command on PATH:
grep, find, pkill. Three. Hard to notice among a hundred and thirty-one, enough to
break a measurement.
The PATH inventory: 26 entries, six of them gone
Past the function layer comes PATH. Every number in this section comes from the shell my
agent runs; the PATH in my own terminal is three directories shorter, and I will get to the
difference at the end.
Twenty-six directories. Six of them do not exist on disk. Three are macOS cryptex
bootstrap paths, one is /pkg/env/global/bin — the /pkg directory it points at is not on
this machine at all. The remaining two are my own doing: ~/development/flutter/bin and
~/.pub-cache/bin. I deleted Flutter months ago; the lines still sit in the first two lines
of my .zshrc.
I also wondered where those directories come from. Twelve of them come not from my own files
but from path_helper. Apple's man page is clear about the job: first "default PATH and
MANPATH values are obtained from the files /etc/paths", then the contents of the files in
/etc/paths.d are appended. On my machine /etc/paths has six lines and /etc/paths.d has
four files with six lines in total.
I looked at those four files one by one, because I was about to wave them away as "leftovers
of tools I installed" and that would not have been true. All four are owned by root. Two
(10-cryptex, 100-rvictl) ship with macOS — rvictl is still sitting in
/Library/Apple/usr/bin. MacGPG2 is dated 2020 but it is not a leftover either: there are
twenty executables in that directory and /usr/local/bin/gpg still links into it. Only
10-pmk-global is genuinely orphaned, and it carries the same creation minute as the system
files; I do not know who put it there, and writing that I do not know beats guessing. A dead
directory on PATH does no measurable harm — it is just enough to make you think something is
installed.
One more detail: path_helper runs through /etc/zprofile on every login shell and puts its
own list in front. That is why a PATH you export in .zshenv ends up behind the system
directories. On macOS this is where people trip most often.
85 commands with the same name: the winner is whoever is earlier
Across the twenty directories that exist on this PATH there are 2,290 distinct command names.
85 of them exist in more than one directory; in 81 cases the copies really are separate
files (in the other four, both paths lead to the same file). So there are eighty-one "which
one?" questions on this machine, and the answer is given by directory order, not by me.
| Command | In how many directories | Winner |
|---|---|---|
python3 |
4 |
/Library/Frameworks/Python.framework/Versions/3.13/bin (3.13.3) |
gpg |
3 | /opt/homebrew/bin |
psql |
2 |
/opt/homebrew/opt/postgresql@16/bin (16.11) |
node |
2 |
~/.nvm/versions/node/v22.16.0/bin (22.16.0) |
git |
2 |
/opt/homebrew/bin (2.48.1) |
That last row stopped me. /usr/bin/git on this machine is 2.54.0 (Apple Git-157), while
Homebrew's git is 2.48.1. For years I have assumed "the system tool is old, the package
manager's is new" and put Homebrew at the front of my PATH. For this command the opposite
happened: my ordering drops me six minor versions back. What is old here is not the system,
it is the package I forgot to update. Three weeks ago I counted
the two packages my package manager refuses to read from its own list;
there the problem was a package not showing up, here the package shows up fine — what is
invisible is which copy runs.
Which pg_dump? My client is 16, my server is 14
The psql row is more serious. Line seven of my .zshrc puts postgresql@16/bin at the
front of PATH, which is why psql and pg_dump come out as 16.11. But the PostgreSQL server
running on this Mac is postgresql@14: in the process list it is
/opt/homebrew/opt/postgresql@14/bin/postgres, and that cluster's PG_VERSION file holds a
single line, 14. (There is also an idle postgresql@16 data directory on disk; that is
what really sets the trap.)
I connected the client and asked the server who it was:
$ /opt/homebrew/opt/postgresql@16/bin/psql -tAc 'select version()' postgres
PostgreSQL 14.18 (Homebrew) on aarch64-apple-darwin24.4.0, ...
It works. And that is no coincidence: PostgreSQL's own documentation says "pg_dump can also
dump from PostgreSQL servers older than its own version", supported back to 9.2. A newer
client can read an older server.
For the other direction the same page is far blunter: "pg_dump cannot dump from PostgreSQL
servers newer than its own major version; it will refuse to even try, rather than risk making
an invalid dump." Today I am on the good side, but I won that by ordering, not by skill. If
that line in .zshrc were ever deleted — or if I had never written it — pg_dump would
quietly fall back to the 14.18 in /opt/homebrew/bin; and on the day I moved the cluster
to 16, my backup command would not have written half a file, it would have politely refused.
The good news is the politeness of that refusal; the bad news is that the only place the
refusal shows up is the log of a scheduled job.
The third context: launchd's four-directory world
After measuring my terminal and my agent's shell, I had not counted the third context at all:
scheduled jobs. There are 27 plists under ~/Library/LaunchAgents — jobs that pull backups,
fetch exchange rates, open tunnels, feed a panel.
launchd states what it hands over, in its own report:
$ launchctl print gui/501/com.erbay.spark-bulut | /usr/bin/grep -A3 'default environment'
default environment = {
PATH => /usr/bin:/bin:/usr/sbin:/sbin
}
Four directories. They are not arbitrary; they are a constant in Apple's Libc:
#define _PATH_STDPATH "/usr/bin:/bin:/usr/sbin:/sbin". I counted: 1,257 command names
resolve on that PATH — 55% of the 2,290 I counted in my shell. There is no node in it, no
npm, no psql, no gh, no brew.
And there is a python3 — but which one? That job's plist invokes its program as
/usr/bin/env python3. I asked with those same four directories:
$ env -i /usr/bin/env python3 --version
Python 3.9.6
In my terminal python3 is 3.13.3. In the scheduler it is 3.9.6. In Python's own developer
guide the status of 3.9 is "end-of-life", dated 2025-10-31 — an interpreter that has not
received a patch in more than a year runs my script every morning at 07:15. Today's run wrote
two lines to its log and both are ordinary; so this is not something broken, it is a place
where the version I installed myself never comes into play.
The day I counted launchd's retries
taught me something similar: launchd does what it is told, not what I think I told it.
Three plists set PATH themselves. One wrote four directories, one wrote six; both are
reasonable. The third froze an agent session's PATH exactly as it stood at that moment: 32
entries, 26 unique, nine entries (seven distinct directories) that no longer exist. One of
them is ~/.codex/tmp/arg0/codex-arg0adUvOF — a temporary directory, deleted weeks ago. That
plist keeps looking for it on every launch. Among the same files I also found a comment I had
written myself: "the directory holding the claude CLI must be on PATH in the launchd
environment." So I had already learned this lesson, noted it in a plist, and then forgotten it.
And a fourth: the job that walks back into the shell
I was going to write "none of these jobs ever sees my shell"; before writing it I read all 27
plists one by one, and I am glad I did. One of them — my daily worktree cleanup — invokes its
program as /bin/zsh -lc <script>. The -l means a login shell: /etc/zprofile,
path_helper and .zprofile all come into play. But -c means non-interactive, so .zshrc
is not read. I rebuilt its environment and asked:
$ env -i HOME=$HOME USER=$USER PATH=/usr/bin:/bin:/usr/sbin:/sbin \
/bin/zsh -lc 'echo $PATH; whence -p node'
# 17 entries
/opt/homebrew/bin/node
Seventeen entries: more than launchd's four, fewer than my terminal's twenty-three. And node
there is Homebrew's 23.11.0 — not the nvm 22.16.0 that wins in my terminal. Because nvm is
set up by .zshrc, and .zshrc is never read in this job. That script only calls git, so it
does no harm today; but the question "what do I get when I type node on this machine" has
three answers here: 22.16.0, 23.11.0, and command not found.
The shebang layer: 53 commands look up their interpreter on PATH
PATH does not only resolve the name I type. When I run a script, that script's interpreter goes
through the same mechanism.
I looked at the first line of all 2,290 commands on my PATH: 1,705 are binaries (no shebang),
583 are scripts, two could not be read. 53 of those 583 scripts begin with #!/usr/bin/env;
that is, they look up their interpreter on PATH at run time. The four largest groups: 17 node,
11 perl, 9 bash, 8 python3. The list includes npm, yarn, wrangler, firebase,
netlify and claude.
This small detail has a nice consequence. Homebrew's npm does not use Homebrew's own node
23.11.0, it uses the nvm node 22.16.0 at the front of PATH — because the first line of
npm-cli.js is #!/usr/bin/env node. So the answer to "which node" depends not on the manager
that installed the package but on whichever PATH is in force; the scheduled job in the previous
section is the proof. The only way to measure the difference between being installed and being
used is to ask for the path, not the name.
The remaining 530 scripts name an absolute interpreter. Two of those point at a
/usr/local/bin/bash that does not exist (test-yaml and test-yaml5.34). Two commands,
quietly unusable for months. Nobody noticed, because nobody looked for them.
I broke my own measurement twice
The most instructive part of this article is the traps I fell into.
The first. To compare my terminal with my agent's shell, I ran zsh -l -c 'echo $PATH' from
inside the agent. The output had 30 entries, four of them duplicated, and 43 command names
resolved to different paths between the two contexts: node, npm, psql, the whole pg_*
family. A perfect headline. I even started sorting out which ones were genuinely separate files
— 35 really were. Then I looked at the rig: I had started zsh -l from the agent's shell, that
shell inherited its PATH, and .zprofile and .zshrc added their own entries on top. The
difference I had found was a product of my own rig.
The second, and the embarrassing one. Next came a clean environment:
env -i HOME=... TERM=... /bin/zsh -lic 'echo $PATH'. The output had 24 entries and one of them
was /usr/ucb. That directory does not exist on this machine, appears in none of my dotfiles,
and never enters my PATH in a Terminal window. Where did it come from? env -i removes PATH
entirely, so zsh falls back to its compile-time default:
$ env -i /bin/zsh -fc 'echo $PATH'
/bin:/usr/bin:/usr/ucb:/usr/local/bin
A real login shell does not find PATH empty: launchd hands it _PATH_STDPATH — the same four
directories I printed two sections earlier. So the environment I thought was clean was
committing the article's own error one floor further down. The correct rig:
$ env -i HOME=$HOME SHELL=/bin/zsh TERM=xterm-256color USER=$USER \
PATH=/usr/bin:/bin:/usr/sbin:/sbin /bin/zsh -lic 'echo $PATH'
The real answer: 23 entries in my terminal (six of them missing from disk), 26 in the
agent's shell, and the number of commands resolving to different paths is zero. The
directories present for the agent but not in my terminal are ~/go/bin, ~/Library/pnpm and
~/.local/bin; of the 10 executables in those three directories, 9 reach me from nowhere else
in my terminal (govulncheck and goimports are two of them — which means that if I call
either in my own terminal, it is not found).
So the only real difference between the two shells is those three functions. Three lines, enough
to demolish an article's surprise; while all forty-three differences were an echo of the rig I
had built. Within twenty-four hours I made three floors of the same mistake: first I failed to
see that my shell had swapped my tool, then I contaminated the environment while comparing, then
I over-cleaned it while cleaning.
The day I counted my shell history the
lesson was close to this one — an assumption about a tool is not a measurement with it.
Questions for your own setup
The numbers from this machine tell you nothing; the questions do.
-
Before writing a counting measurement, run
type <command>. If the output says "shell function" or "alias", your measurement is measuring your shell, not your disk. Print the version in the report too. -
Do not confuse
commandwith an absolute path.command grepskips the function, not the PATH ordering; the only thing that guarantees whichgrepyou get is/usr/bin/grep. -
If you are comparing, reset the environment — but do not over-reset it.
env -iremoves PATH as well, and the shell falls back to its own default; to imitate a real login shell, passPATHwith your system's default value. -
If you changed PATH mid-session,
rehash. Otherwise the shell keeps using the location it noted earlier and your measurement reports the wrong winner. -
Read your PATH end to end, once. Which directories are missing, which line comes from which
file, whose leftovers are sitting in
/etc/paths.d— and are they really leftovers? -
Look for binaries that share a name. Database clients,
python3,node,git: if ordering picks the version, make sure you set that ordering deliberately. - Do not trust PATH in scheduled jobs. Write the program path absolutely; if you must set PATH, write out the three directories you need instead of copying your current shell. A frozen PATH becomes a list of non-existent directories six months later.
-
Wherever you see
#!/usr/bin/env, read it as "the interpreter will be chosen at run time." Fine if that is the behaviour you want in production; not fine if you meant to pin a version.
The name of a command is an intention; the program that runs is a decision
What I type on the keyboard all day is not a command, it is a command name. Four authorities
decide which program runs: aliases, my shell's function table, the order of PATH, and the first
line of the script. All four came from my own hands, but none of them is the work of a single
sitting — one came from a tool's install script, one from the seventh line of my .zshrc, one
from a directory deleted months ago.
That is why "it works on my machine" now feels incomplete to me. Not on which machine but in
which context: in my terminal (23 directories), in my agent's shell (26 directories, three
functions), in the midnight job opened with zsh -lc (17 directories, a different node), or at
07:15 in launchd's four? Same machine, same disk, four different answers.
What yesterday's discarded article gave me turned out to be a habit rather than a lesson: ask for
the tool's identity before you measure with it. If I do not know who shows up when I say grep,
the number I counted is not a measurement, it is an impression.
Official Sources
- zsh — Command Execution (
Doc/Zsh/exec.yo) - zsh —
command,whence,hashbuiltins (Doc/Zsh/builtins.yo) - PostgreSQL 16 — pg_dump, Notes
- Apple shell_cmds —
path_helper(8)man page - Apple Libc —
include/paths.h(_PATH_STDPATH) - Python Developer's Guide — Status of Python versions
- Node.js — Previous Releases
Top comments (0)