Months ago something was written down, and I was certain the file was called
MEMORY.md. Today I went looking for that note.
$ mdfind "kMDItemFSName == 'MEMORY.md'"
$
Empty. No error, no warning, no "there are places I could not look." Just a
blank line and the next prompt. The file was right where I left it; find
turned it up in half a second.
Trusting a search box instead of filing things into folders is a habit of mine
too. Not having to remember where you put something is a pleasant kind of
relief — right up until you learn there is a place the box does not look. I
spent half an hour of the afternoon counting how big that place is, and the
number bothered me.
Every measurement here was taken on 9 October 2026, on macOS 26.6.2 (build
25G83, xnu-12377.161.14, arm64); the file counts are a single snapshot taken
at 14:15 and drift by a few hundred as the machine works. The counting commands
are pure reads; the experiments create their own throwaway files, and one of
them enables indexing on a disposable disk image. Nothing here touches the
machine's own configuration, and the leftovers were deleted afterwards.
I released canaries, and one of them sang
I did not want to draw conclusions from a single file. I wrote the same content
— a random search token — into six different places and counted which ones the
index could see. The first row in that list is my positive control: an
ordinary text file in an ordinary, visible directory. If that one is not found,
my instrument is broken and nothing below is a finding.
$ bash kanarya.sh
LOCATION IN INDEX
gorunur/kanarya.txt YES
.gizli/kanarya.txt no
gorunur/.kanarya-nokta.txt no
yok.noindex/kanarya.txt no
markerli/kanarya.txt YES
/tmp/kanarya.txt no
The positive control sang: the visible file entered the index within fifteen
seconds, by name and by content both. So the instrument works, and the four
remaining misses are real.
Three of them had a dot in common. If the directory name starts with a dot, if
the file name starts with a dot, or if the path runs through a directory ending
in .noindex, the content never enters the index. /tmp is out of scope too —
I had originally set the lab up there, found nothing at all, assumed my
instrument was broken, and moved everything into my home directory.
The fifth row did not come out the way I expected. I will come back to it.
Apple wrote this down twenty years ago
Rather than guess at the behaviour, I went looking for it in the documentation.
The local man mdfind page says nothing: it "consults the central metadata
store" and offers not one word about what that store covers. The right place
turned out to be a much older technical note, Apple's
QA1497.
It is not even about this — it is about files that refuse to be deleted — but
the list I wanted is sitting inside it:
Names that won't get indexed include: Names beginning with a ".", Names
ending with ".noindex"
And among the places that do not get indexed, it counts the standard UNIX
temporary directory, /tmp. The document lists name rules; it does not say
that the whole tree beneath a dot-directory falls outside as well — that part is
my measurement, not Apple's statement. The same note adds that these lists are
not exhaustive and that what Spotlight skips is likely to change over time.
One caveat. That page was published on 17 November 2006, was last revised on
24 September 2008, and carries a banner saying it is no longer being updated.
Leaning on a twenty-year-old archive document to explain today's behaviour would
not normally be good enough. Here the order was reversed: the measurement came
first, the document second. The rule still holds on 26.6.2.
What bothers me is not the rule itself. The rule is reasonable: files beginning
with a dot are traditionally a tool's own private ledger, and you do not want to
search those. But it has a side effect that appears nowhere in the
documentation. The rule stayed fixed while what people put behind a dot changed.
In the era this note was written, a shell configuration file lived there. In
2026 I had to go count what lives there now.
Forty-seven per cent of my home directory
Is one example enough? It is not. I walked my entire home directory twice: once
counting everything, and once excluding anything whose path contains a component
beginning with a dot.
all files under ~ : 2,516,736
files with no dot component in the path: 1,322,015
→ left behind a dot (derived) : 1,194,721 (47.5%)
top-level hidden directories : 66
files inside those 66 directories : 428,233
I am giving the last two lines separately, because not all of that 1.19 million
sits at the top level; dot-directories also nest inside visible folders. The
third line was not measured — it is subtracted from the first two, which is why
it says "derived."
Every hidden directory holding more than five thousand files:
136,570 files 4.9G .rustup
57,706 files 1.0G .nvm
50,224 files 3.4G .cache
46,087 files 1.0G .cargo
41,978 files 844M .pub-cache
24,711 files 472M .dartServer
24,187 files 1.3G .gradle
13,535 files 8.2G .codex
8,787 files 1.2G .local
7,085 files 3.7G .claude
Let me be honest about this list: most of it is cache. I have no wish to search
the hundred and thirty-six thousand files inside .rustup, and I am grateful
Spotlight does not. On its own, 47.5% is not a scandal. It is a scale.
What stopped me is the last row. The .claude directory holds 699 markdown
files:
.md files under ~/.claude : 699
tool documentation (plugins/) : 226
other tool files : 16
memory notes : 457
found by mdfind in the same set : 0
Those 457 files are the memory notes kept on my behalf by the tool I delegated
remembering to: how a project was set up, why a decision went the way it did,
which trap caught someone twice. Their front matter says node_type: memory and
not one of them carries an author field — I did not type these, they were kept
for me. But their purpose is exactly that: not forgetting.
Zero. The oldest note is dated 11 February 2026; for eight months these files
have been accumulating what I should not forget, and for eight months the search
box has never seen them. I handed remembering to a tool, the tool wrote the
notes into a dot-directory, and the search box does not look there.
A sharper version of the same contradiction sits in this blog's own repository.
I keep the repository in two places: a visible working copy, and the working
copies that agents use, underneath the repository's hidden .claude/worktrees/
directory. I counted the same folder, src/content/blog, in both:
visible working copy : 2,586 .mdx on disk → mdfind finds 2,586
one agent work copy : 2,650 .mdx on disk → mdfind finds 0
Same machine, same file format, same moment, nearly the same count. One is
visible in full, the other not at all. The only difference between them is a dot.
The file the internet recommends does nothing
Back to the fifth row of the canary table. Inside markerli/, before writing
the file, I had placed an empty file named .metadata_never_index. That is the
advice that has been circulating for years for hiding a folder from Spotlight.
The file was indexed anyway.
On the first attempt the marker and the content were created in the same second,
so I suspected a race. I set it up again: marker first, eight seconds of
waiting, then the file.
marker placed FIRST, file written AFTERWARDS
→ mdfind: BASE2/markerli/kanarya-markerli-1791542155.txt
Indexed again. In an ordinary folder, the marker accomplishes nothing.
It occurred to me that the marker's real home might be the root of a volume,
so I tested that too. I created and mounted a twenty-megabyte APFS disk image.
The first finding came straight away: a freshly mounted disk image is not
indexed at all by default.
$ mdutil -s /Volumes/SLLAB...
Indexing disabled.
Which means everything you drop into a .dmg is outside search before you lift
a finger. Then I enabled indexing by hand — the positive control worked, both
files entered the index — and after that I put .metadata_never_index at the
volume root and wrote a new file. The third file was indexed as well. The marker
changed nothing, neither in a folder nor at a volume root.
Apple's QA1497 lists dot names and .noindex names; .metadata_never_index
does not appear in it anywhere. I will not claim what the document does not say:
whether this file ever worked, I do not know. What I can report is that it does
not work on this machine today. The documented way to exclude something is the
Privacy list:
you add the folder or disk by hand in System Settings. Removing an item from
that list and adding it back is also
Apple's prescribed way to reindex.
Apple's own warning sits right there too: exclude certain folders and you may
stop being notified when updates are available for some apps.
What I caught was not the tool, but how I was asking
When I first wrote this article, this section claimed something false. The
correction turned out to be more instructive than the rest of the piece.
Reaching for mdls to find out whether a file is in the index feels sensible —
it is the metadata tool, after all. On the first pass I asked it like this,
naming the attributes I was curious about one by one:
$ mdls -name kMDItemContentType -name kMDItemKind -name kMDItemTextContent <file>
# file inside .gizli/ (NOT indexed) # inside gorunur/ (indexed)
kMDItemContentType = "public.plain-text" kMDItemContentType = "public.plain-text"
kMDItemKind = "Plain Text Document" kMDItemKind = "Plain Text Document"
kMDItemTextContent = (null) kMDItemTextContent = (null)
Indistinguishable. From this I concluded that mdls does not reveal index
membership. That was wrong. I asked the same two files again, this time with a
bare mdls and no attribute names at all:
NOT indexed → 15 attributes, every one of them starting with kMDItemFS*:
kMDItemFSName = "kanarya.txt" kMDItemFSSize = 21 kMDItemFSOwnerUserID = 501 …
indexed → 47 attributes, among them:
_kMDItemPrimaryTextEmbedding = ( { "vec_dim" = 1; "vec_id" = 19903; … } )
kMDItemContentType = "public.plain-text"
kMDItemKind = "Plain Text Document"
kMDItemDateAdded = 2026-10-09 10:45:54 +0000
kMDItemTextContent = (null)
Fifteen against forty-seven. A bare mdls tells the two apart instantly: for a
file with no index record there is no kMDItemContentType, no kMDItemKind, no
kMDItemDateAdded — only the file system's own fields. The text embedding
Spotlight keeps for the indexed file is even sitting there in the output.
So why did they look identical on the first attempt? Because when you ask for an
attribute by name with -name, the tool computes it for you on the spot — even
for a file with no record in the index. The way I phrased the question was
quietly turning "this file has no index record" into "here are your attributes."
mdls did not lie to me; I asked the wrong question and got a plausible answer.
Which is this entire article's subject — except that this time the one who fell
into the trap was me, while writing about the trap.
A hard link works, and then goes stale without telling you
So how do we make those notes findable again? I tried the two obvious routes: a
symbolic link from a visible folder to the hidden file, and a hard link to the
same file.
$ mdfind "zqxbaglanti..."
~/sl-gorunur-.../hard-not-....txt
The symbolic link never showed up — neither the one pointing at a file nor the
one pointing at a directory. The hard link, on the other hand, entered the
index, by name and by content. And it is not a copy; it is the same file:
411854173 2 link ~/.sl-kaynak-.../not-....txt
411854173 2 link ~/sl-gorunur-.../hard-not-....txt
The same inode number, two links. The index is not keyed on the file itself but
on the name that leads to it. The moment you give the same content a second,
visible name, it becomes searchable. When I edited the file through the hidden
path, the index caught up within ten seconds.
Then I broke it. Most text editors do not write a file in place; they write to a
temporary file and move it over the original. I imitated that "atomic save"
behaviour by hand:
411854655 1 link ~/.sl-kaynak-.../not-....txt ← new inode
411854173 1 link ~/sl-gorunur-.../hard-not-....txt ← old content
new content in the index : 0
The link snapped. The two files are separate inodes now, each with a link count
of one. The new content does not enter the index, and the visible hard link goes
on serving the old content without mentioning it. Search hands you a result,
the result opens, and what you were looking for is not inside. That is worse
than getting no result at all.
The fix that works: reverse the arrow
Because the hard link is fragile, I tried one more thing, and this is the route
that holds: put the real directory somewhere visible and make the dot path a
symbolic link to it. Reverse the direction of the link. Your tool still writes
to ~/.whatever/notes; the files actually live under ~/notes.
~/.lab-nokta-... -> ~/lab-gercek-... (the dot path is now the symlink)
written through the dot path → mdfind: ~/lab-gercek-.../not.txt ✓
atomically saved through that path → mdfind: ~/lab-gercek-.../not.txt ✓
old content still in the index? → 0 (it does not go stale)
An atomic save breaks nothing here, because the new file is created inside the
visible directory to begin with; the dot sits at the doorway, not on the file.
One caveat: if your tool deletes and recreates its own directory it will take
the symlink with it, so test the arrangement once with a canary after setting
it up.
Four questions worth asking of your own setup
I do not want to reduce this to a recipe, because the right answer depends on
what you keep. These are the questions I put to myself.
Where do the things I delegated remembering to actually live? Run
ls -d ~/.* and look for your note-taking tool's data. If it begins with a dot,
you cannot find those notes from the search box — and if you never tried, there
was no way to notice.
Do I have a second reflex when a search comes back empty? I did not; an
empty result meant "it was never written down" and I moved on. Now I look behind
the dot with rg --hidden or find. Neither one skips hidden directories.
Does my backup cover the same set the search does? They are not the same
set. Those 1.19 million files behind a dot are absent from search but present on
disk, and most of them should be backed up. It is worth comparing the scope
against a file count once.
Did I actually hide the thing I meant to hide? If you placed a
.metadata_never_index, then no. The documented route is the Privacy list — and
after you use it, test it the canary way: write a random word into a file and
look for it with mdfind. It takes thirty seconds.
An empty result is not evidence of absence
What I was left with at the end of the day is not a macOS detail. In the
behaviour measured here, nothing is working badly: Spotlight follows its
documented rule to the letter, mdls answers correctly the question I put to
it, and the hard link does exactly what the file system says it should. The only
thing that was wrong was my taking an empty result for evidence.
We have learned to be wary of tools that throw errors. We have not learned to be
wary of tools that come back empty, because an empty result looks exactly like a
correct one. "It is not there" and "I did not look there" render identically on
screen, and the only way to tell them apart is to deliberately release a canary.
In the mdls section I walked into that same trap myself: I got a plausible
answer and assumed my question had been right.
This machine taught me the same lesson three days ago in a different costume:
there were
146 directories in my home folder that I own and cannot open,
and a file-listing tool was swallowing the error and reporting "no problems."
What went missing there was permissions; here it is names. What they share is
this: the machine handed me an incomplete answer dressed as a complete one.
Will I move the notes? I do not know; where a tool puts its files is not my
decision — but I now know I can put a symbolic link at the doorway. I look at
the search box differently too. It does not tell me what is on my disk — it
tells me what is in the places it is allowed to look. I had never seen the gap
between those two sentences as a gap, because the box never mentioned it. Nor
did it have to; I was the one asking.
Top comments (0)