macOS Made a File That Reports Zero Bytes and Occupies Sixty Kilobytes

I was writing a storage analyser, which means walking every file on the disk
and adding up what it costs. Somewhere around the 140,000th file the progress
line printed a path that does not exist:

/Users/me/Applications/Chrome Apps.localized/localized

There is no localized in that folder. There is a .localized, and there is
one other thing, and that other thing turned out to be two separate problems
wearing one filename.

The file

$ ls -b ~/Applications/Chrome\ Apps.localized/
Icon\r

Not Icon followed by something. The name is the four characters I, c,
o, n, and then byte 0x0D — a carriage return. Python is blunt about it:

>>> os.listdir(d)
['Icon\r', '.localized']

macOS wrote that. Every folder with a custom icon gets one: Finder → Get Info →
paste an image, and this appears. The trailing CR is a fossil from classic Mac
OS, where it was the line terminator, and it is there to keep the file out of
the way of anything that lists a directory naively. It has outlived the reason.

Problem one: it eats your log line

A filename on a POSIX system may contain any byte except / and NUL. That
includes carriage return, newline, escape, and backspace. Almost nothing you
write treats a path as anything other than text you can print, and this is
where that assumption gets paid.

Here is what a progress line actually emits:

$ find box -type f -print0 | while IFS= read -r -d '' f; do
    printf 'scanning %s  42,318 files\n' "$f"
  done | hexdump -C
00000000  73 63 61 6e 6e 69 6e 67  20 62 6f 78 2f 49 63 6f  |scanning box/Ico|
00000010  6e 0d 20 20 34 32 2c 33  31 38 20 66 69 6c 65 73  |n.  42,318 files|

Look at offset 0x11: 6e 0d — the n of Icon, then 0d, the carriage
return. The terminal receives it and does what a terminal has always done with
a CR: it moves the cursor back to column zero. The rest of the line then
overwrites what you just printed. You do not see a corrupted path. You see a
line that is missing its front half, spliced onto whatever came next — which
is precisely as confusing as it sounds when the count in front of it is also
changing.

The same byte does the same damage to a single-line label in a GUI, a CI log,
and anything that later parses its own output. And the CR is the polite one: a
filename containing \n will break find | while read line outright, because
that loop's entire contract is that filenames do not contain newlines. Hence
-print0 and read -d '', which exist for this reason and not for style.

If you render paths anywhere a human will look, strip the control range first:

final safe = path.replaceAll(RegExp(r'[\x00-\x1f\x7f]'), '?');

Problem two: it is zero bytes, and it is sixty kilobytes

This is the one that actually mattered for a disk tool.

stat -f %z         (st_size)        0 bytes
ls -l                              0
os.path.getsize()                  0
------------------------------------------------
st_blocks × 512                61,440 bytes
du -k                              60 KB

The data fork is genuinely empty. The icon — 61,010 bytes of it — lives in an
extended attribute:

$ xattr ~/Applications/Chrome\ Apps.localized/Icon^M
com.apple.FinderInfo
com.apple.ResourceFork
com.apple.provenance
com.apple.quarantine

com.apple.ResourceFork is the resource fork, the other half of a classic Mac
file, kept alive on APFS as an xattr and reachable through a magic path:

$ stat -f %z ~/Applications/.../Icon^M/..namedfork/rsrc
61010

st_size describes the data fork. It always has. So a scanner that sums
st_size — which is the obvious thing to sum, and what os.path.getsize hands
you — reports this file as free. Sixty kilobytes, invisible, once per folder
with a custom icon.

Sixty kilobytes is nothing. The habit is not nothing. st_size is a statement
about content; st_blocks is a statement about the disk, and a disk usage
tool is answering the second question. They come apart in three directions and
resource forks are only one of them:

• Sparse files report a large st_size and occupy almost no blocks.
• Compressed files — macOS transparently compresses much of /System and many app bundles — report the uncompressed size and occupy less.
• Resource forks and xattrs report nothing and occupy real space.

Sum st_size across a volume and your total will not match About This Mac, in
either direction, and the user will believe Apple. Correctly. du has always
used st_blocks; that is the whole reason du and ls -l disagree, and I had
read that fact several times without it meaning anything until a zero-byte file
turned up holding an icon.

The catch, if you write Dart

Knowing to use st_blocks and being able to are different problems. Dart's
FileStat exposes size, mode, type and three timestamps. There is no
blocks. File.length() is st_size as well. The number you want is in the
stat struct the VM already called and did not surface, so reaching it means
going around the standard library:

// dart:ffi against stat(2), or a platform channel to
// URLResourceKey.totalFileAllocatedSizeKey on the Swift side.

Which is the actual reason so many size-summing tools get this wrong. It is not
that st_size is chosen over st_blocks; it is that in most high-level
languages st_size is the only one handed to you, and the difference between
"how big is this file" and "what does this file cost me" never comes up until
something reports zero and takes sixty kilobytes.

Two rules I would now write down before starting a tool like this:

1. Sum st_blocks, not st_size, and if the language will not give you st_blocks, that is a dependency you take on day one rather than a refactor you do after the totals disagree with About This Mac.
2. Strip [\x00-\x1f] from any path before rendering it. Paths are bytes, not strings, and one of them on your Mac right now ends in a carriage return.

The file is still there. It is on your Mac too — find ~ -name 'Icon?' -type f
will tell you, and the ? is the only way to match it that does not require
you to type a carriage return into your shell.

---

Originally published at devshakib.jumyn.com. I write about Flutter, Dart and the parts of shipping that are genuinely awkward — and publish the packages that came out of them at pub.dev/publishers/jumyn.com.