I Trusted Filename Equality for 48 Hours. One café Was NFD.

I spent a full forty-eight hours staring at one static folder that refused to match between two machines. The chat preview showed café-logo.png in both trees, and my tired eyes agreed with that preview. Did I really need another listdir pass after the rename helper landed in the generated patch? I thought a visual match was enough evidence, and that lazy assumption burned two working days.

The loader used a plain Python string as a dict key, then called Path.exists() on a concatenated name. Locally the file looked present, and the unit test that used the same literal stayed green. On the second box the same string missed the asset, and the map lookup returned nothing useful. How can two names look identical in a tab and still hash to different keys?

Field notes from the first wasted day

I treated this like every other agent patch that looked right in the transcript and wrong in production.

1. I copied the relative path out of the chat and pasted it into a Path(p).exists() check.
2. I ran ls in both checkouts and watched two café glyphs sit in the same column.
3. I printed sorted(os.listdir("assets")) on both machines and compared the rendered text.
4. I blamed a CDN cache, then an import-case bug, then a stale __pycache__ directory.

Have you ever watched two print(name) lines agree while name.encode("utf-8") does not? Terminal fonts compose NFC and NFD into the same café, and so do most editor tabs. ls without escapes is a screenshot, not a checksum, and I keep relearning that the hard way.

The fourth ghost was the wrong ghost. Case folding is a real Linux-versus-laptop trap, but this folder was already lowercase ASCII plus one accented letter. I did not need another lecture about Cafe versus cafe. I needed code points, byte lengths, and a filesystem that would store what I actually wrote.

What finally showed the split

I stopped asking the name to look right and asked Python to insult it.

python3 -c "import os; [print(repr(n), n.encode('utf-8').hex()) for n in os.listdir('assets')]"
ls -b assets

One entry printed a short UTF-8 sequence for é. The other printed the base e plus a combining acute, and that second spelling was two bytes longer. Same glyph. Different string. Different dict key. Different Path.exists() result when my source literal was the composed form.

That is Unicode normalization, not mysticism. NFC is the composed café most source files already contain. NFD is the decomposed café that some desktop tools, zip members, and designer exports still emit. Linux ext4 will happily keep both as two directory entries. Some desktop layers will fold, compose, or overwrite, and your eyes will not be invited to the meeting.

The artifact: write both spellings, then refuse to trust either

I now run a tiny script before I trust any rename an agent proposes. It is boring on purpose, and it is meant to be copied into a scratch directory.

#!/usr/bin/env python3
"""Show why two café filenames are not the same Python string.

Reproducible check, not a benchmark. Run it in every environment
that will serve, test, or package the files.
"""
from __future__ import annotations

import os
import unicodedata
from pathlib import Path

TARGET = "café-logo.png"  # what your code, YAML, or HTML refers to


def dump(label: str, value: str) -> None:
    raw = value.encode("utf-8")
    nfc = unicodedata.normalize("NFC", value)
    nfd = unicodedata.normalize("NFD", value)
    if value == nfc == nfd:
        form = "already-identical"
    elif value == nfc:
        form = "NFC"
    elif value == nfd:
        form = "NFD"
    else:
        form = "other"
    points = " ".join(f"U+{ord(ch):04X}" for ch in value)
    print(f"{label}: {value!r}")
    print(f"  utf8={raw.hex()} form={form} chars={len(value)} bytes={len(raw)}")
    print(f"  {points}")


def main() -> None:
    dump("literal in source", TARGET)
    dump("NFC(literal)", unicodedata.normalize("NFC", TARGET))
    dump("NFD(literal)", unicodedata.normalize("NFD", TARGET))

    folder = Path("assets")
    folder.mkdir(exist_ok=True)
    nfc_name = unicodedata.normalize("NFC", TARGET)
    nfd_name = unicodedata.normalize("NFD", TARGET)
    (folder / nfc_name).write_bytes(b"nfc")
    (folder / nfd_name).write_bytes(b"nfd")

    print("\nlistdir:")
    for name in sorted(os.listdir(folder)):
        dump("  entry", name)
        print(f"  == source literal: {name == TARGET}")
        print(f"  exists(literal path): {(folder / TARGET).exists()}")


if __name__ == "__main__":
    main()

Run the same file in every place the asset will live, including CI and the box that actually serves files.

python3 nfc_nfd_check.py
python3 -c "import unicodedata as u; s='café'; print(len(s), len(u.normalize('NFD', s)))"

On a filesystem that treats NFC and NFD as one inode, you may get a single file and a silent overwrite. On a typical Linux volume you may get two files that both render as café-logo.png. Which machine are you deploying to, really?

A decision table I wish I had on hour two

Check	API or command	NFC versus NFD	Why people miss it
Editor tab	open the folder	Looks equal	Fonts compose both forms
print(name)	Python REPL	Looks equal	The terminal composes too
ls	shell listing	Looks equal	No escapes, no code points
ls -b	shell listing	Escapes differ	Easy to skip in a long listing
name == "café-logo.png"	Python ==	False if forms differ	Source literals are usually NFC
dict key / hash(name)	assets[name]	Two keys	Loaders skip one entry
Path.exists()	pathlib	Depends on bytes on disk	Concatenated NFC misses NFD files
zip / S3 / Docker COPY	byte-preserving stores	Two objects	GUIs still show one glyph
unicodedata.normalize("NFC", name)	stdlib	Stable key	You have to opt in

I keep that table next to the asset loader now. Do I love extra process for a PNG? No. Do I love another forty-eight hours of fake missing files? Also no.

Where a second environment actually helped

One environment hid the split by folding the two writes. The clean box kept both directory entries and made the dict collision obvious. I needed a Python process on a second filesystem, not another screenshot from the same laptop.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. I used MonkeyCode's free model access to draft the rename helper, then ran the same nfc_nfd_check.py script on the free server option so I could compare os.listdir against my usual machine. I am not attaching a model name, a quota, or a hardware story, because those claims are not this bug. The useful part was a second filesystem I did not personalize with desktop helpers.

The model produced a perfectly readable patch. The patch used a café literal that matched my source tree visually and never called unicodedata.normalize. Why would it? The screenshot looked fine, and the unit test used the same composed literal as the source file.

A free model is handy for the boring rename boilerplate. A free server is handy because it is not your laptop's filesystem. If you already keep a throwaway Linux box for this class of bug, keep using it and skip the product mention entirely.

What broke when I fixed it the wrong way

I tried three bad fixes before the check script, and each one survived local tests.

• I told the loader to lower() every name. Case was never the bug, so the miss stayed in place.
• I switched to Path.glob("*café*"). Glob still uses the code points you pass in, so NFD files stayed invisible.
• I asked the model to make the names safe. It ASCII-folded café into cafe and broke the design-asset contract with the rest of the site.

The fix that survived both machines is intentionally loud:

import unicodedata
from pathlib import Path


def canon_name(name: str) -> str:
    return unicodedata.normalize("NFC", name)


def list_assets(folder: Path) -> dict[str, Path]:
    out: dict[str, Path] = {}
    for entry in folder.iterdir():
        if not entry.is_file():
            continue
        key = canon_name(entry.name)
        if key in out:
            raise RuntimeError(
                f"collision after NFC: {entry.name!r} vs {out[key].name!r}"
            )
        out[key] = entry
    return out

I canonicalize on the way in, and I refuse to paper over a collision. If both forms exist, I want CI to fail in the same hour, not a random survivor after deploy. Should a loader pick a winner quietly? Only if you enjoy guessing which café your CDN cached.

A test plan I would actually repeat

I would not repeat the wandering through caches and case folding. I would repeat this sequence, in this order, before another rename lands.

1. Write both NFC and NFD spellings into a scratch directory from Python, not from the editor.
2. Run nfc_nfd_check.py locally and on the deploy-like box, then diff the hex lines.
3. Assert that every incoming key, HTML reference, and YAML path uses canon_name.
4. Fail CI if two on-disk entries collapse to one NFC key.
5. Only then let an agent rename files, and re-run the script after the patch.

import unicodedata
from pathlib import Path

# Proposal: paste into pytest against the loader you actually ship.
def test_loader_does_not_split_cafe(tmp_path: Path) -> None:
    nfc = unicodedata.normalize("NFC", "café-logo.png")
    nfd = unicodedata.normalize("NFD", "café-logo.png")
    (tmp_path / nfd).write_bytes(b"nfd")
    assets = list_assets(tmp_path)
    assert nfc in assets
    assert assets[nfc].read_bytes() == b"nfd"

Labeling that honestly: the snippet is a pattern you can paste into pytest. It is not a screenshot of a private suite, and it is not a performance claim. If your loader talks to S3 or a zipfile, add one more test at that boundary, because those stores keep raw bytes too.

Git can participate in the same mess. If you share a repo across machines, inspect core.precomposeunicode and then still print repr(name) in the checkout you build from. Do not treat a green git status as proof that Python will see one name.

Limitations, and who should skip this

NFC is a convention, not a moral victory, and it will not save every path bug.

• Some operating-system APIs may still hand you NFD from os.listdir after you wrote NFC. Test the API you ship, not the API you wished you had.
• Windows can fold more than Unicode form, including case and trailing dots. This script will not save that path.
• Preservation pipelines sometimes must keep original bytes. Normalizing those names is data loss, and you should not silently rewrite them.
• HTML, S3 keys, zip members, and container layers are extra namespaces. Normalize at each boundary, or you will fix only the local folder.
• Do not treat a canonical filename as a cryptographic identifier over raw bytes unless you version that canonical form.

If you only ever build on one machine, with one language, and you never accept files from designers, skip the ritual. You are not the audience for this note. If an agent is allowed to rename assets across two filesystems, you are absolutely the audience.

What I would repeat next time

I would print encodings before I print feelings. I would treat every visually identical name as hostile until encode("utf-8") agrees on both boxes. I would run the same tiny script on a second filesystem before I trust a green local test that used my own source literal.

Would I still use a model to draft the rename helper? Yes, because typing glob patterns by hand is tedious and easy to typo. Would I ship that patch after a chat screenshot of two identical café glyphs? Not again, and not even if the diff rendered cleanly.

The green folder icon is not a checksum. café is not a single Python string. Forty-eight hours is a long time to learn that your eyes compose Unicode before your loader ever runs.