The Directory Listing Showed café.json. Python Still Raised FileNotFoundError.

I spent a 48-hour lab window on a FileNotFoundError that os.listdir kept contradicting in the same folder. The listing showed café-menu.json sitting beside the script, yet Path.read_text insisted that exact path did not exist. Have you ever copied a filename from the terminal, pasted it into a test, and still lost to the disk? This notebook is the reproduction I wish I had run on hour one, before blaming Docker.

Hour zero: the file is sitting right there

I started with the smallest script I could type from memory, because the traceback looked like a bad relative path. The module used pathlib.Path and a UTF-8 source file, so I assumed the editor name matched the disk name. Why would Python deny a file that a plain directory listing could already see?

# lab script — run from the extracted directory
from pathlib import Path

target = Path("café-menu.json")
print("exists", target.exists())
print("cwd json", list(Path(".").glob("*.json")))
print("open attempt")
target.read_text(encoding="utf-8")

On hour zero, glob printed the json file, and exists() returned False inside the same process. That pairing is rude, and it is also a clue. If glob can see a name, the directory is readable, so the failure is equality rather than permission. I had been debugging a missing file. I was actually debugging two strings that rendered as one word.

What I tried before I printed code points

I did the usual path hygiene first, because most missing-file bugs are still boring. The list below is the order I actually ran, not the order I wish I had run.

1. Print Path.cwd() and Path.resolve() for both the script and the target.
2. Switch from a relative name to Path(__file__).resolve().parent / name.
3. Call os.listdir() and repr() on every entry, hunting for a sneaky trailing space.
4. Re-extract the zip that had produced the fixture files in the first place.
5. Blame the container, then blame macOS, then blame my own typing of é.

repr() finally earned its keep, though it still looked innocent in a proportional font. One listing showed 'café-menu.json' while the name in the editor also showed 'café-menu.json' to my eyes. Same glyphs. Different code points. Have you checked code points the last time a filename looked perfect?

import os
import unicodedata

for name in os.listdir("."):
    if "menu" not in name:
        continue
    print(name)
    print([hex(ord(ch)) for ch in name])
    print("is NFC", unicodedata.normalize("NFC", name) == name)
    print("is NFD", unicodedata.normalize("NFD", name) == name)

The extracted file used NFD: Latin e plus combining acute U+0301. The string in my source file used NFC: a single é stored as U+00E9. Python compares those strings by code point, not by how a terminal draws them. The operating system may be stricter or sloppier than Python, which is how the same test flips across machines.

Hour six: a green helper on the other machine

I wanted a second opinion that was not another tab in the same local shell. I pasted the failing snippet into MonkeyCode, which the operator supplies with free model access and a free server option.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The session returned a compact helper that checked Path.exists() and listed nearby files when the open failed. That helper came back green on the Linux server, because the filename I created there was stored in NFC. My laptop extract still used NFD. Did the model fail, or did I ask it to debug a Unicode personality it could not see from that disk?

The second machine was still useful, just not as an oracle. It became the NFC control environment, and the laptop became the NFD experiment. Once I stopped treating a generated helper as a verdict, the split itself turned into the test plan. Would you trust a green exists() that never printed ord()?

The artifact: write both forms and compare

This is the lab I now run whenever a UTF-8 filename looks present and absent at once. It needs only the standard library. It writes one NFD name, then reports which form the platform preserved and which form Python can still open.

"""filename_forms.py — lab helper, not a library."""
from __future__ import annotations

import os
import tempfile
import unicodedata
from pathlib import Path

RAW = "cafe\u0301-menu.json"  # NFD: e + combining acute
NFC_NAME = unicodedata.normalize("NFC", RAW)
NFD_NAME = unicodedata.normalize("NFD", RAW)


def inspect(dir_path: Path) -> dict[str, object]:
    names = os.listdir(dir_path)
    menu_names = [n for n in names if "menu" in n]
    return {
        "nfc_name": NFC_NAME,
        "nfd_name": NFD_NAME,
        "same_python_string": NFC_NAME == NFD_NAME,
        "nfc_in_listing": NFC_NAME in names,
        "nfd_in_listing": NFD_NAME in names,
        "nfc_exists": (dir_path / NFC_NAME).exists(),
        "nfd_exists": (dir_path / NFD_NAME).exists(),
        "listing_codepoints": {
            n: [hex(ord(ch)) for ch in n] for n in menu_names
        },
    }


def run_lab() -> dict[str, object]:
    with tempfile.TemporaryDirectory() as raw:
        root = Path(raw)
        # Write using NFD in the path. The OS may rewrite the stored form.
        (root / NFD_NAME).write_text("{}", encoding="utf-8")
        return inspect(root)


if __name__ == "__main__":
    for key, value in run_lab().items():
        print(f"{key}: {value}")

Run it in two places before you trust a report that says the file is missing.

python filename_forms.py

On a typical Linux ext4 workspace, nfc_exists and nfd_exists often disagree, because the stored bytes are one specific form. On macOS, lookup is often normalization-insensitive, so both exists() calls may return True even while nfc_in_listing stays False. That last sentence is the trap. Tests pass on a laptop and fail in Linux CI, or the reverse, without a logic bug in your parser.

Zip files make the same split portable. If the archive stored NFD names from a Mac extract, Linux open() will not invent NFC for you.

import zipfile
from pathlib import Path

def list_zip_name_forms(archive: Path) -> None:
    with zipfile.ZipFile(archive) as zf:
        for info in zf.infolist():
            name = info.filename
            print(name, [hex(ord(ch)) for ch in name])

A small pytest contract

Label this as a lab test. I ran it as a reproduction; the comments are the contract, not a production suite.

# test_filename_forms.py
import unicodedata
from pathlib import Path


def test_nfc_and_nfd_are_not_python_equal():
    nfc = unicodedata.normalize("NFC", "café")
    nfd = unicodedata.normalize("NFD", "café")
    assert nfc != nfd
    assert [ord(ch) for ch in nfc] == [99, 97, 102, 233]
    assert [ord(ch) for ch in nfd] == [99, 97, 102, 101, 769]


def test_indexes_must_use_one_form(tmp_path: Path):
    nfd = unicodedata.normalize("NFD", "café-menu.json")
    nfc = unicodedata.normalize("NFC", "café-menu.json")
    (tmp_path / nfd).write_text("ok", encoding="utf-8")
    names = {
        unicodedata.normalize("NFC", path.name)
        for path in tmp_path.iterdir()
    }
    assert nfc in names

The second test does not claim Path.exists(nfc) will fail everywhere. It claims your in-memory index should not fork on rendering. That is the bug that survived my first patch.

Decision table from the notes

Signal	Likely form issue?	Next probe
exists is False, glob is True	Yes	Print hex(ord(ch)) for both strings
Failures only in Linux CI	Often	CI stored NFC; laptop lookup was insensitive
Failures only after unzip	Often	Archive stored NFD from a Mac-side zip
Failures after git clone	Sometimes	core.precomposeunicode differs by clone host
ASCII-only names	No	Return to cwd, permissions, and mount points

What broke after the first "fix"

I first normalized only the string I passed to open(). That hid the error in one function and left every dict keyed by filename still split in two. Cache keys, etag maps, and already-processed sets treated NFC and NFD as different files. Have you counted how many filename dicts live in a downloader?

I also considered always calling os.listdir() and picking the unique visual match. That is a security smell on a case-insensitive disk, and it is slow on large directories. Worse, two visually identical names can coexist on Linux. A fuzzy match can open the wrong document, which is a worse incident than FileNotFoundError.

The repeatable rule was narrower than those patches. Normalize to NFC for comparison keys, and log raw code points whenever a name-based lookup fails. Leave the bytes on disk alone unless your code owns the write path and has a migration plan.

What I would repeat

I would still spend the 48 hours, but I would spend them on evidence instead of folklore. The checklist below is what I would rerun on the next visually identical miss.

1. Print code points before printing glyphs, because glyphs lie politely.
2. Keep one NFC control environment and one NFD experiment, even if the second machine is throwaway.
3. Store canonical names in indexes, not whatever os.listdir() returned on today's host.
4. Refuse to treat an assistant's green run as evidence unless it ran against both forms.

Would I send customer-uploaded filenames into a hosted model? No. The useful part of that session was the second operating system, not a clever prompt. If you already have Linux CI and a Mac laptop, you do not need a product in the middle. If you do not, a free server session is a cheap way to meet the other Unicode personality, and that is the only reason it belongs in these notes.

Limitations, and who should skip this

This lab does not tell you which form is morally correct. NFC is a convention for keys, not a Unicode commandment. Backup tools, forensic imagers, and rsync-style copiers should preserve exact directory bytes and not rewrite names underfoot. Windows, macOS, and Linux also disagree about whether lookup is normalization-insensitive, so a green exists() is not proof that listdir will return your literal string.

Skip this approach if you cannot put sample names on a second machine. Skip it if the names themselves are sensitive, because a filename can be personal data. Skip it if your runtime is a browser, because this notebook is about Python and POSIX-ish filesystems. Skip any generated helper that only checks exists() without printing code points. That helper already fooled me once, and it will fool the next review if nobody asks what é actually is.

The file was on disk the whole time. Python compared two different strings that looked like one word. After this lab I stop trusting my eyes for filenames, and I start trusting ord().