I Debugged ZoneInfoNotFoundError for 48 Hours. /usr/share/zoneinfo Was Empty.

Have you ever shipped a timezone helper that looked perfect on your laptop, then died on the first remote run? I spent two days on that exact failure, and the stack trace was almost insultingly short. Local tests were green, but the remote process raised ZoneInfoNotFoundError before it parsed a single timestamp. This field notebook is the checklist I wish I had printed at hour zero.

Hour 0: The Symptom Looked Like Bad Input

I assumed the payload was dirty, because that is the boring explanation that is usually right. Why would a well-known IANA name fail after a tidy wrapper landed in the tree? I printed the incoming strings, dumped repr(), and compared them against the zone list I keep in notes. Nothing was misspelled, and America/New_York was exactly that ASCII string.

Then I did the thing every Python person does under pressure, which is still the right first probe. I ran the same one-liner in two shells and trusted the first result too much.

python -c "from zoneinfo import ZoneInfo; print(ZoneInfo('America/New_York'))"

The laptop printed America/New_York. The remote shell raised before it printed anything useful at all. Same interpreter family, same zone string, and a completely different machine. That gap, not the payload, is the whole story.

What I Tried Before I Looked At tzdata

I did not jump to missing zone files, because four other theories usually own this class of bug. Would you have listed /usr/share/zoneinfo before touching encoding, versions, or the generator? I would now. I did not then.

1. Hidden characters. I checked for a Unicode hyphen that only looked like a hyphen. repr() said the name was plain ASCII.
2. Interpreter drift. Both sides were 3.12+, so zoneinfo lived in the stdlib and was not a missing backport wheel.
3. Silent rewrites. I diffed the helper against the first generated copy, and the wrapper had not changed.
4. Stale image layers. I rebuilt the remote environment once without cache, and the crash stayed put.

Each of those checks was cheap, and none of them were wasted work. They just were not the empty directory that actually broke the run.

Hours 12–24: A Slim Remote Shell Finally Helped

I needed a box that looked more like CI than the laptop I debug on every afternoon. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I used MonkeyCode's free model access to draft the helper, then ran the same script on the free server option so I was not debugging against desktop tz files.

That pairing mattered because a model will happily emit ZoneInfo("America/New_York") in a helper that looks production-ready. Your laptop will happily import that name because desktop images ship an IANA database. A slim Linux box will not, unless tzdata is installed or the PyPI tzdata package is on sys.path. The failure is not "the generated code cannot spell a zone." The failure is "this runtime has no timezone database."

# Remote probes I finally ran after blaming the payload for too long.
ls /usr/share/zoneinfo/America 2>/dev/null | head
python -c "import zoneinfo; print(zoneinfo.TZPATH)"
dpkg -l tzdata 2>/dev/null || rpm -q tzdata 2>/dev/null || true

The America directory was missing, TZPATH pointed at the usual system roots, and no tzdata package was installed. After that, the exception stopped looking mysterious and started looking like an ops problem.

The Repro I Now Drop Into The Repo

Do not trust a chat transcript that says the timezone helper is fine. Pin a test that fails closed when the database is absent, and run it on every machine that will execute the job. The example below is a local pytest module you can copy; treat it as an executable checklist, not as a claim about any hosted quota or image.

# tests/test_tzdata_present.py
from __future__ import annotations

import os
from datetime import datetime, timezone
from pathlib import Path
from zoneinfo import ZoneInfo, ZoneInfoNotFoundError

import pytest

REQUIRED_ZONES = (
    "UTC",
    "America/New_York",
    "Europe/London",
    "Asia/Shanghai",
)


def _zoneinfo_roots() -> list[Path]:
    roots: list[Path] = []
    for raw in os.environ.get("PYTHONTZPATH", "").split(os.pathsep):
        if raw:
            roots.append(Path(raw))
    roots.extend(Path(p) for p in ("/usr/share/zoneinfo", "/etc/zoneinfo"))
    return roots


def test_system_has_some_zoneinfo_files() -> None:
    existing = [p for p in _zoneinfo_roots() if p.exists()]
    assert existing, (
        "No tzdata files on this machine. Install the OS tzdata package "
        "or add the PyPI tzdata extra, then rerun."
    )


@pytest.mark.parametrize("name", REQUIRED_ZONES)
def test_required_zones_resolve(name: str) -> None:
    try:
        zone = ZoneInfo(name)
    except ZoneInfoNotFoundError as exc:
        pytest.fail(
            f"{name} missing ({exc}). Slim images need tzdata; "
            "UTC-only code should use datetime.timezone.utc instead."
        )
    assert str(zone) == name


def test_utcnow_replacement_is_aware() -> None:
    now = datetime.now(timezone.utc)
    assert now.tzinfo is not None
    assert now.utcoffset() is not None

Run the same module on both machines before you generate more clock code.

python -m pytest tests/test_tzdata_present.py -q

On a desktop with tzdata, the file should pass. On a slim server without those files, it should fail on the first assertion. That red bar is the point, because you want it before customer timestamps ever touch astimezone().

Decision Table From Hour One, Written At Hour Forty

Use this when a generator, or a teammate, reaches for timezone names inside application code.

Need	Use	Avoid
Store and compare instants	datetime.now(timezone.utc) and keep values aware	datetime.utcnow() (naive, deprecated since 3.12)
Display in one product timezone	ZoneInfo plus a proven tzdata install	Hard-coded offsets such as -5
Parse Z suffixes only	datetime.fromisoformat after normalizing Z to +00:00	strptime without %z
Minimal container, no tz files	UTC-only helpers, no IANA city names	ZoneInfo("America/New_York")
Must use IANA on slim Linux	OS tzdata package or the PyPI tzdata package	Hoping the stdlib ships the database

datetime.utcnow() still shows up in generated snippets, and it is still the wrong default. It returns a naive datetime, which is why 3.12 marked it deprecated. Prefer datetime.now(timezone.utc) and keep the tzinfo attached.

If you only ever store UTC instants, you do not need IANA names at all. That is the lesson I resisted for a full working day, because city names look more "complete" in a helper.

What Broke When I "Fixed" It The First Time

I pip-installed tzdata into a user site-packages directory and celebrated too early. The next process still crashed, because it ran as a different user and never saw that extra path. Have you ever fixed an import for your own shell and then watched the service user miss it? I have, twice, on this bug.

The second fix added tzdata to requirements.txt and rebuilt the environment, which made the zone names resolve. A display test still drifted, because I mixed naive datetime.now() with an aware ZoneInfo conversion. Python will raise on some of those mixes, and it will let others through if you convert in the wrong order.

The third fix banned naive datetimes in this module. No clever clock math, just a guard that fails closed.

from datetime import datetime, timezone
from zoneinfo import ZoneInfo


def now_utc() -> datetime:
    return datetime.now(timezone.utc)


def to_zone(instant: datetime, name: str) -> datetime:
    if instant.tzinfo is None:
        raise ValueError("refusing naive datetime; pass an aware instant")
    return instant.astimezone(ZoneInfo(name))

Would I let a generator write to_zone without that naive guard? Not again, not even for a sample script.

Hours 36–48: Commands I Will Repeat

This is the list I now run before I trust generated datetime code on any remote box. I run it in the same process identity that will execute the job, not in a throwaway admin shell.

1. Resolve UTC and one city name: python -c "from zoneinfo import ZoneInfo; ZoneInfo('UTC'); ZoneInfo('America/New_York')"
2. If I expect the PyPI extra, print its files: python -c "import tzdata, os; print(os.path.dirname(tzdata.__file__))"
3. Run the pytest module above plus any clock tests in one invocation.
4. Print datetime.now(), datetime.now(timezone.utc), and time.tzname from the same process.
5. Fail the build when a production helper accepts a naive datetime without raising.

I also keep a comment in the bootstrap script: install OS tzdata when the app converts to civil time. If the app is UTC-only, I delete the IANA calls instead of adding a package. Smaller images stay smaller, and the test file still documents the choice.

Who Should Not Use This Approach

Skip the IANA-plus-tzdata path if you only persist UTC instants and never format city-local clocks. Skip remote iteration with a hosted agent if your rules forbid sending source to that environment. Skip ZoneInfo entirely on platforms where you cannot install tzdata and cannot vendor the PyPI package.

This workflow is for people who already generate helpers with a model and then run them somewhere that is not a laptop. It is not a substitute for a CI image that matches production packages, users, and filesystems. The free remote shell helped me see the missing directory. It did not prove the production host was safe.

What I Would Repeat

I would still let a model draft the first wrapper, because the boring UTC helper is not where I want to spend attention. I would not still believe the first green local run, and I would not debug payload JSON for twelve hours when ZoneInfo cannot even construct. I would copy the pytest file above into the repo before generating more clock code, then run it on the Linux shell that actually executes the job.

The crash was never the timestamp string. The crash was an empty path under /usr/share/zoneinfo, which every desktop hides from you. Two days is a long time to discover a missing folder, and it is a short time if you keep the probe.

If you already have a free remote shell, run the zone probe there before you trust another laptop-green datetime helper.