DEV Community

Taylor Wang
Taylor Wang

Posted on

I Blamed urllib for 48 Hours. A Local http.py Was Sitting on sys.path.

Have you ever watched a clean environment reject an import that your laptop treated as obvious? I spent forty-eight hours chasing a http.client failure that never appeared inside my local editor. The traceback kept pointing at http.py, and I read that filename as the standard library.

Hour 0–8: I treated it like a networking problem

Why would urllib.request explode on import while curl against the same host looked completely healthy? I assumed TLS, proxies, or a missing CA bundle, because those failures usually wear this costume. The local virtualenv ran the same Python minor version, so I trusted the import graph blindly. That first quiet mistake cost me an entire evening of packet captures which proved absolutely nothing.

Commands that wasted the morning

I kept proving the network was fine, which was true and also irrelevant:

curl -I https://example.com
python -c "import ssl; print(ssl.OPENSSL_VERSION)"
env | grep -E 'PROXY|CERT|REQUESTS|SSL' || true
python -c "import urllib.request"
Enter fullscreen mode Exit fullscreen mode

None of those checks asked a simpler question: which file did Python actually import for http? I was debugging sockets, certificates, and retries, while the process had not yet created a socket. Does that sound familiar if you have ever stepped through library setup code at three in the morning? I needed a module file path, and I kept collecting evidence about a connection that did not exist.

Hour 8–24: the error moved when I changed directories

Have you noticed how an IDE launch configuration can hide the real working directory from you? Pytest from the repo root added the project directory in a different way than my editor did. Running python tools/probe.py from a nested folder made the failure disappear, which felt like progress. It was not progress; it was sys.path[0] shifting away from a file named http.py.

The layout that bit me

The tree looked harmless, which is how these bugs survive review:

repo/
  http.py          # my tiny helper, honest name, terrible idea
  tools/
    probe.py
  tests/
    test_probe.py
  requirements.txt
Enter fullscreen mode Exit fullscreen mode

I had extracted a tiny HTTP helper into http.py because the name felt honest and short. Python then treated http as a module, so http.client became an attribute lookup that failed. Urllib imports http.client during setup, which is why the stack looked like a networking bug. Once I opened the path in the traceback, the helper I had written was staring back at me.

AttributeError: module 'http' has no attribute 'client'
Enter fullscreen mode Exit fullscreen mode

Want the shortest confirmation that does not involve a packet capture?

python -c "import http, sys; print(http.__file__); print(sys.path[:4])"
find . -name 'http.py' -print
Enter fullscreen mode Exit fullscreen mode

If that __file__ points inside your repo, you are not debugging urllib. You are debugging a name you stole from the standard library, and the network stack is an innocent bystander.

Hour 24–36: a clean machine finally disagreed with my laptop

Did I need another library version, or did I need a process whose path I did not own? I cloned the repo onto MonkeyCode's free server option so the IDE could not rewrite sys.path. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I used free model access only to draft a path-dump harness, and I still verified every line by hand.

The clean clone ran python -m pytest from the repository root, which put the project directory first. My editor had been launching tools/probe.py with a working directory that skipped the colliding file. Same commit, same interpreter family, two different import graphs, and I had spent a day comparing OpenSSL strings that never mattered.

The artifact: a path dump I will actually keep

Would you ship a service without printing the file behind every suspicious import during boot? I would not, not after this, because famous names are not reserved in your working tree. The script below is the reproducible check; run it, do not trust a memory of what import http meant yesterday.

#!/usr/bin/env python3
"""Dump origins for names that should resolve to the standard library."""
from __future__ import annotations

import importlib.util
import sys

DENY_AT_PROJECT_ROOT = (
    "http",
    "email",
    "json",
    "uuid",
    "token",
    "types",
    "logging",
    "queue",
    "code",
    "site",
    "test",
    "csv",
    "argparse",
    "configparser",
)


def dump_path() -> None:
    print("sys.executable:", sys.executable)
    print("sys.path[0]:", repr(sys.path[0]))
    for i, entry in enumerate(sys.path[:8]):
        print(f"sys.path[{i}]: {entry!r}")
    print("--- specs ---")
    for name in DENY_AT_PROJECT_ROOT:
        spec = importlib.util.find_spec(name)
        origin = getattr(spec, "origin", None) if spec else None
        print(f"{name:12} -> {origin}")


if __name__ == "__main__":
    dump_path()
Enter fullscreen mode Exit fullscreen mode

I run that script in CI and on any borrowed machine before I argue about library versions again. The check is boring, and boring is the point when the standard library name is the landmine. If the printed path sits inside the repo, I rename the helper before I touch TLS settings. Save it as tools/dump_import_origins.py and invoke it the same way your tests start, not the way your editor starts.

python tools/dump_import_origins.py
python -m tools.dump_import_origins
python -m pytest -q
Enter fullscreen mode Exit fullscreen mode

Those three launches are not equivalent, which is the whole incident. If only one of them prints a stdlib path, you still have a shadow, and you only proved that one entrypoint is lucky.

Decision table: where sys.path[0] comes from

How you launch What sys.path[0] becomes Shadow risk
python repo/tools/probe.py directory containing the script helpers sitting beside probe.py
python -m pkg.mod from repo root current working directory any colliding *.py at cwd
pytest test path injection plus project root test.py, root helpers, conftest neighbors
IDE Run button whatever cwd the launch config invented the silent one, usually

Read that table before you trust a green local run that you started from a nested folder. The same source tree can be safe or cursed depending on how Python was actually invoked. I still keep this table in the debugging notes, because I will forget the launch distinction. If your team onboards by cloning and running python -m, the root-level helper names become load-bearing.

A tiny test that fails on purpose

Label this as a guard, not as a performance claim; it only asserts filenames at the repo root.

from pathlib import Path

FORBIDDEN = {
    "http.py",
    "email.py",
    "json.py",
    "uuid.py",
    "types.py",
    "token.py",
    "logging.py",
    "queue.py",
    "csv.py",
}

def test_no_stdlib_shadow_at_repo_root():
    root = Path(__file__).resolve().parents[1]
    collisions = sorted(p.name for p in root.glob("*.py") if p.name in FORBIDDEN)
    assert collisions == [], f"stdlib shadow at repo root: {collisions}"
Enter fullscreen mode Exit fullscreen mode

That test is supposed to fail loudly when a forbidden filename exists beside the project entrypoint. I would rather see CI go red on a name collision than watch urllib raise a nonsense import error. Put the test next to the dump script so a future helper named email.py cannot hide behind a green editor run.

Hour 36–48: what I changed and what I refused to change

I renamed http.py to http_helper.py, and the urllib import started resolving to the real stdlib. I added a boot check for a small denylist of names that should never live at the project root. I did not add a rewrite of sys.path in application code, because that hides the next collision. Path hacks feel clever until a test runner, a gunicorn loader, or a notebook starts in another way.

What I now do on hour one, before certificates:

  1. Print sys.executable, sys.path[:5], and module.__file__ for every import the traceback already named.
  2. Search the tree for that basename, including nested packages that accidentally became top-level modules.
  3. Re-run the same dump under python, python -m, and the test runner, because one lucky launch is not evidence.
  4. Rename the helper to a boring, specific name instead of negotiating with import machinery.

What I would repeat

Would I still start with curl, or would I print module files before I blame the network stack? I would reproduce on a machine that does not inherit editor path injections, even if it is slow. I would keep the dump script, the denylist, and the rename, because those survive the next laptop. I would not ask a model to explain a traceback until I know which file was imported.

The repeatable sequence is short enough to keep on a sticky note:

  • Dump origins first, including names you think are too famous to collide.
  • Compare three launch styles before you compare two dependency pins.
  • Rename the local module rather than deleting sys.path entries in application code.
  • Keep secrets out of any assistant prompt that is only supposed to draft a dump script.

Limitations, and who should skip this

This workflow will not catch a shadowed third-party package that you installed under the same name. It also will not save you if the production image uses a different interpreter than your clean box. Do not paste credentials, private URLs, or customer traces into any coding assistant to get a dump. If you cannot run untrusted code on a shared server, skip the remote box and use a local venv.

Teams with strict air-gap rules should copy the script into their own CI, not onto a free host. A borrowed clean path is not a replica of production networking, glibc, init, or installed wheels. Namespace packages, editable installs, and PYTHONPATH set by a task runner can still reorder the graph after the dump looks healthy. If your app is a zipimport, a frozen binary, or a namespace laid out under src/, adjust the test root before you trust the assertion.

Who should not use the remote-box half of this:

  • Anyone whose tree contains secrets that a shared machine should never see.
  • Anyone who needs bit-for-bit production fidelity rather than a second opinion on sys.path.
  • Anyone hoping a model will notice http.py without being shown module.__file__.

Field notes I am willing to reread

The laptop lied because it was convenient, and the traceback lied because I refused to open the file path. Stdlib names are not available just because they feel too famous for anyone to reuse in a repo. If your next failure mentions http, json, email, uuid, or token, ask which file Python opened first. That question would have ended this incident before the packet capture, and I intend to ask it sooner.

If you want a scratch machine that is not your IDE, a free server option was enough for this check. I still would not treat that box as production, and I would not upload .env files to it. The dump script stays in the repo, because that is the part I can defend without a vendor story.

Top comments (0)