I Trusted lru_cache for 48 Hours. The Rotated Token Never Left Hour Zero.

Have you ever rotated a secret, restarted nothing, and still watched the old credential keep winning? I burned forty-eight hours on that failure after an assistant cached a settings helper. The suite stayed green and the logs stayed polite. Every new token I exported seemed to vanish into a process that had already decided.

This write-up is a reconstructed lab notebook, not a customer case study with invented graphs. I wanted a second pair of eyes on a helper that looked too small to fail. Would you have blamed the secret store first, or the deploy pipeline? I blamed both, and I was wrong for a full day.

Hour 0: the symptom that looked like ops

A long-lived worker kept calling an upstream API with a token I had already rotated. Staging dashboards showed the new secret in the environment view. Local curls with the new value succeeded, which made the worker look haunted. Did the platform cache secrets at the edge, or was my process simply rude?

I pasted the settings module into a review session and asked for a faster load path. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I used MonkeyCode as an open-source coding assistant here, mainly the free model access and the free server option, so the review did not depend on a personal API key. I wanted a critique of the helper, not a rewrite of the worker.

The assistant suggested memoizing load_settings because the function only read environment variables. That change looked boring, which is how it survived code review. Have you noticed how the smallest diffs attract the least suspicion?

What I actually shipped, then regretted

The generated shape was roughly this, reconstructed for the lab and labeled as such:

# settings.py — reconstructed lab helper, not production source
from functools import lru_cache
import os

@lru_cache(maxsize=1)
def load_settings():
    """Load process settings. Looks pure. It is not."""
    return {
        "token": os.environ.get("API_TOKEN", ""),
        "base_url": os.environ.get("API_BASE", "https://example.invalid"),
        "timeout": float(os.environ.get("API_TIMEOUT", "5")),
    }

def headers():
    return {"Authorization": f"Bearer {load_settings()['token']}"}

The helper took no arguments, cached one mapping, and parsed the timeout a single time at first call. What could a cache key even be, when the call signature is empty? That question should have stopped the diff, and it showed up twenty hours too late.

Hours 8–24: false leads I would not repeat

I treated this like an operations incident, because the symptoms lived in staging. Here is the messy list, in the order I wasted them.

1. I rotated the secret a second time and waited for eventual consistency that never arrived.
2. I compared os.environ inside an ad-hoc print and saw the new token, then felt clever.
3. I blamed HTTP client defaults, then blamed DNS, then blamed a reverse proxy buffer.
4. I restarted a sidecar and left the Python worker untouched, which kept the cache warm.
5. I asked the assistant to add retries, which hid nothing and made traces noisier.

Why did the debug print show the new token while the client still sent the old one? Because I printed os.environ["API_TOKEN"] and never printed load_settings()["token"]. That mismatch is embarrassing in hindsight, and it is also extremely common. Would a structured log of a token fingerprint have saved me? Yes, and I still refuse to log secrets in the clear.

The print that lied to me

This was the exact shape of the noisy debug I added, reconstructed:

import os
from settings import load_settings, headers

def debug_auth_view():
    raw = os.environ.get("API_TOKEN", "")
    cached = load_settings()["token"]
    # Lab only: never log live secrets. Fingerprint them.
    print("raw_len", len(raw), "cached_len", len(cached), "same", raw == cached)
    print("header_keys", sorted(headers().keys()))

When raw_len changed and cached_len did not, the incident stopped being an ops story. Until that line existed, every dashboard agreed with the environment and disagreed with the process.

Hours 24–36: the tests that protected the bug

CI stayed green, which is the part that made me stubborn. The tests used pytest monkeypatch, and they looked responsible at a glance. Why would I reopen a helper that already had coverage?

# test_settings.py — this can pass for the wrong reason
import os
import settings

def test_token_from_environment(monkeypatch):
    monkeypatch.setenv("API_TOKEN", "gamma")
    # Import already happened at module top.
    # lru_cache may already hold hour-zero state.
    value = settings.load_settings()["token"]
    assert value in {"gamma", os.environ.get("API_TOKEN")}

That last assertion is a gift to false confidence. If the cache holds an empty string from import time, the in clause can still pass by reading os.environ directly. Did the assistant write that looser clause after I complained about flakes? In this reconstructed notebook, yes.

I moved the import below the patch, which helped one test and created a new mess around singleton clients.

import importlib

def test_token_after_import_is_a_trap(monkeypatch):
    monkeypatch.setenv("API_TOKEN", "gamma")
    import settings as settings_mod
    importlib.reload(settings_mod)
    assert settings_mod.load_settings()["token"] == "gamma"

Is reload a fix, or a way to launder import order until the suite order changes again? I would not ship reload as a production invalidation strategy. It also duplicates module globals, which is how HTTP clients get created twice and then leak.

Hours 36–48: prove the cache, do not argue with it

The turning point was a two-call script in a clean interpreter. No pytest. No worker. Just the helper and a shell that I actually controlled.

# repro_lru_env.py — run this as its own process
import os
from settings import load_settings

os.environ["API_TOKEN"] = "alpha"
# If the module was already imported, move this assignment above the import.
print("first", load_settings()["token"])

os.environ["API_TOKEN"] = "beta"
print("second", load_settings()["token"])  # still alpha if cached
print("env", os.environ["API_TOKEN"])      # beta
print("cache_info", load_settings.cache_info())

load_settings.cache_clear()
print("cleared", load_settings()["token"])  # beta

Commands I ran in the lab, in this order:

python -m venv .venv
source .venv/bin/activate
pip install pytest
API_TOKEN=alpha python repro_lru_env.py

The second print staying at alpha while env showed beta ended the mystery. Does maxsize=1 matter when the function takes no arguments? It does not. lru_cache keys on the argument tuple, and the empty tuple is a single immortal slot until cache_clear or process exit.

cache_info() made the lie numeric. After two calls I saw a hit count climb while the environment kept changing. Have you watched a hit counter celebrate a bug? I have now.

Artifact: a test that fails when the cache lies

I want a test that hurts when someone memoizes environment reads. This is the contract I would keep, and it is meant to fail on the cached helper above.

# test_settings_contract.py
import importlib
import sys

def load_fresh_settings_module():
    sys.modules.pop("settings", None)
    return importlib.import_module("settings")

def test_load_settings_follows_env_without_manual_clear(monkeypatch):
    monkeypatch.setenv("API_TOKEN", "alpha")
    mod = load_fresh_settings_module()
    assert mod.load_settings()["token"] == "alpha"

    monkeypatch.setenv("API_TOKEN", "beta")
    # Production will not call cache_clear() after a sidecar rotation.
    observed = mod.load_settings()["token"]
    assert observed == "beta", (
        f"settings froze at {observed!r}; "
        "do not lru_cache zero-argument env readers"
    )

Run it like this:

pytest -q test_settings_contract.py -vv

If the helper still wears @lru_cache, this test fails on the second assertion. That failure is the whole point of the notebook. Should I weaken the test so CI stays green? No. Green tests that follow the cache are how I lost a day and a night.

A safer shape, still labeled as a proposal

I would pass settings in, or rebuild them on a named version key. This is a proposal, not a library:

# proposal only — explicit snapshot, no hidden process cache
from dataclasses import dataclass
import os

@dataclass(frozen=True)
class Settings:
    token: str
    base_url: str
    timeout: float

def read_settings() -> Settings:
    return Settings(
        token=os.environ.get("API_TOKEN", ""),
        base_url=os.environ.get("API_BASE", "https://example.invalid"),
        timeout=float(os.environ.get("API_TIMEOUT", "5")),
    )

Call read_settings() in main(), then pass the frozen object downward. If a test needs beta, it constructs Settings(token="beta", ...) and never fights import order. Is that slower than lru_cache? Yes, by an amount I still have not bothered to measure, because reading a few environment variables was never the bottleneck.

Decision table: when memoizing settings is allowed

I needed a boring table more than another retry wrapper.

• Process-boot constants that never change — caching is optional; document that a restart is the invalidation story.
• Tokens, feature flags, base URLs from env — do not cache, or cache a snapshot object you rebuild on a known signal.
• Per-request overrides from headers — never use a process-wide lru_cache.
• File-backed config — hash the bytes you read; do not pretend the function is pure.
• Tests that call setenv — import after the patch, or pop sys.modules, and skip reload theater.

If your platform restarts every worker when a secret rotates, this bug stays latent. If you have long-lived processes, it becomes an outage with excellent unit tests. Which fleet do you actually run?

What I would repeat next time

I would still ask an assistant for a review, but I would forbid drive-by memoization on zero-argument functions. The useful part was not a rewrite. It was forcing the two-call script into the conversation before anyone touched retries.

Repeatable checklist:

1. Print the cached object and the raw os.environ value on the same line, as fingerprints.
2. Run one repro in a fresh process before you touch pytest or staging restarts.
3. Add a contract test that changes env twice without calling cache_clear.
4. Reject @lru_cache on zero-argument functions that read the world.
5. If you must cache, put a version key in the call, and test a bump.

Would I ban lru_cache in the codebase? No. I would ban it on functions whose outputs depend on hidden process state. The decorator is fine when the arguments already name every input that matters.

Limitations, and who should ignore this

This notebook does not measure latency, and it does not claim that reading os.environ is expensive enough to care. It also does not cover multiprocess managers, secret volumes that change on disk, or typed settings libraries with their own reload hooks. If you already inject an immutable settings object at process start, and you restart on every rotation, @lru_cache is redundant rather than deadly.

Do not use this approach if you need live reload of secrets inside a request path without a design for invalidation. Do not paste real tokens into any assistant. Do not treat importlib.reload as a concurrency primitive. And please do not "fix" a frozen cache by adding time.sleep around the rotator.

I am still going to ask for help on tiny helpers. I am not going to accept a cache on a function with no arguments unless the docstring names the process as the cache key. If you want a cheap second pass on a similar two-call repro, the free models are enough to interrogate that script before you restart staging again.