Pin Overlay Order Before One Settings Extract

Do not extract a Settings class from the mess yet.
Pin overlay order on the live messy repo first.
Record which layer wins for every configuration key.

A Settings extract looks like harmless cleanup work.
It often changes resolution for one quiet key.
A dump of winners catches that shift early.

Why overlay order breaks Settings extracts

Messy CLIs read one key from several layers.
Hardcoded defaults sit under a dotenv file.
Process environment sits under argparse flags next.

Developers fold those layers into one Settings object.
The new object applies one guessed precedence table.
Production then points at the wrong database URL.

Empty string is not a missing key.
Missing is not the hardcoded default value.
Those three states diverge under real flag matrices.

What to freeze before the extract

Freeze four observables for every configuration key.
Do not freeze opinions about class shape yet.

1. Record the resolved value after full startup.
2. Record the winning layer name for that key.
3. Distinguish present, empty-string, and fully missing states.
4. Record type coercion after the last parser runs.

Skip log prettiness. Skip folder layout. Skip new names.
The overlay matrix is the only contract that matters.

Numbered workflow

Follow this sequence on one repository only.
Do not start with a new config library import.

1. Inventory keys the process reads during startup.
   Search environ reads, argparse flags, and dotenv loads.
   Write the key list into overlay_keys.txt first.

2. Build cases that mix missing and empty values.
   Include dotenv-only, environ-only, and flag-only rows.
   Include one row where every layer sets the key.

3. Invoke the real entrypoint, not a rewritten stub.
   Add a temporary dump print if stdout has no config.
   Print JSON with resolved values for the listed keys.

4. Infer winners by peeling layers from the child process.
   Do not re-code overlay logic inside the probe script.
   Commit overlay_dump.json before any Settings class exists.

5. Extract the smallest object that matches those winners.
   Move one layer's reads, then rerun the same dump.
   Stop if winners change for any listed key.

Inventory commands

Run a boring search before you touch imports.
Keep the output next to the later dump file.

grep -RIn -E 'os\.environ|getenv|load_dotenv|add_argument' --include='*.py' .

Copy matching key names into overlay_keys.txt.
Drop comments and dead parser stubs from that list.
One process start path is enough for this slice.

Artifact: peel the child, infer the winner

Label the following scripts as unexecuted proposals.
Wire each script to your existing loader path.
Do not reimplement overlay logic inside the probe.

The probe only needs a resolved dict from startup.
Winner labels come from peeling flags, env, then dotenv.
Same values on two layers will hide the higher layer.
Give every layer a distinct value in those rows.

#!/usr/bin/env python3
"""Peel config layers. Infer winners from resolved dumps.

Unexecuted proposal. Point ENTRY at your real CLI.
The child must print one JSON object on stdout.
"""
from __future__ import annotations

import json
import os
import subprocess
import sys
from pathlib import Path

KEYS = ("DATABASE_URL", "LOG_LEVEL", "FEATURE_X", "TIMEOUT_S")
ENTRY = Path("dump_resolved.py")
DOTENV = Path(".env.characterization")


def base_env(extra: dict[str, str] | None = None) -> dict[str, str]:
    env = os.environ.copy()
    for key in KEYS:
        env.pop(key, None)
    if extra:
        env.update(extra)
    return env


def write_dotenv(values: dict[str, str]) -> None:
    body = "".join(f"{k}={v}\n" for k, v in values.items())
    DOTENV.write_text(body, encoding="utf-8")


def run_child(
    *,
    dotenv: dict[str, str],
    environ: dict[str, str],
    flags: list[str],
) -> dict:
    write_dotenv(dotenv)
    proc = subprocess.run(
        [sys.executable, str(ENTRY), *flags],
        cwd=str(Path.cwd()),
        env=base_env(environ),
        capture_output=True,
        text=True,
        check=False,
    )
    if proc.returncode != 0:
        raise RuntimeError(proc.stderr or proc.stdout)
    payload = json.loads(proc.stdout)
    return {k: payload[k] for k in KEYS}


def infer_winners(full: dict, peeled: dict) -> dict[str, str]:
    winners = {}
    for key in KEYS:
        if full[key] != peeled["no_flags"][key]:
            winners[key] = "flag"
        elif full[key] != peeled["no_environ"][key]:
            winners[key] = "environ"
        elif full[key] != peeled["no_dotenv"][key]:
            winners[key] = "dotenv"
        else:
            winners[key] = "hardcoded"
    return winners


def run_case(case: dict) -> dict:
    full = run_child(
        dotenv=case["dotenv"],
        environ=case["environ"],
        flags=case["flags"],
    )
    peeled = {
        "no_flags": run_child(
            dotenv=case["dotenv"],
            environ=case["environ"],
            flags=[],
        ),
        "no_environ": run_child(
            dotenv=case["dotenv"],
            environ={},
            flags=case["flags"],
        ),
        "no_dotenv": run_child(
            dotenv={},
            environ=case["environ"],
            flags=case["flags"],
        ),
    }
    return {
        "name": case["name"],
        "resolved": full,
        "winners": infer_winners(full, peeled),
    }


def main() -> None:
    cases = [
        {
            "name": "all_absent",
            "dotenv": {},
            "environ": {},
            "flags": [],
        },
        {
            "name": "dotenv_only",
            "dotenv": {"LOG_LEVEL": "DEBUG", "TIMEOUT_S": "5"},
            "environ": {},
            "flags": [],
        },
        {
            "name": "environ_overrides_dotenv",
            "dotenv": {"LOG_LEVEL": "DEBUG"},
            "environ": {"LOG_LEVEL": "WARNING"},
            "flags": [],
        },
        {
            "name": "flag_overrides_environ",
            "dotenv": {"LOG_LEVEL": "DEBUG"},
            "environ": {"LOG_LEVEL": "WARNING"},
            "flags": ["--log-level", "ERROR"],
        },
        {
            "name": "empty_env_vs_missing",
            "dotenv": {},
            "environ": {"DATABASE_URL": ""},
            "flags": [],
        },
        {
            "name": "empty_flag_vs_default",
            "dotenv": {},
            "environ": {},
            "flags": ["--database-url", ""],
        },
    ]
    rows = [run_case(case) for case in cases]
    Path("overlay_dump.json").write_text(
        json.dumps(rows, indent=2, sort_keys=True) + "\n",
        encoding="utf-8",
    )


if __name__ == "__main__":
    main()

Keep dump_resolved.py thin. Call the live loader.
Do not copy dotenv parsing into that file.

#!/usr/bin/env python3
"""Print resolved config from the current messy path.

Proposal only. Replace load_runtime_config with your function.
Keep stdout as one JSON object. Send logs to stderr.
"""
from __future__ import annotations

import json
import sys

# Proposal: from your_cli import load_runtime_config


def load_runtime_config(argv: list[str]) -> dict:
    raise NotImplementedError("wire this to the live loader")


def main() -> None:
    resolved = load_runtime_config(sys.argv[1:])
    print(json.dumps(resolved, sort_keys=True))


if __name__ == "__main__":
    main()

Commit the first dump before you create Settings.
The file is the overlay contract for this slice.
Later reruns must match that file byte for byte.

python peel_overlay.py
git add overlay_keys.txt overlay_dump.json
git commit -m "Pin overlay winners before Settings extract"

How to read one dump row

Read winners before you debate class design.
The empty-string row is the usual footgun case.

{
  "name": "empty_env_vs_missing",
  "resolved": {
    "DATABASE_URL": "",
    "FEATURE_X": "0",
    "LOG_LEVEL": "INFO",
    "TIMEOUT_S": "30"
  },
  "winners": {
    "DATABASE_URL": "environ",
    "FEATURE_X": "hardcoded",
    "LOG_LEVEL": "hardcoded",
    "TIMEOUT_S": "hardcoded"
  }
}

DATABASE_URL won from environ as an empty string.
A Settings class may coerce that empty string away.
The dump makes that coercion visible before merge.

Decision table for the first extract

Use this table as the first pytest parametrize list.
Each table row becomes one named characterization case.
Distinct layer values keep peel inference honest.

Case	dotenv	environ	flag	Expected winner
all_absent	absent	absent	absent	hardcoded
dotenv_only	set	absent	absent	dotenv
environ_overrides_dotenv	set	set	absent	environ
flag_overrides_environ	set	set	set	flag
empty_env_vs_missing	absent	empty string	absent	environ
empty_flag_vs_default	absent	absent	empty string	flag

Do not collapse empty string into None in this table.
Do not collapse missing into the hardcoded default either.
Those collapses belong in a later, separate diff.

Lock the dump with one contract test

Add a test that reruns the peel script.
Compare JSON as text after stable sorting.
Avoid hand-written assertions per key at this stage.

# test_overlay_contract.py — unexecuted proposal
import json
import subprocess
import sys
from pathlib import Path

ROOT = Path(__file__).resolve().parent
DUMP = ROOT / "overlay_dump.json"


def test_overlay_dump_matches_committed_contract():
    subprocess.run(
        [sys.executable, str(ROOT / "peel_overlay.py")],
        cwd=str(ROOT),
        check=True,
    )
    committed = DUMP.read_text(encoding="utf-8")
    current = json.dumps(
        json.loads(committed),
        indent=2,
        sort_keys=True,
    ) + "\n"
    assert committed == current

That test is weak until peel_overlay.py writes atomically.
Write to a temp file, then replace overlay_dump.json.
Otherwise a crashed run can destroy the contract.

pytest -q test_overlay_contract.py

Smallest safe change after winners are pinned

After winners are pinned, change one thing.
Move dotenv reads behind a single function.
Leave argparse in the original main module.

Do not rename keys in the same diff.
Do not add type validation in the same diff.
Do not delete unused flags in the same diff.

Re-run the dump after that single movement.
If overlay_dump.json changes, revert the movement.
Investigate the winner table before any retry.

A Settings class may still be the second diff.
The first diff only centralizes one losing layer.
Winning flags stay in the original parse path.

Optional draft step after the dump exists

Disclosure: This article was prepared as part of MonkeyCode's product outreach.
MonkeyCode provides free model access and a free server option.
After overlay_dump.json exists, a free model can draft pytest wrappers from that file.
Run the peel on the free server when local environ is dirty.
Do not ask the model to invent overlay winners.

Limitations

This harness does not see import-time reads.
Child subprocess env can still leak extra keys.
The probe clears only the listed KEYS set.

Nested YAML merge order is out of scope here.
Windows environment names fold case. Linux names do not.
Do not copy a Linux dump onto a Windows runner.

Peel inference fails when two layers share a value.
Force distinct values in the override matrix rows.
Keep empty-string rows out of that distinct-value set.

Secrets will land in overlay_dump.json if present.
Redact database passwords before you commit the dump.
Treat the file as local contract data, not docs.

A Settings class still can change runtime mutation.
The dump captures startup resolution only, once.
Re-run after any parser or dotenv version bump.

Who should not use this

Skip this if the app has one config source.
Skip this if the CLI cannot run headless.
Skip this if you cannot inventory the keys.

Greenfield services with pydantic-settings already pinned should skip.
Do not use the dump as a public document.
It is a local contract for one refactor slice.

The Settings class is not the source of truth.
The committed overlay dump is the source of truth.
Extract code only after those winners stay stable.