Nested Settings Extracts Fail Until Merge Order Is Pinned

A settings extract without pinned merge tests will lie.
Later keys, None values, and env strings all collide.
Characterization tests must land before any loader extract.

The failure, not the helper

Many messy repos grow one global settings mapping.
YAML files, process env, and CLI flags write it.
A clean load_settings helper looks like progress.
It is unsafe without a frozen merge contract.

Three collisions appear after the first green refactor.
Merge order flips a nested key without a traceback.
Explicit None deletes a default that missing keys keep.

Env strings also keep "false" as a truthy string.
Callers then branch on the wrong debug flag.
The extract still looks tidy in review.

What to pin before any extract

Pin five observables in one local pytest harness.
Do not extract until every row stays stable.

1. Overlay order: file, then env, then CLI last.
2. Nested dict keys: deep merge, never shallow replace.
3. Nested lists: replace the whole list, never extend.
4. Missing key versus explicit JSON null or None.
5. Env coercion: bool, int, float, and raw strings.

Skip logger noise and pretty-print diffs at this stage.
They hide the contract you must freeze first.
Cache identity belongs in the same harness.

Example messy module

The following module is a compact teaching fixture.
It is not a vendor SDK and not live telemetry.

# settings_messy.py
from __future__ import annotations

import copy
import json
import os
from pathlib import Path
from typing import Any

_CACHE: dict[str, Any] | None = None
_DEFAULTS = {
    "service": {"name": "api", "port": 8080, "debug": False},
    "features": ["alpha"],
    "retries": 3,
    "token": "local-dev",
}


def _deep_merge(base: dict, overlay: dict) -> dict:
    out = copy.deepcopy(base)
    for key, value in overlay.items():
        if (
            key in out
            and isinstance(out[key], dict)
            and isinstance(value, dict)
        ):
            out[key] = _deep_merge(out[key], value)
        else:
            out[key] = copy.deepcopy(value)
    return out


def _coerce(raw: str) -> Any:
    lowered = raw.strip().lower()
    if lowered in {"true", "false"}:
        return lowered == "true"
    if lowered.isdigit() or (
        lowered.startswith("-") and lowered[1:].isdigit()
    ):
        return int(lowered)
    try:
        return float(raw)
    except ValueError:
        return raw


def load_settings(
    path: Path | None = None,
    environ: dict[str, str] | None = None,
    cli: dict[str, Any] | None = None,
) -> dict[str, Any]:
    global _CACHE
    if _CACHE is not None:
        return _CACHE

    data = copy.deepcopy(_DEFAULTS)
    if path and path.exists():
        file_data = json.loads(path.read_text(encoding="utf-8"))
        data = _deep_merge(data, file_data)

    env = environ if environ is not None else os.environ
    prefix = "APP_"
    for key, raw in env.items():
        if not key.startswith(prefix):
            continue
        parts = key[len(prefix):].lower().split("__")
        cursor = data
        for part in parts[:-1]:
            nxt = cursor.get(part)
            if not isinstance(nxt, dict):
                nxt = {}
                cursor[part] = nxt
            cursor = nxt
        cursor[parts[-1]] = _coerce(raw)

    if cli:
        data = _deep_merge(data, cli)

    _CACHE = data
    return _CACHE


def reset_cache() -> None:
    global _CACHE
    _CACHE = None

That module-level cache is the usual landmine.
The second call returns a mutated mapping.
Tests that skip reset_cache() pin the wrong object.

Characterization harness

Run these tests against the current messy module first.
Do not rewrite production files until this file is green.

# test_settings_characterization.py
from __future__ import annotations

import json
from pathlib import Path

import pytest

from settings_messy import load_settings, reset_cache


@pytest.fixture(autouse=True)
def _clear():
    reset_cache()
    yield
    reset_cache()


def test_file_overrides_defaults(tmp_path: Path):
    p = tmp_path / "s.json"
    p.write_text(json.dumps({"service": {"port": 9090}}), encoding="utf-8")
    got = load_settings(path=p, environ={}, cli=None)
    assert got["service"]["port"] == 9090
    assert got["service"]["name"] == "api"
    assert got["service"]["debug"] is False


def test_env_overrides_file(tmp_path: Path):
    p = tmp_path / "s.json"
    p.write_text(json.dumps({"retries": 9}), encoding="utf-8")
    got = load_settings(
        path=p,
        environ={"APP_RETRIES": "2", "APP_SERVICE__DEBUG": "true"},
        cli=None,
    )
    assert got["retries"] == 2
    assert got["service"]["debug"] is True


def test_cli_overrides_env(tmp_path: Path):
    p = tmp_path / "s.json"
    p.write_text("{}", encoding="utf-8")
    got = load_settings(
        path=p,
        environ={"APP_TOKEN": "from-env"},
        cli={"token": "from-cli"},
    )
    assert got["token"] == "from-cli"


def test_explicit_none_replaces_default(tmp_path: Path):
    p = tmp_path / "s.json"
    p.write_text(json.dumps({"token": None}), encoding="utf-8")
    got = load_settings(path=p, environ={}, cli=None)
    assert "token" in got
    assert got["token"] is None


def test_missing_file_key_keeps_default(tmp_path: Path):
    p = tmp_path / "s.json"
    p.write_text("{}", encoding="utf-8")
    got = load_settings(path=p, environ={}, cli=None)
    assert got["token"] == "local-dev"


def test_list_replace_not_extend(tmp_path: Path):
    p = tmp_path / "s.json"
    p.write_text(json.dumps({"features": ["beta"]}), encoding="utf-8")
    got = load_settings(path=p, environ={}, cli=None)
    assert got["features"] == ["beta"]


def test_second_call_returns_cached_object(tmp_path: Path):
    p = tmp_path / "s.json"
    p.write_text("{}", encoding="utf-8")
    first = load_settings(path=p, environ={}, cli=None)
    first["retries"] = 99
    second = load_settings(path=p, environ={"APP_RETRIES": "1"}, cli=None)
    assert second["retries"] == 99
    assert first is second


def test_false_string_is_boolean_false(tmp_path: Path):
    p = tmp_path / "s.json"
    p.write_text("{}", encoding="utf-8")
    got = load_settings(
        path=p,
        environ={"APP_SERVICE__DEBUG": "false"},
        cli=None,
    )
    assert got["service"]["debug"] is False

Use one command and keep the output in the review.

python -m pytest test_settings_characterization.py -q --tb=short

The cache test documents a defect on purpose.
Do not fix that defect inside the extract patch.
Pin the defect. Extract the loader. Fix cache later.

How to read a red row

A red assertion is data, not a prompt to rewrite tests.
Classify the miss before touching production files.

1. Overlay miss: CLI lost to env, or env lost to file.
2. Shape miss: a nested dict was replaced, not merged.
3. List miss: features extended instead of replaced.
4. None miss: missing key and explicit null collapsed.
5. Coercion miss: "false" stayed a non-empty string.
6. Cache miss: second call observed new env too early.

Write the class in the pull request, then stop.
Do not bundle a behavior fix with the file move.
Mixed patches hide which contract actually changed.

Decision table

Use this table as the merge oracle.
Update a cell only when product intent changes.

Source	Nested dict	Nested list	Explicit None	Env "false"	Second call
File after defaults	deep merge	replace list	keep None	n/a	cached object
Env after file	deep merge via __	replace scalar	n/a	boolean False	ignored
CLI after env	deep merge	replace list	keep None	n/a	ignored

If a proposed helper disagrees with one cell, reject it.
The helper is wrong. The table is the contract.
Do not negotiate cells during an extract review.

Golden dump for nested maps

Add one freeze helper when nested diffs get noisy.
Keep it as a debug aid, not the only assertion.

import json

def freeze(data) -> str:
    return json.dumps(
        data,
        sort_keys=True,
        separators=(",", ":"),
        default=str,
    )

Assert specific keys first, then the frozen dump.
Key asserts catch merge bugs faster than blobs.
The dump still catches unknown extra keys.

python -c "from test_settings_characterization import freeze; print('ok')"

Do not sort lists that encode priority order.
features replacement must stay exact, not sorted.
Sorting would hide a replace-versus-extend bug.

Smallest safe change

Follow this order. Do not skip a row.

1. Land the characterization file on the messy module.
2. Paste pytest output into the pull request body.
3. Move load_settings into settings_loader.py unchanged.
4. Re-export the same names from settings_messy.py.
5. Re-run the same tests with no assertion edits.
6. Only later consider a copy-on-return cache fix.

The extract is a move plus a re-export.
It is not a rewrite of merge rules.
New behavior belongs in a second, smaller patch.

# settings_messy.py
from settings_loader import load_settings, reset_cache

__all__ = ["load_settings", "reset_cache"]

Callers keep the old import path after the move.
That is the point of the smallest safe change.
Import churn is a separate, measurable patch.

Env key mapping rules

Document the parser before anyone "simplifies" it.
Small parser changes silently retarget nested keys.

1. Read only keys that start with APP_.
2. Strip the prefix, then lowercase the remainder.
3. Split nested path segments on __, not _.
4. Coerce the leaf, then write through the path.
5. Ignore unrelated process environment keys.

APP_SERVICE_DEBUG is not service.debug.
APP_SERVICE__DEBUG is the nested form.
Pin both spellings if both appear in scripts.

APP_SERVICE__DEBUG=false APP_RETRIES=2 python -c "print('env-pin')"

Pass a fake environ mapping in unit tests.
Do not mutate os.environ inside characterization cases.
Process-global env makes test order flaky.

Where a free coding model fits

A model can draft the move after tests exist.
It cannot invent the merge order from comments.
Feed it the characterization file and the table.
Ask for a file move, not a clever redesign.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.
MonkeyCode offers free model access and a free server option.
Those two facts are the only product claims used here.
No model names, quotas, or hardware details are implied.

A local pytest run remains the source of truth.
The free server helps when the laptop is busy.
Run the same command. Compare the same assertion set.
Discard any patch that edits tests to stay green.

Limitations

This harness does not cover YAML anchors or substitution.
It does not cover Windows environment case folding.
It does not cover concurrent first-load races.
It does not prove JSON schema validity.

Float coercion will accept "1e2" as 100.0.
That may surprise a token or version string.
Add a denylist of keys that must stay strings.
Pin those keys with extra tests before extracting.

Empty env values also need an explicit policy.
APP_TOKEN= may mean empty string or missing.
The fixture above stores an empty coerced string.
Record that choice before changing _coerce.

Who should not use this approach

Skip this workflow for greenfield settings code.
Write an explicit schema and a typed loader instead.
Skip it when secrets must never enter env dumps.
Skip it when merge order is still a product debate.

Do not extract during an incident window.
Pin tests on a quiet branch first.
Do not extract two loaders in one change.
One function move is the batch size.

Checklist

1. Five merge observables have stable tests.
2. Cache identity is asserted, not ignored.
3. Extract is a file move plus re-export only.
4. No assertion was rewritten to match new code.
5. A later patch may copy the cache on return.

The core conclusion stays after the file move.
Pinned merge order is the refactor. The helper is leftover.
If the laptop is busy, the free server option can run this same pytest harness.