Freeze the Output Tree Before You Extract a Writer

Freeze the output tree before you extract a writer. Path joins and open calls leak working-directory assumptions. Those leaks survive helper tests and break callers later.

A tree snapshot is the missing oracle for writers. It records relative paths, modes, and content hashes. Extract one function only after that oracle is green.

The failure mode

Messy report modules mix parsing, path math, and writes. Engineers extract write_json to clean the messy file. The helper then writes beside a new cwd.

Relative paths resolve against the test runner directory. File modes often pick up a different umask. Byte content gains a trailing newline from a formatter.

None of those changes fail a mocked open test. The directory on disk is the public surface. Pin that on-disk surface before any extract.

What to pin

Pin four facts for every path the module writes. Record the working directory used during the run. Record each relative path from a chosen root. Record permission bits and a sha256 of bytes.

Also record empty directories that the module creates. Skip caches and pycache with an allowlist root. Do not snapshot the whole repository in this oracle.

This harness is a labeled proposal for local use. It is not a production benchmark or vendor test. Tune ignore rules before you trust the fixture.

Artifact: tree snapshot harness

Use one root directory for each characterization run. Force cwd with os.chdir inside the test. Then walk the root and emit sorted JSON.

"""tree_oracle.py — proposal harness, not a measured benchmark."""
from __future__ import annotations

import hashlib
import json
import os
import stat
from pathlib import Path
from typing import Any

IGNORE_NAMES = {".git", "__pycache__", ".pytest_cache", ".mypy_cache"}


def file_sha256(path: Path) -> str:
    digest = hashlib.sha256()
    with path.open("rb") as handle:
        for chunk in iter(lambda: handle.read(65536), b""):
            digest.update(chunk)
    return digest.hexdigest()


def snapshot_tree(root: Path) -> list[dict[str, Any]]:
    root = root.resolve()
    rows: list[dict[str, Any]] = []
    for dirpath, dirnames, filenames in os.walk(root):
        dirnames[:] = sorted(n for n in dirnames if n not in IGNORE_NAMES)
        current = Path(dirpath)
        rel_dir = current.relative_to(root).as_posix() or "."
        mode = stat.S_IMODE(current.stat().st_mode)
        rows.append(
            {
                "path": rel_dir,
                "kind": "dir",
                "mode": oct(mode),
                "sha256": None,
            }
        )
        for name in sorted(filenames):
            if name in IGNORE_NAMES:
                continue
            file_path = current / name
            rel = file_path.relative_to(root).as_posix()
            file_mode = stat.S_IMODE(file_path.stat().st_mode)
            rows.append(
                {
                    "path": rel,
                    "kind": "file",
                    "mode": oct(file_mode),
                    "sha256": file_sha256(file_path),
                }
            )
    rows.sort(key=lambda row: (row["path"], row["kind"]))
    return rows


def write_snapshot(root: Path, out_file: Path) -> None:
    payload = {"root_name": root.name, "rows": snapshot_tree(root)}
    out_file.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n")

Run the module only under a temporary root. Keep HOME and TMPDIR inside that same root. Then write tree.json next to the characterization test.

umask 022
TZ=UTC python -m pytest tests/test_tree_oracle.py -q

Characterization test

The characterization test should create a temp directory. It should chdir into that directory before calls. It should invoke the messy module exactly once.

After the call, snapshot the temporary root on disk. Compare the JSON to a committed fixture file. Fail on path, mode, or content hash drift.

"""tests/test_tree_oracle.py — proposal characterization test."""
from __future__ import annotations

import json
import os
from pathlib import Path

import pytest

from tree_oracle import snapshot_tree

FIXTURE = Path(__file__).parent / "fixtures" / "report_tree.json"


def run_messy_report(root: Path) -> None:
    # Stand-in. Wire the real module entry before relying on this.
    (root / "out").mkdir()
    report = root / "out" / "summary.json"
    report.write_text('{"ok": true}\n', encoding="utf-8")


def test_report_output_tree(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
    monkeypatch.chdir(tmp_path)
    monkeypatch.setenv("HOME", str(tmp_path / "home"))
    monkeypatch.setenv("TMPDIR", str(tmp_path / "tmp"))
    monkeypatch.setenv("TZ", "UTC")
    (tmp_path / "home").mkdir()
    (tmp_path / "tmp").mkdir()
    os.umask(0o022)
    run_messy_report(tmp_path)
    actual = {"rows": snapshot_tree(tmp_path)}
    expected = json.loads(FIXTURE.read_text())
    assert actual["rows"] == expected["rows"]

Label the run_messy_report stub as a stand-in. Wire your real entrypoint before relying on this. Commit the fixture only after a manual review.

Control cwd, env, umask, and clock

Tree drift often comes from process context, not code. Pin context in the test, not in production config.

Set umask in the test process before writes. Set TZ to UTC for any stamped filenames. Redirect HOME, TMPDIR, and XDG dirs into tmp_path.

monkeypatch.setenv("XDG_CACHE_HOME", str(tmp_path / "cache"))
monkeypatch.setenv("SOURCE_DATE_EPOCH", "1700000000")
(tmp_path / "cache").mkdir()

Do not freeze wall-clock values inside file bytes. Inject a clock function before hashing JSON reports. Unstable timestamps make the oracle flaky and useless.

Numbered workflow

Follow this order and do not skip the freeze.

1. Pick one module that writes files to disk.
2. List path-building expressions in the target module.
3. Wrap the entrypoint with a forced cwd and env.
4. Generate report_tree.json from one representative input file.
5. Commit the fixture and the equality assertion together.
6. Extract one function only in the next commit.
7. Re-run the tree test and revert unexpected row changes.
8. Update the fixture only when layout change is intended.

Step six is the smallest safe change in this method. Two extracts in one diff hide the real cause. Path math and I/O belong in separate commits.

Decision table

Use the table before you open a refactor pull request.

Observation	Extract now?	Pin first
Paths built from Path.cwd()	No	cwd plus relative rows
Files created with default umask	No	mode bits per file
JSON bytes include timestamps	No	clock or hash of stable fields
Writes go through one open	Maybe	that path and those bytes
Helper would both join and write	No	split into two later commits
Output is a socket or pipe	No	this oracle does not apply
Filenames include random UUIDs	No	inject a clock or id factory first

Maybe still requires a green tree test first. Absence of a fixture means you must not extract.

Smallest safe change

Keep the extract boring and easy to reverse. A good first extract returns a Path object. It does not write bytes in that commit.

# proposal: extract path math only
def report_path(root: Path, name: str) -> Path:
    if not name.endswith(".json"):
        name = f"{name}.json"
    return root / "out" / name

Leave write_text in the original module for now. The tree oracle should stay green after extract. A second commit can move the write call.

If you must move the write, keep path math. Do not move both behaviors in one diff. The oracle then points at exactly one cause.

Reviewing fixture rows

Treat each JSON row like a tiny public API. Unexpected new paths are bugs until proven otherwise. Mode changes often mean umask leaked into the helper.

Hash changes mean bytes changed, not just names. Open both files and diff the raw bytes. Formatter noise and key reordering show up here.

{
  "rows": [
    {"path": ".", "kind": "dir", "mode": "0o755", "sha256": null},
    {"path": "out", "kind": "dir", "mode": "0o755", "sha256": null},
    {
      "path": "out/summary.json",
      "kind": "file",
      "mode": "0o644",
      "sha256": "9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08"
    }
  ]
}

The hash above is a placeholder for your real digest. Replace it from a reviewed local run. Do not copy hashes from this article into production fixtures.

Delete rows that sit outside the allowlist root. Never edit hashes to make a red test pass. Either restore the writer or justify a layout change.

After the oracle is green

A coding model can draft the extract after freeze. It cannot replace the committed tree fixture at all. Generate the helper against tests that fail on drift.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode provides free model access and a free server option. Those two facts are the only product claims here.

Use a free session only after the fixture exists. Paste the snapshot test and the messy function. Ask for one path-builder extract and nothing else. Reject any diff that edits the fixture silently.

The local pytest run remains the only merge judge. Remote suggestions are drafts until the oracle agrees. Merge nothing that the tree oracle did not see.

Limitations

This oracle ignores file timestamps as inode metadata. It still hashes file bytes on every walk. Timestamped JSON will churn the hash on each run.

It does not model concurrent writers or races. Races can produce extra files between two walks. Those races need locks, not a snapshot fixture.

Network filesystems may report different mode bits than local disks. Container umask can differ from developer laptops at runtime. Run the oracle in one pinned environment only.

Symlinks need an explicit recording policy in the harness. This proposal records them as files or skips them. Pick one policy and document it in the test.

Large binary outputs make committed fixtures too heavy. Hash them, but store hashes only in git. Do not commit multi-megabyte copies as oracle files.

Who should not use this

Do not use this on modules that must be nondeterministic. Skip it for live network captures and sockets. Skip it for encrypted payloads with unique IVs.

Do not use it as a substitute for schema tests. Tree equality does not prove JSON field meaning. Pair it with a parser assertion when content matters.

Teams without temp-dir discipline should not start here. Fix cwd and env leaks first in those codebases. Then add the snapshot once roots are stable.

What done looks like

The fixture is reviewed and committed in git. One extract lives in a separate follow-up commit. Pytest shows the same rows before and after.

Callers still import the original module after extract. No new CLI flags landed in the same diff. The output tree did not move on disk.

Freeze the tree, then change one function. Stop when the oracle still matches the fixture.

If the fixture already pins your writer, reuse it. A free coding session can draft the next extract. Keep that draft against the same tree oracle.