Pin Walk Order, Symlink Loops, and EACCES Before One Extract

Do not extract a directory walker until three outputs freeze. Freeze relative path order, symlink cycle behavior, and EACCES handling. Characterization tests make the later extract a mechanical change.

Messy CLIs hide walk logic inside report commands. Mixed os.walk, glob, and ad hoc skips drift over time. One helper extract without goldens will change file order.

The observable contract

A walker is not a vague "list files" helper. Callers depend on relative paths, not inode order. They also depend on skip rules for unreadable nodes.

Pin these three observables before any function rename. Sorted relative POSIX paths must match for every regular file.

Cycle-safe behavior must hold when a symlink points upward. PermissionError on a subdirectory must skip, not crash.

Do not pin mtime, size, or absolute prefixes. Those values move across machines and temp directories. They also change when CI creates a fresh sandbox.

Filesystem order is not an API

Directory iteration order is not a stable contract. ext4, APFS, and tmpfs return different child sequences. CI hosts will shuffle your report if you skip sort.

Pin a lexicographic sort on the relative POSIX strings. Sort once at the end, not per directory only. Per-directory sort plus later append still needs a final sort.

Build a tiny messy fixture

Create a sandbox that encodes the real hazards. Keep the layout documented for later local reproduction. Label this fixture as unexecuted until you create it.

walk_sandbox/
  keep/a.txt
  keep/b.txt
  skip_perm/secret.txt
  loop/up -> ..
  nested/deep/c.txt
  .hidden.txt

The loop/up entry is a symlink to the parent directory. Unreadable skip_perm needs a chmod step on POSIX hosts. Hidden files appear only if the current contract includes them.

Record intended relative paths in one frozen tuple. Sort that tuple using plain lexicographic string order. Use forward slashes even when helpers run on Windows.

Characterization tests first

Write tests against the current messy function first. Do not rewrite production code during this pinning step. The tests should fail only when observable behavior changes.

Treat the pytest module below as a labeled template. Run it only after the sandbox exists on disk.

# test_walk_characterize.py — proposed template, unexecuted until you run it
from __future__ import annotations

import os
from pathlib import Path

import pytest

from messy_report import collect_files  # existing mixed helper

FROZEN_RELPATHS = (
    "keep/a.txt",
    "keep/b.txt",
    "nested/deep/c.txt",
)


def test_relative_paths_are_sorted_posix():
    root = Path("walk_sandbox")
    got = collect_files(root)
    assert got == list(FROZEN_RELPATHS)


def test_symlink_cycle_does_not_loop_or_duplicate():
    root = Path("walk_sandbox")
    got = collect_files(root)
    assert got == list(FROZEN_RELPATHS)
    assert len(got) == len(set(got))


@pytest.mark.skipif(os.name != "posix", reason="chmod directory bits are POSIX-only")
def test_unreadable_dir_is_skipped(tmp_path):
    keep = tmp_path / "keep"
    keep.mkdir()
    (keep / "ok.txt").write_text("x", encoding="utf-8")
    blocked = tmp_path / "blocked"
    blocked.mkdir()
    (blocked / "nope.txt").write_text("x", encoding="utf-8")
    blocked.chmod(0o000)
    try:
        got = collect_files(tmp_path)
        assert got == ["keep/ok.txt"]
    finally:
        blocked.chmod(0o755)

Execute one command and keep the raw pytest output.

python -m pytest test_walk_characterize.py -q

If the hidden file currently appears, extend FROZEN_RELPATHS. Do not "fix" hidden-file policy in this pass. Characterization captures today's contract, not the ideal one.

Reproduce the EACCES row

Create the blocked directory inside a pytest tmp_path. Write one file, then chmod the directory to 0o000. Always restore mode in a finally block after the assertion.

Mark the test skipped when the host is not POSIX. Windows access control lists do not map to chmod bits. A false green on Windows would hide a real skip bug.

Decision table for skip rules

Use a table so reviewers argue on rows. Do not argue from memory of yesterday's walk.

Input node	Current messy behavior	Pin in test?
Regular file	Emit relative POSIX path	Yes
Directory	Recurse into children	Yes
Symlink to file	Follow or emit; pick the live result	Yes
Symlink to dir, acyclic	Follow once	Yes
Symlink cycle	Stop without hang or duplicate	Yes
Unreadable dir (EACCES)	Skip and continue	Yes
Name .hidden.txt	Include or exclude; pick the live result	Yes
mtime or size	Do not compare	No
Absolute path prefix	Do not compare	No

Fill the "pick one" rows from the live helper. Copy the observed result into the frozen tuple. Do not invent a cleaner policy during pinning.

Four-step sequence

Follow four steps and do not skip the goldens. Each step produces a concrete artifact you can review.

1. Capture live relative paths from the messy helper.
2. Freeze symlink and EACCES behavior in pytest.
3. Extract one walker after those tests pass.
4. Reject any draft that changes the frozen tuple.

Smallest safe change

Wait until the three tests pass on messy code. Then extract one function and leave other helpers untouched.

Move only the walk, skip, and sort logic. Leave report formatting inside the original caller module.

Treat the extract below as unexecuted until tests stay green.

# walk_files.py — proposed extract, unexecuted until goldens stay green
from __future__ import annotations

from pathlib import Path


def collect_files(root: Path) -> list[str]:
    root = root.resolve()
    out: list[str] = []
    seen: set[Path] = set()

    def walk(dir_path: Path) -> None:
        try:
            real = dir_path.resolve()
        except OSError:
            return
        if real in seen:
            return
        seen.add(real)
        try:
            children = sorted(dir_path.iterdir(), key=lambda p: p.name)
        except PermissionError:
            return
        for child in children:
            try:
                if child.is_symlink() and child.is_dir():
                    walk(child)
                    continue
                if child.is_dir():
                    walk(child)
                    continue
                if child.is_file():
                    rel = child.relative_to(root).as_posix()
                    out.append(rel)
            except PermissionError:
                continue

    walk(root)
    out.sort()
    return out

Swap the import inside the report command next. Run the same pytest module against the new helper. The extract is done when FROZEN_RELPATHS still matches.

Do not add ignore-file parsing in the same diff. Do not add content hashing in that same change. Keep one behavior surface per extract, then stop.

Command log for the extract

Keep a short command log in the pull request body. Reviewers should replay the same three local commands.

python -m pytest test_walk_characterize.py -q
python -m pytest test_walk_characterize.py -q --tb=short
python -c "from walk_files import collect_files; from pathlib import Path; print(collect_files(Path('walk_sandbox')))"

That last print is a manual sanity check, not a test. Do not commit its output as a golden file. The pytest tuple remains the only frozen artifact.

Where a free model can help

A model is useful after goldens exist, not before. It can rewrite the nested walk into the helper. It cannot invent the skip policy from a prompt.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode offers free model access and a free server option. Use that pair only to draft the extract against frozen tests.

Paste the characterization file and the current messy function. Ask for a move that keeps the assertions unchanged.

Then run pytest on your machine before any merge. Discard any diff that changes FROZEN_RELPATHS or follow rules. The characterization tests remain the only source of truth.

One local review command is enough after the draft.

python -m pytest test_walk_characterize.py -q --tb=short

What the model must not change

Give the model a short reject list. Store that list beside the characterization test module.

1. Do not sort collected paths by mtime or size.
2. Do not return Path objects to existing callers.
3. Do not emit absolute prefixes in the result list.
4. Do not raise on EACCES for nested subdirectories.
5. Do not follow a symlink cycle during recursion.

If the draft violates one rule, reject the whole patch. Do not hand-merge a partial extract from a bad draft. Partial merges often reintroduce silent path order bugs.

Limitations

This workflow pins a POSIX-leaning relative path contract. Windows junctions and drive letters need extra table rows. Network filesystems can raise OSError subclasses you never froze.

chmod 0o000 is not a portable permission story. Some platforms ignore directory execute bits in tests. Skip the EACCES case on those hosts, or xfail it.

Symlink resolve() can escape the intended sandbox root. The seen-set of resolved paths is the cycle guard. If resolve() fails, the walk skips that node.

That skip becomes part of the frozen contract. Document it in the test name so later readers notice.

Hidden-file policy is whatever the messy code already did. Changing it is a product change, not a refactor. Ship that change later with a new explicit test row.

Who should not use this approach

Do not use this extract on crawlers that must raise EACCES. Security scanners often need the exception, not a skip. Do not use sorted relative paths when callers need inode order.

Do not send production trees to a remote model. The sandbox fixture is the only prompt input. Redact real customer paths before any model request.

Skip this method when the walk already has public docs. Characterization fits undocumented mixed helpers, not stable APIs. Documented walkers need explicit design, not frozen accidents.

Close

Freeze order, cycles, and EACCES before any rename. Extract one walker after those three tests stay green. Keep ignore files and hashes out of that diff.

The tests are the refactor permit for this change. Optional model drafting starts only after that permit exists.