Snapshot Exit, Stdout, and Stderr Before One Flag Extract

Messy CLIs fail at the process boundary, not helpers.
Snapshot three channels before extracting one flag parser.
Exit code, stdout shape, and stderr tokens form the contract.

Why the process boundary comes first

A flag extract looks local inside the editor.
The process contract is not local at all.

Callers wrap your CLI in scripts and CI jobs.
Those jobs parse JSON keys from stdout streams.
They also grep stderr and branch on exit codes.

Rename a helper and those wrappers can still pass.
Change one token and the same wrappers fail loudly.
Characterization tests freeze the boundary before any extract.

This writeup is not a production war story.
Every snippet below is a labeled, unexecuted example.
Run the harness on your tree before you trust it.

Official references for the tools used:

• subprocess
• pytest

The messy module under test

The sample CLI mixes parsing, formatting, and logging.
It lives in one file on purpose today.
That layout is the starting mess, not the goal.

# example_cli.py — labeled example, not production code
import json
import sys

def main(argv):
    fmt = "json"
    ids = []
    for a in argv:
        if a.startswith("--format="):
            fmt = a.split("=", 1)[1]
        elif a.startswith("--id="):
            ids.append(a.split("=", 1)[1])
        elif a in ("-h", "--help"):
            sys.stderr.write("usage: example_cli [--format=json|text] --id=N\n")
            return 2
    if not ids:
        sys.stderr.write("error: missing --id\n")
        return 2
    rows = [{"id": i, "ok": True} for i in ids]
    if fmt == "json":
        sys.stdout.write(json.dumps({"rows": rows, "count": len(rows)}) + "\n")
        return 0
    if fmt == "text":
        sys.stdout.write("count={0}\n".format(len(rows)))
        return 0
    sys.stderr.write("error: unknown format\n")
    return 2

if __name__ == "__main__":
    sys.exit(main(sys.argv[1:]))

Do not extract parse_flags from this file yet.
You do not own the observable contract yet.
Capture that contract with a process harness first.

Decision table: freeze wrappers, ignore internals

Record only observables a wrapper can see.
Skip internal function names during this pass.
Skip private dict identity and helper layout.

Channel	Freeze now	Ignore now
Exit code	0 versus 2 versus any other	Python traceback wording
stdout	JSON keys, types, and text lines	Indentation inside JSON values
stderr	Stable tokens error: and usage:	Absolute paths and timestamps
argv	--format= and --id= forms in use	Flags nobody calls today

Three rules keep this table small and honest.
Freeze behavior that CI or scripts already assert.
Ignore noise that changes on every single run.

If a table cell is unknown, add a probe.
Do not guess the contract by reading source.
Source comments drift. Process bytes do not.

1. Inventory current callers

Search the repo for the CLI file name.
Note who decodes stdout as JSON objects.
Note who greps stderr for a fixed token.

rg -n "example_cli" --glob '!*.pyc'
rg -n "error: missing --id"
rg -n '"count"' --glob '*test*'

Write every hit into a short caller list.
That list is the characterization scope today.
Anything absent from the list stays unfrozen.

2. Build a subprocess snapshot harness

Run the real file through a new Python process.
Do not import main for this first layer.
Wrappers do not import main. They execute the file.

# test_cli_triad.py — labeled example harness
import json
import subprocess
import sys
from pathlib import Path

CLI = Path(__file__).resolve().parent / "example_cli.py"

def run_cli(args):
    proc = subprocess.run(
        [sys.executable, str(CLI), *args],
        capture_output=True,
        text=True,
    )
    return proc.returncode, proc.stdout, proc.stderr

def test_json_success_triad():
    code, out, err = run_cli(["--format=json", "--id=7"])
    assert code == 0
    assert err == ""
    payload = json.loads(out)
    assert list(payload.keys()) == ["rows", "count"]
    assert payload["count"] == 1
    assert payload["rows"] == [{"id": "7", "ok": True}]

def test_missing_id_stderr_token():
    code, out, err = run_cli(["--format=json"])
    assert code == 2
    assert out == ""
    assert "error: missing --id" in err

def test_help_uses_usage_token():
    code, out, err = run_cli(["--help"])
    assert code == 2
    assert out == ""
    assert err.startswith("usage:")

def test_unknown_format_keeps_exit_two():
    code, out, err = run_cli(["--format=xml", "--id=1"])
    assert code == 2
    assert out == ""
    assert "error: unknown format" in err

Run that harness before any parser extract exists.

python -m pytest test_cli_triad.py -q

Green results mean the triad is now pinned.
Red results mean your mental model was wrong.
Fix the tests to match today's real behavior.

3. Golden-file text only when parsers are strict

json.loads already pins keys and value types.
Skip a raw stdout golden unless byte order matters.
Callers that decode JSON do not care about spaces.

If a caller greps one text line, pin that line.
Use a stripped golden, never a timestamped dump.
Keep one format inside each test function.

def test_text_success_line():
    code, out, err = run_cli(["--format=text", "--id=7"])
    assert code == 0
    assert err == ""
    assert out == "count=1\n"

Do not combine JSON and text in one case.
Failures must name the single broken channel.
Mixed asserts hide which wrapper you just broke.

4. Extract one flag parser, nothing else

The smallest safe change is one pure function.
It returns format, ids, and two boolean flags.
It does not print. It does not call sys.exit.

def parse_flags(argv):
    fmt = "json"
    ids = []
    help_requested = False
    unknown_format = False
    for a in argv:
        if a.startswith("--format="):
            fmt = a.split("=", 1)[1]
            if fmt not in ("json", "text"):
                unknown_format = True
        elif a.startswith("--id="):
            ids.append(a.split("=", 1)[1])
        elif a in ("-h", "--help"):
            help_requested = True
    return {
        "fmt": fmt,
        "ids": ids,
        "help_requested": help_requested,
        "unknown_format": unknown_format,
    }

Wire that dict back into main with no new behavior.
Keep every stderr string byte-identical after the move.
Keep JSON key insertion order identical after the move.

Re-run the triad tests on the same command line.
If they fail, revert the extract before more edits.
The tests own the merge decision, not the diff size.

5. Refuse the second extract on the same branch

Do not extract a formatter in this same patch.
Do not relocate stderr writes in this same patch.
One behavior-preserving extract per branch is the cap.

If a generated diff also rewrites help text, drop it.
The harness does not cover formatter internals yet.
Uncovered internals are not ready for movement today.

Where a free coding model belongs

A model can draft tests from captured process output.
It should not rewrite example_cli.py as step one.
That order is the method, not a style preference.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode's free model access and free server option can host that draft step.
Paste the captured triad, not a wish list of clean code.
Ask for pytest cases that assert today's bytes and codes.

Then run those cases on your own checkout.
The server does not replace the subprocess harness.
The model does not own the merge or the revert.

Reject any patch that rewrites stderr tokens.
Reject any patch that reorders the JSON keys.
Reject any patch that maps user-error exit 2 to 1.

Failure analysis the triad is built to catch

Case A: the extract switches parsing to argparse.
argparse writes its own usage text on help.
The usage: test fails before any caller does.

Case B: the extract starts pretty-printing JSON output.
Key order or wrapping can shift under indent=2.
The key-list assert fails on the next pytest run.

Case C: the extract returns exit 1 for user errors.
Scripts that check code == 2 now take the wrong branch.
The missing-id test fails first and names that channel.

Case D: the extract lowercases or punctuates stderr.
A grep for error: missing --id now misses the line.
The token test fails first and blocks the merge.

Each failure names one channel on purpose.
That is why these are four tests, not one blob.
A single catch-all snapshot hides the broken wrapper.

Limitations

This harness does not pin wall-clock timestamps.
It does not pin random identifiers or UUIDs.
It does not pin Unicode normalization on Windows consoles.

It does not replace unit tests for pure helpers.
It does not prove thread safety under parallel invokes.
It does not prove latency, memory, or startup cost.

Subprocess tests are slower than direct function tests.
Keep the triad set under a dozen focused cases.
Add helper unit tests only after the extract stays green.

JSON key order is pinned because this sample uses literals.
CPython preserves insertion order for those dicts.
If construction order varies, assert key sets instead.

Who should not use this approach

Do not snapshot CLIs that print secrets or tokens.
Those bytes will land in git and in CI logs.
Rotate anything that already leaked into a fixture.

Do not characterize intentionally unstable streams first.
Live prices and clocks need fakes before snapshots.
Freeze time, then pin the three channels around it.

Do not treat a frozen bad contract as good design.
A stable mistake is still a mistake for callers.
Schedule a versioned break after the extract is safe.

Teams with a real boundary suite can skip this pass.
The method is for messy repos without process tests.
If the triad is already pinned, extract the parser now.

Checklist before the extract PR

1. Caller inventory exists in a ticket or comment.
2. Triad tests pass on the current main branch.
3. The diff extracts one parser and nothing else.
4. Stderr tokens match the pre-change captured bytes.
5. Exit codes match the decision table above.
6. JSON keys and types match the snapshot asserts.
7. No second helper moved inside the same patch.

If any box is unchecked, stop the extract.
The parser move is not the bottleneck here.
The missing snapshot is the actual risk.