DEV Community

Dakota Huang
Dakota Huang

Posted on

Capture the Child Process Transcript. Then Extract One Runner.

A child-process wrapper hides a four-part contract. Argv, cwd, env, and streams must stay stable. Extract a runner only after transcript tests lock them.

Messy modules wrap subprocess.run in ad-hoc helpers. Those helpers then grow flags and path hacks. Reviewers later accept a cleanup that reorders argv.

Production then fails on a missing env key. Stderr order can also flip under check=True. The failure looks random without a frozen transcript.

This workflow records one golden transcript first. It then allows one runner extract. The method stays local, testable, and intentionally small.

The four fields that actually leak

Most suites pin only the process return code. That single integer is not the contract. A runner also leaks cwd and a filtered environment.

Stdout encoding and stderr line order also matter. A later cleanup can flip check=True silently. Characterization tests must freeze all four fields together.

Field Pin this observable Leave this unpinned
argv exact token list a reconstructed shell string
cwd resolved path or None unrelated parent directories
env allowlisted keys only a full os.environ dump
streams stdout, stderr, returncode wall-clock duration

Treat wall-clock duration as a non-goal here. Retry jitter belongs in a later dedicated test. Timing noise should never fail a transcript lock.

Step 1. Isolate the call site

Find the function that shells out today. Do not extract that function yet. Copy the live call into a short fixture note.

# inventory.py — messy on purpose; do not extract yet
import os
import subprocess
from pathlib import Path

def list_skus(root: Path, extra: str | None = None) -> list[str]:
    env = os.environ.copy()
    env["SKU_ROOT"] = str(root)
    cmd = ["sku-tool", "list", "--json"]
    if extra:
        cmd.extend(["--filter", extra])
    proc = subprocess.run(
        cmd,
        cwd=root,
        env=env,
        capture_output=True,
        text=True,
        check=False,
    )
    if proc.returncode != 0:
        raise RuntimeError(proc.stderr.strip() or "sku-tool failed")
    return [line for line in proc.stdout.splitlines() if line]
Enter fullscreen mode Exit fullscreen mode

Record three facts beside that call. Note the argv token order. Note that cwd equals the given root.

Also note that SKU_ROOT is injected every time. Those three notes become later assertions. They are observations, not a redesign.

Step 2. Capture one real transcript

Run the messy function against a stub binary. Do not mock inside the module yet. Put a tiny script early on PATH.

mkdir -p /tmp/sku-fixture/bin
cat > /tmp/sku-fixture/bin/sku-tool <<'EOF'
#!/usr/bin/env python3
import json, os, sys
print("SKU-100")
print("SKU-200")
EOF
chmod +x /tmp/sku-fixture/bin/sku-tool
export PATH="/tmp/sku-fixture/bin:$PATH"
python3 -c "from pathlib import Path; from inventory import list_skus; print(list_skus(Path('/tmp/sku-fixture')))"
Enter fullscreen mode Exit fullscreen mode

Save stdout, stderr, and returncode from that run. That triple is the golden transcript. Store it next to tests, never in chat history.

{
  "argv": ["sku-tool", "list", "--json"],
  "cwd_is_root": true,
  "env_sku_root_is_root": true,
  "stdout_lines": ["SKU-100", "SKU-200"],
  "stderr": "",
  "returncode": 0,
  "text": true,
  "check": false
}
Enter fullscreen mode Exit fullscreen mode

Label this fixture as observed, not designed. Tighten tokens only after a second capture. Do not invent flags the binary never received.

Step 3. Wrap subprocess with a recording fake

Replace the live binary inside tests only. Keep the production module byte-stable. Patch subprocess.run and record every kwarg.

# tests/test_inventory_transcript.py
from __future__ import annotations

import json
import subprocess
from pathlib import Path
from unittest.mock import patch

import pytest

from inventory import list_skus

GOLDEN = json.loads(Path("tests/golden/sku_list.json").read_text())

class Transcript:
    def __init__(self) -> None:
        self.calls: list[dict] = []

    def run(self, argv, cwd=None, env=None, capture_output=True, text=True, check=False):
        self.calls.append(
            {
                "argv": list(argv),
                "cwd": None if cwd is None else str(cwd),
                "sku_root": None if env is None else env.get("SKU_ROOT"),
                "capture_output": capture_output,
                "text": text,
                "check": check,
            }
        )
        stdout = "SKU-100\nSKU-200\n"
        return subprocess.CompletedProcess(argv, 0, stdout, "")

def test_list_skus_pins_child_transcript(tmp_path: Path) -> None:
    fake = Transcript()
    with patch("inventory.subprocess.run", fake.run):
        rows = list_skus(tmp_path, extra=None)
    assert rows == GOLDEN["stdout_lines"]
    assert len(fake.calls) == 1
    call = fake.calls[0]
    assert call["argv"] == GOLDEN["argv"]
    assert call["cwd"] == str(tmp_path)
    assert call["sku_root"] == str(tmp_path)
    assert call["capture_output"] is True
    assert call["text"] is True
    assert call["check"] is False
Enter fullscreen mode Exit fullscreen mode

This test is a characterization test only. It does not claim a clean design. It freezes observed coupling before any extract.

Add a second case for the filter tokens. Pin those extra argv items alone. Do not merge both cases into one assertion block.

def test_list_skus_pins_filter_tokens(tmp_path: Path) -> None:
    fake = Transcript()
    with patch("inventory.subprocess.run", fake.run):
        list_skus(tmp_path, extra="A*")
    assert fake.calls[0]["argv"] == [
        "sku-tool",
        "list",
        "--json",
        "--filter",
        "A*",
    ]
Enter fullscreen mode Exit fullscreen mode

Scan the module for Popen as well. Patching run will miss Popen sites. Freeze each family in a separate test file.

rg -n "subprocess\.(run|Popen|check_call|check_output)" inventory.py
Enter fullscreen mode Exit fullscreen mode

Step 4. Pin the failure transcript too

Happy-path locks are incomplete on purpose. Record the nonzero return code next. Record the exact stderr text as well.

class FailingTranscript(Transcript):
    def run(self, argv, cwd=None, env=None, capture_output=True, text=True, check=False):
        self.calls.append({"argv": list(argv), "check": check})
        return subprocess.CompletedProcess(argv, 2, "", "denied\n")

def test_list_skus_raises_on_stderr(tmp_path: Path) -> None:
    fake = FailingTranscript()
    with patch("inventory.subprocess.run", fake.run):
        with pytest.raises(RuntimeError, match="denied"):
            list_skus(tmp_path)
    assert fake.calls[0]["check"] is False
Enter fullscreen mode Exit fullscreen mode

Do not rewrite the exception type in this PR. That rewrite is a later behavior change. Characterization first. Typed errors second.

Keep check=False pinned in the failure case. Flipping check changes exception class and traceback. That flip is not a runner extract.

Step 5. Extract one runner, nothing else

The tests now own the four-field contract. Extract a single function after they pass. Keep argv construction in the original caller.

# inventory.py — after the smallest extract
import os
import subprocess
from pathlib import Path

def run_sku_tool(argv: list[str], root: Path) -> subprocess.CompletedProcess:
    env = os.environ.copy()
    env["SKU_ROOT"] = str(root)
    return subprocess.run(
        argv,
        cwd=root,
        env=env,
        capture_output=True,
        text=True,
        check=False,
    )

def list_skus(root: Path, extra: str | None = None) -> list[str]:
    cmd = ["sku-tool", "list", "--json"]
    if extra:
        cmd.extend(["--filter", extra])
    proc = run_sku_tool(cmd, root)
    if proc.returncode != 0:
        raise RuntimeError(proc.stderr.strip() or "sku-tool failed")
    return [line for line in proc.stdout.splitlines() if line]
Enter fullscreen mode Exit fullscreen mode

Retarget nothing if the call stayed in-module. Patch inventory.subprocess.run still intercepts it. That stable patch path is the point of one extract.

If you move run_sku_tool to procutil.py, stop. Update the patch in a second PR. One extract, one module, one patch path.

Step 6. Commands that prove the lock

Run the narrow test file first. Then inspect the production diff only. Avoid a full-suite first pass on a messy repo.

python -m pytest tests/test_inventory_transcript.py -q --tb=short
git diff --stat -- inventory.py tests/test_inventory_transcript.py
git diff -- inventory.py
Enter fullscreen mode Exit fullscreen mode

The production diff should add one function. It should not retune flags or retries. It should not change check, text, or cwd.

Reject a diff that also reformats JSON parsing. Parsing is a second contract. Handle it after the runner extract merges.

Where a free model and free server fit

Drafting the fake from a captured log is tedious. A free coding model can propose Transcript.run from that log.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode offers free model access and a free server option. You can draft the first fixture there, then rerun pytest on the free server if your laptop lacks that interpreter. Treat both outputs as untrusted until the golden JSON matches a local rerun.

Do not accept a model-written extract that also improves argv. Reject any diff that touches more than the runner function. The model proposes; the transcript decides.

Decision rule for the next PR

Use this table before you merge anything.

Observation Action Stop if
transcript test is red fix fixture data only production changes first
runner extract is the only edit merge argv tokens also moved
a new flag appears add one test row old flags get cleaned
patch path broke restore the path a second extract appears

The rule is numeric and small. One red test. One extract. Zero extra flags.

Count subprocess call sites after merge as well. The number must stay constant. A hidden second run() is a new contract.

rg -c "subprocess.run" inventory.py
Enter fullscreen mode Exit fullscreen mode

Limitations

This workflow does not prove the child binary is correct. It only pins how Python wraps that binary. Binary upgrades still need fresh captured fixtures.

It also ignores timing, signals, and partial pipe reads. Streaming CLIs need a different harness. Interactive prompts need a different harness too.

Environment allowlists can drift without failing GOLDEN. A newly required key will not appear there. Add an explicit env test when the binary documents that key.

text=True hides encoding defects on mixed bytes. If the tool emits non-UTF8, pin bytes instead. Do not convert encoding during the extract PR.

Who should not use this

Do not use this method on secret-bearing wrappers. Putting tokens in argv is a different defect. Characterization would freeze that leak as if it were desired.

Do not use it as a license to skip design. A god runner with twenty kwargs remains a trap. Extract one function, not a process framework.

Skip this if the module already has contract tests. Do not stack golden files on a clean API. Characterization is for unknown coupling only.

Close

Pin argv, cwd, env, and streams first. Then extract one runner function. Leave retries, parsing, and module moves for later PRs.

The transcript is the contract you actually have. The extract is the only change that PR should contain. Keep those two steps in that order.

Top comments (0)