Derive OSS Review Scope From the Crash Transcript

OSS patch review should start from a crash transcript. The transcript defines files, claims, and comments. A model that cannot cite that transcript should not review the diff.

Whole-tree prompts invite drive-by refactors. Review threads then drift into style. The original failure leaves the discussion. A frozen worktree plus a captured transcript keeps the bug in view.

This article proposes a local gate. Stack frames become the file scope. Review notes must cite transcript line numbers. The scripts below are a workflow, not a production study.

Why a transcript beats a repo dump

A crash log names the path that actually failed. Maintainers already trust that path. A model does not need the rest of the tree.

Issue text is often incomplete. Stack frames are harder to fake. The patch should touch those frames first. Comments that ignore the log are noise.

This gate does not replace tests. It only binds review scope to one reproduction. Human judgment still owns the merge.

What the gate records

The gate writes four local artifacts. Each file is small and replayable.

1. worktree.lock stores the frozen commit SHA.
2. repro.cmd stores the exact failing command.
3. transcript.txt stores stdout, stderr, and exit code.
4. scope.paths stores files parsed from stack frames.

The patch may change only files in scope.paths, plus tests. Review comments must cite transcript.txt line numbers. Anything else fails the gate.

Step 1: Freeze a disposable worktree

Do not review on the contributor clone. Create a worktree from the issue SHA. Keep main untouched.

#!/usr/bin/env bash
set -euo pipefail
# Proposed local workflow. Run from the maintainer clone.
ISSUE_SHA="${1:?need issue sha}"
BUG_ID="${2:?need bug id}"
ROOT="$(git rev-parse --show-toplevel)"
WT="${ROOT}/.review-worktrees/${BUG_ID}"

git fetch --quiet origin "${ISSUE_SHA}"
git worktree add --detach "${WT}" "${ISSUE_SHA}"
printf '%s\n' "${ISSUE_SHA}" > "${WT}/worktree.lock"
echo "frozen ${ISSUE_SHA} at ${WT}"

The worktree dies after the review. No model process should write it. Use a second clone if the first tool is untrusted.

Step 2: Capture one reproduction transcript

Run the command the issue claims. Capture both streams. Store the exit code. Do not edit the log by hand.

#!/usr/bin/env bash
set -euo pipefail
WT="${1:?need worktree}"
shift
CMD=( "$@" )
LOG="${WT}/transcript.txt"
printf 'cmd: %q ' "${CMD[@]}" > "${LOG}"
printf '\n---\n' >> "${LOG}"
set +e
( cd "${WT}" && "${CMD[@]}" ) >> "${LOG}" 2>&1
STATUS=$?
set -e
printf '\n---\nexit: %s\n' "${STATUS}" >> "${LOG}"
printf '%s\n' "${CMD[*]}" > "${WT}/repro.cmd"
if [[ "${STATUS}" -eq 0 ]]; then
  echo "reproduction did not fail; stop the review" >&2
  exit 2
fi

A green command is not a review input. The bug is not proven. Stop before any model sees a diff.

Step 3: Derive file scope from stack frames

Parse the transcript for source paths. Keep paths that exist in the worktree. Drop site-packages and generated noise.

#!/usr/bin/env python3
"""Proposed parser for Python and similar 'File "path", line N' frames."""
from pathlib import Path
import re
import sys

FRAME = re.compile(r'File "([^"]+)", line (\d+)')
SKIP_PARTS = (".venv", "site-packages", "dist-packages", "__pycache__")


def existing_repo_file(worktree: Path, raw: str) -> Path | None:
    candidate = Path(raw)
    if not candidate.is_absolute():
        candidate = worktree / candidate
    try:
        resolved = candidate.resolve()
        resolved.relative_to(worktree.resolve())
    except (OSError, ValueError):
        return None
    if not resolved.is_file():
        return None
    if any(part in SKIP_PARTS for part in resolved.parts):
        return None
    return resolved.relative_to(worktree.resolve())


def main() -> int:
    worktree = Path(sys.argv[1])
    text = (worktree / "transcript.txt").read_text(encoding="utf-8", errors="replace")
    found: list[str] = []
    for match in FRAME.finditer(text):
        rel = existing_repo_file(worktree, match.group(1))
        if rel is None:
            continue
        key = rel.as_posix()
        if key not in found:
            found.append(key)
    if not found:
        print("no in-repo stack frames; refuse model review", file=sys.stderr)
        return 2
    out = worktree / "scope.paths"
    out.write_text("\n".join(found) + "\n", encoding="utf-8")
    print(f"wrote {out} ({len(found)} paths)")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

The parser is language-shaped. Java, Go, and Rust need different frame regexes. Empty scope is a hard stop, not a prompt to guess.

Step 4: Bound the incoming patch

Apply the patch inside the worktree. Then list changed files. Reject extras before review text exists.

#!/usr/bin/env bash
set -euo pipefail
WT="${1:?need worktree}"
PATCH="${2:?need patch file}"
cd "${WT}"
git apply --check "${PATCH}"
git apply "${PATCH}"
mapfile -t CHANGED < <(git diff --name-only)
mapfile -t SCOPE < <(grep -v '^$' scope.paths)
# tests may change even when they are not in the crash frames
allow() {
  local f="$1"
  [[ "${f}" == tests/* || "${f}" == test/* || "${f}" == *_test.py ]] && return 0
  printf '%s\n' "${SCOPE[@]}" | grep -Fxq "${f}"
}
for f in "${CHANGED[@]}"; do
  if ! allow "${f}"; then
    echo "out-of-scope path: ${f}" >&2
    exit 3
  fi
done
echo "patch stays inside transcript scope"

Drive-by cleanup fails here. Format-only files fail here. A later model cannot excuse them.

Step 5: Demand transcript citations in the review

Review notes live in review.md. Each claim needs a transcript: cite. The checker reads the log length. Invalid lines fail the packet.

#!/usr/bin/env python3
"""Fail review notes that do not cite transcript.txt line numbers."""
from pathlib import Path
import re
import sys

CITE = re.compile(r"transcript:(\d+)(?:-(\d+))?")
CLAIM = re.compile(r"(?m)^-\s+")


def main() -> int:
    worktree = Path(sys.argv[1])
    notes = (worktree / "review.md").read_text(encoding="utf-8")
    log_lines = (worktree / "transcript.txt").read_text(encoding="utf-8").splitlines()
    n = len(log_lines)
    claims = CLAIM.findall(notes)
    cites = CITE.findall(notes)
    if not claims:
        print("review.md has no claim bullets", file=sys.stderr)
        return 2
    if len(cites) < len(claims):
        print("every claim needs a transcript:N cite", file=sys.stderr)
        return 3
    for start, end in cites:
        a = int(start)
        b = int(end) if end else a
        if a < 1 or b < a or b > n:
            print(f"cite transcript:{a}-{b} is outside 1-{n}", file=sys.stderr)
            return 4
    print(f"accepted {len(claims)} claims against {n} transcript lines")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

A valid review.md looks like this. Short claims. Concrete cites. No repo tour.

# Review packet for BUG-1842

- The parser still raises on empty tokens. transcript:14-21
- The patch guards that branch in `pkg/scan.py`. transcript:18
- The new unit test replays the same command. transcript:2

A rejected note looks like this. It may be true. It still fails the gate.

- Consider renaming helpers for clarity.
- Also reformat imports in adjacent modules.

Those sentences do not point at the crash. They do not belong in this review.

Step 6: Replay after the patch

The same command must change outcome. Capture a second transcript. Diff the two logs. Keep both files.

#!/usr/bin/env bash
set -euo pipefail
WT="${1:?need worktree}"
CMD_FILE="${WT}/repro.cmd"
read -r -a CMD < "${CMD_FILE}"
AFTER="${WT}/transcript.after.txt"
set +e
( cd "${WT}" && "${CMD[@]}" ) > "${AFTER}" 2>&1
STATUS=$?
set -e
if [[ "${STATUS}" -ne 0 ]]; then
  echo "command still fails after patch" >&2
  exit 4
fi
diff -u "${WT}/transcript.txt" "${AFTER}" > "${WT}/transcript.delta" || true
echo "wrote transcript.delta"

A silent success is not enough. The delta shows what vanished. Reviewers can read the delta without opening the tree.

Where a free model fits

The model reads three files only. Those files are transcript.txt, transcript.delta, and the patch. It writes review.md. The citation checker still runs locally.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode provides free model access and a free server option that can host this narrow review loop. The scripts do not require that product. They fail closed without any model at all.

Do not paste secrets into the prompt. Do not upload .env files. Do not send the full worktree. The transcript is the budget.

A compact decision table

Use this table before inviting a model. It is a process check, not a score.

Signal	Action
Reproduction exits zero	Stop. No review.
No in-repo stack frames	Stop. Fix the parser or the bug report.
Patch touches extra files	Reject. Ask for a scoped revision.
review.md lacks cites	Reject the comments, not necessarily the patch.
After-patch command still fails	Reject the patch.
Delta is empty after a claimed fix	Reject. The log did not change.
Cite points at setup noise	Ask for a tighter command.

Humans still read security-sensitive diffs. This table does not bless cryptography changes. It only blocks unscoped chatter.

Limitations

The Python frame regex misses many languages. A Go panic needs another parser. A C abort may print only addresses.

Flaky tests poison the transcript. One green rerun can fake a fix. Pin the seed when the suite allows it.

Generated sources confuse scope. Vendor trees and protobuf outputs look like project files. Extend SKIP_PARTS before trusting the list.

The citation checker is syntactic. A model can cite the right lines and still be wrong. Maintainers read the claims.

Networked reproduction is out of scope. The gate assumes a local command. External fixtures need a recorded snapshot, not a live API.

Who should not use this

New contributors who need mentoring should not face this gate first. Docs-only patches have no crash log. Do not invent a transcript for them.

Security embargo patches should stay off shared model hosts. The frozen worktree still helps. The model step should not.

Huge monorepos with weak traces will reject most patches. Fix tracing first. Do not loosen cites to keep a bot busy.

Teams without a failing command should pick another process. This workflow has nothing to cite.

Close the loop

Delete the worktree after the decision. Keep the four artifacts beside the issue. The next reviewer can replay without a chat history.

git worktree remove --force "${WT}"

The crash transcript remains the contract. Models may draft review.md. They do not choose the scope. A spare free server can run the same commands. Keep the worktree disposable either way.