DEV Community

Quinn Sun
Quinn Sun

Posted on

The Pairing Transcript Became the Diff Allowlist

A flaky webhook retry was supposed to be a thirty-minute fix. The coding agent turned it into a worker pool, a new helper module, and a README lecture on backoff theory.

The junior had asked for a one-line guard. The senior asked for a pairing session instead. What follows is a reconstructed walkthrough of that session: the questions asked, the dead ends, and the single decision that survived.

This is a worked example, not a production postmortem. No latency numbers or model leaderboards are claimed.

The scene

The repository was a small Node service that retried a webhook once and then dropped it. Production needed a second attempt with jitter. Nothing else.

A free coding agent was already in the loop because the team wanted cheap iteration, not a new architecture. Cheap generation made extra files feel free. Extra files become debt the team owns on Monday.

The senior did not start with a prompt. The senior started with a notebook.

Questions the senior actually asked

The junior wanted to paste the stack trace and let the model run. The senior slowed the session down. Every question was spoken out loud, then written into pairing-transcript.md before any tool call.

  1. Which file owns the retry today?
  2. Which test already describes the current one-shot behavior?
  3. What counts as success, in one sentence?
  4. Which paths have already been tried and rejected?
  5. What is the agent forbidden to touch, even if the patch looks cleaner?

Those questions were not ceremony. They became the only input the agent was allowed to see besides the two files named in the answers.

The transcript started like this:

# Pairing transcript — webhook retry jitter

## Goal
Add a second retry with jitter to `src/webhooks/retry.js`.
Keep the existing payload shape. Do not introduce a queue.

## Files in play
- src/webhooks/retry.js
- test/webhooks/retry.test.js

## Forbidden
- package.json
- src/workers/
- README.md

## Questions asked
- Where does retry count live? Answer: `attempt` on the in-memory job.
- Is there a shared sleep helper? Answer: `src/lib/sleep.js`, out of scope.
- Can we reuse the worker pool? Answer: no; rejected last quarter.

## Dead ends (do not retry)
- Extracting a generic RetryPolicy class
- Switching to an external queue library
- Changing the webhook HTTP client

## Decision kept
Patch `retry.js` and extend the existing test.
Fail the session if any other path changes.
Enter fullscreen mode Exit fullscreen mode

The agent had not been launched yet. That was the point.

Dead end 1: a conservative prompt

The junior's first idea was a stronger system prompt. Make the smallest change. Do not refactor. Do not add files.

The agent agreed in prose. Then it added src/lib/retryPolicy.js and re-exported it, because a helper would be cleaner. A prompt is a preference. It is not a gate.

Dead end 2: trusting the agent's summary

The second idea was to read the model's recap. The recap said it had updated retry.js only. git status showed three extra files and a lockfile bump.

Summaries are not diffs. The senior refused to review chat when git was available.

git diff --name-only main
# src/lib/retryPolicy.js
# src/webhooks/retry.js
# src/webhooks/index.js
# test/webhooks/retry.test.js
Enter fullscreen mode Exit fullscreen mode

The session had already failed its own goal. The extra files were reverted before anyone argued about taste.

git checkout -- src/lib/retryPolicy.js src/webhooks/index.js
Enter fullscreen mode Exit fullscreen mode

Dead end 3: a stale path list in CI

The junior then proposed a hard-coded allowlist in GitHub Actions. Allowed files: retry.js and its test. That list would rot the next time the bug lived in a different folder.

The senior rejected a repo-wide allowlist. The allowlist had to be born in the pairing session, next to the bug, and discarded with the branch.

The decision they kept

The pairing transcript is the allowlist. If git diff names a file that the transcript does not list under Files in play, the check fails. Dead ends stay in the transcript so the next session cannot rediscover the same queue library.

The rule is small. It is also mechanical. Mechanical rules survive tired reviewers.

Artifact: a blast-radius checker

The script below is a worked example. It reads pairing-transcript.md from the repo root, collects bullet paths under a ## Files in play heading, and compares them to git diff --name-only against main.

#!/usr/bin/env python3
"""Fail if the working tree touches files outside the pairing transcript."""
from __future__ import annotations

import subprocess
import sys
from pathlib import Path

TRANSCRIPT = Path("pairing-transcript.md")


def files_in_play(text: str) -> set[str]:
    wanted: set[str] = set()
    capture = False
    for raw in text.splitlines():
        line = raw.strip()
        lower = line.lower()
        if lower.startswith("## ") and "files in play" in lower:
            capture = True
            continue
        if capture and line.startswith("## "):
            break
        if capture and line.startswith("- "):
            path = line[2:].strip().strip("`")
            if path:
                wanted.add(path)
    return wanted


def changed_files(base: str = "main") -> set[str]:
    out = subprocess.check_output(
        ["git", "diff", "--name-only", base],
        text=True,
    )
    return {line.strip() for line in out.splitlines() if line.strip()}


def main() -> int:
    if not TRANSCRIPT.exists():
        print("missing pairing-transcript.md", file=sys.stderr)
        return 2
    allowed = files_in_play(TRANSCRIPT.read_text())
    allowed.add("pairing-transcript.md")
    if not allowed - {"pairing-transcript.md"}:
        print("Files in play section is empty", file=sys.stderr)
        return 2
    extra = sorted(changed_files() - allowed)
    unused = sorted(
        allowed - changed_files() - {"pairing-transcript.md"}
    )
    if extra:
        print("blast radius exceeded:")
        for path in extra:
            print(f"  extra: {path}")
        return 1
    print("blast radius ok")
    if unused:
        print("listed but unchanged:")
        for path in unused:
            print(f"  unused: {path}")
    return 0


if __name__ == "__main__":
    sys.exit(main())
Enter fullscreen mode Exit fullscreen mode

Run it before the pull request is opened:

python3 scripts/check_blast_radius.py
echo $?
Enter fullscreen mode Exit fullscreen mode

A failing run looks like this:

blast radius exceeded:
  extra: src/lib/retryPolicy.js
  extra: src/webhooks/index.js
Enter fullscreen mode Exit fullscreen mode

The unused-file warning is intentional. If the transcript lists a test that never changed, the pair probably forgot to prove the behavior.

Wire it to the branch, not a global policy

A local check is enough for a pairing session. Teams that want the same contract on the server can add a job that only cares when the transcript exists.

# .github/workflows/blast-radius.yml
name: pairing-blast-radius
on: pull_request
jobs:
  allowlist:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - name: Compare diff to pairing transcript
        run: python3 scripts/check_blast_radius.py
Enter fullscreen mode Exit fullscreen mode

The workflow does not try to understand architecture. It only enforces the contract the senior wrote down before the agent started.

A tiny wrapper keeps the command in muscle memory:

#!/usr/bin/env bash
set -euo pipefail
test -f pairing-transcript.md
python3 scripts/check_blast_radius.py
git diff --stat main
Enter fullscreen mode Exit fullscreen mode

Where a free model and a free server actually helped

The pair did not need a larger model. They needed a session both people could replay without copying source onto a personal laptop.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode's free model access and free server option were used as the remote place to run the agent against a redacted checkout. The transcript lived in the repo. The model stayed on the server. The blast-radius script stayed in CI. Remove the product name and the method still holds: put the agent where the checkout already is, and make the pairing notes the allowlist.

No model names, quotas, or hardware claims belong here. Those change. The gate does not.

Decision table from the session

Idea What happened Kept?
Stronger "be small" prompt Agent still added a helper file No
Trust the chat recap Recap omitted two paths No
Repo-wide allowed-path list Would rot on the next bug No
Transcript as per-branch allowlist Extra files failed before review Yes
Record rejected paths in the same file Next prompt could not rediscover a queue Yes

Limitations

This gate does not catch a bad edit inside an allowed file. A one-line retry change can still become a rewrite of retry.js if that file is listed. Teams that need shape control still need tests, review, or a stricter AST check.

The parser is deliberately dumb. It understands markdown bullets, not glob trees. If someone writes src/webhooks/**, the script will look for a file with that literal name and fail in a confusing way. List concrete paths.

The check assumes main is the merge base. Feature branches that diverge heavily should pass an explicit base ref.

Do not use this approach when:

  • The work is an intentional cross-cutting refactor
  • The repository cannot add a markdown contract file
  • Secrets still live in the checkout that the agent can read
  • The team will ignore a red CI job and merge anyway

A pairing transcript is not a substitute for ownership. It is a way to keep cheap generation from silently expanding the change.

What the senior refused to drop

The junior wanted to delete the dead-end section after the pull request merged. The senior kept it. Rejected paths are the cheapest form of memory a team has.

The next pairing session started from that list. The agent did not propose a queue. The diff stayed inside two files. The retry gained jitter, and nothing else.

Cheap models make extra architecture feel free. A transcript that fails the build is how this pair made extra architecture expensive again. If a team already has a free model endpoint and a server it can actually run, the smallest experiment is one bugfix branch with pairing-transcript.md and the script above.

Top comments (0)