The Pairing Session That Would Not Score a Patch in a Chat Window

The pairing session started with a failing webhook. A payment provider retried a POST the service had already acknowledged, and the agent in the IDE offered a forty-line helper that wrapped fetch in exponential backoff.

The mid-level engineer liked the shape of the helper. The senior did not look at the helper first. The senior looked at the room: a laptop with production .env files one directory up, a Slack thread with customer ids, and a public model tab that already held yesterday's stack trace.

That was the actual problem. Not the retry math.

The patch on the table

The agent had produced something that compiled in the editor. It also reached for process.env keys the webhook handler had never owned, imported a retry package the lockfile did not list, and logged the raw JSON body "for debugging."

The pair froze the diff in a scratch file before anyone pressed Apply. The file was not a merge candidate. It was evidence.

// proposed-retry.js — agent draft, not merged
import { retry } from 'async-retry'; // not in package-lock.json

export async function deliverWebhook(url, payload, secret) {
  return retry(async () => {
    const res = await fetch(url, {
      method: 'POST',
      headers: {
        'content-type': 'application/json',
        'x-signature': process.env.WEBHOOK_SIGNING_KEY, // new env, wrong owner
      },
      body: JSON.stringify(payload),
    });
    console.log('webhook payload', payload); // leaks PII into pairing logs
    if (!res.ok) throw new Error(`status ${res.status}`);
    return res.json();
  }, { retries: 5 });
}

The senior wrote three constraints on the whiteboard. No new dependency. No new environment key. No payload in logs. The mid-level engineer still wanted a second opinion from a model. The argument was not whether models can write retries. The argument was where that opinion would live.

Dead end one: the chat window

The first proposal was the fastest one. Paste deliverWebhook, the fixture, and the production error into a browser chat and ask whether the helper was safe.

The senior stopped the paste. The chat window had no retention contract the pair could name. The stack trace already contained a merchant id. The draft imported a package name that would have been enough for a copycat lockfile attack if the transcript leaked.

They kept a rule instead of a transcript. Pairing diffs that mention secrets, customer payloads, or lockfile gaps do not leave the desk through a consumer chat box. A model that cannot see the diff is inconvenient. A model that keeps the diff is a different incident.

Dead end two: the pairing laptop

The second proposal was to run a local coding model on the same machine as Slack, the password manager, and the checkout clone.

That failed for a quieter reason. The laptop was a conversation surface, not an eval host. Browser cookies, SSH agents, and .env files sit one ls away from a tool-calling loop. A pairing session that also hosts inference mixes two jobs that fail in opposite ways. Conversation wants history. Eval wants a machine the pair can shut down without losing the afternoon's mail.

They rejected laptop inference without benchmarking a single token. Ownership beat latency.

Dead end three: fix-until-green

The third proposal was to let the agent loop against the existing test file until the suite went green. The suite tested HTTP status codes. It did not test logging, lockfile churn, or new environment keys.

Green tests would have hidden the leak. The senior called that a false pairing success. A model that is "better at coding than most developers" in a public essay still cannot invent the invariant the suite forgot to name. The pair had to write that invariant by hand before any remote token moved.

Questions the senior put on the whiteboard

The senior did not run a ceremony. The senior asked five things and waited for answers that could be typed into a file.

1. Which exact failure must change after this helper lands.
2. Which existing callers must keep the same function signature.
3. Where the model will process the diff, and who can delete that host in five minutes.
4. Which files the model is forbidden to mention, including .env and the lockfile.
5. Who wrote the fixture — the agent or the pair.

The mid-level engineer answered the first two from the bug ticket. The third answer was empty. The fourth answer was a shrug. The fifth answer was "the agent," which the senior treated as a failed gate, not a style note.

They rewrote the fixture themselves. The model would see a redacted shape, not the production handler.

The artifact they kept: a pairing shape gate

The pair added a tiny eval kit next to the scratch diff. The kit is a proposal, not a production framework. It does not claim to prove correctness. It fails closed when the model's reply violates the whiteboard.

pairing-fixture.json names the human invariants:

{
  "functionName": "deliverWebhook",
  "mustKeepSignature": ["url", "payload", "secret"],
  "forbiddenImports": ["async-retry", "p-retry", "got"],
  "forbiddenIdentifiers": ["process.env", "WEBHOOK_SIGNING_KEY", "console.log"],
  "requiredBehavior": [
    "retry on non-2xx",
    "use the secret argument for signing, not a new env key",
    "do not log payload"
  ],
  "redactedContext": "Payment webhook delivery; payload is PII; lockfile is frozen for this sprint."
}

pairing-eval.mjs reads the fixture, the frozen draft, and an optional model completion. It scores the completion as text. No tool loop. No repository write.

// pairing-eval.mjs — pairing desk example, unexecuted in this article
import { readFileSync } from 'node:fs';

const fixture = JSON.parse(readFileSync('pairing-fixture.json', 'utf8'));
const draft = readFileSync(process.argv[2] || 'proposed-retry.js', 'utf8');

const failures = [];

for (const name of fixture.forbiddenImports) {
  if (draft.includes(name)) failures.push(`forbidden import: ${name}`);
}
for (const id of fixture.forbiddenIdentifiers) {
  if (draft.includes(id)) failures.push(`forbidden identifier: ${id}`);
}
for (const arg of fixture.mustKeepSignature) {
  if (!draft.includes(arg)) failures.push(`missing signature piece: ${arg}`);
}
if (!draft.includes(fixture.functionName)) {
  failures.push(`missing function ${fixture.functionName}`);
}

if (failures.length) {
  console.error('SHAPE GATE FAILED');
  for (const f of failures) console.error(`- ${f}`);
  process.exit(1);
}
console.log('SHAPE GATE PASSED');

A thin optional client can ask a model to rewrite the draft against the fixture. The pair labeled it optional because the session already had a human rewrite path. Environment variables stay outside the repo.

# commands the pair typed; keys never committed
export MODEL_BASE_URL="http://127.0.0.1:8080/v1"
export MODEL_API_KEY="$MODEL_API_KEY"
export MODEL_NAME="$MODEL_NAME"

node pairing-eval.mjs proposed-retry.js
# expected: SHAPE GATE FAILED with import, env, and console.log lines

The first run failed on all three whiteboard rules. That failure was the useful output. The agent draft did not survive pairing. A later human edit that retried with the existing secret argument, kept fetch, and logged only res.status passed the same script.

Decision table from the session

Option	Why it looked fast	Why the pair dropped it	What they kept instead
Public chat paste	Instant second opinion	No retention contract; PII already in the trace	Redacted fixture only
Inference on the pairing laptop	No extra host	Mixes Slack, SSH agent, and .env with a tool loop	Dedicated eval host the pair can power off
Agent loops until tests are green	Feels like progress	Suite never named logging or lockfile invariants	Human fixture plus a shape gate
Apply the forty-line draft	Compiles in the IDE	New dependency, new env, payload logs	Signature-preserving retry, no extra package

The decision that survived the dead ends was narrow. The pair owns the fixture. The model may propose text. The eval host is disposable. Production secrets never join the prompt.

Where the model was allowed to live

After the fixture existed, the pair still needed a place that was not the laptop and not a consumer chat tab. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access and free server option were relevant here only as a throwaway eval surface: a host the pair could image, run node pairing-eval.mjs on, and delete without touching the checkout clone.

They did not treat that host as a benchmark lab. They did not record tokens, hardware, or model names in the ticket. The ticket recorded the whiteboard rules and the path to pairing-fixture.json. If the free server vanished tomorrow, the fixture and the shape gate would still be the pairing output.

What they merged

The merged helper was shorter than the agent draft. It kept the original signature. It retried on non-2xx with a hard cap of three attempts. Signing used the secret argument. Logs used status codes only.

export async function deliverWebhook(url, payload, secret) {
  const body = JSON.stringify(payload);
  let lastError;
  for (let attempt = 1; attempt <= 3; attempt += 1) {
    const res = await fetch(url, {
      method: 'POST',
      headers: {
        'content-type': 'application/json',
        'x-signature': secret,
      },
      body,
    });
    if (res.ok) return res.json();
    lastError = new Error(`webhook status ${res.status} attempt ${attempt}`);
    await new Promise((r) => setTimeout(r, 200 * attempt));
  }
  throw lastError;
}

The shape gate passed. The original status-code tests still passed. The senior required one extra test the agent had not written: a spy that asserts console.log never received payload. That test was a pairing product, not a model product.

Limitations

The shape gate is a string filter. It will miss an obfuscated import, a dynamic env lookup, or a logger wrapped in a helper. It will also fail a valid patch that mentions a forbidden identifier in a comment. The pair accepted both failure modes because the gate sits in front of review, not in place of it.

The workflow assumes the pair can name invariants before prompting. Teams that cannot write pairing-fixture.json in ten minutes will only generate a second, vaguer agent loop. The free eval host does not reduce that work. It only keeps the work off the laptop and out of a chat window.

Time-sensitive claims about model ranking, "AI already better than most developers," or public leaderboard scores were left out of the ticket. The session did not measure those claims. It measured whether a specific draft violated a whiteboard the pair could still read after lunch.

Who should not use this

Skip the eval-host path when the diff is already public and contains no customer data. A README typo does not need a disposable server. Skip the shape gate when the change is a generated parser with no stable identifiers to forbid. Skip the whole pairing freeze when an on-call incident needs a one-line revert rather than a retry helper.

Staff engineers who already run isolated CI runners with secret scanning may find the kit redundant. The value showed up on a noisy pairing desk, not in a mature platform org.

The session ended with a boring merge. The interesting part never reached main. It stayed in the three dead ends, the five whiteboard questions, and the decision the pair refused to reopen: the fixture stays human, and the model never scores a patch from a chat window on the same laptop that holds production env files.