The patch looked harmless. Forty lines, a retry helper, tests green on the laptop. Then the senior stopped on a single identifier the repository had never declared: BILLING_QUEUE_URL. The agent had invented a queue the service did not own.
This walkthrough reconstructs that pairing session. It is an illustrative pairing, not a postmortem from a named production team. The failure mode is ordinary in current pairing rooms: a coding agent fills every gap with a plausible name, and the diff still compiles.
The scene
A mid-level engineer had asked a coding agent to add backoff around a webhook sender. The stack was a small Node service plus a Python script the team already used for lint-on-patch. The agent produced a tidy WebhookSender.ts, imported amqplib, and read four environment variables. Only one of those variables existed in the deployment chart.
The senior did not start with a prompt tweak. The senior started with a freeze. The rest of the hour was spent defending that choice against three polite dead ends.
Questions the senior asked out loud
The mid-level engineer wanted to tell the model not to invent things. The senior treated that as a wish, not a control. These were the questions that actually ran the session. They were spoken at the keyboard and copied into the pairing doc:
- Which names are already true in production, and who owns them?
- Which names is this change allowed to introduce, if any?
- Which values must stay test doubles and never appear in the agent runtime?
- What will we grep tomorrow to prove the patch did not grow a shadow environment?
None of those prompts were for the model. They were for the humans who were about to keep or reject the patch.
Dead end 1: a longer system prompt
They tried a system prompt that forbade invented environment variables, packages, and hostnames. The next patch renamed BILLING_QUEUE_URL to BILLING_AMQP_URL and added REDIS_URL because a comment mentioned cache. The model had complied with the vibe and violated the world.
Prompt text is not a lockfile. The senior said that once, then stopped editing the preamble.
Dead end 2: review the diff like a normal PR
They read the TypeScript. The retry math was fine. The undeclared queue was easy to catch once someone hunted names. The sneaky part was a new import:
import * as amqp from "amqplib";
package.json did not list amqplib. The unit test mocked the module, so npm test stayed green. Diff review that hunts logic misses dependency gravity. The senior called that a pairing failure, not a model failure.
Dead end 3: ask the agent to list assumptions in chat
The agent produced a tidy bullet list in the transcript. Ten minutes later the patch used process.env.WEBHOOK_TIMEOUT_MS while the list still said timeouts stay hardcoded. Chat drifted. The transcript was not the artifact. The senior refused to treat conversation as source of truth.
The decision they kept
They kept one rule for the rest of the session. No agent patch until an assumption ledger is committed. No merge if the patch mentions a name the ledger does not grant.
The ledger was a small YAML file, checked in next to the service, boring on purpose.
# assumptions.lock.yml
# Illustrative pairing artifact. Not exported from a live system.
service: webhook-sender
allow_new_env: false
env:
WEBHOOK_ENDPOINT:
owner: platform-config
in_production: true
test_double: "https://example.test/webhook"
WEBHOOK_SIGNING_KEY:
owner: secrets-manager
in_production: true
test_double: "test-signing-key"
packages:
allow_add: []
existing:
- undici
hosts:
allow:
- example.test
forbidden_prefixes:
- BILLING_
- REDIS_
- AWS_
notes:
- Queue transport is out of scope for this change.
- Timeouts stay as numeric literals in code.
The pairing moved in this order. They did not skip steps.
- A human writes or updates
assumptions.lock.yml. - A human seeds a disposable runtime with only the test doubles.
- The agent may edit code under that freeze.
- A checker runs against the patch, not against the chat.
- The senior reads the checker output before reading the implementation.
That order was the decision. Everything else was technique.
Decision table from the session
| What showed up | Response they abandoned | Response they kept |
|---|---|---|
| Invented env name | "Do not invent names" in the system prompt | Declare env in YAML; fail the patch on unknowns |
| New runtime import | Trust green unit tests |
packages.existing is the only import grant |
| Assumptions in chat | Re-ask the model to summarize | YAML is the only source of truth |
| Real secrets file on the laptop | Hope the agent does not read it | Agent runs on a scratch machine seeded from test doubles |
Artifact: a patch checker for the freeze
The following Python is an example checker. It is meant to be run on a unified diff, not on the whole tree. It looks for process.env members, os.environ keys, import lines, and obvious host strings. It is deliberately shallow. Shallow was the point. The senior wanted a gate that failed loud on invented names, not a static analyzer that needed a week.
#!/usr/bin/env python3
"""assumption_freeze.py — example pairing checker, not a security scanner."""
from __future__ import annotations
import argparse
import re
import sys
from pathlib import Path
try:
import yaml
except ImportError:
print("Install pyyaml before running this example.", file=sys.stderr)
sys.exit(2)
ENV_PATTERNS = [
re.compile(r"process\.env\.([A-Z][A-Z0-9_]+)"),
re.compile(r"process\.env\[['"]([A-Z][A-Z0-9_]+)['"]\]"),
re.compile(r"os\.environ\[['"]([A-Z][A-Z0-9_]+)['"]\]"),
re.compile(r"getenv\(['"]([A-Z][A-Z0-9_]+)['"]\)"),
]
IMPORT_RE = re.compile(
r"^(?:import\s+(?:.+?\s+from\s+)?['"]([^'"]+)['"]|from\s+['"]([^'"]+)['"])",
re.M,
)
HOST_RE = re.compile(r"https?://([a-z0-9.-]+)", re.I)
def load_lock(path: Path) -> dict:
data = yaml.safe_load(path.read_text()) or {}
if "env" not in data or "packages" not in data:
raise SystemExit("assumptions.lock.yml needs env and packages keys")
return data
def read_patch(path: Path) -> str:
added = []
for line in path.read_text(errors="replace").splitlines():
if line.startswith("+") and not line.startswith("+++"):
added.append(line[1:])
return "\n".join(added)
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--lock", default="assumptions.lock.yml")
parser.add_argument("--patch", required=True, help="unified diff to inspect")
args = parser.parse_args()
lock = load_lock(Path(args.lock))
added = read_patch(Path(args.patch))
allowed_env = set(lock.get("env", {}))
forbidden = tuple(lock.get("forbidden_prefixes") or [])
allow_new_env = bool(lock.get("allow_new_env"))
allowed_pkgs = set(lock.get("packages", {}).get("existing") or [])
allow_add = set(lock.get("packages", {}).get("allow_add") or [])
allowed_hosts = set(lock.get("hosts", {}).get("allow") or [])
failures: list[str] = []
used_env = []
for pat in ENV_PATTERNS:
used_env.extend(pat.findall(added))
for name in sorted(set(used_env)):
if name.startswith(forbidden):
failures.append(f"forbidden env prefix: {name}")
elif name not in allowed_env and not allow_new_env:
failures.append(f"undeclared env: {name}")
for match in IMPORT_RE.finditer(added):
pkg = match.group(1) or match.group(2)
if not pkg or pkg.startswith("."):
continue
root = pkg.split("/")[0]
if root.startswith("node:"):
continue
if root not in allowed_pkgs and root not in allow_add:
failures.append(f"undeclared import: {pkg}")
for host in sorted(set(HOST_RE.findall(added))):
if host not in allowed_hosts:
failures.append(f"undeclared host: {host}")
if failures:
print("assumption freeze failed:")
for item in failures:
print(f" - {item}")
return 1
print("assumption freeze ok")
return 0
if __name__ == "__main__":
raise SystemExit(main())
Commands they kept on the shared notes:
git diff --cached > /tmp/pair.patch
python3 assumption_freeze.py --lock assumptions.lock.yml --patch /tmp/pair.patch
A failing run looked like this:
assumption freeze failed:
- undeclared env: BILLING_QUEUE_URL
- undeclared import: amqplib
When the checker failed, they did not ask the agent to try again. They updated the ledger or they deleted the line. That binary choice killed a long class of polite, wrong patches.
Seed the scratch runtime from the same file. The pairing notes used a tiny loader so the agent never received a human's real environment.
# seed_env.py — example only
from pathlib import Path
import os
import yaml
lock = yaml.safe_load(Path("assumptions.lock.yml").read_text())
for name, meta in (lock.get("env") or {}).items():
os.environ[name] = str(meta.get("test_double", ""))
print("seeded", sorted(k for k in lock.get("env", {})))
Where a free model and a free server sat in the method
The agent still needed a place to run. The senior refused to let it execute on the laptop that held the real secrets file. A disposable remote, seeded with only the test doubles, was the runtime. A cheap model pass did one job: draft a first assumptions.lock.yml from the pairing notes. A human edited that draft before any code patch.
Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access and free server option map onto that split — draft the ledger with a free model, run the agent on a free scratch server — and the same freeze works on any spare VM. Teams that already want the agent half of pairing off the laptop can try that split there.
What the checker will not catch
The script will not catch semantic lies, such as declaring WEBHOOK_ENDPOINT and then sending the wrong business payload. It will not catch dynamic lookups like process.env[key] where key is computed. It will not catch require("amqplib") or transitive dependencies pulled by a package the ledger already allows.
It will not strip secrets from prompts. It will not see generated files a diff tool ignores. The senior called the checker a seatbelt, not a driver. Name freeze is not proof of correctness.
Who should not use this
- Teams that want an unsupervised agent merging to main. The ledger assumes a human writes the freeze.
- Changes that must introduce many new env keys in one sitting. Turning on
allow_new_envdeletes the value of the ritual. - Regulated pipelines that need a reviewed SAST tool. This script is an example, not an audit.
- Pairing sessions shorter than the time to write YAML. A one-line typo fix does not deserve a ceremony.
What survived after they closed the laptop
They kept the file, the checker, and the order. They threw away the longer system prompt, the faith in green unit tests, and the chat bullet list.
The agent still wrote most of the TypeScript. The senior still rejected the queue. The mid-level engineer left with a cheaper lesson than a production incident. Plausible names are not dependencies. Freeze the world first. Then let the model type.
Top comments (0)