Why Your Coding Agent Agrees With Its Own Bugs: Building cold-jury, a Zero-Dep Cross-CLI Review Gate

Javier García Camacho — Mon, 09 Nov 2026 08:58:40 +0000

Applying Google's Elephant-Goldfish pattern and arXiv:2607.21656 to run sterile, cross-model hostile code reviews in Node.js

If you ask the same coding agent session that just wrote a 300-line change to review its own git diff before committing, it will almost always rubber-stamp it.

That is not a prompt engineering problem. It is a structural context problem: over the course of the conversation, the model has already rationalized every shortcut, extra abstraction, and edge-case assumption in its context window. Asking it to audit that same thread asks it to contradict its own recent reasoning.

Recent empirical benchmarks confirm how steep this self-bias is. In Cross-Model LLM Code Review (arXiv:2607.21656, evaluated across 116 tasks on LiveCodeBench):

Having Claude Sonnet 4 review its own drafts yielded +0.0 percentage points of improvement (91.4% -> 91.4%).
Having Claude Sonnet 4 review drafts written by OpenAI Codex jumped pass rates from 71.6% to 89.7% (+18.1 percentage points).
Even a weaker model (GPT-4o-mini, 69.0% baseline) reviewing Claude Sonnet 4 improved Claude's final score from 91.4% to 94.0% (+2.6 percentage points) because uncorrelated error distributions expose blind spots.

To automate that cross-examination without cloud lock-in or heavy wrappers, I built and open-sourced cold-jury (MIT).

What `cold-jury` Does

cold-jury is a zero-dependency Node.js (>=20.0.0) CLI and universal AAIF Skill (SKILL.md, agents/openai.yaml, .claude-plugin) that implements Google's Elephant-Goldfish Model (EGM):

Your coding agent is the Elephant: it holds full repository memory, user history, and task context.
cold-jury spawns three ephemeral Goldfish jurors in a different CLI (Writer != Juror) inside isolated temporary working directories (fs.mkdtempSync) with no conversation memory, no project dotfiles, and a stripped environment variable allowlist.

Each juror receives only two inputs:

A single-sentence scope specification (--spec).
The redacted unified git diff.

The 4-Stage Pipeline

1. Local Pre-Flight Secret Shield (Before Any Subprocess Spawns)

Sending a raw git diff or --spec string to an external CLI risks leaking live credentials that accidentally landed in the working tree. Before cold-jury spawns any child process, a local scanner inspects both the diff and the --spec text across 17 credential families:

Private keys (-----BEGIN ... PRIVATE KEY-----)
AWS (AKIA..., ASIA...), Stripe (sk_live_..., rk_live_...), Anthropic (sk-ant-...), OpenAI (sk-proj-...), GitHub (ghp_..., github_pat_...), Google (AIza...), Slack (xoxb-...), SendGrid (SG....)
.env secret assignments, Authorization: Bearer headers, JWTs, database URIs with embedded passwords (postgres://user:pass@...), and high-entropy secret assignments.

If a P0 secret is detected, cold-jury short-circuits immediately with exit code 1 (BLOCK) before a single byte leaves your machine. Lower-severity tokens are replaced in-memory with [REDACTED:<RULE_ID>].

2. Cross-Vendor Juror Selection (`Writer != Juror`)

cold-jury doctor discovers which CLIs are installed and authenticated on your machine across five supported engines:

claude (Anthropic Claude Code)
codex (OpenAI Codex CLI)
gemini (Google Gemini CLI)
opencode (OpenCode CLI)
ollama (Local Ollama models, verified via live HTTP /api/tags check so stopped daemons are marked OFFLINE)

When you run cold-jury review --author codex, the selector automatically picks the highest-leverage available non-author CLI (prioritizing claude <-> codex cross-family pairings).

3. Three Parallel Adversarial Probes in Sterile Rooms

Instead of asking one generic "please review this diff" prompt, cold-jury runs three narrow, hostile probes concurrently (Promise.allSettled):

appsec: CWE-mapped vulnerabilities (command/SQL injection, SSRF, XSS, path traversal, auth bypass, hardcoded secrets).
correctness: logic bugs, unhandled rejections, race conditions, off-by-one errors, null/undefined dereferences, broken edge cases.
surgical: YAGNI and scope discipline (ponytail-surgical-coding) — flags drive-by refactors, unrequested abstractions, and dead code outside --spec.

To prevent prompt injection from untrusted strings inside the diff itself, the diff is wrapped in per-run random nonce fences (<<<DIFF_8f3a...>>>), and every child process is spawned with shell: false (plus explicit cmd.exe /d /s /c metacharacter validation on Windows .cmd shims to mitigate CVE-2024-27980, and --end-of-options on git diff ranges to prevent CWE-88 argument injection).

4. Deterministic Gate (`PASS = 0`, `BLOCK = 1`)

Findings are deduplicated across probes and evaluated against your severity threshold (--fail-on P1 by default). Any P0 or P1 finding—or any failed/timed-out probe (fail-closed)—returns exit code 1.

Quickstart (Zero Install or Universal Skill)

You can run it directly from GitHub on any Node >=20 machine:

# 1. Check installed CLIs and cross-review matrix
npx --yes github:Dimoncelli/cold-jury doctor

# 2. Run a cross-CLI hostile review on your current git diff
npx --yes github:Dimoncelli/cold-jury review \
  --author claude \
  --spec "Validate JWT audience claim and reject expired refresh tokens"

# 3. Install a non-bypassable git pre-push hook
npx --yes github:Dimoncelli/cold-jury init-hook

Or install it as a native Agent Skill for Claude Code, Codex, Gemini CLI, Cursor, or OpenCode:

npx skills add Dimoncelli/cold-jury

The repository is MIT-licensed, ships with 0 runtime npm dependencies (24 files, 31.6 kB packed), and runs 164 automated tests across Linux, macOS, and Windows:

👉 GitHub Repository: https://github.com/Dimoncelli/cold-jury

I would love feedback from anyone running multi-agent coding workflows—especially on additional secret patterns, probe heuristics, or local Ollama model pairings you find effective for code review.

DEV Community: Javier García Camacho