The claim worth verifying
old-coder (MIT, by AmazingAng) is a markdown skill for coding agents — Claude Code, Codex CLI, Cursor, Aider, or custom agent loops. Its strategy: don't read the code your agent wrote; make it run the gauntlet. Concretely, the human approves a test plan (SPEC) before any code exists and reviews an EVIDENCE report afterward, instead of reading the diff. The README cites Robert C. Martin: "My current strategy is to not read any of the code written by my agents."
The workflow
SPEC → RED → GREEN → REFACTOR → GAUNTLET → EVIDENCE. Each scenario maps 1:1 to at least one automated test. All numbers in the evidence report come from one final fresh run, the entry command is recorded so a human can rerun everything, and source state is pinned via commit SHA or tree hash (the demo pins commit d6e17b1, tree hash 50433e0a4acc8507).
The 9-layer gauntlet (reusable checklist)
From skills/old-coder/references/gauntlet.md, layers run in order and halt at the first failure: tests → types → lint → changed-line coverage → mutation → property-based tests → real execution → supply chain/secrets scan → suite health (randomized order). Per-ecosystem tool tables exist for Python, JS/TS, Go, Rust, Java, Scala, SQL, and Emacs Lisp, with the rule "prefer whatever the project already uses" and pinned tool versions for reproducible reruns. Extended layers are selected by risk: concurrency only when the failure model names races, performance only when the spec states a budget, UI checks only when the change touches user-facing UI, version matrix only when the project claims multi-version support. A typo fix gets a couple of checks; changes touching money, logins, data, or concurrency get everything plus hostile-input self-attacks.
The honesty rules (the actual differentiator)
- Never weaken a test to make it pass.
- Never report a check that did not run — unverified never equals pass.
- Fail-closed gates:
set -eat the top, no|| true, no2>/dev/null; a must-find-nothing grep passes only on rc 1. - Prove each home-grown check can fail with a one-off negative control.
- Every mutant must make at least one test fail — a survivor means a weak or vacuous assertion.
- Record the dependency diff and the reasons, item by item.
These rules make "all green" auditable instead of self-attested. That is the part ordinary CI pipelines do not give you.
What the demo evidence file admits
The rate-limiter demo (demo-rate-limiter/evidence.md) reports 17/17 tests passing (also in randomized order), 29/29 statements and 10/10 branches covered, and 8/8 mutation kills — and then openly states that the property tests alone only killed 3/8, that one kill (M2) was flaky until spec revision 2 pinned the exact-boundary behavior with a deterministic test, and that the spec was never approved by a human (autonomous run). It also documents a real bug caught by the loop: a window_seconds=NaN value slipped past the original <= 0 validation and was fixed via a spec revision, a watched RED test, and a finiteness check (later killed as mutant M7). Known uncovered modes are listed: not thread-safe; a NaN-returning clock fails closed but is not rejected.
Limits
The gauntlet proves the code meets the spec — it cannot prove the spec covers everything that matters. The skill presumes a test base exists; mutation testing is a real time cost, which the skill itself acknowledges by scaling effort to risk. Treat the demo's numbers as the author's report, not independent results — rerun ./tools/gauntlet.sh to verify.
Not tested/not run by me: I did not execute the skill or any of its scripts; everything above was read from the repository's own files (README, gauntlet.md, evidence.md).
Repo (MIT): https://github.com/AmazingAng/old-coder · Gauntlet reference: https://github.com/AmazingAng/old-coder/blob/main/skills/old-coder/references/gauntlet.md · Demo evidence: https://github.com/AmazingAng/old-coder/blob/main/demo-rate-limiter/evidence.md
Top comments (1)
I like this framing because reading generated code is not the same as trusting it. A gauntlet turns review into evidence: tests, type checks, lint, security checks, fixtures, and runtime behavior. Then the human can spend attention on design judgment instead of line-by-line suspicion.