This week the framework caught one of its own security checks doing absolutely nothing.
The private-key scanner had been reporting “No private keys detected” on every single run. It looked healthy. It was green in CI. And it was completely broken.
That’s the kind of quiet failure that only shows up when the review chain is willing to dig past the green checkmark. It became the headline of v4.1.1. But the story actually started a few days earlier with a more ordinary kind of drift.
agentic-framework is a Claude Code plugin that turns Claude into an orchestrated team: 21 specialist subagents, 10 slash commands, 9 skills, and 5 hooks — including a peer-review Stop gate that refuses to let a session end when committed work hasn’t been through review. The repo also validates itself. A consistency battery keeps agent frontmatter, the model registry, MCP tool grants, and hook wiring in agreement. When that agreement breaks, the battery is supposed to notice.
v4.1.0 — cleaning up the drift
Most of the week went into fixing configuration that had slowly diverged from what the validators actually enforced.
I canonicalized effort: across all 21 agents, added mcpServers: to 17 of them, wrote explicit tools: allowlists for the seven language experts, and moved eight agents (the six language experts plus security-specialist and spec-compliance-reviewer) from sonnet to opus. The model registry in claude.json was updated in the same commit so the model-parity check stayed green.
To keep this from happening again, I added validator check 15. It now confirms that effort is a known tier, that every mcpServers entry exists in the MCP plugin manifest, that every mcp__<server>__ tool in an allowlist has its server declared, and that any serena tool grant includes serena’s two required bootstrap tools. The rule exists because a real version of this defect had already shipped mid-branch and taken two follow-up commits to clean up. The check came after the pain.
The other half of the release was hardening migrate-legacy, the script that moves a legacy ~/.claude install onto the plugin pipeline. It had two quiet bugs: it could delete tracked hook files during its own run, dirty the checkout, and still exit 0; and it dead-ended for anyone whose tracked CLAUDE.md contained personal edits. I added a live dirty-guard filtered by a protected-paths list (CLAUDE.md included) and by paths the run itself touches, changed the exit code to 2 with the offending paths named, and wrote four test topologies covering tracked hooks, a modified CLAUDE.md that must survive byte-exact, genuine dirt that must abort, and unpushed commits that must also abort.
I ran the migration on my own machine as the final test. The legacy checkout is gone. Everything is now plugin-served. The peer-review Stop gate is live — a real reviewer run wrote the per-session verdict marker the gate actually reads.
PR #40 landed as twelve commits. The adversarial peer review took three rounds. It first found a false positive in the new check, then found that the fix for that false positive had opened a recall hole — a spelling of the tools list that let the original defect slip back through. It only approved after the tokenizer treated every single-line spelling the same way.
v4.1.1 — the check that never ran
The smaller release fixed three things and uncovered one that had been broken the entire time.
The stats table in the team-presentation doc became a fully generated block; every row, including the validator check count, is now read at runtime from the check registry. The hand-maintained assertion-count row that had drifted three times in a single branch was deleted. Check 15 received a real frontmatter parser after review discovered the first draft silently skipped validation on malformed shapes — it now fails loudly on a missing or unterminated fence. And the secret scanner stopped flagging quoted values that start with $ (variable expansions and command substitutions), after a variable named tool_tokens burned a full CI round.
Then came the private-key rule.
The scanner contained this line:
grep -r --exclude-dir=.git "-----BEGIN.*PRIVATE KEY-----" .
grep treats the leading dashes as an option cluster and exits with status 2. The fail-open redirect treated that exit code as “clean.” So the script printed “No private keys detected” on every run — including runs that contained a planted PEM sitting in the tree.
The security-specialist found it. The peer-review-critic then reproduced the failure at root cause before approving the fix. The change was simple: add -e, and exclude the scanner’s own source now that the pattern actually matches its own lines. We verified it red with a planted key and green on a clean directory.
PR #41 was five commits.
Process note
Both pull requests travelled the same path: code-review-gatekeeper, then security-specialist for a focused pass, then peer-review-critic doing multi-round adversarial review that verifies its own findings with probes, then all four CI jobs green (validate + tests on Ubuntu, the lint job, and the macOS and Windows hook legs).
This framework’s own Claude Code agents wrote and reviewed these changes, through the framework’s own gates. No claim beyond that.
Current measured state: 15 validator checks, 88 assertions in the consistency suite, 103 in the migration suite, 34 in plugin-manifests. All exit 0.
Repo: https://github.com/tomas-rampas/agentic-framework
text
Top comments (0)