DEV Community

Skillselion
Skillselion

Posted on Originally published at skillselion.com

The agent review loop that doubled runtime and changed nothing

obra removed the subagent review loop from the brainstorming skill in superpowers v5.0.6, dated 24 March 2026. The release notes report that dispatching a fresh agent to review plans added about 25 minutes per run and produced identical quality scores across five versions with five trials each. An inline self-review checklist replaced it.

Disclosure before anything else. We run Skillselion, an independent catalog of agent skills and MCP servers, and we are not affiliated with Anthropic, OpenAI, Cursor, Claude, Claude Code or Codex. This piece comes out of our deep dive on the Brainstorming skill listing, written on 20 August 2026 against v6.3.0 of the file. v6.4.1, whose release notes are dated 18 September 2026, has since rewritten the approval gate, so the deep dive still shows the older gate wording. Every quote in this article comes from the current file at commit 8ca22db.

What did the release notes actually say?

Most agent tooling gets louder over time. Another reviewer pass, another orchestration layer, another agent watching the first agent. Here is a maintainer going the other way and showing his working.

The subagent review loop (dispatching a fresh agent to review plans/specs) doubled execution time (~25 min overhead) without measurably improving plan quality. Regression testing across 5 versions with 5 trials each showed identical quality scores regardless of whether the review loop ran.

Source: RELEASE-NOTES.md, v5.0.6, read 27 September 2026.

Five versions, five trials each, one metric, one decision. That is a small experiment by any lab standard, and it is far more than almost any agent workflow gets before it ships. The fancier design lost.

What replaced the review loop?

A checklist the agent runs on its own output, in the same turn. The skill tells the agent to reread the spec it just wrote:

After writing the spec document, look at it with fresh eyes:

Source: skills/brainstorming/SKILL.md, line 247, verified 27 September 2026.

Four items follow: a placeholder scan for TBDs and vague requirements, an internal consistency check, a scope check, and an ambiguity check. Each item is a question the agent has to answer about its own text. The last two do the most work:

Is this focused enough for a single implementation plan, or does it need decomposition?

Could any requirement be interpreted two different ways? If so, pick one and make it explicit.

Source: skills/brainstorming/SKILL.md, lines 251 and 252, verified 27 September 2026.

Then comes the line that decides the cost of the whole thing:

Fix any issues inline. No need to re-review [...]

Source: skills/brainstorming/SKILL.md, line 254, verified 27 September 2026.

No second pass, no iteration cap, no convergence loop. The release notes give the payoff in the same units as the cost: "Self-review catches 3-5 real bugs per run in ~30s instead of ~25 min, with comparable defect rates to the subagent approach". The earlier design carried a three-iteration cap and a subagent dispatch. Both went.

Where does the brainstorming skill refuse to move?

The measurement deleted the review step. It did not touch the approval step, and the skill is emphatic about the difference. The gate sits near the top of the file, inside a HARD-GATE block:

Before taking any implementation action, including invoking an implementation skill, writing product code, scaffolding, installing product dependencies, or creating an external project, complete the selected path's prerequisites:

A reply approves the stage actually presented. Approval of an idea or feature scope does not approve artifacts that do not exist yet.

Source: skills/brainstorming/SKILL.md, hard gate, read 27 September 2026.

Read those two decisions together and a rule falls out. The part that was measured and found inert got cut. The part that protects a human decision stayed, and stayed at full strength on every path. The gate has moved further in that direction since our deep dive was written. v6.4.1, released 18 September 2026, splits approval into stages on the architectural path, where the written spec and then the implementation plan each get a human review.

Why is the path ratchet one-way?

The skill sorts a request into one of three paths before the first question: Spike for a feasibility answer you will throw away, Bounded for a change to a flow that already exists in the repo, Architectural for new subsystems and interface changes. Classification is not a private judgment, and the agent has to say it out loud so a human can override it.

Bounded has a test rather than a vibe. "If there is no existing flow to change, the task is not bounded." That single sentence kills the most common failure, where an agent decides a greenfield feature is small because it can picture the shape of it.

Movement between paths runs in one direction only:

When in doubt between two paths, take the heavier one. The ratchet is one-way: hidden complexity discovered mid-task upgrades the path [...] Nothing downgrades mid-task.

Source: skills/brainstorming/SKILL.md, path selection, read 27 September 2026.

An agent that can talk itself down a tier will do it, because the lighter tier is where the work is allowed to start. Removing the downgrade removes the incentive.

Is a small change exempt from approval?

No, and the skill gives that exemption its own section titled as an anti-pattern.

Every path ends with your human partner approving the required design before implementation. A bounded change may need only two sentences in chat. A new todo-list project is architectural and requires the written spec and planning handoffs.

Source: skills/brainstorming/SKILL.md, lines 92 to 95, verified 27 September 2026.

Two sentences in chat is a real approval, and the gate applies to it in full. If you have ever watched an agent decide that a config change was too trivial to mention and then rewrite three files, you know which half of that sentence is doing the work.

Does this contradict subagent-driven-development?

Not really. The same pack ships the Subagent-Driven Development skill, which holds 197,517 installs in our catalog as of 7 September 2026, and v6.4.1 rebuilt executing-plans so that it ends with one fresh whole-branch review. The maintainer has not sworn off subagents. That skill is about executing a plan task by task. What v5.0.6 removed was a subagent reading a plan or a spec and grading it before implementation started. Both use subagents, for different jobs.

The claim worth stealing is narrow: on this codebase, on these five versions, a review subagent did not beat a 30 second self-check on plan quality. Your workflow might measure differently. The point is that it got measured at all.

What should you copy from this skill?

Three things survive the trip out of this repo.

  • Put the gate near the top of the file, above the path definitions. The same rule in a references section at the bottom reads as a suggestion.
  • Give the classification a falsifiable test. "If there is no existing flow to change, the task is not bounded" beats any adjective.
  • Time your review step before you defend it. Twenty five minutes of overhead is easy to mistake for rigour when nobody has checked what it buys.

If you want to compare gate designs across skills, the same pack's Systematic Debugging skill listing and its Test Driven Development skill both refuse to let the agent proceed on different evidence, and all of them sit together on the obra/superpowers pack page. For the wider field we keep a ranked shortlist of Testing and Review skills and the Dev Tools skills category.

FAQ

Is the brainstorming skill official Anthropic software?

No. It ships in obra/superpowers, a community MIT-licensed repository, which GitHub reported at 292,010 stars and 26,133 forks on 27 September 2026. Skillselion is an independent catalog and has no affiliation with Anthropic.

How popular is the skill compared with the rest of the pack?

Our catalog records 353,451 installs for brainstorming as of 7 September 2026. Inside the same pack it leads systematic-debugging at 250,011 and the Writing Plans skill at 241,042. Source: Skillselion catalog listings, data as of 7 September 2026.

Does deleting the review loop mean plan review is a waste of time?

It means one specific implementation of it was. The replacement is still a review, run inline by the same agent, with a four item checklist and no re-review pass. What the measurement removed was the second agent and the 25 minutes, not the act of checking.

Where do I read the whole file?

The full section by section reading is on the Brainstorming skill page under the deep dive heading. That reading predates v6.4.1, so use the GitHub links above for the current gate wording. The rest of the series is indexed at our skill and MCP deep dives. How long skills like this stay maintained is the subject of the Agent Skill Maintenance Census.

Last updated: 27 September 2026. Every quote above was checked byte for byte against obra/superpowers at commit 8ca22db on that date. The most recent commit to touch skills/brainstorming is the v6.4.1 release commit 5bf4e78 (committed 19 September 2026 UTC; the release notes date v6.4.1 to 18 September 2026). Install figures are Skillselion catalog values, dated where they appear.

Top comments (0)