DEV Community: AI Dive

Your agent commits every spec. Nobody owns them.

AI Dive — Mon, 14 Sep 2026 16:57:11 +0000

In June, the maintainers of okama, a Python finance library, found their internal implementation plans published on Read the Docs. Eight markdown files an agent had written, committed exactly as the plugin intends, rendered as public web pages. They noticed after the release.

The plugin was Superpowers. By default every spec and every plan it writes lands in docs/superpowers/, and the brainstorming skill commits the design doc before you've even seen the plan.

So who owns those files, and when is one dead?

Every tool commits the files

This isn't a Superpowers quirk. spec-kit keeps one numbered folder per feature on that feature's branch. Kiro keeps .kiro/specs/, one folder per feature. Anthropic's playbook goes further and commits an artifact at every stage: intent, spec, plan, diff, review findings, incident record.

Whether to commit is settled. Ownership and lifecycle aren't.

Picture a front-end team of 10 people in four squads sharing one repo, with every developer generating these files. Three failures are already documented:

A docs generator renders every markdown file under docs/, listed in the table of contents or not. That's the okama leak.
Issue #1246: the brainstorm commits the plan straight to main. One user counts 10 to 15 commits per session.
Drift, the quiet one. Birgitta Böckeler at Thoughtworks separates spec-first (written, used once) from spec-anchored (maintained for the feature's life). Superpowers writes spec-first documents and keeps them forever, so you get anchored storage with first-draft maintenance. Nobody updates the file, and an agent reads it next quarter as truth.

Rule 1: the folder has an owner, and it's a squad

Since v5, Superpowers honors the instructions in your CLAUDE.md over its own defaults. A table of paths isn't enough though. In issue #939 a user wrote an output paths table and the model followed the concrete path in the skill instead. One bold imperative line, naming the default as the thing to avoid, fixed it on the first attempt:

IMPORTANT: specs MUST be saved under docs/superpowers/<squad>/specs/
and plans under docs/superpowers/<squad>/plans/, NOT docs/superpowers/specs/.

In a monorepo, put that line in the package's own CLAUDE.md. Claude Code loads a subdirectory's file when it reads there. Then let the platform enforce the owner:

# CODEOWNERS
docs/superpowers/checkout/        @org/squad-checkout
docs/superpowers/search/          @org/squad-search
docs/superpowers/design-system/   @org/squad-design-system

Every spec review now lands with the people who'll live with it. One caveat: the override is a prompt, not a setting. Check the first spec after every plugin release.

Rule 2: not every plan is a team artifact

Personal redirects go in CLAUDE.local.md, which loads after the shared file and which you git-ignore yourself. For the folder itself, .git/info/exclude ignores it per clone without touching the shared tree.

The shared side is whatever passed the review gate the brainstorm already runs. Approved by a second person, it moves into the squad folder. Not approved, it stays local. Both are written on a branch, never on main. Two lines in the shared CLAUDE.md do it:

Treat specs and plans as the start of a feature.
Create the worktree before writing them.

There's a limit. The okama maintainers tried ignoring the folder first and reverted, because plans stopped syncing between machines and agents. Personal means personal.

Rule 3: a spec has a status, and a dead spec says so

Michael Nygard wrote this down for architecture decision records in 2011. A decision is proposed, then accepted, and when a later record changes it the old one is marked superseded instead of deleted.

So every shared spec opens with a short header, plus a review date if your team uses one:

---
status: accepted
superseded-by:
owner: @org/squad-checkout
touches: src/checkout/retry.ts, src/checkout/api.ts
---

Never edit an accepted spec into a different decision. Write the next one and point it back. The one exception is in Anthropic's playbook: when implementation departs from the plan, update the plan in the same commit.

The weak spot is that a status is metadata a human sets. The agent won't mark its own spec superseded unless your CLAUDE.md tells it to.

Rule 4: a spec that lies about the tree fails the build

For every spec whose header says accepted, pull the paths it names (in backticks or on the touches line) and check that each one exists. Proposed and superseded specs are skipped, because only accepted specs make claims about the tree.

#!/usr/bin/env bash
# drift gate: accepted specs may only name paths that exist
shopt -s globstar
fail=0
for f in docs/superpowers/**/specs/*.md; do
  grep -q '^status: accepted' "$f" || continue
  paths=$( { grep -oE '`[^` ]+/[^` ]+`' "$f" | tr -d '`'
             grep -m1 '^touches:' "$f" | sed 's/^touches://' | tr ',' '\n'; } | tr -d ' ')
  for p in $paths; do
    if [ ! -e "$p" ]; then
      echo "drift: $f names $p, which no longer exists"
      fail=1
    fi
  done
done
exit $fail

People already do the manual version. One commenter on r/ClaudeCode has Claude compare the spec files to the codebase and file tickets for what's missing. This does that on every pull request.

While you're in CI, add the guard okama shipped: exclude the folder from the docs build. In Sphinx that's one entry in exclude_patterns, and the files stay in git without reaching the HTML.

The check catches deleted files, not changed behavior. A spec can name files that all exist and still describe an API that's gone. That part is still review's job.

Rule 5: most work doesn't deserve a spec

Marmelab ran spec-kit on a feature that shows the current date and got 8 files and 1,300 lines of specification. The OpenSpec bake-off ran the same requirements twice, with the spec tool and with Claude Code alone. The spec run produced 50% more code with 50% more cyclomatic complexity, took twice as long and cost three times as much. It's one experiment, and the author says so. It also found three gaps the plain run missed.

A team that ran spec-driven development for months couldn't prove it improved the code: two to three times the tokens, about twice as long.

My rule: write a spec when the work crosses a squad boundary or will be read again in 90 days, and keep it near 300 lines. Everything else is a prompt. The spec buys coverage, not speed.

What this doesn't fix

None of these rules is enforced by a tool today. The override issue is open, the main branch issue is open, and the pull request that reordered the instruction was closed without merging. And if your team won't review a code owner request, it won't review a spec either. In that case git-ignoring the folder is the honest choice.

The full walkthrough

The video puts the issues, the ADR source and the okama config on screen. The written version with every source is at https://aidive.dev/videos/spec-plan-governance/.

Do you commit your agent's specs, and has a stale one misled an agent yet?

I used AI tools to help edit this piece.

Everything Claude Code costs 27,000 tokens before you type. I kept five pieces.

AI Dive — Tue, 08 Sep 2026 17:52:58 +0000

Someone asked on r/ClaudeCode whether everything-claude-code is really that good. The thread got one reply saying the token cost of skill descriptions is "minimal". Nobody had counted.

So I cloned it, version 2.2.1, and counted. It has 250,000 stars, 68 agents, 286 skills, 94 slash commands, 122 rule files and 23 hooks. The README that describes all of it is 2,200 lines long.

The question I wanted answered: what does a full install cost you before you type a word, and which parts are worth taking?

The part the repo tours skip

A skill's body only loads when the skill fires. Its name and description do not. They sit in your context from the first token of every session, and the same is true of every subagent.

That metadata is the price of admission, so I measured it on the fresh clone:

286 skill names + descriptions   85,000 chars   ~21,000 tokens
68 subagent descriptions         20,000 chars    ~5,000 tokens
the repo's own instruction file                  ~1,000 tokens
------------------------------------------------------------
standing context, full install                  ~27,000 tokens

Against a 200,000 token window that is 14% gone every session. It sounds survivable until you remember what shares that window: your files, your diffs, your tool results and the conversation itself. Every skill you will never call competes with the code you are working on.

The repo's own README says "optimize the context window, persist everything else". Then it ships 286 skills.

Two caveats before anyone quotes this. Tokens are estimated at four characters each, so the real number moves with the tokenizer. And none of this says the skills are bad. It says they are not free, and most of them are for stacks you don't write in.

What I kept

Five things. The first two don't need the repo installed at all.

1. AgentShield

A security auditor for agent configurations. It ships inside ECC and also lives on npm as its own package, ecc-agentshield (5,500 downloads last week).

npx ecc-agentshield scan --path .

It audits your agent files, hooks, MCP servers, permissions and secrets, with flags for output format, severity floor and safe auto-fixes. Its README justifies itself with numbers from January: 12% of one skill marketplace was malicious, 341 skills out of 2,800. Those are the package's claims, not mine.

The catch is a date. AgentShield was last published in March. The repo it ships with was updated yesterday, and the scanner's README still describes the ecosystem at 42,000 stars. Six months behind the thing it audits.

2. The settings block

The README publishes four changes to .claude/settings.json: default model to Sonnet, thinking budget from 32,000 down to 10,000, compaction threshold from 95% down to 50%, subagents on the cheapest model. It claims about 60% off model cost and 70% off hidden thinking cost. Its figures, not mine, and they are a trade. A smaller default model will cost you on the hard reviews.

3. Instincts

This is the one idea in the repo I have not seen anywhere else. An instinct is a small learned behaviour with a confidence score between 0.3 and 0.9. Not a rule you wrote. A pattern the system noticed you repeating and wrote down for you.

It watches before and after every tool call, runs the analysis in a background agent on the cheap model, and scopes instincts per project so your React habits stay in the React project. When the same instinct shows up in two projects it gets promoted to global. The system decides what is universal by watching instead of asking.

The tool behind it is 2,200 lines with six commands, and instinct libraries are exportable. Hand yours to a teammate and they import it, which is the first sane onboarding story I've seen for an agent setup.

String attached: all of it rides on the hook runtime, which is the exact part of the repo you are asked to opt into on purpose. No hooks, no instincts.

4. The three context skills

A context budget, a compaction adviser, a token budget adviser. The repo that costs you 14% also ships the tools to see it. The compaction rule alone is worth copying: compact after research and before you implement, after a milestone, after debugging, after an approach fails, and never in the middle of an implementation where you lose the file paths and the half-finished state.

5. The installer, used one skill at a time

Seven profiles, and the smallest says in its own description that it leaves the hook runtime out. You can skip profiles entirely and name skills one by one, and an adviser takes a plain sentence about your work and returns matching components with a preview before anything is written. Any install that would put hooks on your machine stops and waits for a yes.

Install that way and the bill stops being 14%. I'm not giving a figure for the fraction because I did not measure every combination. Even the smallest profile still carries the agents and the rules, so naming skills by hand is the only way to pay for exactly what you use.

What stays in the repo

The 68 agents as a set: about 5,000 tokens of standing context, mostly language reviewers and build fixers for C++, C#, Django and Kotlin. Take the two that match your work.

The 94 slash commands: the README itself calls them temporary entry points while the repo moves to skills, and commands are the easiest thing here to write yourself.

The hooks file: 23 hooks, 291 lines, 40 KB, every hook squeezed onto a single line, the longest at 2,000 characters. The descriptions beside them are readable and some are good (one blocks the agent from editing your linter config and says why). The description is the only part you can read.

The ballast: config for 19 harnesses and 15 MB of docs in 22 languages, for a tool you run in one language, in one place.

What this doesn't measure

Whether the skills make the answers better. I measured what they cost you to have. Those are different questions. Also, version 2.2.1 had 110 open pull requests the day I cloned it, so the counts above are already old.

Takeaways

A full install of ECC parks about 27,000 tokens in context before you type, 14% of a 200k window.
Skill bodies are lazy. Skill metadata is not.
The licence is MIT. Copying five directories out is the intended use, not a workaround.
AgentShield is worth running once, and worth checking the publish date of.
Instincts are the only new idea; they need the hook runtime.

Full write-up with the tables and chapter timestamps: https://aidive.dev/videos/ecc-what-to-steal/

Are you running the full install, a profile, or hand-picked skills? And what does your context sit at when a session starts?

I use AI tools to help edit my writing. The counts, the install and the opinions are mine.

JetBrains says rtk costs more. My dashboard says 11.6M saved. Both are right.

AI Dive — Sun, 06 Sep 2026 15:33:37 +0000

JetBrains spent about $320 of API credit in July to test rtk, the shell proxy tens of thousands of us installed to save tokens in Claude Code. Their sessions came out 7.6% more expensive per task. Two weeks earlier the same team measured caveman, the skill that advertises a 65% cut, and got 8.5%.

I read both posts with rtk gain open in another tab. It said 11.6 million tokens saved over 25,599 commands.

So who is wrong? Nobody. The two numbers count different things, and once you see which slice of the bill each tool touches, the four tools I run sort themselves into a ranking. That ranking is the useful part, so here it is.

What I actually tested

Four tools, all free, 575,612 GitHub stars between them on September 2: graphify, rtk, Superpowers and caveman. For each one I took the number it advertises, the number JetBrains measured in their paired benchmarks (claude-sonnet-5, 82 to 86 agentic tasks, run twice), and what my own machine shows after months of daily Claude Code use.

The frame that made it click comes from rtk's own docs. A Claude Code bill has two sides. Input tokens are everything the model reads: shell output, your prompt, the system prompt, and the whole conversation replayed on every call. Output tokens are everything it writes. JetBrains replayed 83 sessions and split the reading side:

1.9M characters of tool output, by origin
  19.7%  shell output rtk can compress
  46.3%  shell output rtk has no rule for
  34.0%  Read / Grep / Glob tools that never touch the shell

Only a fifth of what the model reads can pass through a shell proxy at all. Keep that table in mind for the rest.

rtk shrinks shell bytes, and shell bytes are not the bill

rtk sits between Claude Code and your shell. A PreToolUse hook rewrites git status into rtk git status, and the compact output comes back instead of the noisy one. My dashboard after 25,599 commands:

rtk find      354 calls   2.2M saved   46.6%
rtk read    3,504 calls   2.2M saved   10.4%
rtk grep    2,760 calls   1.9M saved   47.7%
rtk ps aux     24 calls   1.1M saved   98.0%
total      25,599 calls  11.6M saved   41.6%

That 11.6M is bytes divided by four, counted on the commands that had a rule. JetBrains found a rule for 349 of 1,056 shell commands, one in three, and combined with the table above that puts the ceiling around 3% of a session's cost. Their measured result was +7.6% per task at low reasoning effort (p=0.004), with 13.8% more turns, and no difference at high effort. Quality unchanged.

The rtk README now says it in one line: up to 90% of the bash output, which is not the same as cutting your bill by 90%. I still run it. One-line test output is nicer to read and it costs nothing. I just stopped expecting it to move the invoice.

caveman trims the smallest slice

caveman makes the agent talk like a caveman: drop articles, filler, pleasantries. Code, paths and error messages are never touched. Its own table shows 1,214 output tokens per answer without and 294 with, the 65% figure.

JetBrains ran it on 82 paired agentic tasks and got 8.5% of output tokens, 592k down to 542k, with no detectable quality change (sign test p=0.82). The reason is structural. An agent's output is mostly code and tool calls, which caveman leaves alone on purpose, so only the prose around them shrinks. Meanwhile the skill's rules cost about 1 to 1.5k input tokens on every turn.

Claude Code has shipped a built-in equivalent since v2.1.237, the Concise output style under /config. Same effect, no plugin. Both apply to the main conversation only; subagents keep their own prompt.

graphify moved the most tokens on a large repo

graphify parses the project with tree-sitter, locally, and builds a knowledge graph the agent queries instead of grepping files. Zero model calls for the code pass. I ran its built-in benchmark on one of my projects:

nodes                  34,031
edges                  56,865
naive full read     2,268,733 tokens
average graph query    24,702 tokens
                        91.8x fewer per query

Two caveats that the number hides. The comparison is against reading everything, and a grep-driven session was never that expensive, so the real gain is smaller. And the graph goes stale; you rebuild it with graphify update or the git hooks, and the semantic pass over docs and PDFs does spend tokens. On a small repo the gap shrinks to nothing.

Install: uv tool install graphifyy (double y), then graphify install.

Superpowers moved the bill, without compressing anything

Superpowers is Jesse Vincent's plugin, 280,792 stars, fourteen skills. Nothing in it compresses output. It changes what gets read and which model reads it.

A brainstorming gate blocks any code until the intent is approved. The plan skill cuts work into steps of two to five minutes: failing test, minimal code, tests green, commit. Each step then runs in a fresh subagent whose context holds only that task, never the session history. And the rule that pays for everything, quoted from the skill file: "Use the least powerful model that can handle each role to conserve cost."

mechanical, well specified   ->  small model
multi-file, debugging        ->  standard model
architecture, final review   ->  most capable model

That is what makes Opus and Fable usable on a $20 Pro plan. The expensive model touches a handful of tasks instead of the whole session. The cost is ceremony: the gate fires on a one-line fix too, and that is tokens as well.

What this doesn't prove

My graphify and rtk numbers come from my own projects, and repo shape changes them a lot. JetBrains measured on Sonnet 5 with 82 to 86 tasks; a different model at a different effort level can move the rtk result in either direction, as their own high-effort arm shows. I did not benchmark Superpowers with a paired test; its effect is on model selection, which is visible on the invoice but harder to isolate per task.

The full run

The video walks through the JetBrains tables and my dashboards on screen, with the four tools ranked at the end: Superpowers, graphify, rtk, caveman.

What do you run to save tokens, and did you ever check it against the bill? A) a shell proxy B) a code graph C) task and model discipline D) never measured, just installed

English isn't my first language; I used an editing tool to polish the wording. The measurements and opinions are mine.

Fable 5.1 cut cache reads by 75%. Cost per task went up 20% anyway.

AI Dive — Fri, 04 Sep 2026 15:34:54 +0000

In the same week that Claude Fable 5.1 shipped, two things happened that look like they cannot both be true.

Millennium had a piece of code that crashed about one run in a million. Their team had not explained it in four to five years. Every model they tried missed it, Fable 5 included. Fable 5.1 found it.

That same week a creator titled his review "I'm asking for a refund", and Artificial Analysis, which ranks these models for a living, put Fable 5.1 at the top of its intelligence index while measuring it at 20% more cost per task than Fable 5.

Both are accurate, and the reason is not in the benchmarks. It is in the pricing page and in how you pay for the model.

What I actually did

I did not run my own benchmark suite. There are plenty of those already and they mostly disagree.

What I did was read the release the way you would read a spec sheet before buying hardware: the announcement, the model docs, the system card, the prompting guide, the pricing page, and the support articles about plan limits. Then the independent measurements from Artificial Analysis, ARC Prize, FrontierSWE and Simon Willison, because the vendor's own table is not evidence about the vendor.

That turns out to be the differentiator. Almost every review this week ran demos and picked a side. The interesting material was sitting in the footnotes.

The cut is one line item

Fable 5.1 is not broadly cheaper. Input is still $10 per million tokens and output is still $50, unchanged from Fable 5.

One line moved. Cache reads, meaning the tokens the model rereads from a prefix it already processed, went from $1 to $0.25 per million. That is the entire price cut.

It matters more than it sounds, because in a long agent session most of your tokens are re-reads of the same context. Anthropic measured four weeks of real August traffic and reports roughly 25% off a typical bill and up to 45% off a heavily agentic one.

It also produces the one situation where the expensive model undercuts the cheaper one:

Cache read, per million tokens
Fable 5.1   $0.25
Opus 5      $0.50

Everything else on Fable costs double what Opus costs. A cache heavy loop really can be cheaper on Fable. Nothing else can.

Why the per-task number still went up

The second dial is effort, and it moves the bill much harder than the cache line does.

Simon Willison drew the same pelican at every effort level. Low cost him 10 cents. High cost 13 cents. Xhigh cost $1.83. Max cost $3.30, producing 65,927 output tokens over 14 minutes. Same prompt, 33 times the price.

Artificial Analysis runs everything at max. So they measured $3.76 per task against $3.14 for Fable 5, which is the 20% increase people are quoting. The cache cut is why it was not worse: without it the same measurement would have landed at $5.16.

So the honest version of the headline is that the model got cheaper to reread and more expensive to think, and which of those dominates depends on your workload. If you are running one shot requests at max effort, you are the case they measured. If you are running a review loop over the same repository for two hours, you are the case Anthropic measured.

The plan is a different meter entirely

None of that per-token math applies if you are on a subscription, and this is where the refund videos come from.

On Pro, Fable is not part of the plan's usage limits at all. It runs on pay as you go usage credits, and unlike previous launches there is no one-time credit this time.

On Max it is included, for up to 50% of your weekly limits. The same support page adds that these models consume those limits faster than other Claude models. That sentence is doing a lot of work and almost nobody quotes it.

Then the counter itself. The pricing page says Max gives 5x or 20x more usage per 5-hour session than Pro. Per session, with weekly limits stacked on top. A lawsuit filed June 15 alleges the real weekly multipliers are far below the advertised ones. And on September 14 a permanent 25% raise of weekly limits takes effect, which Anthropic itself clarified works out to a 17% reduction compared to today, because a temporary 50% boost ends the day before.

The model is not more expensive per token on a plan. It spends a budget faster than that budget was ever advertised to be spent.

Three things break if you call the API today

If you have a working Fable 5 integration, three changes fail it rather than degrade it.

Forced tool use returns a 400. Setting tool_choice to any or to a named tool is rejected. Thinking is always on now, and forcing a call would skip it.

Thinking blocks record which model wrote them, one way. Fable 5.1 reads Opus's thinking. Nothing reads Fable 5.1's.

Conversations are append only. Edit an earlier turn, or rebuild your system prompt or tools array between requests, and the next call fails with a message about the block being bound to a different conversation. This is enforced for accounts created on or after August 31. It breaks every harness that injects a reminder and deletes it on the following turn, which is a very common pattern.

Five things arrive at the same time, and the useful one is that you can change effort per message without losing your cache.

Then there are the documented behavior regressions, which are the ones burning people's quota right now. It rewrites whole files for small edits. It issues one tool call per turn in coding loops where Fable 5 batched several. It sometimes describes what it would do next instead of doing it. Each of those has a one line fix in the prompting guide, and each of them costs output tokens until you apply it.

What this doesn't prove

This is a documentation read, not a capability test. I cannot tell you whether the model is better at your work, and I did not try to.

The benchmark table is also weaker evidence than it looks. Anthropic's own footnote gives a standard error of 3.5 to 4.5 points per model, which puts the three point coding lead over Opus 5 inside the error bar. The rows missing from the marketing graphic are the ones where Opus 5 wins: 90.42 against 90.0 on ARC-AGI-2, and 59.4 against 54.7 on SWE-bench multimodal.

One more thing worth knowing before you send anything sensitive: Fable 5.1 carries 30-day retention and is not available under zero data retention unless Anthropic authorizes it. Enterprise Frontier Safeguards is the way out, and it ships this fall.

The full run

The video walks the actual pages, so you can see where each of these numbers lives rather than taking my word for the quote.

If you are on Max: did your 5-hour window survive the first week of Fable 5.1, and did you change your prompts before or after finding out?

English is not my first language, so I use a tool to polish the writing.

Superpowers fixes Claude Code. Then it bills you for every two-line fix.

AI Dive — Wed, 02 Sep 2026 16:40:40 +0000

Every few weeks the same thread comes back on r/ClaudeCode. Someone installed the Superpowers plugin because everybody recommended it, checked their usage stats, and found it sitting at 1 to 3 percent. No visible change in the code either. Is it worth it, or is it a tokens killer?

The plugin has 280,000 stars, 25,000 forks and is on its sixth major version in under a year, so this is not a prompt pack somebody abandoned after launch. Jesse Vincent, who has been shipping open source developer tools since the 90s, built it by mining 2,249 markdown files of lessons his own agents had learned.

I installed it, read the 14 skill files line by line, and ran it on my own projects. The short answer to the Reddit question: both camps are right, and which one you land in depends on what you feed it.

What I actually looked at

Superpowers is a free plugin on Anthropic's plugin marketplace, one command to install. The core is 14 skills: markdown instruction files the agent loads whenever a situation matches. Brainstorming, plan writing, subagent driven development, test driven development, systematic debugging, verification before completion, and so on.

A hook at session start forces Claude to check, before every task, whether one of these skills applies. The entry skill is blunt about it: if there is even a 1 percent chance a skill is relevant, load it. So this behaves less like a toolbox and more like a development methodology injected into your agent.

I read every file, then watched what each one did on real feature work and on real small fixes.

The gain sits upstream of the code

The brainstorming skill opens on a hard gate: no code, no scaffolding, no implementation skill of any kind until you have approved an explicit intent. Every request gets sorted into one of three paths. A spike is a feasibility question whose output is an answer, not code you keep. Bounded is a small change to a flow that already exists. Architectural is anything that restructures how the project fits together. The agent says its classification out loud so you can override it, and hidden complexity found mid task can upgrade the path, never downgrade it.

On my own project this phase has already killed features I would have built for nothing. That alone paid for the install.

Out of the architectural path come two documents: a spec you validate, then an implementation plan. The plan skill's opening instruction sets the tone. Write it for a skilled developer with zero context on the codebase and, in the file's own words, questionable taste. The work gets cut into tasks of two to five minutes each:

write the failing test
run it, confirm it fails
write the minimal code that passes
run the tests again
commit

Each task lists the exact files to touch, down to line numbers. This sizing is the reliability argument in one line: a short task means an agent that finishes with a context window still mostly empty. It never reaches the moment where the session overflows and the agent starts inventing functions that do not exist.

The cost is the execution loop, and it never switches off

At execution time your main session stops coding and becomes an orchestrator. It dispatches one fresh subagent per task, with exactly the context that task needs and none of your session history. When the subagent is done, the orchestrator runs a two part review: spec compliance first, then code quality. If the review fails, a fix loop starts, capped at five rounds. The first three resume the original implementer, round four dispatches a fresh one on a more capable model, and at round five a breaker trips and the orchestrator rules on every open finding itself.

That is a real code review process running on its own, task after task. It is also where the tokens go. Dispatch briefs, two reviews per task and the completion ledger are paid every single time, and you feel it on the smallest tasks.

The brainstorming file is explicit that the ceremony scales with the task. The human approval step does not. On a two line fix, that means answering framing questions, approving a two sentence design, then waiting for the full cycle. For a typo in a config file, the complete process is simply slower than fixing it yourself.

And here is the answer to the 1 to 3 percent question. If your usage stats show the plugin that low, your requests almost never trigger the skills. You are paying the session start check every time without ever touching the gains.

The setting that decides whether it pays

The execution skill has a model selection section that starts with one rule: use the least powerful model that can handle each role. A well specified mechanical task touching a file or two goes to a small model. Coordination across several files and debugging go to a standard model. Architecture and the final branch review go to the most capable one.

Two subtleties in that file I had not seen anywhere else.

First, always name the model explicitly when you dispatch. A subagent without one inherits your session's model, often the most expensive one, which silently defeats the whole section.

Second, turn count beats token price. The cheapest models take more turns on multi step work and end up costing more overall, so reviewers and implementers working from prose get a floor one tier up.

This is what makes running Opus or Fable on the 20 dollar Pro plan workable with the plugin: the expensive model only touches the few decisions that deserve it.

The win nobody installs it for

Specs and plans are not chat messages that vanish. They are markdown files saved in a dated plans folder inside the repo and committed with the work. The spec travels with the plan, and conflicts resolve against the spec, so the document is the authority rather than the agent's memory.

Six months later you mention the plan file in a prompt and the agent picks the feature's context right back up. Your git history stops telling you only what changed and starts telling you why. Documentation falls out of the process without anyone sitting down to write it.

What this does not prove

I read the files and used the plugin on my own projects. I did not run a controlled benchmark with token counts per task, so "it burns quota on small fixes" is an observation, not a measurement. The repo also moves fast: sixth major version in under a year, 125 open issues, so the skills you read today will have changed by the next update.

The plugin plans its own exit, though. Its instructions put your directives above the skills, so telling the agent to skip the process works.

The rule I ended up with

Superpowers on by default for any feature work. An explicit skip for tiny fixes.

If you build features that take hours with Claude Code, install it. If your usage is throwaway scripts and small fixes, walk on by. In between, install it and learn to say skip, or let just the brainstorming skill run for a few days. It carries most of the gain and the other skills graft on after it.

The full read

The video shows the actual skill files on screen, section by section, which is hard to convey in prose.

Which camp are you in? A) On all the time B) On for features, off for fixes C) Deleted it, saved my quota D) Never installed it

The reading, the runs and the opinions are mine. I used an editing tool to polish the English.

Anthropic graded 400,000 Claude Code sessions. Managers beat the engineers.

AI Dive — Mon, 31 Aug 2026 15:29:55 +0000

Anthropic ran a classifier over roughly 400,000 Claude Code sessions from 235,000 people, recorded between October 2025 and April 2026, to figure out who actually succeeds with a coding agent. The finding that made me read the whole thing: management occupations reach verified success slightly more often than software engineers, and the ten biggest occupation groups all land within seven points of the engineers.

I made a video that walks through the study and then tests its main claim live. Here is the short version.

What "verified success" means

The numbers only make sense with the definitions. Nobody at Anthropic read the transcripts. A classifier built on Sonnet 4.6 graded each session, and its grades were cross-checked against telemetry (commits, code changes, test results). On sessions that touch code, classifier and telemetry agree more than 90% of the time. A success only counts as verified when there is hard evidence, like passing tests or committed work.

The five levels

Each session also gets an expertise rating from novice to expert, based on how the person works, not on their job title. A lawyer who knows exactly what a contract script must catch can rate as an expert. A senior engineer poking at an unfamiliar language can rate as a novice.

Novice-rated sessions reach verified success 15% of the time. Intermediate through expert sit at 28 to 33%. Almost the entire gain is between novice and intermediate, which is the encouraging part: you do not need mastery, you need to stop prompting like a novice.

The output gap is just as stark. One instruction from a novice triggers about 5 actions and 600 words of output. The same instruction from an expert triggers about 12 actions and 3,200 words.

The part I tested myself

In the video I run the same task twice in two real Claude Code sessions, once written like the novice prompts in the study and once like the expert ones. The expert prompt contains zero code. It packs a repo map, constraints, and a test plan, and the session behaves almost exactly like the study predicts.

Three habits the data rewards, if you want the actionable bit: frame directions precisely up front, ask the agent to verify its own work, and redirect a failing session instead of watching it sink. When things break, 19% of novices quit with zero lines written; everyone else quits 5 to 7% of the time.

Caveats

Verified success caps out around a third even for experts. The data is self-selected Claude Code users. And the occupation ranking may partly reflect who confirms out loud when they got what they asked for. The study is still the best evidence I have seen that knowing your domain now matters more than knowing how to code.

Full write-up: https://aidive.dev/videos/anthropic-400k-expertise/