Anthropic ran a classifier over roughly 400,000 Claude Code sessions from 235,000 people, recorded between October 2025 and April 2026, to figure out who actually succeeds with a coding agent. The finding that made me read the whole thing: management occupations reach verified success slightly more often than software engineers, and the ten biggest occupation groups all land within seven points of the engineers.
I made a video that walks through the study and then tests its main claim live. Here is the short version.
What "verified success" means
The numbers only make sense with the definitions. Nobody at Anthropic read the transcripts. A classifier built on Sonnet 4.6 graded each session, and its grades were cross-checked against telemetry (commits, code changes, test results). On sessions that touch code, classifier and telemetry agree more than 90% of the time. A success only counts as verified when there is hard evidence, like passing tests or committed work.
The five levels
Each session also gets an expertise rating from novice to expert, based on how the person works, not on their job title. A lawyer who knows exactly what a contract script must catch can rate as an expert. A senior engineer poking at an unfamiliar language can rate as a novice.
Novice-rated sessions reach verified success 15% of the time. Intermediate through expert sit at 28 to 33%. Almost the entire gain is between novice and intermediate, which is the encouraging part: you do not need mastery, you need to stop prompting like a novice.
The output gap is just as stark. One instruction from a novice triggers about 5 actions and 600 words of output. The same instruction from an expert triggers about 12 actions and 3,200 words.
The part I tested myself
In the video I run the same task twice in two real Claude Code sessions, once written like the novice prompts in the study and once like the expert ones. The expert prompt contains zero code. It packs a repo map, constraints, and a test plan, and the session behaves almost exactly like the study predicts.
Three habits the data rewards, if you want the actionable bit: frame directions precisely up front, ask the agent to verify its own work, and redirect a failing session instead of watching it sink. When things break, 19% of novices quit with zero lines written; everyone else quits 5 to 7% of the time.
Caveats
Verified success caps out around a third even for experts. The data is self-selected Claude Code users. And the occupation ranking may partly reflect who confirms out loud when they got what they asked for. The study is still the best evidence I have seen that knowing your domain now matters more than knowing how to code.
Full write-up: https://aidive.dev/videos/anthropic-400k-expertise/
Top comments (0)