Alberta published a case study last week about scanning 466 million lines of code across 1,280 applications using around 50 specialized Claude agents - red-team agents mapping exploit paths, blue-team agents writing remediation plans, code-quality agents, content-clarity agents - all running in parallel. The scan that would have taken six years traditionally took 20 hours.
Everyone shared the 466 million number. The throughput is impressive.
But that's not the number I kept thinking about.
The number I kept thinking about: how many of those 50 agents needed to trust the output of another agent before they could do their job?
And the question nobody asked: who owned that handoff?
A Fleet Is Not One Agent Scaled Up
When you run one agent, you supervise it directly. You read the output, spot the gap, course-correct. The failure mode is visible. The loop closes fast.
When you run a fleet of specialized agents, something changes. You're not supervising each agent anymore. You're designing how they relate to each other. You're thinking in interfaces, not prompts.
In Alberta's setup, a red-team agent identifies a vulnerability. That finding becomes the input to a blue-team agent that writes a remediation plan. Two specialized agents, two distinct roles, one handoff between them.
The first time I chained two agents in my own setup, the second one confidently built on the first one's mistake - because I never defined what "done" meant at the handoff. The first agent's output looked complete. It wasn't. The second agent had no way to know.
That's the asymmetry a fleet creates. Individual agents fail loudly. Handoffs fail quietly.
The Handoff Is Where the Risk Moved
When agents work in sequence, one agent's output becomes the next agent's input. On the surface this looks like efficiency - specialization, parallelism, faster throughput. Underneath, it creates a compounding problem.
Each agent is good at its slice. The red-team agent knows how to map exploit paths. The blue-team agent knows how to write remediation plans. Neither agent is responsible for checking whether the transition between them worked.
Errors introduced in step one get treated as ground truth in step two. By the time you're looking at step three's output, the mistake is three layers deep and doesn't look like a mistake anymore - it looks like a conclusion.
I've watched this happen with much smaller chains. An agent summarizing a document slightly overstates a conclusion. The next agent builds a recommendation on that summary. By the time a human looks at the recommendation, the original overstatement is invisible. The recommendation looks well-reasoned.
The throughput is fast. The error is silent. Those two things together are the actual risk profile of a fleet.
Output Qualification Is the Missing Job
If the handoff is where the risk lives, then output qualification is the job that doesn't exist yet in most agent setups.
Output qualification isn't the same as error-checking. It's not about catching mistakes after the fact. It's about defining - upfront, at design time - what "good enough to proceed" looks like at each handoff point.
In the Alberta setup, this might look like: before the blue-team agent acts on the red-team agent's exploit mapping, something checks that the mapping includes the required fields, crosses a confidence threshold, and flags ambiguous cases for human review. That "something" can be an evaluation agent, a structured schema check, or a human gate. The form matters less than the fact that it exists.
Three things I've started doing when designing agent chains:
Define acceptance criteria at the handoff, not just the output. It's easy to write a prompt that says "produce a detailed analysis." It's harder - and more useful - to write criteria that say "this analysis is ready for the next agent when it includes X, cites Y, and has a confidence score above Z."
Build the qualification layer before you scale. One agent failing is a debugging problem. Five agents downstream from one failure is an incident. The cost of adding a qualification check at the handoff stays roughly constant as your fleet grows. The cost of not having one compounds.
Name the specialized roles explicitly. In Alberta's setup, "red-team agent" and "blue-team agent" are clear role definitions with implied scope boundaries. Each agent knows what it's responsible for. Most internal setups skip this. You end up with agents that have overlapping scope, no clean handoffs, and failures that fall in the gaps between them.
The Number Nobody Asked About
The 466 million lines in 20 hours is the headline number. It's the right number for a press release.
The more interesting number is how many of those 50 agents were receiving output from another agent rather than from a human - and how many qualification checks existed at those handoffs.
A fleet isn't one agent scaled up. The throughput lives in the parallelism. The risk lives in the seams.
When you design a fleet, you're not writing prompts. You're designing an org chart - role definitions, handoff policies, and qualification gates at every point where one agent's output becomes another agent's input.
If you've chained agents: what broke first?
Top comments (2)
The handoff point is more important than the number of agents. Once one model’s output becomes another model’s assumed truth, the system needs the same controls we would expect between human teams: a defined contract, required evidence, validation and a clear owner who can reject the output.
I would be cautious about relying on self-reported confidence though, as models can be confidently wrong. I've seen this way too many times. For higher-risk work, structured checks against the original source or a human approval point seem safer than asking another agent whether the first one was correct.
the assumed truth problem is the one that got us. built a handoff where agent B was supposed to validate agent A’s output and it took two weeks to notice it wasn’t actually rejecting anything - just passing everything through with a slightly different format.
adding rejection criteria to the handoff spec helped more than I expected. framing it as "reject if" rather than "pass if" changed the behavior noticeably. and yeah, scrubbing the confidence score was the right call - ours was anchoring on it instead of running independent checks.