A pipeline of AI agents that had run fine for months started getting slower. Same jobs, same sources, same extraction code. Median run time doubled, from 5.7 minutes to 12.2. Then it started failing on rate limits, because it had begun generating enough traffic to throttle itself.
My instinct was to profile the agent doing the heavy lifting. That instinct cost me six weeks.
Here is the thing I should have done on day one, and it takes about an hour.
Do not time anything. Count raw output volume, grouped by role.
Four days of logs, 619 agent turns, 4.5 million characters of agent text. Then a single group-by.
The orchestrator produced 67.8% of it.
The orchestrator fetches nothing. It publishes nothing. It has no tools pointed at the outside world at all. Two thirds of everything my system said was a manager talking about work rather than doing it.
That number was invisible to every profiler I could have pointed at the problem, because nothing was slow. Everything was just saying more.
Then do a second pass and categorise what the talking is about.
I labelled the orchestrator's review turns. 76% concerned the internal consistency of data keys the agents had invented themselves. 3% concerned whether the data was actually correct.
Seventy-six against three. The reviewer was not checking the work. It was checking the paperwork, and the paperwork was not even a spec anyone had written.
Why this compounds instead of plateauing.
Every review produced a list of inconsistencies. Fixing them meant re-emitting the payload. Every re-emission was a chance to add an explanatory field or restructure something that was fine. Payloads grew six to thirteen times over, one lane hitting 800 keys in a single output. More keys meant more surface for the next review to find fault in.
The review loop was manufacturing the defects it kept discovering.
If you are running anything multi-agent, run the group-by. It is one query and it will tell you something a flame graph cannot.
The full write-up covers the structural fix, the numbers before and after, the role conflict it created, and a detector I wrote, tested and deleted because it was wrong in both directions.
https://openred.space/blog/multi-agent-middle-manager-problem.html
Top comments (0)