DEV Community

Maksims Gavrilovs
Maksims Gavrilovs

Posted on • Originally published at Medium

Why Agent Orchestration Sucks and the Loop Wins

#agents #llm #ai #architecture

Eight agents, 298 tickets, 62 videos. Every failure came from the same place: everything was readable, and nothing was delivered.

This is a retraction. I built a faceless AI YouTube channel and closed it into a single autopilot loop. Then I tore that loop out and replaced it with eight agents passing tickets, and wrote it up like it was progress. This is what running that company taught me. It stands on its own; you don't need the earlier posts.

Data status: real-now. Every count is measured from the live system. Open source.

I built an eight-agent AI company to make 60-second videos. It needed 298 tickets to produce 62 of them. Four and a third handoffs per minute of finished content, and not one of those handoffs put a frame on screen.

The architecture looked fine on paper: an orchestrator, specialist workers for scripting, production, QA, analytics, publishing, all coordinating by ticket. A textbook star topology. It is what the frameworks push you toward.

Then I ran it, and every distinct failure turned out to be the same failure. Not a missing store, either: the runtime shares almost everything. Comments, documents, the goal tree, an append-only activity log, and the whole repo are readable by any agent that asks.

That is the trap. Shared storage is not shared context. Nothing is pushed to an agent when it wakes. It gets its role prompt and a ticket id, and from there every fact about the world is something it must decide to go fetch. So each agent reconstructs the world from whatever it thought to look up, and a cold process does not know what it does not know.

Everything below follows from that.

One caveat: no wall-clock numbers. I was in that loop, so stall times measure my availability as much as the architecture. Structural failures and counts only.

Metric Value
Agents 8, each its own LLM session
Tickets created 298 for 62 videos, or 4.3 per video
Worst single video 11 tickets
Cancelled tickets 21, plus 1 permanently blocked
Agent spend tracked $0.00, across all eight agents

1. Every agent reasons from a stale snapshot

My QA agent returned a clean verdict on one video: "Gate 2 = FAIL, final video does not exist."

It was right. A separate ticket had sent the script back for a rewrite, so the render was running again and the directory was nearly empty. QA woke a second time, re-checked, and doubled down. The file appeared later. The verdict never re-evaluated, so the ticket carried a failing grade for work that had succeeded. The video it says does not exist is 23MB on my disk.

None of this is an LLM failure; the same code with no model in it fails the same way. QA read a directory mid-rewrite and then persisted a conclusion instead of an observation: not "absent when I looked" but "does not exist", a claim with no shelf life and nobody responsible for invalidating it. The correcting event was unobservable anyway, because a file appearing is not one of the four things that can wake an agent. Pulling context gives you a sample; staying correct needs a subscription, and there was no way to express one.

Same shape, one level up. My CEO agent added a video to the backlog by hand, overriding the Growth Lead's duplicate check, because the CEO was reading the goal and the Growth Lead was reading the backlog and neither could see the other's screen. The subject was already published. The journal entry records the whole thing:

Duplicate of j0037 (Banach-Tarski Paradox already published, SLO-93). Board added this manually overriding Growth Lead dedup check.

Two agents, two views of the same channel, one of them authoritative and wrong. The check existed and worked. It just wasn't where the decision got made.

Neither agent was dumb. Both were locally correct given what they could see. That is the whole problem.

2. Nobody's job is noticing

The daily cap was three videos. I found it publishing one and assumed something was throttling it. Nothing was: the backlog had run empty and it was producing everything it could. Eight agents, one of them a planner, and none noticed the pipeline was dry. Every agent did its job correctly. Nobody's job was noticing.

The runtime does keep a company-wide activity log, an append-only stream of everything every agent did. No agent was ever told to read it. Instead I solved org-awareness by hiring for it: two roles whose whole job was reading everything and writing me a digest. A view of the system became a staffing decision rather than a query.

That gap hides everything else. An agent flipped into an error state with no message after finishing its work, with nothing useful in 268MB of logs. Research puts ~75% of multi-agent failures in this silent category, which matches what I saw. The system almost never told me it was broken. I had to go ask.

Two more of the same: eight agents shared one git checkout and stepped on each other's uncommitted changes, and a half-abandoned second agent company on the same runtime quietly captured all my notification digests. I was blind without knowing it.

3. Handoffs stop dead and nothing says so

Work moves only when an agent wakes, and there are exactly four ways to wake one: assign a ticket, fire a cron, tick an idle heartbeat, or @mention an exact handle. A comment with no mention moves nothing. A file appearing on disk moves nothing.

So when a run died mid-task, its ticket kept a lock pointing at a run that no longer existed. The assignee tried to pick up its own ticket, hit a conflict, and stopped. No timeout, no retry. I found two finished videos sitting encoded on disk while their tickets sat stuck behind a dead run's lock.

I had caused it myself. Heartbeats were 240 no-op LLM calls a day and the biggest line in my token bill, so I killed them. Heartbeats were also the only thing that would have retried a stranded ticket. Every knob in an orchestrated system is load-bearing for something you aren't thinking about.

4. Prose has no schema

Three tickets stalled because a handoff tagged @QACritic. The real handle was @qa-critic.

No error, no warning, no "unknown recipient." The mention matched nobody, the wake never fired. Nothing threw, so there was no stack trace. No call was made, so there was no failed call to find. The system did not do the wrong thing. It did nothing, correctly, and I found it by diffing prose against configuration by eye.

These bugs also don't reproduce. Re-run the stage, the agent phrases the handoff differently, and the bug moves or evaporates. Nothing to bisect, nothing to regression-test.

The deeper version of this is what a handoff actually carries. Most of mine created a new ticket for the next owner rather than reassigning the old one, so the accumulated comment thread stayed behind and only the fields the sender chose to retype travelled forward. Which means I ended up writing, by hand, a copy-forward schema for every hop:

Entry id: / Channel: / Idea: / Target keyword: / Title candidate: / Hook: / Assumption: / Goal: / Theme/tags: / Duration: / Constraints:

There is one of those in every agent's instructions, each slightly different, all of them prose, none of them validated. That is an interface definition with no compiler. Miss a field and the next agent doesn't error, it just proceeds with a hole in its picture.

The same softness hit the gates. Some fired after the thing they were meant to gate had already run, and reported success anyway. One critic declined every script it saw, including ones that shipped and did well. A gate that always says no is noise, and everyone downstream learns to route around it.

5. You pay to re-explain the problem, every hop

Every handoff means the receiving agent reads the ticket, reconstructs the situation, re-derives the state, and decides what to do, all from cold. At 4.3 tickets per video that is 4.3 cold starts per minute of finished content, and coordination produces no artifact. An agent that stays resident pays that cost once.

Retrieval is the expensive half. Because nothing arrives with the wake, every agent needs instructions for what to go fetch, and those instructions grow with every bug you find. Mine ended up telling the planner to grep the entire journal on every single ideation, because trusting a window had already burned me. That is the pattern: each missed fact adds another mandatory lookup to somebody's prompt. The prompts get longer, the wake gets slower, the token bill goes up, and the system is still fragile, because the next gap is one you haven't hit yet. You are hand-writing a cache-warming strategy in English and paying to re-run it at every hop.

The latency has the same source. The system was never compute-bound, it was wake-bound: trigger fires, agent starts cold, works, hands off, exits, next one waits its turn. Nothing happens in between, and "in between" is most of the pipeline.

And the bill is split eight ways, so nobody sees it. The org's own per-agent spend tracking read $0.00 after hundreds of LLM calls, and I didn't catch it, because there was no one place where the cost of a video added up.

When orchestration is right

Four cases, none rare. Real parallelism, where subtasks are independent and fan-out is the point. Adversarial separation, where the reviewer must not share the author's context, because a critic that watched you write the code will rationalize it. Isolation as a requirement: untrusted code, per-tenant boundaries. Genuine scale, where the problem does not fit in one context window.

My pipeline was none of these. Script, images, clips, voice, stitch, publish is a strictly sequential chain. No parallelism to exploit, no isolation to satisfy. I took a dependency chain, dressed it in an org chart, and paid for coordination I had no use for.

"But you used the wrong framework"

Fair, and worth being precise about.

LangGraph would have prevented most of what I hit, structurally. State is a typed schema passed into each node rather than something a cold process has to go fetch. Edges are deterministic code, not an @mention that can be misspelled. Checkpointers make a dead run resumable instead of stranded, and parallel branches merge through explicit reducers. Temporal solves the durability half even more thoroughly: heartbeats, timeouts, retries, and deterministic replay are what durable execution engines have done since well before any of this got called an agent. I have not run either on this pipeline, so read that as reasoning about their design rather than a report from production.

But notice what those tools are. LangGraph is a state machine with LLM calls in the nodes, where the routing is Python rather than persuasion. Temporal is a workflow engine. Neither is a company of agents, and reaching for one is agreeing with the argument here rather than refuting it.

The frameworks that are companies of agents share my failure mode, because they share my substrate. CrewAI gives you roles and tasks. AutoGen gives you conversable agents in a group chat. The OpenAI Agents SDK gives you handoffs the model itself decides to take. All three are better engineered than what I assembled, and the good ones carry the message thread forward so less falls on the floor. But coordination is still natural language moving between peers, so there is still no schema to validate it, no compiler to catch a bad address, and no way to fail loudly when a message means nothing.

So the line isn't agents versus loops. It's whether coordination is a schema you can typecheck or prose you have to hope lands. Pick a tool on the right side of that line and the specific tool matters much less than the line does.

The shape that actually works

One main agent holds the context for the whole job. Subagents do work and return results into that context. They never decide anything.

That's it. The difference isn't a better store, it's that context is resident instead of retrieved. The main agent watched the video get produced, so it cannot file a FAIL saying the video doesn't exist. It never restarts cold, so it isn't paying to re-derive what it already knows. And it needs no copy-forward schema, no mandatory-lookup list, no instructions about what to go fetch, because nothing is handed off.

The subagent's job is keeping bloat out of that context, which is the part people get backwards. A subagent is not a coworker, it's a filter. Send it at work whose output you need but whose intermediate tokens you don't: scan a 268MB log, sweep a codebase, search 200 files. It burns 50,000 tokens and returns 200. The main agent gets the answer without the noise.

The direction matters. Results flow in, decisions stay put. Nothing is negotiated between peers, so there is no handoff protocol to mistype and no gate to rubber-stamp. There is only retrieval.

And whatever coordination genuinely remains gets a mechanical fix, not an agent. My stranded-lock janitor is 150 lines of Python on a cron, and it beats an agent at the job because it is deterministic, free, and never misreads the situation.

The critic makes the point better, because it survived. My QA Critic agent had a real job: score a script against four criteria before I spend anything rendering it. That judgment was worth keeping, so I kept it as a stage in the pipeline. It is the same model scoring the same four criteria, except the verdict comes back as an object with the rewrite notes in a field, and those notes go straight into the next script generation. Nothing has to wake up, nothing is addressed by handle, and no verdict can outlive the artifact it judged. The critic didn't need to be an agent. It needed to be a function.

I already went back. The channel now runs on the autopilot I replaced: one pure function, plan(journal, now), returning the single most useful action per tick. It is more robust, more effective, and far easier to reason about than the org that replaced it. No ticket to strand, no handle to mistype, no snapshot to go stale. When it breaks I get a stack trace with a line number, which after months of reading agent transcripts felt like a luxury.

The proof was in the fix itself. Diagnosing those two stuck tickets, writing the janitor, dry-running it, installing the cron, and clearing both videos happened in one sitting, by one agent holding the whole problem at once. The eight-agent company never noticed those tickets, and couldn't: no agent in it could see ticket state, files on disk, and agent health at the same time.

What fixed the agent company was not an agent company.

An org chart is not an architecture. Multi-agent topology buys parallelism and isolation, and it costs you context, tokens, and truth. If your work is a dependency chain, you don't need a company of agents. You need one agent that remembers everything, and workers that hand it answers instead of opinions.

References

Prior write-ups of the same system, for anyone who wants the build rather than the postmortem.

Top comments (0)