Intro
In a demo, one agent that "does everything" looks like impressive economy: one prompt, one API call, one mental model. Nobody's testing it against thousands of edge cases a day, and nobody's paying for the tokens burned parsing forty unrelated instructions to answer a two-line question. In production, that same design is where teams spend their first real outage. Below are four failure modes that show up once a monolithic agent hits real traffic. (Illustrative composites, not case studies from a specific client.)
The system prompt that nobody can safely edit
Picture a support agent that started as "answer product questions" and grew, feature request by feature request, into a 4,000-word system prompt covering billing, refunds, technical troubleshooting, and tone guidelines for three different customer segments. Adding a new instruction for edge case #41 quietly changes how the model handles edge case #12, because both are competing for the same attention budget in the same call. Nobody can point to which line caused the regression, because there's no isolation between concerns, only proximity.
Every request pays for capabilities it doesn't use
A single do-everything agent charges the same token bill for a request that just needs a database lookup as it does for one that needs careful reasoning about a refund policy exception. Say a team's simplest, highest-volume intent, "what's my order status," routes through the full monolithic prompt anyway. That's paying premium reasoning costs, on every call, for a task a five-line function could handle for a fraction of the price.
One bad instruction degrades everything downstream
Because a monolithic agent handles routing, task execution, and formatting in a single inference pass, a subtle drift in one part of its behavior, say it starts hedging more on refund questions after an unrelated prompt tweak, has no boundary to stay contained inside. There's no seam where you could catch it before it reaches the user, because the "component" that's misbehaving is the entire agent.
Testing becomes a probability exercise, not an engineering one
With a single-purpose function, you write a test, you know the input, you assert the output. With a 4,000-word do-everything prompt, "testing" often means running the same conversation a dozen times and hoping the failure rate stays under some tolerable threshold. That's not a test suite. That's a weather forecast.
A multi-purpose agent isn't one decision. It's twenty, made by an LLM, at inference time, with nothing watching any of them.
Every one of these failure modes comes from the same root cause: collapsing routing, task execution, and state management into a single non-deterministic call and hoping the model sorts it out consistently, every time, forever. It won't, not because the model is bad, but because that's not what a single inference pass was ever built to guarantee.
Route it, don't merge it
The fix isn't a smarter prompt. It's separating what genuinely benefits from a language model's judgment from what doesn't. A small classifier decides intent. Specialized, single-purpose functions handle each isolated task, several of which may not need an LLM at all. Deterministic code owns routing, state, and final formatting, the parts where you actually want predictability, not creativity. Each piece is small enough to unit test on its own, which is the property the monolith could never give you.
Where's the line for you, at what point does "just add another instruction to the prompt" stop being the pragmatic choice and start being the thing you'll be debugging at 2am?
Top comments (2)
The 2am debugging point for us was when the same prompt section started controlling both routing logic and tone, so fixing a customer segment's tone broke routing for another. The seam you describe - splitting routing/intent classification from task execution - is what finally gave us reproducible test cases: intent in, intent out, no LLM ambiguity. The token cost argument is underrated too; once we routed the high-volume, simple-lookup intents to deterministic functions, our inference spend dropped without touching accuracy. The last holdout for the monolith is usually the engineer who wrote the original prompt - nobody else feels safe editing it, which is itself a signal.
it is intresting
...