Most agent projects do not start as agent projects. They start as a chatbot that answers a question, then someone asks it to also update the record, then someone asks it to check back in an hour, and by the third request you are maintaining a scheduler, a retry policy and a permissions model that nobody scoped. The rewrite does not announce itself, it accumulates.
It is worth being able to spot the moment in advance, because the two architectures are not the same thing with a different amount of effort. They are different shapes.
The Verb In The Ticket
The fastest test I know is to read the verb in the requirement and ignore everything else.
"Answer questions about our returns policy" is a chatbot. The output is a message, and the message is where the work ends.
"Watch the returns queue, spot the repeat offenders, and open a ticket" is an agent. The output is a change in state somewhere outside the conversation.
That single line predicts almost everything downstream, because every expensive part of an agent exists to make that state change safe to perform without a human watching.
The Four Layers A Chat Loop Never Needed
A chatbot is a small system. Parse the input, call the model, render the reply. One request in, one response out, and the whole thing fits in your head.
An agent adds four components that a chat loop has no reason to carry.
Tool use, so it can reach APIs, databases, file systems and browsers rather than only generating text. Persistent memory, so it accumulates across sessions instead of resetting when the window closes. Planning, so a goal decomposes into ordered subtasks with real dependencies between them. And reflection, so it can evaluate its own output, notice a failed step, and take another route instead of returning a confident wrong answer.
Remove any one of those and you have a chatbot with extra steps. This is why attaching a function call to a chat endpoint rarely behaves like the demos, the loop is missing the part that decides what to do when the call fails. The side by side breakdown of both architectures is worth reading if you want each layer laid out against its chatbot equivalent.
The Cost Model Changes Shape
The token bill is the number everyone estimates first and the one that matters least.
A chatbot's cost is roughly linear in conversations. Double the users, double the spend, and you can forecast it on a napkin.
An agent's cost is steps per goal, and steps per goal is really a function of how often it retries. A planner that picks the wrong tool on its first attempt does not cost you one extra call, it costs you every call in the branch underneath that choice. Two agents with identical prompts and identical models can differ several times over in monthly spend purely on how often their first attempt is right.
The second cost never appears on a pricing page. Once software can modify a database or send mail on its own, you need permissions, an audit trail and a way to answer what did it touch last Tuesday. Teams who skip that ship quickly and then stall, because nobody will sign off on production access for a system that cannot explain itself.
Routing Is The Real Product
The systems that work well in production are almost never pure. A chatbot front end handles the predictable majority, the FAQ, the order lookup, the guided flow, at low cost and low latency. Behind it sits an agent that picks up the requests genuinely needing several steps and real tool access.
The routing decision between the two is where the engineering actually lives, and it is the part that gets the least attention. Route too eagerly and you pay agent prices to answer questions a database lookup would have handled. Route too conservatively and users hit the wall the chatbot cannot cross, which is the exact experience that teaches people to stop trusting the feature entirely.
Worth noting that the routing layer is also the cheapest place to change your mind later. Getting it wrong inside the agent means reworking the planner.
What To Decide First
Do not start from the label. Read the verb, decide whether the output is a message or a state change, and let that pick the architecture for you.
If it is a state change, the first design question is not which framework. It is which decisions are you willing to let this thing make unattended, and what happens on the ones you are not. Answer that early and the retries, permissions and audit trail stop feeling like overhead, because they are the only reason the agent gets to run at all.
Top comments (0)