Most AI projects inside a business start the same way. Someone ships a chatbot, it demos well, and then the real questions arrive. Can it actually update the order? Can it tell us which tickets are going to escalate? What happens when it does not know the answer?
None of those are chatbot problems. They are the other three layers of a stack nobody plans for up front, and the order you build them in decides how much of the early work gets thrown away.
The Conversation Layer Is The Easy Part
Retrieval over your own documents, a configurable personality, an embeddable widget, voice in and voice out. This used to be the whole project. It is close to commodity now, and that is a good thing, because it means the interesting decisions move somewhere else.
The trap is treating the conversation layer as the product. A chatbot that can only answer is a search box with better manners. What separates it from a FAQ page is whether it can reach into the systems behind it and change something.
Automation Is Where The Value Actually Lands
The second layer is a workflow engine: call an API, branch on the result, loop over a list, write to a database, run on a schedule. Boring infrastructure, and it is the part that turns an answer into an outcome.
This is also where the honest cost estimate lives. Wiring a model to a workflow is straightforward. Deciding what it is allowed to do without a human, and what happens when a step fails halfway through, is the actual engineering. Build the workflow layer with retries and a run history you can read, or you will be debugging by screenshot for the next year.
Prediction Is Not The Same As Generation
This is the layer teams skip, and it is usually the cheapest win available.
A lot of what gets sent to a language model is not a generation problem at all. "Which team owns this ticket", "is this order likely to be refunded", "does this reading look abnormal" are classification, regression and anomaly detection. They have had good answers for twenty years, and those answers run locally in milliseconds for no per request cost.
The practical difference shows up at volume. A language model call to sort a ticket costs money every time, varies between runs, and cannot easily tell you why it chose what it chose. A classifier trained on a few hundred of your own labeled rows costs nothing per call, returns the same answer for the same input, and points at the training rows that shaped it. Our own platform overview covers how the prediction layer sits next to the generative one rather than underneath it, with eighteen algorithm types available for the jobs where a model beats a prompt.
Use the language model for the open ended work. Use a trained classifier for the repeated judgement calls. Most stacks get this backwards because the language model was already there.
The Human Handoff Decides Whether People Trust It
The last layer is the least glamorous. When the system does not know, or the customer asks for a person, somebody has to pick up the thread with the full history already loaded, from whatever channel the conversation started on.
Get this wrong and every other layer inherits the blame. Users forgive an AI that says it is not sure and passes them to someone who already knows the context. They do not forgive being asked to explain the whole problem again to a human who cannot see the last ten minutes.
The Takeaway
Build the conversation layer because it is cheap and expected. Build the automation layer because it is where an answer becomes an outcome. Build the prediction layer because it quietly removes the most repetitive work in the building. Build the handoff because it is what makes people willing to use the other three.
The order matters less than the honesty about which layer a given problem belongs to. Most of the frustration with business AI comes from asking one layer to do another layer's job.
Top comments (0)