A user asks a question that support has answered before. An agent reads the message, clarifies the details, finds the right instructions, and explains what to do. A little later, another person asks something similar. For them, the problem is new. For the team, it is another trip down a familiar path.
This is an illustrative scene, but it captures exactly the routine we wanted to automate in our support operation. The goal was practical: answer standard user requests as automatically as possible and reduce support costs in the process. When a request already has a clear procedure, you want a person to step in where their knowledge or authority is actually needed.
That led to a service connecting language models to support work. Messages arrive from Telegram and Discord, while agents manage conversations in Chatwoot. But the most interesting part turned out to be what happens between the incoming question and the outgoing reply: what knowledge the model receives, what decision it returns, and what the service does with that decision.
At first, this path did not end with a message to the user at all. The assistant left suggestions in private notes visible to agents. We could see how it handled a request before giving it permission to respond on its own.
This is the first article in a series on automating support with AI. We will follow a single request from context assembly through an LLM call, result validation, and the choice of a next action. Later parts will look more closely at the knowledge base and conversation history, model tiers and processing costs, human handoff, and protocol failures. Chatwoot and the messaging platforms provide a concrete example; the core decisions also apply to other support systems.
The conversation already lives somewhere
Users write wherever it suits them. In our case, that means Telegram or Discord. Agents, meanwhile, need to see the request, its history, and its current state. Chatwoot handles that part.
The integrations deliver messages and connect them to the right conversation. Processing begins after that: is an automated response allowed right now, what information should go to the model, and what should happen to its result? In autonomous response mode, a Chatwoot agent-bot event triggers this processing. The trial mode starts from the channel integration and saves the result as a private note. These are alternative ways to trigger processing.
Looking inside an autonomous response gives us this flow:
Telegram / Discord message → Chatwoot conversation
↓
processing eligibility checks
↓
history + knowledge base + response rules
↓
LLM call at T1
↓
parse result and select action
↙ ↓ ↘
reply next tier human agent
Each subsequent tier also returns a result that needs parsing. Sending a message is just one possible outcome. Chatwoot remains the agent's workspace, with the history, request state, and means to continue the conversation already in place.
A human agent saw the first result
In shadow mode, the assistant receives the message, the knowledge base, and the available conversation history. It then proposes a response or a decision to hand the request to a person. The result is saved in Chatwoot as a private note. The user cannot see it, and sending a reply remains the agent's responsibility.
That is how we started. Before allowing the assistant to speak on behalf of support, we gave it a place beside the person already handling the conversation.
For an engineer, this offers a useful observation point. You can open a request and see what the assistant would have suggested in that context: did it find the right information, understand the question, or start answering where a person was needed? Alongside the result, you can see the processing tier, token usage, and cost, if the provider returned it.
The presence of a note does not, by itself, mean the answer was good. The agent may not have read it, may have used just one sentence, or may have resolved the issue in a completely different way. To measure suggestion quality, we need to agree separately on how to evaluate it. A count of generated notes is no substitute.
Still, this starting point has an important practical benefit: we can examine the assistant's work on support requests without turning every generation mistake into a message to the user. Data access restrictions and the terms under which context is sent to a model remain part of the system's design. A private note changes who can see the result; it does not eliminate the external service call.
Assemble the task before calling the model
A message saying “it isn't working again” rarely tells you exactly what happened. Earlier in the conversation, the user may already have described the problem and the agent may have asked them to run a check. If the model receives only the latest sentence, it has to guess at part of the conversation.
The service fetches history from Chatwoot. It includes user messages and public replies, while excluding private notes. Among other things, this prevents a trial response from the assistant from automatically returning in the next request as though it had already been part of the conversation with the user.
The second component is a knowledge base from Confluence. It contains information needed for answers, alongside separate instructions about when the bot should respond and when it should hand the request to an agent. These materials serve different purposes. Knowing a procedure does not mean the assistant has permission to make every decision associated with it.
In the path examined here, the knowledge base text is loaded and cached inside the service process. Connecting Confluence does not, on its own, turn the system into vector search: what matters is which data actually enters the request. History length, knowledge base completeness, and update frequency affect both answer quality and the amount of context you pay for.
That is why I would also begin work on such an assistant by looking at our own documentation: does the team have a sufficiently clear answer to a typical request? If the instructions contradict themselves, the model may faithfully restate that contradiction. A more expensive tier cannot create a missing product rule.
An LLM call must end in an actionable decision
For autonomous operation, a string containing well-written text is not enough. The service needs to know what to do next. The model's result therefore contains an action and the data needed to carry it out.
The code provides four options:
| Decision | Service action |
|---|---|
| Reply to the user | Send the prepared message |
| Hand off to an agent | Initiate a human handoff |
| Reply and hand off | Send the message and continue the handoff |
| Request the next tier | Repeat processing with the next model configuration |
This contract makes behavior inspectable: we can see which command arrived and which handler executed it. A restricted command list does not guarantee the right choice, however. The model can still propose an inappropriate response. Code checks, prompt rules, and quality evaluation address different parts of that problem.
There is another step between the API response and command execution: the service must parse the JSON and interpret its fields. This is where a tidy diagram meets everyday integration work. A response may be invalid JSON, or valid JSON with an unexpected structure. Those are different situations, and they are handled differently.
For broken JSON, there is a separate request to repair the format. It receives the previous call's output, without the knowledge base or conversation history: its job is to restore the structure. If the JSON is readable but the command is expressed differently, the handler tries to recover the action from known fields. This tolerance has a cost: a recovered command can result in a message being sent to the user. Recovery rules therefore deserve as much attention as the normal response path.
When to call the next model
There are three processing tiers. Each can be configured with a separate model. It is also possible to use one model with different reasoning depths, which is what the defaults do. It is therefore more useful to think of T1, T2, and T3 as processing configurations than as a mandatory list of three different providers.
The reasoning behind a cheap-to-expensive strategy is straightforward: a routine question may need only the first call, while additional resources should be spent where they can help. In the current loop, however, a specific action triggers the transition: call_next_model. A numerical score is recorded, but it does not itself act as a switching threshold. If the first tier decides to answer or call a human agent, the loop ends there. If the third tier requests another tier, the request goes to a person.
This is an important boundary: there is a limit to how long computation can continue. And needing a human agent does not always mean the model was insufficiently capable. Sometimes the answer requires access or a decision the assistant simply does not have.
After handoff, the bot must yield the conversation
When an agent takes over, writing a log entry saying processing has finished is not enough. The user needs to understand what is happening to their request, and the agent needs to know why it was handed over and where to pick up the work.
In this service, handoff includes notifying the user, adding a private note for the agent, and changing the conversation state. Separate checks take the handoff label and participation of a human agent into account. Their purpose is to stop the assistant from continuing to answer over the person.
This is now a distributed-system state problem. Events arrive through a webhook, a message can be repeated, and an agent can join between two processing steps. Message deduplication exists at the entry point, but that mechanism alone does not prove that all possible overlaps have been eliminated. Handoff needs to be treated as a scenario of its own, with its own checks.
This is exactly where the simple picture of “receive text, call a model, send a response” stops explaining the service. We need to account for who is currently handling the conversation and whether a decision made a few seconds ago is still valid.
Savings begin with the cost of the whole request
We came to this problem to automate routine work and reduce costs. Looking only at the price of the first request would therefore be insufficient.
One support request may end after T1. Another may pass through several tiers. A third may need an additional call to repair the response format and still end up with a human agent. The code collects token usage, duration, and available call costs, but human work falls outside the model's meter.
To understand the outcome, we need a wider view: was the issue resolved, did the user have to contact support again, how long did they wait, and how much work remained for the agent? Cheap generation is useful until the cost of fixing its consequences outweighs it.
I will not claim a percentage saving in this article; that conclusion requires comparable data. What matters here is showing a design that lets us gradually automate standard requests and examine what happens at each step.
For us, that path began with a fairly modest result: a note the user never saw. Autonomous replies followed, with a limited set of actions and a way to hand the conversation to a person. Around the model call, the main engineering work gradually revealed itself: context, conversation state, routing, and result validation.
In the next part, “Context for AI in Support: Knowledge Base, History, and Response Rules,” we will begin with a short user message that is not enough on its own to resolve the issue. We will explore how to assemble the necessary information around it, what to leave out of the request, and why more context does not automatically make support better.
I also wrote about setting context and task boundaries in “How to Give AI a Task in an Existing Project”.
Top comments (0)