A model that writes a fluent, on-topic reply is not impressive anymore. That part is basically solved. If the hard problem in a conversational agent were generating good replies, we would all be done.
I build an autonomous chat agent for creator agencies. It runs the top of a sales funnel on real messaging channels: open a cold lead, hold a real back-and-forth over days, work out who is actually going to buy, and bring in a human when it counts. What I did not expect is how little of the difficulty lives in any single reply. Almost all of it lives in the things that only exist across a long conversation.
Two of them turned out to be the whole game, and both are long-conversation problems, not single-message ones: staying the same believable person for days, and reading who is actually going to buy from how they behave over time.
Staying in character for days
One reply in character is easy. The two-hundredth reply, on day 3, still sounding like the same person who sent the first one, is not.
An agent running on someone else's brand cannot drift into generic-assistant voice. It has a specific tone, specific boundaries, specific things it would and would not say. Holding that steady across a long conversation is a different problem from producing one good line. Tone slips, earlier context falls out of view, it contradicts something it said yesterday, and the illusion breaks the moment it does.
That is the part the demos never show. A first message that nails the voice proves almost nothing. The real question is whether it is still the same person a hundred messages and 3 days later, and that is where most agents quietly come apart.
Knowing who will actually buy
The second problem is knowing who is worth a human's time, and you cannot get it from keywords.
The naive version watches for phrases like "how much." It barely works. The people most likely to buy rarely announce it, and the ones who ask the price first are often the ones who never pay. Whatever signal exists is in behavior across the whole conversation: whether someone gives real answers or one-word replies, whether their responses get more substantive, how consistently they show up across days.
None of that is legible from a single message. The thing you actually care about, real intent, is a property of the whole conversation, not any turn inside it. A smarter individual reply does not help you see it.
Both problems have the same root
Put those two side by side and they are the same shape. Persona coherence and reading intent both fall apart the instant a conversation is treated as a stream of independent replies. Neither one lives in a message. Both live in the accumulation: who this person is, what has already been said, where the whole thing is heading.
That is the uncomfortable part of building here. The unit that matters is the entire conversation, and a conversation is long, messy, and spread across days. The single turn is the one piece that is genuinely easy, and it is the one piece everyone benchmarks.
The handoff is not the hard part
There is a lot of hand-wringing about when an agent should stop and pass the conversation to a human. In practice that is the least of it. By the time a conversation is truly ready for a person, the hard part is already behind you: the character held, the intent became readable, the context is all there. The handoff is just acting on what the conversation already told you.
It only looks hard if the rest went wrong. If the persona drifted and the intent was never really read, then deciding when to bring in a human is a guess. When the long-conversation part holds up, that moment is obvious.
Where this goes
None of this feels solved to me, and I do not think it is close. The reply was the first 10%, and it is the 10% the whole field benchmarks. The interesting work is in the other 90%: keeping an agent coherent, honest about who is actually going to buy, and consistent as a single character across a long, real, human conversation.
That is the problem we built ChatSan to solve.
Top comments (1)
I appreciated how you highlighted the challenges of maintaining a consistent persona across a long conversation, as this is indeed a crucial aspect of building a believable autonomous chat agent. The point about tone slips and earlier context falling out of view resonated with me, as I've experienced similar issues in my own projects. I'm curious to know more about how you've addressed these challenges in your implementation, particularly in terms of using context retention mechanisms or persona modeling techniques to ensure consistency over time. Have you explored the use of any specific architectures or algorithms to mitigate these issues?