DEV Community

Peter Odero
Peter Odero

Posted on Originally published at atomisc.com

When an AI should stop talking and fetch a human

Originally published on atomisc.com.

Most AI customer-facing projects treat escalation as an exception path. Something to add later, once the happy path works.

That is backwards, and you can usually tell within one conversation which way round a system was built. An agent designed to hand off to a human when it should says "let me check with a colleague". An agent designed happy-path-first produces a confident paragraph with no basis behind it, because nothing in its design ever told it that not knowing was an option.

The one-line version

An AI that never escalates is not confident. It is unsupervised.

When should an AI hand off to a human: the five triggers

These are the ones worth wiring in before launch, not after the first incident.

1. Retrieval came back empty or weak

The most important one, and the one most often missing.

If the system retrieved nothing relevant from your own documentation, the honest answer is that it does not know. A model asked a question with no supporting material will still produce an answer, and that answer will read exactly as fluently as a correct one. There is no tell in the prose. The only place to catch it is at the retrieval step, before the drafting starts.

2. The person asks for a human, in any wording

"Can I speak to someone", "is there a real person there", "put me through to sales", or just growing terseness.

This one is easy to implement and easy to implement badly. The failure is matching on an exact phrase. People ask in a hundred ways and an unmatched request for a human is the fastest route to a complaint.

3. Frustration or a complaint is detected

Separate from an explicit request. A customer who is annoyed and has not yet said so is the one worth catching early, because the cost of getting this wrong is not a lost sale, it is a public review.

4. The opportunity crosses a value threshold you set

Not every enquiry deserves the same handling. Above a number you choose, a person should be involved regardless of how well the conversation is going. This is a commercial decision, not a technical one, and it belongs to you rather than to whoever built the system.

5. Anything touching legal, credit, warranty disputes or safety

Non-negotiable. These are categories where a confidently wrong answer creates liability rather than just embarrassment. An agent should not be forming a position on a warranty dispute at any hour, at any confidence level.

What a handoff has to actually do

Firing the trigger is the easy half. For an AI to hand off to a human properly, the system must do all four of these things. A handoff that does not do all four is not really a handoff.

  • Tell the customer plainly that a colleague is picking it up. Going silent mid-conversation is worse than the wrong answer, because the customer cannot tell the difference between escalation and abandonment.
  • Assign the record to the right person in the CRM, with the full conversation attached. Not an email to a shared inbox. A named owner, with context, in the system they already work in.
  • Stop replying on that thread until a human releases it back. An agent that keeps answering alongside a person who has taken over is a specific and very visible kind of embarrassing.
  • Log the reason. This is the one everybody skips, and it is the most valuable of the four.

Why logging the reason is the whole game

Every escalation is a recorded instance of something your system could not handle. Logged with a reason and counted, they stop being anecdotes and become a ranked list.

After a month you are not guessing at what to improve. You have a precise, evidence-backed inventory of the gaps in your own documentation, ordered by how often customers hit them. The most common escalation reason tells you exactly what to write down next.

That is why the monthly report worth having includes what the system could not answer. Volume, response times and booking rates tell you it is working. The failure column tells you what to do next, and it is usually the only part of the report that changes anybody's behaviour.

A reasonable target

Escalation rate should fall over the first few months as the documentation gaps get filled, then flatten out at whatever proportion of your enquiries genuinely needs a person. If it is falling toward zero, something is wrong: either the triggers have been loosened, or the agent has started answering things it should not.

The uncomfortable part

Designing escalation properly means accepting that a meaningful share of conversations will not be handled by the AI, and saying so before you buy.

Anyone selling you a system that handles everything is either not describing escalation, or has set the thresholds so loose that the escalations are not happening. The second is worse, because it looks better on the dashboard.

Sources


Atomisc builds and runs HubSpot and AI lead response for growing sales teams, with published prices. More at atomisc.com/blog.

Top comments (0)