Your AI Support Agent's Best Feature Is Knowing When to Give Up

Most teams building an AI support agent obsess over deflection rate. Wrong metric to optimize first.

The agents that actually work in production aren't the ones that answer everything. They're the ones that answer what they can confidently handle and hand off the rest cleanly, with context, before the customer gets frustrated.

A bad escalation is worse than no automation. The customer repeats their whole problem to a human, the human has zero context, and now you've added friction instead of removing it. Let's build one that doesn't do that.

Start with the escalation policy, not the prompt

Before you write a single prompt, decide what the agent is not allowed to resolve alone. This is a business decision, not a technical one.

Typical hard-escalation triggers:

• Billing disputes and refunds over a threshold
• Account security or access loss
• Legal, compliance, or cancellation requests
• Angry sentiment (churn risk)
• Anything the agent has already failed at twice

Write these down as explicit rules. You don't want the LLM "deciding" whether a refund is appropriate. You want it to recognize the category and route it.

The three signals that should trigger a handoff

1. Low confidence

Don't trust a model that says "I'm 95% sure." Models are confidently wrong all the time. Instead, gate on whether the retrieval step actually found relevant knowledge.

If your RAG pipeline returns no chunks above a similarity threshold, the agent has nothing to stand on. Escalate.

def should_escalate(query, retrieved_docs, sentiment, attempts):
    top_score = max((d.score for d in retrieved_docs), default=0)

    if top_score < 0.72:
        return "low_knowledge_match"
    if sentiment == "negative" and attempts >= 1:
        return "frustrated_customer"
    if attempts >= 2:
        return "repeated_failure"
    if is_restricted_topic(query):
        return "policy_restricted"
    return None

2. Repeated failure

Track turns. If the customer rephrases the same question twice, the agent is looping. Loops erode trust faster than a slow human reply. Hand off after the second miss, not the fifth.

3. Sentiment shift

Run lightweight sentiment on each customer message. A person who starts polite and turns short is about to escalate themselves - beat them to it. Nothing kills goodwill like a cheerful bot arguing with someone who's already angry.

Build the handoff, not just the trigger

Here's where most implementations fall apart. The trigger fires, and then... the customer gets dumped into a generic queue.

A good handoff packages everything the human needs:

async function escalateToHuman(session, reason) {
  const summary = await llm.summarize({
    messages: session.transcript,
    instruction: "Summarize the issue, what the customer wants, " +
                 "and what the agent already tried. Be concise."
  });

  await ticketing.createOrUpdate({
    channel: session.channel,
    customerId: session.customerId,
    priority: reason === "frustrated_customer" ? "high" : "normal",
    escalationReason: reason,
    aiSummary: summary,
    fullTranscript: session.transcript,
    suggestedResponse: await draftReply(session)
  });

  return "I'm connecting you with a specialist who can help. " +
         "They'll have everything we've discussed.";
}

Two things matter here.

First, the AI summary. The human agent should read three sentences, not scroll a chat log. "Customer wants a refund on order #4821, says it arrived damaged. Bot confirmed the order but can't issue refunds. Customer has photos ready."

Second, the suggested response. Even when escalating, the agent can draft what it would say if it were allowed. The human edits and sends. That turns a full handoff into a 20-second review.

Tell the customer the truth

Don't fake being human. Don't stall. The moment you decide to escalate, say so plainly and set expectations: "A specialist will reply within 30 minutes." Then make sure that's actually true.

The worst pattern is the bot that pretends to help while quietly doing nothing. Customers can smell it.

Close the loop with data

Every escalation is training data. Log the reason, the transcript, and how the human ultimately resolved it. Review weekly.

You'll find patterns fast:

• Escalations from low_knowledge_match on the same topic mean your knowledge base has a gap. Fill it, and those stop escalating.
• Escalations that a human resolves with a canned answer mean your policy is too conservative. Loosen it.
• Escalations that humans also struggle with mean the process itself is broken. Fix upstream.

Over a few weeks, this loop tightens. Deflection rate climbs because the agent gets smarter about what it knows, not because you forced it to guess.

The mental model

Think of the AI agent as a well-trained junior support rep. It handles the routine volume, knows the docs cold, and - critically - knows when a problem is above its pay grade. A junior who escalates cleanly with context is an asset. One who improvises on refund policy is a liability.

Build for the escalation first. The deflection takes care of itself.

If you're designing this kind of workflow and want the handoff logic, routing, and knowledge base wired into your actual tools, that's the work we do at Michael AI.

---

Originally published at getmichaelai.com