DEV Community

Shubham Palriwala
Shubham Palriwala

Posted on Originally published at agnost.ai

Chatbot resolution rate vs containment rate: definitions, formulas and benchmarks (2026)

Containment rate is the share of AI conversations that ended without a human handoff. Resolution rate is the share where the customer's problem was actually solved. Every resolved conversation is contained, but many contained conversations are not resolved: the customer gave up, closed the chat, and called or emailed instead. Containment tells you where a conversation ended. Resolution tells you whether it worked. Measure resolution by having a judge read each whole conversation and by checking for repeat contact, and treat containment as a cost metric, not a quality metric.

This guide gives exact formulas for containment, resolution (confirmed, assumed and verified), escalation, deflection, abandonment and CSAT. It shows how Intercom, Zendesk, Ada and Dimension Labs define them in their own docs, lists the only benchmarks a primary source actually publishes, and ends with a dashboard spec. It is updated for 2026: the biggest change since 2025 is that Zendesk now splits automated resolutions into Contained and LLM-Verified tiers (introduced May 18, 2026), so "resolution" no longer means the same thing across vendors.

Disclosure: We make Agnost AI. Facts about every other product come from that vendor's own docs, pricing or product pages, checked on November 18, 2026. Confirm details with each vendor.

The formulas

Use one denominator throughout: eligible conversations, meaning conversations where a customer actually asked for help. Drop greetings with nothing after them, spam and internal tests. Ada calls these "Excluded" and leaves them out of its resolution rate.

containment_rate     = eligible conversations with no human handoff / eligible conversations
escalation_rate      = eligible conversations handed to a human / eligible conversations
                     (= 1 - containment_rate when handoff is the only way out)

confirmed_resolution = conversations where the customer confirmed the answer helped / eligible conversations
assumed_resolution   = contained conversations where the customer left without asking for more help / eligible conversations
verified_resolution  = conversations a judge says met the customer's goal, with no handoff / eligible conversations

true_resolution      = verified resolutions with no repeat contact about the same issue within N days / eligible conversations
abandonment_rate     = contained conversations that ended with no confirmation and no judged resolution / eligible conversations
deflection_rate      = help requests answered by self-service that did not become a ticket or call / all help requests
csat                 = positive ratings / rated conversations   (always report response rate next to it)
Enter fullscreen mode Exit fullscreen mode

Three points the formulas make concrete:

  • Containment minus resolution is your silent-failure pool. Conversations that stayed with the bot but did not solve anything are the ones no dashboard flags. That gap is where to look first.
  • Assumed resolution is close to containment. "Left without asking for more help" includes everyone who gave up. If your vendor's resolution number includes assumed resolutions, it is closer to containment than to verified resolution.
  • Repeat contact is the honest check. If the same customer comes back about the same issue within a few days, the first conversation was not resolved, whatever the bot dashboard says. Pick N per channel (for reference, Zendesk treats an email conversation as over 72 hours after the first email) and keep it fixed.

For the rest of the KPI set (task success, silent failures, frustration, tool errors, latency and cost per resolved conversation), see AI agent SLOs and KPIs for production. This page goes deeper on the resolution family only.

Why containment counts silent give-ups as success

Containment is computed from one event: was there a handoff? A customer who types "this is useless" and closes the tab produces no handoff, so the conversation counts as contained. So does a customer who reads a confident wrong answer, believes it, and churns a month later.

Dimension Labs says it plainly on its product page: "A conversation that ends without a handoff counts as contained, including the ones where the customer quit and called instead," and "Containment counts abandonment as success." Zendesk's own guide on automated resolution rate tells you to exclude "'Contained' interactions where the user gives up" from resolution. Ada's docs describe automated resolution as built on containment precisely because containment cannot tell satisfied customers from frustrated ones.

Containment is still useful. It is the right input for staffing and cost, because it tells you how many conversations a human did not have to touch. It is the wrong number to report as quality, and the wrong number to pay a vendor on.

How vendors define resolution

Vendor Resolution metric What counts as resolved Who decides Containment metric Billing unit Benchmark published
Intercom Fin Resolution rate Confirmed ("Ok thanks", "That helped") or assumed (customer leaves without asking for more help) after Fin's last answer Customer behavior Not separately defined in the resolutions article $0.99 per resolution Vendor-reported 76% average on fin.ai
Zendesk Automated resolution tiers Verified resolution: an LLM confirms the AI agent resolved it. Contained resolution: no further help requested, not verified LLM at conversation end Contained resolution tier Verified resolutions draw from the allowance None
Ada Automated resolution rate Relevant, accurate, safe and contained AI language understanding after the conversation Containment rate (no handoff) Not covered in the docs we cite None
Dimension Labs Not defined on the product page Joins conversations to resolution, handoff and account data Analytics on your data "True containment" ranked by effect From $299/month None
Agnost AI Your own rule Whatever your plain-language rule says, judged on every conversation LLM judge after 15 min idle Count handoff events Events Not applicable

Intercom Fin

Intercom counts a resolution after Fin's last answer in a conversation, when the customer either confirms the answer was satisfactory (a confirmed resolution, such as "Ok thanks" or "That helped") or "exits the conversation without requesting further assistance" (an assumed resolution). Both are billed. Fin's pricing page sums it up as "No further help is requested after Fin's last answer," at $0.99 per resolution, charged at most once per conversation.

The details that matter for measurement:

  • If the customer comes back to the same conversation for more help, even in a later billing period, the resolution is deducted.
  • A reply to a greeting is not an answer or a resolution.
  • If Fin asks a clarifying question and the customer doesn't reply, the conversation auto-closes with no resolution recorded.
  • If Fin says it has no answer and a teammate replies, it is not billed.

Intercom describes resolution rate as "the percentage of conversations that Fin resolves without human involvement" and does not publish a formula. Because assumed resolutions are counted and billed, ask Intercom for the confirmed and assumed split separately if you want to know how many customers actually said the answer worked.

Zendesk automated resolution tiers

Since May 18, 2026, Zendesk classifies AI agent conversations into three tiers:

  • Assisted escalation: "The AI agent contributed to the interaction before a human agent completed the resolution."
  • Contained resolution: "The AI agent handled the interaction to completion without the customer requesting further assistance."
  • Verified resolution: "The AI agent successfully resolved the interaction."

An LLM runs a verification step at the end of each conversation. Conversations that pass are Verified; those that don't are Contained. Only Verified resolutions draw from your resolution allowance. A conversation ends 72 hours after the first email, 2 hours after the first message on messaging by default (extendable to 72 hours), or on hangup for voice.

This is the clearest public example of a vendor separating containment from resolution, and it is the model to copy for your own agent.

Ada

Ada's automated resolution rate is Resolved / (Resolved + Not Resolved). Excluded conversations (spam, a standalone greeting, no request for help) are left out. After a conversation ends, Ada's AI agent uses "AI language understanding" to assess the customer's inquiry and its own responses. A conversation counts as resolved only if it was relevant, accurate, safe and contained.

Ada defines containment rate separately as "The percent of conversations that did not result in a handoff to a human agent," and CSAT as the percent of conversations rated positively out of all rated ones. Ada's docs say 100% automated resolution is not a realistic goal, because some customers ask for a human straight away and others send off-topic or abusive messages.

Dimension Labs

Dimension Labs is an analytics layer, not a support bot. It enriches each conversation with typed fields such as bot containment and AI failure mode, joins them to intent, resolution, handoff and account data, and ranks issues by "measured effect on true containment, escalation and churn." It produces scheduled reports or a live monitor and exports to BI tools. Pricing starts at $299 a month. It is the right shape of tool if you want analysts to ask questions of conversation data across several bots.

Agnost AI

Agnost AI monitors production chat and voice agents (voice via transcripts) that you build yourself, rather than a vendor bot. It has no built-in resolution score. You write resolution as a plain-language rule, for example "Resolve the customer's request, or say clearly that it can't be done and offer a next step." Once a conversation has been idle for 15 minutes, an LLM judge reads the whole conversation in a batch and marks each assistant message pass or fail against every active rule. There is no sampling: every conversation is judged up to your plan's event limit. Each user message also gets an intent and a sentiment, and Agnost flags user frustration, tool-call errors and loops.

Limits. Verdicts are pass or fail, not scores. Judging is batched after 15 minutes of idle time, not real time. Alerts go to Slack or email. Agnost does no CI gating and has no offline datasets.

Pricing. Free covers 1,000 events a month. Starter is $49 a month (10,000 events). Pro is $499 a month (1,000,000 events). Enterprise is custom.

Benchmarks: what is actually published

We looked for resolution, containment and escalation benchmarks on each vendor's own pages. Here is everything a primary source publishes:

Metric Published figure Source and caveats
Resolution rate 76% average across 12,000+ customers, "many seeing over 85%", rising "1% every month" Intercom's fin.ai homepage. Vendor-reported, undated, no method given. Intercom's billing definition counts assumed resolutions.
Automated resolution rate No industry average Zendesk's guide gives the formula and a worked example (150 of 200 = 75%), not a benchmark.
Automated resolution rate No benchmark; 100% "not realistic" Ada docs.
Containment rate No public benchmark Not published by Intercom, Zendesk, Ada or Dimension Labs on the pages we checked.
Escalation rate No public benchmark Not published by any of the above.
CSAT on AI conversations No public benchmark Ada defines it; none of the above publish an average.

So: there is no public, independent benchmark for chatbot containment or escalation, and the one resolution number is a vendor describing its own customers with its own definition. Numbers you see in listicles without a primary source are not benchmarks. Set targets from two weeks of your own traffic, per intent, and compare yourself to last month. A password-reset intent and a billing-dispute intent will never have the same resolution rate.

How to measure resolution on your own agent

If you built the agent yourself, you have no vendor dashboard. Do what Zendesk and Ada do: judge each finished conversation.

  1. Log conversations with stable IDs. Put a conversation ID and user ID on every span or event, and send an explicit event when a handoff happens. Containment and escalation come straight from those events.
  2. Write the resolution rule per intent. "Resolved a billing question" and "booked a meeting" need different definitions. Keep each one to a sentence a support lead would agree with. How to track intent resolution rate covers writing these.
  3. Judge whole conversations, not single messages. A reply can look fine on its own and still leave the customer stuck three turns later. Run the judge once the conversation is finished. In Agnost AI, that is a rule in online evals; count a conversation as unresolved if any assistant message fails the resolution rule.
  4. Split the three resolution types. Report confirmed (the customer said so), assumed (they left) and verified (the judge says the goal was met). The gap between assumed and verified is your silent-failure rate.
  5. Join repeat contact. Match the same user coming back about the same intent within N days, in chat, email or phone. Subtract those from resolved.
  6. Calibrate weekly. Read 30 to 50 conversations by hand, label them, and check the judge agrees. See how to calibrate an LLM judge.
  7. Alert on change, not on a fixed number. Alert when a rule's failure count jumps after a prompt or model change. Agnost sends these to Slack or email, and with Auto Improve connected, a fired alert opens a pull request with a fix tested against the affected past conversations, for a person to review.

A resolution dashboard spec

One page, weekly by default, every panel filterable by intent, channel and agent version.

Panel Metric Chart Why it's there
Headline Verified resolution rate, containment rate, and the gap between them Three numbers with last week's value The gap is the silent-failure pool
Resolution split Confirmed / assumed / verified, stacked Stacked bar by week Shows how much of "resolved" is just "left"
By intent Verified resolution rate per intent, with conversation count Sorted bar, worst first The lowest intent with real volume is the roadmap
Escalations Escalation rate, and share of escalations judged late, missing context or unnecessary Line plus number Raw escalation rate has no good direction; bad handoffs do
Abandonment Contained conversations with no confirmation and no judged resolution Line Rises before CSAT moves
Repeat contact Same user, same intent, within N days Line The honest check on resolution and containment
Frustration Conversations with frustration signals Line Leading indicator of unresolved conversations
CSAT Positive / rated, with response rate Number plus response rate Useful only with its response rate next to it
Evidence The five most recent failed conversations for the worst intent Linked list So someone reads them

Keep the panels that come from events (containment, escalation, repeat contact) separate from the ones a judge produces (verified resolution, abandonment, frustration), and label which is which. Judged metrics arrive later and carry judge error. For behavioral signals beyond these, see how to measure AI chatbot performance.

How to choose

If you… Use
Run Intercom Fin Fin's resolution reporting, and ask for confirmed vs assumed separately
Run Zendesk AI agents The Verified resolution tier as your quality number, Contained as cost
Run Ada Automated resolution rate; it already requires relevant, accurate, safe and contained
Have several bots and want analysts to query conversation data Dimension Labs
Built your own agent and want every conversation judged against your own definition of resolved Agnost AI, with handoff events for containment
Need resolution as part of a full SLO set with alerting Start from the SLO and KPI framework

Whatever you use, report verified resolution and containment side by side, never containment alone.

Related reading

Frequently asked questions

What is the difference between resolution rate and containment rate?

Containment rate is the share of AI conversations that ended without a handoff to a human. Resolution rate is the share where the customer's problem was actually solved. Every resolved conversation is contained, but a contained conversation can still be unresolved: the customer may have given up, left, and called or emailed instead. Containment measures where the conversation ended; resolution measures whether it worked.

How do you calculate AI chatbot resolution rate?

Divide resolved conversations by eligible conversations, meaning conversations where the customer actually asked for help (exclude greetings, spam and tests). Decide up front what counts as resolved: confirmed by the customer, assumed because they left without asking for more help, or verified by a judge reading the conversation. Report the three separately, and check for repeat contact about the same issue within a few days before you count a conversation as resolved.

What is a confirmed resolution versus an assumed resolution?

In Intercom Fin's definition, a confirmed resolution is when the customer says the answer was satisfactory after Fin's last answer, for example 'Ok thanks' or 'That helped'. An assumed resolution is when the customer exits the conversation without requesting further assistance. Intercom bills both at $0.99 per resolution, and deducts the charge if the customer comes back to the same conversation for more help.

What is a good containment rate for an AI chatbot?

There is no public, independent benchmark for containment. Intercom, Zendesk, Ada and Dimension Labs do not publish a containment benchmark on their docs or product pages. Containment alone is also a weak target, because it counts customers who quit as successes. Set a target from your own baseline, per intent, and track it next to resolution and repeat contact.

What is a good resolution rate for an AI customer support agent?

The only published figure we found is vendor-reported: Intercom says Fin averages a 76% resolution rate across more than 12,000 customers. The page does not give a date or method, and Intercom's billing definition of resolution includes assumed resolutions. Zendesk's own guide on automated resolution rate gives a formula but no industry average. Treat any single number as a sanity check, not a target.

What is a good escalation rate for an AI chatbot?

No primary source publishes an escalation benchmark. Escalation rate has no good direction on its own: too low can mean the bot traps people who need a human, too high means it is not doing its job. Track the share of escalations that were late, missing context or unnecessary instead, and alert on sudden changes.

Is deflection rate the same as containment rate?

They are often used interchangeably, but they measure different things. Containment counts chat conversations that ended without a human handoff. Deflection usually counts support requests that never became a ticket or a call because self-service handled them. Both share the same flaw: a customer who gives up and does not come back is counted as a win.

How do I measure resolution automatically without surveys?

Have an LLM judge read each finished conversation and decide whether the customer's goal was met, then join that verdict to repeat-contact data. Zendesk and Ada both use an LLM or AI language understanding to verify resolutions, and you can do the same on your own agent with online evals. Calibrate the judge against a weekly sample of conversations a person has read.


Originally published on agnost.ai.

Top comments (0)