DEV Community: Mike Wei

Your AI resolution rate went up and CSAT went down. That's not a tradeoff — it's a measurement bug.

Mike Wei — Wed, 19 Aug 2026 18:01:50 +0000

Disclosure: I work at Aissist.io, which builds AI agents for customer support. This is adapted from a longer piece on our site. The argument stands on its own; judge it on the logic.

Every support team that deploys an AI agent eventually hits the same wall. Resolution rate climbs from 40% to 60% to 75%, the dashboard turns green, and then CSAT starts sliding. The conclusion looks obvious: automation and satisfaction pull against each other, so pick a number you can live with and stop pushing.

That conclusion is usually wrong. What most teams are looking at isn't a tradeoff curve. It's two different curves that happen to overlap on the left side of the chart, and almost everyone is standing on the worse one without knowing it.

Two curves, not one

The capability curve. Resolution goes up because the AI genuinely got better at handling more intent types. CSAT rises with it — customers get correct answers instantly instead of waiting four hours for a human to say the same thing. On this curve, satisfaction tends to peak somewhere in the 60–80% resolution range and then declines gently, because the last 20% of tickets are genuinely the ones that need judgment.

The deflection curve. Resolution goes up because the path to a human got harder to find. The number on the dashboard rises identically. CSAT doesn't decline gently — it collapses.

The vertical gap between those two curves is the deflection penalty, and it's the thing people mistake for a law of physics.

Here's the part that makes this hard to see from the inside: CSAT is a hill, not a ramp. It's low at both ends. Under-automate and people wait too long. Over-automate and people get trapped. If you only know your CSAT is falling, you can't tell which end you're on from the metric alone.

Where the deflection penalty comes from

Six failure modes, and most deployments have several running at once:

Suppressed escalation. The handoff exists but is buried, rate-limited, or gated behind three "are you sure?" prompts.
Overreach. The agent attempts regulated, high-stakes, or genuinely ambiguous issues it has no business attempting.
Confidently wrong answers. The ticket closes. The problem doesn't.
False resolutions. A customer gives up and closes the tab. The system logs a win.
Effort and looping. Three rephrasings of the same question before anything works. Customers rate effort, not accuracy.
Lost empathy on edge cases. The people with the worst experiences are disproportionately the ones who fill out surveys.

Deflection is not resolution

This is the crux. Deflection counts any conversation that didn't reach a human — including the ones where the customer rage-quit. Genuine resolution excludes escalations and verifies the ticket actually stayed closed.

For the same deployment, those two numbers can differ by 20–30 points. So when a vendor quotes you a resolution rate, you're not looking at a performance figure until you know which definition produced it. Half the "AI CSAT problem" discourse is people comparing a deflection number to a satisfaction number and drawing a causal arrow between them.

Three design choices that break the tradeoff

Confidence-gate the automation. Auto-resolve only high-confidence intents. Route ambiguity to a human by default. The instinct is to push the confidence threshold down to lift the resolution number — that's the exact move that slides you onto the deflection curve.

Make escalation instant and graceful. Treat a handoff as a feature, not a failure. Pass full context so the customer never repeats themselves. A fast, clean escalation produces a higher CSAT than a mediocre AI answer, which means the honest version of your funnel is also the better-performing one.

Measure genuine resolution. Exclude escalations. Verify the ticket stayed closed inside a defined window (72 hours is a reasonable default). Measure CSAT specifically on AI-handled tickets, not blended across every channel — blending is how a struggling AI hides behind a good phone team.

Five questions for any vendor

Does your "resolution" metric include escalations or workflow handoffs?
Are re-opens within 72 hours counted as resolution failures?
Is CSAT measured on AI-handled tickets, or blended across all channels?
Is resolution defined consistently across channels?
Can escalation rate and resolution rate appear on the same report?

That last one is the tell. If the two numbers can't sit next to each other, someone has decided you shouldn't see them together.

The takeaway

Before you accept that automation costs you satisfaction, check which curve you're on. Recompute resolution with escalations excluded and re-opens counted as failures, then look at CSAT on AI-handled tickets only. If genuine resolution comes in well below the deflection figure, you don't have a tradeoff to manage. You have a measurement problem to fix — and fixing it usually moves both numbers in the right direction at once.

I compared what 18 AI support vendors actually charge per resolved conversation. The spread is 25 .

Mike Wei — Wed, 19 Aug 2026 17:37:01 +0000

I compared what 18 AI support vendors actually charge per resolved conversation. The spread is 25×.

Disclosure: I work at Aissist.io, which is one of the 18 vendors in this benchmark. All figures come from public pricing pages (verified July 2026) or third-party procurement data. Full methodology and the complete table are in the original writeup.

Headline prices for AI support agents cluster politely between $0.50 and $2.00. That number is close to useless. The only figure that matters is effective cost per resolved conversation — total spend divided by conversations actually resolved, including seats and platform fees.

Once you normalize for that, the range runs from about $0.13 to $3.33+.

The pricing model matters more than the rate

Five models are in play:

Per resolution — you pay when the AI finishes the job. Everything hinges on the definition of "resolved."
Per conversation — you pay whether or not it worked. The vendor gets paid for failures.
Per session — one real issue often spans 3–4 billable sessions.
Flat per ticket — cheap, but it's an add-on layer; your helpdesk bill continues underneath.
Enterprise custom — platform fee plus negotiated usage, typically $50K–$600K+/year.

At a 60% resolution rate, a $2.00 per-conversation fee is $3.33 per actual resolution. The headline lied by 67%.

What 10,000 monthly conversations at 70% resolution actually costs

Vendor	Est. monthly	Effective $/resolution
My AskAI	~$1,000 + helpdesk	~$0.14 + helpdesk
eesel AI	~$4,000 + helpdesk	~$0.57 + helpdesk
Aissist.io	~$4,200	~$0.60
HubSpot Breeze	~$4,400	~$0.63
Intercom Fin	~$7,780	~$1.11
Gorgias	~$10,000+	~$1.43
Decagon	~$14,070	~$2.01
Zendesk AI	~$14,500	~$2.07
Sierra	~$15,000–20,000	~$2.14–2.86
Salesforce Agentforce	~$22,000+	~$3.14+

Same 7,000 resolved conversations. Over $200,000/year of difference.

The fine print that moves the number

Assumed resolutions — Intercom Fin bills when a customer stops replying, which includes the ones who gave up.
Double billing — Gorgias charges an AI-resolved conversation as both a ticket and a resolution unless it's escalated within 72 hours.
Session inflation — Freddy's per-session rate looks like $0.10–$0.49, but published resolution rates of 23–30% mean 3–4 sessions per real fix.
Uncapped overages — Zendesk has auto-billed everything above committed volume at full rate since January 2026, no cap, no warning.

Six questions to ask any vendor

What exactly triggers a charge? Get it in writing.
What happens when the AI fails — do you still pay?
Is there a floor (minimums, annual commitments)?
Is there a ceiling on overages?
What's the all-in stack: seats, platform, onboarding, helpdesk underneath?
Who audits the resolution rate? If the vendor grades its own homework, ask for independent reporting.

For context: a human-handled ticket runs $6–$12. Most of these are still a bargain. But normalize every quote to cost per resolution first — then ask who's counting.

Nothing here is a quote or a contractual commitment. Verify with the vendor before signing.

A benchmark tool for AI customer service — would love this community to poke holes in it

Mike Wei — Sun, 12 Jul 2026 18:01:15 +0000

How is your AI support actually doing?

Not the number in the vendor's slide deck. Not the demo. Yours — on your real traffic, this quarter.

If you run AI customer service, you've probably tried to answer some version of these four questions:

→ Is it truly resolving tickets, or just deflecting them until the customer gives up or a human quietly cleans it up?
→ What's the ROI really looking like once you net out the cost of the platform, the setup, and the escalations?
→ Is there still room to improve — or are you already near the ceiling for the kind of traffic you handle?
→ And how do you actually compare with everyone else running the same play in your industry?

Here's the uncomfortable part: almost nobody can answer these with confidence. And it's not because support leaders aren't paying attention. It's because the ground truth doesn't exist in any shared, honest form.

Start with the word "resolution." It sounds precise. It isn't. One vendor counts a ticket as resolved the moment the bot sends a reply. Another counts it only if the customer never writes back. A third quietly folds deflection — the customer bouncing off a help article — into the same bucket and calls it automation. So when you see "85% resolution" on a landing page, you have no idea whether that's a genuinely closed issue or a cleverly drawn chart. The definition bends to flatter whoever is publishing it.

That makes comparison almost impossible. If you're sitting at 60%, is that world-class for your vertical, or embarrassing? You genuinely can't tell — because there's no shared denominator, no neutral reference point, and every benchmark you can find was published by someone with a reason to make their own number look good.

That lack of transparency is the problem we set out to close.

We built a benchmark tool, and we deliberately backed it with two different kinds of data:

Public data — independent 2026 cross-program aggregates, pulled from sources that aren't trying to sell you anything.
Private data — real field deployments, where we can see what actually happens once a system is live on messy, real-world traffic.

The reason we use both matters. Public data keeps us honest and broad. Private data keeps us grounded in what really happens after launch — not the pilot, not the demo, but month six, when the easy intents are automated and the hard ones are all that's left.

Here's how it works. You pick your industry — ecommerce, fintech, SaaS, travel, healthcare, telecom, and so on — and the tool shows you the realistic resolution and CSAT range for that vertical. Not a single vanity figure, but a band: the low end, the median, and the top-10% world-class mark. Then you can enter your own current resolution rate and CSAT, and it places you directly on that scale. In about ten seconds you can see whether you're lagging, sitting at the median, or genuinely at the front of your industry.

And because a number without context is just anxiety, the tool also breaks down the four factors that decide where any given deployment actually lands:

→ Industry complexity. How ambiguous, regulated, emotional, or multi-step your intents are. This sets the ceiling. Structured, data-rich intents like order status resolve high; regulated or emotional ones resolve far lower, no matter which vendor you use.

→ System capability. How strong the underlying platform is — multi-agent reasoning, backend actions, retrieval, guardrails. A more capable system reaches higher within the same industry band and holds quality as complexity rises.

→ Maturity of your assets and playbooks. The state of your knowledge base, SOPs, and escalation rules. Most deployments launch around 40–50% and climb past 60% over six to twelve months as those assets mature. If you're new, low isn't failure — it's the starting line.

→ Transparency of your data. How accessible the orders, accounts, and history are that the AI needs to actually resolve a ticket. When the system can see everything, it resolves. When data is siloed, even simple intents stall.

The honest caveat: these ranges are directional, not audited. Definitions of "resolution" vary by source, so we treat the numbers as a realistic target band — something to steer by — rather than a certified figure. We'd rather tell you that plainly than pretend to a precision the whole industry can't actually deliver yet.

This is a living benchmark. We'll keep updating it as more public data lands and more deployments mature, and we expect the ranges to sharpen over time. If your team has been looking for a bit of honest transparency in a space that badly lacks it, we hope this gives you a real reference point to plan against — whether or not you ever talk to us.

Try it here: https://aissist.io/tools/benchmark

And I'd genuinely value your feedback. If the range for your industry feels off compared to what you're seeing in your own numbers, tell me — that's exactly the kind of signal that makes the next version better for everyone.

The fix for bad AI isn't a bigger brain. It's better structure.

Mike Wei — Thu, 09 Jul 2026 23:30:25 +0000

Most AI support bots top out at 30-50% resolution and get stuck around 3.5/5.0 CSAT.

Customers describe them with one word: robotic.

We kept hitting the same wall, and the reason turned out to be structural. A single generic model tries to force every refund, every connectivity issue, every warranty claim, every shipping question through one path. But real business problems aren't linear. They're full of ambiguity, overlapping systems, and incomplete information. Rigid decision trees and one big model both break the moment real complexity shows up.

So we stopped trying to build one model that knows everything.

Instead: divide and conquer. A sub-agent is a specialist focused on one domain — refunds, connectivity, shipping, warranty, finance, product damage. On each execution, a super agent plans the work, activates the right specialists, gathers facts from them, and makes the final call. Multiple sub-agents can contribute to the same request and cross-check each other — which is exactly why the output is more reliable than any single-agent system.

The results across Q1 2026 deployments:

83% resolution (vs. the 30-50% ceiling)
4.8/5.0 CSAT (vs. ~3.5)

And because each sub-agent is measurable — traffic, resolution, NPS, CSAT at the specialist level — you don't just learn whether the AI works. You learn about your own products, customers, and processes.

The counterintuitive part: running multiple agents per request should cost more than a single model. After continuous engine optimization, it's often more cost-effective than single-model alternatives — while doing deeper reasoning.

The lesson we keep relearning: reliability in AI doesn't come from a bigger model. It comes from better structure.

How the architecture works: https://aissist.io/technology/multi-agent-platform

The "Moore's Law for AI" everyone promised you? It didn't happen.

Mike Wei — Thu, 09 Jul 2026 23:24:39 +0000

Everyone building AI agents is about to learn the same expensive lesson: capability and economics are two different problems.

I've spent 15+ years building large-scale ML systems, and here's the uncomfortable math behind agentic AI in 2026:

→ Token prices haven't fallen in a straight line. The cheapest models have flattened out, and flagship pricing turned upward again this year (GPT-5.4 launched at 2x GPT-5's input price).

→ Reasoning tokens are billed as output tokens — deeper thinking raises cost even when the visible answer stays short.

→ One agentic request expands into planning, retrieval, verification, and tool calls. More useful work, way more tokens.

Betting your unit economics on "models will get cheaper" is not a strategy. Power constraints alone make that assumption shaky — AI demand is now colliding with the electricity grid, not just GPU supply.

What actually works is treating token efficiency as an architecture problem:

System design is the strongest lever. A system tightly coupled to the problem spends less because it already knows the workflow, the boundaries, and the next step. Generic agents pay a "rediscovery tax" on every single run.

Task optimization comes second. Routing, classification, extraction — these don't need frontier reasoning. The first big savings usually come from stopping the model from doing work it never needed to do.

Custom models come last, and only at scale. Trade a recurring inference bill for a predictable training cost — but only when the task is stable and you have real evaluation discipline.

This is why we treat cost as a core design constraint at Aissist, not a cleanup task for later. AI that can't scale economically fails the business case no matter how smart it is.

Full analysis: https://aissist.io/technology/token-efficiency