A commercial cleaning company — 22 people, about 900 recurring accounts — found this one in a chargeback notice, which is the most expensive mailbox in business. A property manager had been invoiced $312 for a November service visit she believed was cancelled. On Nov 3 at 8:12 AM she emailed the support address with one sentence: "This invoice is wrong, I cancelled that visit, please fix it."
The AI replied in 51 seconds. It was polite. It was structured. It had three bullet points. And it did the one thing that turns a routine billing question into a formal dispute: it re-litigated the charge.
"I've reviewed the visit record and can confirm the service was completed on Nov 1 from 9:00 AM to 12:00 PM. The invoice of $312 reflects 3 hours at the standard rate of $104/hour. Cancellations require 24 hours' notice per our policy."
The customer had, in fact, called the office on Oct 30 — the cancellation was real, logged in a phone system the agent couldn't see. But the agent didn't say "let me check." It said "I can confirm," attached the policy, and defended the invoice. She replied. The agent defended again — 47 times over nine days, each reply slightly more confident, citing the same visit record like it was a court exhibit. On day nine, she filed the chargeback, attached both threads, and added: "Their bot argued with me for a week."
Then they pulled 90 days of logs. 1,204 conversations contained a billing dispute phrase — "wrong charge," "this invoice," "I didn't authorize," "refund this." In 289 of them the agent's first response took a position on the money: confirmed the charge, justified the amount, or restated policy. In 63 of those 289, the dispute was later found valid — the charge was actually wrong. The agent hadn't just failed to help those 63. It had built the paper trail the customer used against them.
The support lead's first instinct was to make the bot firmer. The second instinct, after finance weighed in, was to recognize the pattern: a billing dispute is not a support question with an answer. It's a claim with an investigation — and the agent was acting as the defense attorney for the invoice before the evidence came in.
A dispute is not a conversation. It's a case.
Every other topic the agent handles rewards speed and confidence. Disputes punish both, because the agent is answering from a dataset that — in every deployment I've audited — is missing at least one of the systems that decides who's right:
| What decides the dispute | Where it lives | Can the agent see it? |
|---|---|---|
| The invoice and its line items | Billing system | Usually yes |
| The visit / order / delivery record | Ops system | Usually yes |
| The cancellation or change request | Phone log, email, voicemail, verbal | Usually no |
| Prior credits or promises made by humans | CRM notes, call notes | Sometimes |
| The contract's actual terms | Signed PDF, not the FAQ | Almost never |
That third row is the entire story of the $312. The customer's claim was true, it was recorded in a system the agent has never had access to, and the agent answered as if the visit record were the complete evidence. In dispute conversations, confidence is not a feature. It's a liability with a timestamp — every confident reply is a sentence the chargeback team or a regulator can read later.
And it compounds worse than ordinary bugs. A wrong FAQ answer annoys a customer once. A wrong dispute stance gets quoted back: the customer screenshots the agent defending the charge, and now your own AI is exhibit A. Support's dashboard shows 1,204 "resolved" dispute conversations. Nobody joins that table to the chargeback one until the fees arrive.
Step 1: Make the dispute a case with a status
You can't gate a claim the agent treats as small talk. The first fix is structural: any conversation containing a dispute phrase stops being a chat and becomes a case record:
{
"dispute_case": {
"case_id": "disp_88410",
"account_id": "acct_33127",
"invoice_id": "inv_2026-11-0417",
"amount_disputed": 312,
"currency": "USD",
"dispute_phrase_matched": "this invoice is wrong",
"customer_claim": "service was cancelled on Oct 30 by phone",
"status": "open | investigating | resolved_valid | resolved_invalid | escalated",
"evidence_gathered": [],
"first_response_mode": "acknowledge_only",
"deadline": "first_human_or_resolution_within_24h",
"created_at": "2026-11-03T08:12:00Z"
},
"rules": {
"first_response_takes_no_position_on_money": true,
"agent_may_not_confirm_or_defend_a_disputed_charge": true,
"resolution_requires_evidence_from_all_systems": ["billing", "ops", "comms_log", "contract"],
"unresolved_after_24h_must_reach_a_human": true
}
}
The field that does the heaviest lifting is first_response_mode: acknowledge_only — it's the kill switch for the whole class of bug. The first reply to a dispute may confirm receipt, restate what the customer claims, and commit to a check. It may not confirm the charge, justify the amount, or recite the cancellation policy. At the cleaning company, that one rule alone would have prevented all 47 re-litigation messages, because none of them would have been written.
resolution_requires_evidence_from_all_systems exists because disputes are decided at the intersection the agent can't see. The visit record says "completed." The comms log says "cancelled Oct 30." Only when both are in the case do you know what the truth is — and the agent's job is to collect, not to judge.
unresolved_after_24h_must_reach_a_human is the clock. Disputes age badly: the same claim that's a refund on day two is a chargeback on day nine.
Step 2: The first-response gate
Before any reply to a detected dispute message, the agent routes:
{
"dispute_first_response_gate": {
"trigger": "inbound_message_matches_dispute_pattern",
"detect": ["wrong charge", "this invoice", "didn't authorize", "refund this",
"never received", "double billed", "dispute", "why was I charged"],
"resolve": "create_or_load_dispute_case_then_check_evidence_status",
"branches": {
"no_case_yet": "acknowledge_claim_restate_it_back_commit_to_investigation_give_deadline",
"case_open_evidence_incomplete": "state_what_was_checked_and_what_is_pending_never_take_position",
"case_open_evidence_supports_customer": "offer_correction_outright_before_customer_escalates",
"case_open_evidence_supports_charge": "escalate_to_human_never_send_defense_from_bot",
"case_resolved_valid": "reference_the_correction_and_refund_confirmation_not_the_original_invoice",
"repeat_contact_on_open_case": "give_case_status_and_deadline_never_reopen_the_argument"
},
"forbidden": [
"confirming_or_justifying_a_disputed_charge_before_evidence_is_complete",
"reciting_cancellation_or_refund_policy_at_a_disputing_customer",
"arguing_with_a_customer_who_has_repeated_a_claim",
"closing_a_dispute_because_the_customer_stopped_replying"
],
"style_rule": "the_first_reply_wins_or_loses_the_chargeback"
}
}
The no_case_yet branch is where the wording does all the work. What the cleaning company's bot said feels thorough; the honest version feels shorter and lands completely differently: "Got it — you're saying the November 1st visit was cancelled on October 30th and the $312 invoice shouldn't exist. Let me check that properly instead of guessing. I'm opening case disp_88410 and I'll have an answer — or a human — by tomorrow 9 AM. If the cancellation checks out, the invoice gets corrected today."
That's 63 words. Notice what it does that the original didn't: it restates the claim back (so the record shows the company heard it), refuses to take a position without pretending to ("check that properly instead of guessing" is not an admission), and names a deadline — which is the single strongest de-escalator in billing, because disputes get louder when they feel like they're floating.
Read the forbidden list again, because each line is a documented failure. Reciting the policy is the one agents do most: the customer says "this is wrong" and gets the cancellation terms paragraph. That reply has never once resolved a dispute. It reads as "we heard you, and the answer is no" — which is a position the company never actually took.
And closing_a_dispute_because_the_customer_stopped_replying is the quiet one: agents treat silence as resolution. Silence on a dispute is a customer writing their chargeback letter.
Step 3: Replace the three replies that generate chargebacks
| Never say | Say instead |
|---|---|
| "I can confirm the service was completed and the invoice is correct." | "Let me verify that against every system — including the phone log from October 30 — before I tell you anything." |
| "Per our policy, cancellations require 24 hours' notice." | "I'm checking whether your October 30th call was recorded as a cancellation. Policy aside, that's the fact that decides this." |
| "Is there anything else I can help you with?" | "Case disp_88410 is open. You'll have an answer by tomorrow 9 AM — here's what happens if it checks out." |
The middle row is the most expensive sentence in the log at the cleaning company — it appeared 41 times across the nine days. A policy recitation to a disputing customer doesn't just fail to help; it signals that the company considers the case closed before looking, and that's exactly the framing the chargeback narrative needs.
Step 4: Audit for defended-then-lost disputes
The bug is invisible from support's side — every conversation looks "handled" — so audit it from the join between first-response stances and final outcomes:
-- disputes where the bot's first reply took a money position, vs the final finding
SELECT d.case_id,
d.amount_disputed,
d.first_response_took_position,
d.final_finding,
d.chargeback_filed,
d.days_to_resolution
FROM dispute_cases d
WHERE d.created_at > now() - interval '30 days'
AND d.final_finding = 'resolved_valid'
AND d.first_response_took_position = true;
Every row is a chargeback that was preventable by a sentence the company already knew how to write — a valid claim, defended by the bot before the evidence arrived.
| # | Check | Method | Healthy threshold |
|---|---|---|---|
| 1 | Valid disputes whose first bot reply took a money position | Query above | 0 |
| 2 | Dispute conversations with >3 bot replies defending the charge | Transcript pattern match | 0 |
| 3 | Policy recitations sent to disputing customers | Transcript pattern match, 5 sampled/day | 0 |
| 4 | Disputes open >24h with no human touch | Case age vs assignment log | 0 |
| 5 | Chargebacks citing bot messages as evidence | Chargeback reason codes, monthly | Trend down |
Check 2 has a number attached to it at the cleaning company: 47. Whatever your version of that number is, it should be zero — a dispute defended more than three times by a bot is a company arguing with its own customer in public.
The numbers, for the internal pitch
- 63 of 289 first-response money positions were later proven wrong (21.8%) at the cleaning company. Across three dispute-heavy deployments I've audited, the range was 14–24%.
- The 47-message thread became the customer's chargeback evidence. The filing cited it directly. The bank ruled for the customer in 11 days; the $312 plus the $15 fee left the account.
-
The fix was one mode field, six branches, one join.
first_response_mode: acknowledge_onlyalone would have prevented all of it. - After the gate: 60 days, zero bot-defended disputes later found valid. Median dispute resolution time dropped from 4.2 days to 1.1 — because "acknowledge, investigate, deadline" is faster than "defend, defend, defend, escalate."
- Chargebacks citing bot messages went from 3–5 per month to zero. Total dispute volume didn't drop much — but disputes stopped becoming chargebacks, which is the part with the fee and the bank ratio attached.
The vendor shipped the gate on a Thursday. The property manager got a call from the operations lead that week — invoice corrected to zero, the chargeback withdrawn, and per the ops lead she's still a customer. That's the good ending. The ordinary ending is the 62 other valid claims, most of whose owners stopped arguing with the bot and started arguing with the bank.
If you want the working versions — the dispute case schema, the first-response gate config, the three-phrase replacement table, and the defended-then-lost audit query I run on every deployment that invoices customers — it's here: https://whop.com/agentfast-04b6/
Top comments (0)