When a chatbot promises you a discount that doesn't actually exist, who's responsible: the company or the bot? A court made the company pay it anyway.
This is what should cause sleepless nights for your legal team. Not that an agent deleted a database, but that it made an unapproved promise. 😳
The story
In November 2022, Jake Moffatt hastily reserved a flight from Vancouver to Toronto. His grandmother had recently passed away.
He asked the chatbot about bereavement fares, and it told him he could book a full-price ticket immediately and then apply for a reduced bereavement rate refund within 90 days of the date the ticket was issued.
No, that was not true. The airline's actual bereavement policy was different.
Believing in the accuracy of the information provided by the chatbot, Moffatt went ahead and booked the flights, paying $1,630.36 CAD. He submitted the refund request with his grandmother's death certificate on November 17 but Air Canada rejected the claim.
The ruling that changes things
Moffatt lodged a claim with the British Columbia Civil Resolution Tribunal. His claim was successful.
On February 14, 2024, Tribunal Member Christopher C. Rivers ruled entirely in his favor and found Air Canada guilty of negligent misrepresentation. The case is Moffatt v. Air Canada (2024 BCCRT 149).
Air Canada went even further by claiming that because a chatbot is not a legal person, it cannot be held responsible for a contract.
Rivers described it as "a remarkable submission" and stated that "the chatbot is still just a part of Air Canada's website." Voilà !
Air Canada was required by the tribunal to compensate Moffatt in the amount of $812.02 CAD, which consisted of $650.88 for the difference in the fares, $36.14 in interest before the judgement, and $125 tribunal fees.
The fine was insignificant, but it set a huge precedent.
Why this is the scary agent story
All our fear reserve is dedicated to agents that perform actions. Agents that deploy code, delete data, or mistakenly send an email to 40,000 recipients.
But that's not the point here. The point is the person who actually vocalizes things.
Destroying something is like a noisy action. You notice the mistake, undo it, and then log an incident.
A promise made under a hallucination is legally unenforceable. It seems like run-of-the-mill, good-natured assistance until a client drags you into court over it.
Let's break down what was really going wrong with the Air Canada case. At its core, it was a context drift failure of Retrieval-Augmented Generation.
→ The bot retrieved the correct policy document.
→ It distorted the synthesis on the way out.
→ The output sounded confident and specific.
→ Nobody caught it because it wasn't an error, it was a sentence.
Exactly, that summarizes the issue. Your guardrails are there to look out for anomalies, not to catch a well-written incorrect paragraph.
What I'd actually change
If you are deploying a chatbot, do not think of it as a search bar. Treat it as a new employee that can sign contracts.
That's because, according to the law, it is what was previously determined by a tribunal.
Let me give you my perspective on this:
→ Any statement about money, refunds, or entitlements needs a hard-coded template, not a generated one.
→ Log every promise the bot makes, not just the errors it throws. The promises are the liability.
→ Put a verification step between "retrieved the policy" and "told the customer." Context drift lives in that gap.
→ Assume a customer will quote your bot back to you in a legal filing. Write your prompts like they will.
I'm aware of the friction explanation. Canned responses are more noticeable and less natural than fluid replies.
However, "fluent" is what ultimately cost Air Canada the lawsuit. The chatbot did not hesitate, it did not stutter, it spoke clearly and confidently about a policy that was not there.
The takeaway
Credit for bringing this pattern to light belongs in part to Suresh Kumar Ariya Gowder, whose Medium post described the hallucinated bereavement discount as a promise problem not a bug problem.
The most risky capability of your agent is not that it can take action. It is actually that it can give its word in a way that was never approved by your lawyers.
This is a question for you: Would you be aware, before the customers, if your support bot assured them that they would receive something the next day that your policy clearly states they would not receive?
Top comments (1)
This case highlights an important shift: the biggest legal risk isn’t autonomous actions—it’s unauthorized commitments.
I think the next generation of support AI needs a Policy Decision Layer, where responses about refunds, pricing, contracts, warranties, or regulatory obligations are validated against approved policy before they’re ever shown to a customer. In these cases, fluency should never outrank determinism.
Audit logs should record not only what the model answered, but also which policy version authorized that answer. If you can’t prove why a promise was made, you’re already in a difficult position.