Originally published at ictdesk.net
A clinic put a chatbot on its website to book appointments and answer questions. Someone asked whether its doctors were specialists. The bot said yes and named two qualifications that neither doctor actually held. Nobody typed that claim. The model produced it.
On 12 May 2026 the Higher Regional Court of Hamm decided that the clinic was on the hook for it anyway, in case 4 UKI 3/25. If you run live support software with any AI answering in front of customers, that is the ruling to read this month.
What the court actually decided
The reasoning is short and uncomfortable. The chatbot is not an independent third party. It is a tool the operator chose, deployed and controls. So its statements are attributed to the operator directly, and the misleading answer counted as an unfair commercial practice.
The court went further on the point most teams would have leaned on. Liability applies even where the model hallucinates, and even where the operator configured things carefully. The reason given is that the operator defines the bot's scope and can change it at any time. That control is what makes the words yours.
Three defences went in. None came out.
Worth keeping in proportion: this is a German higher regional court, and an appeal to the Federal Court of Justice was admitted, so it may yet move. A related Munich decision pushed the same way on AI generated summaries. My view is that you plan as though it stands, because the cost of being wrong is asymmetric and the fix is cheap.
Why this is different from the disclosure rules
We wrote about the EU AI Act transparency obligation when it landed, and it is easy to file this ruling in the same drawer. It does not belong there.
Disclosure is about telling people they are talking to a machine. This is about being bound by what the machine then says. You can satisfy the first perfectly, with a clear "you are chatting with an AI assistant" banner, and still lose on the second, because the visitor was told it was a bot and was still misled about a qualification, a price or a refund.
Put bluntly, the banner protects the user's right to know. It does not protect you.
Scope is the control you actually have
You cannot make a language model stop inventing things. You can decide which subjects it is allowed to speak about unaided, and that is where the work belongs.
The way I would structure it is three lanes. Some topics the bot can answer freely, because a wrong answer is cheap and correctable. Some it may answer only by quoting a source it retrieved, and it should show that source in the reply. And some it must never answer, only route.
The third lane is the one that matters legally. Everything a customer could act on belongs there.
The sorting rule is not difficulty. It is reliance. Opening hours are safe not because they are simple but because nobody structures their affairs around them. Anything touching credentials, entitlements, money or contract terms goes in the escalation lane no matter how confidently the model handles it in testing.
Pricing sits awkwardly in the middle, and I would put it in the quote-only lane rather than the free lane. A bot that says "our Business plan is 49 a month" from its training data is guessing. A bot that pulls the current price list and quotes it, with a link, is repeating something you published.
It is not only chatbots
A second German decision pushed in the same direction from a different angle. A Munich court held Google directly liable for false statements in its AI Overviews, on the reasoning that the overview is Google's own content rather than a protected list of somebody else's links.
Different defendant, different technology, same underlying move: when your system generates a statement instead of pointing at someone else's, you are the author. For support teams the read across is obvious. An AI written summary at the top of a knowledge base article is your content. A suggested reply that an agent sends without editing is your content. An auto generated release note is your content.
The comfortable distinction people draw between "the bot chatting" and "AI helping internally" gets thinner the moment the output reaches a customer unedited.
What your help desk should be logging
If a customer claims the bot promised them something, the argument is evidential, not philosophical. You need the transcript, and you need to know which lane produced the reply and what source it used.
That is close to what we argued about authorization records for AI agents, but aimed at a different question. That post asked whether the agent was allowed to take an action. This one asks whether it was allowed to make a claim, and what it based the claim on.
Keep the retrieved source alongside the answer, not just the answer. Six months later, "the bot quoted the policy page as it read on 3 March" is a defence. "The bot said something" is not.
If someone else built the bot
Plenty of support teams did not train anything. They switched on a feature inside a helpdesk product, or handed the channel to an outsourced provider. The ruling does not care much, because the operator is the one whose name is on the website.
That makes the contract worth a read you probably have not given it. Two questions matter more than the rest. Who decides what the bot may discuss, and can you change that scope yourself without raising a ticket with your vendor? If the answer to the second is no, you are carrying a liability you cannot directly control, which is a strange place to be.
Ask for the transcript export too. If your provider keeps conversations in a system you cannot query, your evidence lives on somebody else's retention schedule. That is fine right up until the week you need it.
A short exercise worth running this week
Take your twenty most common inbound questions. Sort them into the three lanes. My guess is that two or three currently sit in the free lane that should not, and that at least one of those is about money.
Then ask the bot the ten questions you would least like a customer to ask, in the rudest phrasing a frustrated person would use. Not the polished test set. The real ones. What comes back tells you more than any policy document.
Frequently asked questions
Does this ruling apply outside Germany?
Directly, no. It binds in its own jurisdiction and is under appeal. But the reasoning, that an operator controls the tool and therefore owns its output, is not unique to German law, and similar conclusions are appearing elsewhere. Treat it as a direction of travel rather than a foreign curiosity.
Does a disclaimer saying the bot may be inaccurate protect us?
On the facts of this case, careful configuration did not help, and a disclaimer is weaker than configuration. It may affect damages at the margins. Do not build your position on it.
Should we just turn the AI off?
No, and I would push back on anyone who suggests it. The ruling penalises unbounded scope, not automation. A bot confined to lookups and published sources is both useful and defensible.
What about a bot that only answers from our own documentation?
That is much the safer design, which is why the quote-only lane exists. The residual risk is a bot that retrieves the right document and then summarises it into something the document never said, so keep the source visible in the reply.
How long should we keep chat transcripts?
Long enough to cover the complaint window you realistically face, balanced against your data retention policy. The transcripts are the evidence, so deleting them early removes your own defence.
Related resources
ICTDesk runs the AI bot on your own content and keeps every conversation searchable, which is the combination this ruling rewards. If you are reviewing what your bot is allowed to promise, take a look at how ICTDesk handles it.
Top comments (0)