A med-spa owner hired me because of a screenshot. A prospect had asked, in the website chat, whether the laser treatment hurt. The AI receptionist answered: "Most clients describe it as mild. You'll be pain-free in one session — that's the whole point of the package."
Fourteen words that turn a chat widget into a marketing claim. In her jurisdiction, promising a clinical outcome in advertising is a licensing problem, not a copy problem. The screenshot ended up in a competitor's Instagram story.
She assumed the fix was a longer banned-words list. It isn't. I audited 4,180 outbound AI messages across six weeks for her and two other clients in similar regulated-adjacent categories (dental, physio, home care). 231 of them — 5.5% — contained something I would not have let a human send. Only 34 of those matched a banned keyword.
That gap is the entire problem. 94 out of 100 bad AI messages use no forbidden word at all.
Why the blocklist always loses
Blocklists are built from nouns. Real risk lives in verbs and promises.
| Message | Banned word? | Actually risky? |
|---|---|---|
| "Results vary by individual." | no | no |
| "You'll be pain-free in one session." | no | yes — outcome promise |
| "We can definitely fit you in tomorrow." | no | yes — commitment without capacity check |
| "The doctor said it's nothing to worry about." | no | yes — clinical opinion invented |
| "This is the cheapest option in the city." | no | yes — unsubstantiated comparative |
| "Our competitor down the road botched a case like yours." | no | yes — defamation-adjacent |
| "Just take two of the ones you already have." | no | yes — dosage instruction |
Every one of those passes a ban list built from words like guarantee, cure, best, free, lawsuit. The agent isn't evading the list; the list was never measuring the thing that hurts.
There's a second failure mode: the list makes the agent sound like it's reading from a list. "I'm sorry, I can't make that claim" is a terrible customer experience when the customer asked a normal question. You end up with an assistant that refuses to say anything useful, which is how moderation projects get quietly switched off two months in.
So the gate has to be about categories of commitment, not vocabulary. Five of them.
Gate 1 — Outcome claims
Any sentence that predicts a result for this specific person. Not "our clients often report" — that's still a claim, but a permitted one if it's sourced. The gate is on the second person: you will, you'll, your case will.
FAIL: You'll be [result] in [timeframe].
FAIL: This will fix / cure / clear / eliminate [condition].
FAIL: You don't need to worry about [risk].
PASS: Most clients describe [X]. Individual results vary, and the clinician
will confirm what's realistic at your consult.
The pass version isn't evasive. It's what a trained receptionist says, because it's true.
Gate 2 — Commitments the agent can't authorise
Price held, slot reserved, refund approved, deadline promised, exception granted. An agent that says "yes, we can do that" without reading capacity is creating a contract with no counterparty.
Make the agent emit intent, not agreement:
FAIL: Consider it booked — see you Tuesday at 3.
PASS: I can request Tuesday 3pm for you. The team confirms within one
business hour and I'll message you the moment it's locked.
Rule of thumb: the agent may describe what will happen next, never assert that it has happened. "Requested" is a state. "Booked" is a liability.
Gate 3 — Facts about a person
This is the one that gets companies in real trouble rather than reputational trouble. The agent must never state a diagnosis, an eligibility determination, a legal position, or a credit decision. It can restate what the customer told it, and it can quote a document. It cannot conclude.
I found six instances in that audit where the agent inferred something from context and stated it as fact: "since you're on blood thinners, we'll skip the numbing." Nobody told it that. It assembled it from a prior message about a prescription and a table about contraindications. That's exactly the behaviour you want in a diagnostic tool and exactly what you cannot have in a receptionist.
FAIL: Based on what you've described, it sounds like [condition].
FAIL: You qualify for [programme / coverage / discount].
PASS: That's a clinical question — I'll flag it for [role] and they'll
come back to you today. Here's what I've passed along: [summary].
Gate 4 — Data leakage across the boundary
Three leaks, all common, none caught by a word list:
- Internal identifiers in customer copy. Ticket IDs, rep names, internal notes ("VIP — do not discount"), margin figures. The agent quotes the CRM row because the CRM row is in context.
- Other customers' data. Retrieval pulls a document that mentions someone else. It is astonishing how often "similar case" means "a different person's case."
- Silent PII collection. The agent asks for a date of birth or an ID number in a channel that isn't approved for it, and now it's in a transcript store with a different retention policy than the form.
The fix is a context boundary, not a filter: whatever the agent is given as retrieval, it gets a redacted view, and the redaction happens before the prompt, not after the reply.
Gate 5 — Tone under pressure
Moderation usually means "don't say bad things." Half the incidents I see are the agent saying a true thing badly, to someone who is upset. Sarcasm read literally. A cheerful "Great!" after a complaint. An emoji in a bereavement reply. A policy citation where an apology belonged.
You cannot check this per sentence. You check it per conversation state: if the customer's last two turns scored as distressed or angry, the agent's register must change and its length must drop. Short, plain, no exclamation marks, one concrete next step.
The policy file that actually runs
Write the five gates as data, not prose, so both the agent and your reviewer read the same thing:
{
"version": "2026-10-18",
"owner": "ops@company.com",
"gates": {
"outcome_claim": { "action": "rewrite", "fallback": "clinician_review" },
"unauthorised_commit":{ "action": "rewrite", "fallback": "handoff_human" },
"personal_fact": { "action": "block", "fallback": "handoff_role" },
"data_leak": { "action": "block", "fallback": "safe_template" },
"tone_mismatch": { "action": "rewrite", "fallback": "short_plain" }
},
"never_say": ["pain-free", "guaranteed", "cure", "cheapest in"],
"always_include_when": {
"health_topic": "Individual results vary."
}
}
Two design notes. action is not binary. Blocking everything is how you get a useless agent; rewriting is usually right for claims and tone, blocking only for personal facts and data. And fallback names a human role, not "error" — every moderation decision needs somewhere for the conversation to go.
Measuring it without reading every transcript
You will not read 4,000 messages. Sample deliberately and count five numbers weekly.
Week of: ____
Outbound AI messages: ____
Sampled (min 10%, plus 100% of: new intents, complaints, price mentions): ____
Gate 1 outcome claims: ____ (target: 0 in sample)
Gate 2 unauthorised commitments: ____ (target: 0)
Gate 3 personal facts: ____ (target: 0)
Gate 4 data leaks: ____ (target: 0)
Gate 5 tone mismatches: ____ (target: <2%)
Rewrites that broke the answer: ____ (the cost of over-blocking)
Escalations caused by moderation:____
That last pair matters more than the violation counts. A moderation layer that hits zero violations by refusing to answer is a failed deployment. Track the false-positive rate or you'll ship a mute bot and call it safe.
At the med-spa, after the five gates and a weekly 10% sample, Gate 1 hits went from 47 in six weeks to 2 in the following six, and both were caught by the reviewer before send. Average reply length dropped 9% — the agent stopped elaborating, which customers read as competence.
The 12-line card I leave with the team
[ ] One owner named for the policy file
[ ] Every gate has a rewrite path, not just a block
[ ] Second-person outcome sentences are rewritten, not deleted
[ ] Agent describes next steps, never asserts completion
[ ] No diagnosis, eligibility, or legal conclusion — ever
[ ] Retrieval is redacted before it reaches the prompt
[ ] Internal notes and margins are stripped from agent context
[ ] Distress state shortens and flattens the reply
[ ] Never_say list is reviewed monthly (it ages fast)
[ ] 10% sample pulled weekly, complaints sampled at 100%
[ ] False-positive rate tracked alongside violations
[ ] Every fallback routes to a named human role
If you only do one thing this week, do the first line. A policy file with no owner is a document, not a control.
The full policy schema, the five gate prompts, the sampling sheet and the reviewer card are in the kit I run on deployments: Agent Audit Pack
Top comments (0)