A common way to give a support bot knowledge is to let your agents save their good answers. A ticket gets solved, the agent clicks save, and the answer joins a set the bot retrieves from when a similar question comes in. Many support tools offer some version of it. We built ours.
On October 8, we read every one of those saved answers as if our bot were about to quote them to a stranger. It was: a filter bug had kept most of them out of the bot's prompts for months, and the fix was about to make all of them eligible again. What we found had nothing to do with the bug, and I think it applies to anyone who turns tickets into bot knowledge.
Some context. I run engineering at GoodBarber, an app platform. Our AI assistant drafts answers to customer support questions; I wrote about it a few weeks ago. It draws on two sources: our help center, and the saved answers. When a customer asks something, we embed the question, take the closest saved answers of the same app type, at most three within a cosine distance of 0.5, and put them in the prompt.
Where a saved answer comes from
When an agent creates a saved answer from a ticket, the form is pre-filled with the ticket itself: its subject, the customer's last messages, and the agent's last reply. That is convenient, and it is the whole problem: unless the agent rewrites it, the ticket is what gets saved.
What gets saved is a reply written to one person, about their account, at a given moment. A colleague reading it as an example would adapt it. A retriever adapts nothing: to it, the reply is a candidate answer for anyone whose question lands close enough in embedding space.
What a ticket leaves in an answer
Claude read all 65 saved answers, about 98,000 characters, and listed everything that would be wrong if the bot repeated it to someone else. Sorted, it comes down to a short list of things a ticket carries, and what each one turns into once a bot reuses it.
| what the ticket left | example (anonymized) | what the bot can do with it |
|---|---|---|
| Identity | first names, last names, emails of customers and of third parties, app names, domains, greetings and agent signatures | tell the next customer someone else's name or email |
| Links into one account | about a dozen answers linked to a specific customer's back office; one linked to a support agent's test project | send the next customer into an account that isn't theirs |
| Things done once | "The issue has been resolved." "When I test it, I can see the thumbnail." "Your analysis is correct." | claim work nobody did, on an account nobody looked at |
| Statements on someone's behalf | an app store appeal letter, ready to send, in one customer's name, declaring "under penalty of perjury" that they were the sole owner | hand a stranger a sworn statement written for someone else |
| Time | "this past Sunday", a verification deadline that had already passed | give dates that were true once |
| Assumptions about the setup | answers that took the customer's publishing setup for granted, with no condition | give the right answer to the wrong customer |
The ticket also leaves its shape. Titles copied from another ticket. An answer that covered one of the four questions asked. Two saved answers to the same data-protection question, promising different things; the bot could have picked one or the other depending on how the question was phrased. And help links to articles that don't exist, or to the wrong one.
Most of this isn't carelessness: as replies to the original customer, much of it made sense. The problem is the save button: it turns a reply into knowledge without asking anyone to rewrite it for a reader who isn't that customer.
Rewriting them
We went through them one at a time, the most dangerous first. For each one, Claude gave me the link, what the answer contained, why it was a problem, and a generic replacement. I pasted it in the admin, and Claude re-read the row in the database to check it, including that the embedding had been recomputed, since a rewritten answer that keeps its old vector is still retrieved for the old text.
Most answers had a reusable core worth keeping:
- The perjury letter became the list of proof the store expects, written for anyone.
- "The issue has been resolved" kept the only advice that applied to everyone: save again, then check the preview.
- A detailed diagnosis of one customer's site kept its technical explanation and lost the site.
- One answer stated two different rules in three lines, and the documentation didn't say which one was true. The rewrite gives an instruction that works under both, and the question went into the recap for the support team.
By the end of the day, about thirty of the 65 had been rewritten and one deleted.
Rules for the next one
The cleanup only holds if the next answers are written differently, so we wrote ten rules for the support team. The ones that matter most:
- A saved answer is generic. Reread it asking: if the bot copies this to another customer, is it still right?
- No customer data: no name, email, app name, domain, ID, greeting or signature.
- No links into an account. Write the menu path instead.
- Don't describe what you did. "I tested it", "it's fixed": the bot won't have done it.
- No dates. A saved answer lives for months.
- When the answer depends on the case, write both branches instead of assuming.
- The question is what retrieval matches on. Make it clear and complete, and make the answer cover everything it asks.
- One topic, one saved answer. If two answer the same question, keep one.
The first rule is the only one you really need. The others are what it looks like in practice.
Clean answers make a loose threshold cheap
The next morning, with the dangerous answers rewritten and the set actually reaching the bot, we looked at how it was used. Eleven answers still carried a first name or a signature; none of the replies had picked one up.
- 42% of replies carried at least one saved answer, drawn from more than twenty different ones.
- The match is often loose. Reading the 72 questions that received one: under a distance of 0.40, the saved answer was relevant 9 times out of 12. Between 0.40 and 0.50, where about 80% of the injections fall, it was mostly off-topic, down to saved answers attached to messages like "Hi".
- The model mostly ignores the off-topic ones. Ten ordinary questions about push notifications received a saved answer about a different feature; none of the ten replies mentioned it.
- Whether they make replies better is not measured yet. Votes are identical with and without, 11 up and 6 down each. Replies with a saved answer send people to the ticket form less, 18.6% against 34.0%, but they also come from more specific questions.
Put those together and the asymmetry is the point. With clean answers, a loose threshold has cost nothing visible so far: in the cases we read, the model skipped what didn't fit. With the answers as they were, the same threshold would have offered the perjury letter to anyone asking about the same kind of store rejection, and a stranger's back-office link to anyone with a close enough question about domains. In our case, the risk was in what went into the set, not in the threshold.
What I'd change in the tool
- Don't pre-fill a saved answer with the raw ticket. Pre-fill a template that asks for a generic question and a generic answer, or at least show the one-question test next to the save button.
- Check before saving. Names, emails, domains, links into an account, dates, "I tested", "resolved": all of these can be flagged automatically before the answer joins the set.
- Review the set before anything widens retrieval. A bug fix, a looser threshold, a new embedding model: each one puts answers in front of customers that nobody has read in that role.
If you turn tickets into knowledge for a bot: what does your save button do to the ticket?
Top comments (0)