The clearest sign a small business needs more than a packaged bot is simple: customers keep asking things that need live data. Stock, available slots, order status, a delivery date. A static knowledge base cannot answer those.
You can measure it with a transcript export and twenty lines of Python. It turns the custom AI or off-the-shelf chatbot question for small businesses into a number from your own data.
Rules file
Keep the rules outside the code so non-developers can edit them. rules.json:
{
"stock": ["in stock", "available", "size", "colour", "back in"],
"slots": ["appointment", "slot", "book", "tomorrow", "reschedule"],
"orders": ["order", "tracking", "delivered", "where is my", "refund"]
}
Tagger
import csv, json, re, sys
from collections import Counter
rules = json.load(open("rules.json"))
patterns = {tag: re.compile("|".join(re.escape(w) for w in words), re.I)
for tag, words in rules.items()}
def tags_for(text):
return [t for t, p in patterns.items() if p.search(text)]
counts, total, examples = Counter(), 0, {}
with open(sys.argv[1], newline="") as f:
for row in csv.DictReader(f):
total += 1
found = tags_for(row["customer_messages"])
for t in found:
counts[t] += 1
examples.setdefault(t, row["conversation_id"])
if not found:
counts["static_only"] += 1
print("conversations:", total)
for tag, n in counts.most_common():
NaN
Run it as python tag.py transcripts.csv.
Reading the output
-
static_onlydominates: a packaged bot with a good knowledge base is fine. -
slotsorordersare large: the bot needs to read, and maybe write, your systems. That is custom territory, or at least an integration. - Open the example IDs. Keyword rules are crude; check a few by hand before deciding anything.
Caveats
This is a triage tool, not a classifier. It will miss paraphrases and catch false positives. That is fine for a first look.
I work at NxFlowAI; we run something like this during our 72-hour audit. More hands-on posts are on the NxFlowAI engineering and workflow blog.
Top comments (0)