Small businesses get the same ten questions all day: opening hours, delivery areas, how to book. Sending each one to a language model is wasted money. A small cache is one of the simplest ways to control AI automation costs for a small business.
Normalise, then hash
import hashlib, re, sqlite3, time
db = sqlite3.connect("answers.db")
db.execute("""CREATE TABLE IF NOT EXISTS cache(
key TEXT PRIMARY KEY, question TEXT, answer TEXT, created REAL, hits INTEGER DEFAULT 0)""")
def normalise(q: str) -> str:
q = q.lower()
q = re.sub(r"[^a-z0-9 ]+", " ", q)
return " ".join(q.split())
def key(q: str) -> str:
return hashlib.sha256(normalise(q).encode()).hexdigest()
Read through the cache
MAX_AGE = 7 * 24 * 3600 # refresh weekly so answers do not go stale
def answer(q, call_model):
k = key(q)
row = db.execute("SELECT answer, created FROM cache WHERE key=?", (k,)).fetchone()
if row and time.time() - row[1] < MAX_AGE:
db.execute("UPDATE cache SET hits = hits + 1 WHERE key=?", (k,))
db.commit()
return row[0], "cache"
a = call_model(q)
db.execute("REPLACE INTO cache(key, question, answer, created) VALUES (?,?,?,?)",
(k, q, a, time.time()))
db.commit()
return a, "model"
Rules that keep it safe
- Cache only general questions. Never cache answers that include a customer's name, order or account details.
- Expire entries when prices, hours or policies change (clear the table on any website update).
- Log
cacheversusmodelso you can see what the cache actually saves.
Going further
Exact matching catches more than you expect once text is normalised. Fuzzy or embedding matching adds complexity; measure first.
Why we often prefer small custom pieces like this over packaged bots: why small custom pieces often beat packaged bots.
Disclosure: I co-founded NxFlowAI, an automation agency.
Top comments (0)