Workshop: Cut Repeat Requests With a Prompt Fingerprint Cache in 90 Minutes

Core conclusion first: on a free model tier, the most reliable cost saving is not a shorter prompt, but a cache that refuses to send the same request twice, and a group of students can build and verify one in a single 90-minute session. The artifact below is an exact-match fingerprint gate in roughly sixty lines of Python, backed by SQLite and three assertions that prove the hit path actually works. Every exercise is designed to be rerun on a laptop without network access, so a trainer can deliver it in a classroom or a study group.

This is a workshop outline, not a benchmark report. Timing blocks are proposals measured in facilitation minutes rather than production performance, and the sample outputs come from a local stub model rather than a hosted endpoint.

Who this workshop is for, and what it deliberately skips

• Audience: developers already calling a hosted model from a script, a notebook, or a small batch job, who have noticed that re-running the same work costs the same as running it the first time.
• Prerequisites: Python 3.10 or newer, comfort with virtual environments, and the ability to read a short sqlite3 snippet without panic.
• Skipped on purpose: semantic similarity caches, vector stores, prompt compression, and any discussion of model quality. Those are separate workshops with different failure modes.

If the room cannot yet run a script end to end, spend the first ten minutes on environment setup and shorten the final retro block instead.

Why duplicate requests dominate early-stage free-tier usage

Before any optimization, most teams assume their spend comes from long prompts. When you instrument an actual workload, a different pattern usually appears: the same request is sent repeatedly because of retries after timeouts, batch reruns after a crash, notebook cells executed twice, or evaluation harnesses restarted from the top. None of these repeats carry new information, and each one consumes the same allowance as the original call.

The first workshop exercise therefore produces a number rather than a fix. Students take a log of fifty requests they already sent, extract a canonical key for each, and report the repeat rate as a percentage. Do not set a threshold in advance, because the point of the exercise is that the decision to ship a cache should follow the measurement rather than precede it.

The artifact: a fingerprint gate in about sixty lines

The cache below keys on a SHA-256 fingerprint of a canonical JSON representation of the request. Store the file as cache_gate.py; the only imports are standard library modules.

# cache_gate.py - exact-match prompt cache keyed by request fingerprint
import hashlib
import json
import sqlite3
import time

SCHEMA = """
CREATE TABLE IF NOT EXISTS replies (
  fingerprint TEXT PRIMARY KEY,
  model       TEXT NOT NULL,
  reply       TEXT NOT NULL,
  created_at  REAL NOT NULL
);
"""


def canonical(request: dict) -> str:
    """Stable text for a request: sorted keys, no incidental whitespace."""
    return json.dumps(request, sort_keys=True, separators=(",", ":"), ensure_ascii=False)


def fingerprint(request: dict) -> str:
    return hashlib.sha256(canonical(request).encode("utf-8")).hexdigest()


class Gate:
    """Wraps a model callable so identical requests never reach it twice."""

    def __init__(self, path: str = "cache.db", ttl_seconds: int = 86_400):
        self.db = sqlite3.connect(path)
        self.db.executescript(SCHEMA)
        self.ttl = ttl_seconds
        self.upstream_calls = 0

    def complete(self, request: dict, call_model) -> tuple[str, str]:
        key = fingerprint(request)
        row = self.db.execute(
            "SELECT reply, created_at FROM replies WHERE fingerprint = ?", (key,)
        ).fetchone()
        if row and (time.time() - row[1]) < self.ttl:
            return row[0], "hit"
        self.upstream_calls += 1
        reply = call_model(request)
        self.db.execute(
            "INSERT OR REPLACE INTO replies VALUES (?, ?, ?, ?)",
            (key, request.get("model", "unknown"), reply, time.time()),
        )
        self.db.commit()
        return reply, "miss"


def stub_model(request: dict) -> str:
    """Local stand-in for a hosted endpoint; labels every real call."""
    text = request["messages"][-1]["content"][:24]
    return f"echo:{text}"


if __name__ == "__main__":
    gate = Gate(path="cache.db")
    req = {"model": "M", "messages": [{"role": "user", "content": "Summarize this log line."}]}
    print(gate.complete(req, stub_model))         # ('echo:Summarize this log line.', 'miss')
    print(gate.complete(dict(req), stub_model))   # ('echo:Summarize this log line.', 'hit')
    print("upstream calls:", gate.upstream_calls) # upstream calls: 1

The canonicalization rule is the whole trick

json.dumps with sort_keys=True and compact separators makes field order irrelevant, which matters because different call sites rarely build the dictionary in the same sequence. A trailing space in a user message, however, must remain a different request, so the canonical form removes formatting noise without normalizing content.

from cache_gate import fingerprint

a = {"model": "M", "messages": [{"role": "user", "content": "hi"}], "max_tokens": 64}
b = {"max_tokens": 64, "messages": [{"role": "user", "content": "hi"}], "model": "M"}
c = {"model": "M", "messages": [{"role": "user", "content": "hi "}], "max_tokens": 64}

assert fingerprint(a) == fingerprint(b)   # field order is not part of the request
assert fingerprint(a) != fingerprint(c)   # a trailing space is

Where free access and a free server change the exercise

The operator describes MonkeyCode as an open-source project that currently offers free model access, with the operator citing a ten-million-token allowance, plus a free server option. Terms, allowances, and availability can change, so confirm the current details on the project's own page before you build a plan or a syllabus around them, and treat this article's mention as a description of an offer rather than a guarantee.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

For the workshop, both claims matter for logistics rather than for the caching logic. Free model access means duplicate calls compete directly with real work for the same allowance, which makes the repeat-rate exercise land harder in the room. A free server option gives students a place to run the cache process beside the model calls, so the lab does not depend on anyone's laptop staying awake.

The 90-minute plan

Block	Minutes	Exercise	Deliverable
1	0-08	Record the baseline	Repeat rate as a percentage from a 50-request log
2	08-22	Key hygiene audit	List of unstable fields removed from the key
3	22-45	Build the gate	cache_gate.py returning hits and misses
4	45-55	Test the hit path	Three passing assertions
5	55-70	Add TTL and tenant scoping	Two policy decisions written down
6	70-82	Report and decide	Filled decision table plus one recommendation
7	82-90	Retro	List of workloads that should not use this

Blocks 2 and 5 are the ones trainers usually rush, and they are also the ones that decide whether the cache is correct rather than merely fast to build.

Worked example students can rerun

Run the whole lab from an empty directory, so everyone sees the same file layout.

python -m venv .venv && . .venv/bin/activate
python cache_gate.py
# ('echo:Summarize this log line.', 'miss')
# ('echo:Summarize this log line.', 'hit')
# upstream calls: 1

Two identical requests, one upstream call, and one row in cache.db. Ask the room to delete cache.db, rerun the script, and confirm that the first line returns miss again, because a cache that survives a deliberate reset is usually a cache the student does not understand yet.

Exercise: prove the hit path with tests

# test_cache_gate.py
from cache_gate import Gate, fingerprint, stub_model


def test_second_identical_call_is_a_hit(tmp_path):
    gate = Gate(path=str(tmp_path / "c.db"))
    req = {"model": "M", "messages": [{"role": "user", "content": "ping"}]}
    assert gate.complete(req, stub_model)[1] == "miss"
    assert gate.complete(req, stub_model)[1] == "hit"
    assert gate.upstream_calls == 1


def test_field_order_does_not_change_the_key():
    left = {"model": "M", "temperature": 0, "messages": [{"role": "user", "content": "x"}]}
    right = {"messages": [{"role": "user", "content": "x"}], "model": "M", "temperature": 0}
    assert fingerprint(left) == fingerprint(right)

pip install pytest && pytest -q test_cache_gate.py
# 2 passed

Decision table: cache, cache with conditions, or do not cache

Situation	Decision	Reason
Fixed system prompt plus identical user text, one sample	Cache	Deterministic request, frequently repeated by reruns
temperature > 0 where variety is the product feature	Do not cache	A cached reply freezes a single sample
Response embeds per-user private data	Cache only with a tenant-scoped key	A shared key leaks answers across users
Time-sensitive answers such as prices or schedules	Cache with a short TTL	TTL turns correctness into a stated policy
Request contains a random id or timestamp	Cache after key hygiene	Unstable keys make every request a miss
Tool calls that write to an external system	Do not cache	Replaying a write is not free

Failure modes to debug in the room

1. Key drift. A generated request id or timestamp silently turns every call into a miss, and the hit rate sits at zero while the code looks correct.
2. Whitespace normalization. Aggressively trimming user content merges two requests that genuinely differ, and students cannot reproduce the collision without a printed key diff.
3. Tenant leakage. A single shared table serves answers across users until the key includes a tenant or project identifier.
4. TTL overreach. A long TTL keeps serving stale answers long after the underlying data changed.
5. SQLite contention. Opening a new connection per request produces locked-database errors under concurrency; keep one connection per worker process.

Limitations, and who should not use this approach

• Do not ship this if your traffic is nearly unique, because the cache adds a lookup, a file, and a policy surface while removing few calls.
• Do not ship this where sampling variance is the feature, such as brainstorming tools or creative generation with a temperature the user controls.
• Do not treat the cache as a substitute for backoff, timeouts, or retry budgets, since it addresses repeats rather than failures.
• Do not store untrusted or regulated content in a plain SQLite file without reviewing your own data-handling requirements.
• Exact matching is the deliberate boundary here; a semantic cache needs its own evaluation set before anyone can claim it is safe.

Homework and a closing note

Before the next session, each student should rerun the baseline exercise against a real workload and bring the measured repeat rate, the TTL they chose, and one workload they decided not to cache. If you want a place to run the lab without wiring up billing first, the operator's free server option is the simplest starting point, but read the current terms carefully before you depend on it.

The workshop succeeds when students leave with a number they measured, a decision they justified, and a list of cases where the cache would hurt.