Cache the Canonical Form: A Normalizing Cache for Metered Model Calls

A repeated prompt is a wasted request. Free model quota burns on identical questions. Most teams fix this with a database. A database is overkill.

This tutorial builds a normalizing cache. It canonicalizes prompts. It matches exact duplicates. It skips the network call entirely.

You need Node 18+ and one empty folder. No vector database. No external services. One JSON file.

MonkeyCode's free model access can be the metered endpoint. Its free server option can host the cache process. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

What you will build

1. A canonicalizer that normalizes prompt text.
2. A TTL cache with a size cap.
3. A client wrapper that checks cache before calling.
4. A hit-rate reporter that proves the savings.

Stage 1: Scaffold

Create the project folder. Initialize Node. Confirm the runtime.

mkdir prompt-cache && cd prompt-cache
npm init -y
node --version

Verify: node --version prints 18 or higher. Upgrade if needed.

Stage 2: Build the canonicalizer

The same question arrives in many shapes. Extra spaces. Different quote styles. Reordered JSON keys. The canonicalizer reduces them to one form.

// canonical.mjs
export function canonicalize(input) {
  if (typeof input === 'string') {
    return input
      .replace(/\s+/g, ' ')
      .replace(/["'`]/g, '"')
      .trim()
      .toLowerCase();
  }
  if (Array.isArray(input)) {
    return input.map(canonicalize);
  }
  if (input && typeof input === 'object') {
    const out = {};
    for (const key of Object.keys(input).sort()) {
      out[key] = canonicalize(input[key]);
    }
    return out;
  }
  return input;
}

export function canonicalKey(prompt) {
  return JSON.stringify(canonicalize(prompt));
}

Test the canonicalizer. Feed it three different phrasings of one question.

// smoke.mjs
import { canonicalKey } from './canonical.mjs';

const a = canonicalKey('What is DNS?');
const b = canonicalKey('  what   is dns? ');
const c = canonicalKey({ prompt: 'What is DNS?' });

console.log(a);
console.log(b);
console.log(c);
console.log('Match:', a === b && b === c);

node smoke.mjs

Verify: all three lines print the same key. The console prints Match: true. If not, inspect the normalization rules.

Stage 3: Build the cache store

The cache maps canonical keys to responses. Entries expire after a TTL. The store caps its size to bound memory.

// cache.mjs
import { readFileSync, writeFileSync, existsSync } from 'node:fs';

const CACHE_FILE = 'cache.json';

export class PromptCache {
  constructor({ ttlMs = 3600000, maxEntries = 500 } = {}) {
    this.ttlMs = ttlMs;
    this.maxEntries = maxEntries;
    this.data = this.load();
  }

  load() {
    if (!existsSync(CACHE_FILE)) return {};
    return JSON.parse(readFileSync(CACHE_FILE, 'utf8'));
  }

  save() {
    writeFileSync(CACHE_FILE, JSON.stringify(this.data));
  }

  get(key) {
    const entry = this.data[key];
    if (!entry) return null;
    if (Date.now() - entry.at > this.ttlMs) {
      delete this.data[key];
      this.save();
      return null;
    }
    return entry.response;
  }

  set(key, response) {
    const keys = Object.keys(this.data);
    if (keys.length >= this.maxEntries && !this.data[key]) {
      const oldest = keys.sort((a, b) => this.data[a].at - this.data[b].at)[0];
      delete this.data[oldest];
    }
    this.data[key] = { at: Date.now(), response };
    this.save();
  }
}

Smoke test the store. Set a value. Get it back. Confirm the round trip.

// smoke2.mjs
import { PromptCache } from './cache.mjs';
const cache = new PromptCache();
cache.set('key-1', 'DNS maps names to IP addresses.');
console.log(cache.get('key-1'));

node smoke2.mjs
cat cache.json

Verify: the console prints the response. The JSON file contains one entry with a timestamp.

Stage 4: Wire the cache into the client

The wrapper checks the cache first. A hit returns instantly. A miss calls the model and stores the result.

// client.mjs
import { canonicalKey } from './canonical.mjs';
import { PromptCache } from './cache.mjs';

const cache = new PromptCache();

export async function callWithCache(url, prompt, options = {}) {
  const key = canonicalKey(prompt);
  const cached = cache.get(key);
  if (cached) {
    return { ...cached, fromCache: true, key };
  }

  const response = await fetch(url, {
    method: 'POST',
    headers: { 'content-type': 'application/json' },
    body: JSON.stringify({ prompt, ...options }),
  });
  const data = await response.json();
  cache.set(key, { status: response.status, data, at: Date.now() });
  return { status: response.status, data, fromCache: false, key };
}

Run the same prompt twice. Watch the second call skip the network.

// run.mjs
import { callWithCache } from './client.mjs';

const url = 'https://your-endpoint.example/v1/complete';
const prompt = 'Explain the OSI model in one sentence.';

const first = await callWithCache(url, prompt);
const second = await callWithCache(url, prompt);
console.log('First from cache:', first.fromCache);
console.log('Second from cache:', second.fromCache);

node run.mjs

Verify: the first call prints false. The second prints true. The second call took near-zero time. Check the cache file for the stored response.

Stage 5: Measure the hit rate

A cache without metrics is a guess. Add a counter. Track hits and misses across the process lifetime.

// metrics.mjs
let hits = 0;
let misses = 0;

export function recordHit() { hits += 1; }
export function recordMiss() { misses += 1; }
export function hitRate() {
  const total = hits + misses;
  return total ? hits / total : 0;
}

Update the client to record every decision.

// client.mjs (updated)
import { canonicalKey } from './canonical.mjs';
import { PromptCache } from './cache.mjs';
import { recordHit, recordMiss, hitRate } from './metrics.mjs';

const cache = new PromptCache();

export async function callWithCache(url, prompt, options = {}) {
  const key = canonicalKey(prompt);
  const cached = cache.get(key);
  if (cached) {
    recordHit();
    return { ...cached, fromCache: true, key };
  }
  recordMiss();
  const response = await fetch(url, {
    method: 'POST',
    headers: { 'content-type': 'application/json' },
    body: JSON.stringify({ prompt, ...options }),
  });
  const data = await response.json();
  cache.set(key, { status: response.status, data, at: Date.now() });
  return { status: response.status, data, fromCache: false, key };
}

export function report() {
  return { hitRate: hitRate(), hits, misses };
}

Load-test with a mixed workload. Ten unique prompts. Then repeat them.

// bench.mjs
import { callWithCache, report } from './client.mjs';

const url = 'https://your-endpoint.example/v1/complete';
const prompts = [
  'What is a reverse proxy?',
  'Explain TCP handshake.',
  'What is a JWT?',
  'How does DNS work?',
  'What is idempotency?',
  'Explain a CDN.',
  'What is a websocket?',
  'How do cookies work?',
  'What is a hash?',
  'Explain load balancing.'
];

for (let round = 0; round < 3; round += 1) {
  for (const prompt of prompts) {
    await callWithCache(url, prompt);
  }
}

console.log(report());

node bench.mjs

Verify: the hit rate is roughly 66 percent. Ten misses on the first round. Twenty hits on the next two rounds. If the rate is lower, your prompts vary more than expected. Inspect the canonical keys.

Stage 6: Deploy

The cache is a plain Node process. It runs anywhere Node 18+ exists. Deploy the folder to your host.

MonkeyCode's free server option can host the cache. Check the storage behavior first. Ephemeral filesystems wipe the cache on restart. A cold cache means a full round of misses.

Verify: after deploy, run one prompt twice. Confirm the second call returns fromCache: true. Confirm cache.json exists on the server.

Verification checklist

• [ ] canonicalKey returns identical keys for equivalent prompts.
• [ ] The second identical call returns fromCache: true.
• [ ] cache.json persists across restarts.
• [ ] The hit rate matches the workload's repetition ratio.
• [ ] TTL expiry removes stale entries.

Limitations

This cache matches normalized text. It does not understand meaning. Two different questions about the same topic produce different keys. That is correct behavior for a conservative cache.

The cache stores raw responses. Sensitive data persists on disk. Encrypt the cache file or exclude it from backups if your prompts contain secrets.

TTL is a fixed window. A long TTL serves stale answers. A short TTL wastes quota. Tune the window to your data's change rate.

The single JSON file works for small workloads. High-throughput deployments need a real store. This cache is a starting point, not a database replacement.

Who should skip this

Teams with highly dynamic prompts get near-zero hit rates. The overhead is not worth it. Teams with strict data retention rules should not persist responses. Teams that already proxy through an API gateway may have caching built in. Check your existing layer first.

Start with one endpoint. Run the bench. Read the hit rate. Let the numbers decide.