The Cache Key Is Not a Prompt Output

A checkout service mixed two tenants inside cached carts. The on-call thread blamed a rare cache race. The root cause was not a cache race. A helper asked a model to summarize each request. That summary then became the production cache key. Similar carts then produced nearly identical key phrases. The key space then collapsed across tenant accounts.

This article treats that scene as a composite pattern. It is not a claim about one named outage. The lesson still holds for agent backends. Free models can draft helpers off the request path. They must not mint keys that other nodes will trust.

Why prompt-shaped keys fail

A cache key is a contract, not a caption. It must stay stable across retries and replicas. It must separate tenants with cryptographic boredom. It must refuse to encode secrets in the key string. Natural language output meets none of those rules.

Free models also drift across days and providers. The same prompt can yield a different noun phrase. A changed phrase silently splits one logical object. Or it joins two objects that must never meet. That is a data-isolation bug, not a style issue.

Semantic likeness is the wrong clustering tool here. Caches need equality, not a nearby caption. Search indexes can group similar carts on purpose. Response caches cannot merge them by accident. Those two jobs must not share a key function.

Red flags: when not to mint keys this way

Do not ship a model-minted key in these cases.

1. The key covers more than one tenant or workspace.
2. The key gates cart, session, or entitlement state.
3. The value holds PII, payments, health, or location data.
4. Replicas must agree without a shared model call.
5. The key is also an ETag, lock name, or topic name.
6. A miss is cheap, but a wrong hit is catastrophic.
7. The model lane has no pinned weights or seed.
8. The compute box is preemptible or not durable.

Any one flag is enough to stop the design. Two flags mean the design is already unsafe. A spike on fake data can ignore a single flag. A first real tenant cannot.

What a production key must guarantee

A production cache key needs four boring properties.

• Determinism: identical inputs yield identical keys.
• Tenant prefix: every key starts with a stable tenant id.
• Explicit schema: version, entity, and fields are named.
• No model text: zero tokens from a generative call.

Add a fifth property for HTTP caches. An ETag must hash bytes, not a caption. If-None-Match then stays a pure equality check. A model paraphrase must never flip cache freshness.

A pinned key function

The next snippet is a proposed Node helper. It has not been run against a named production cluster. Treat it as a copy-paste starting point. Keep the model client out of this file.

// cacheKey.js — pinned, no model I/O
const crypto = require("node:crypto");

const KEY_SCHEMA = "cart-v3";

function assertTenantId(tenantId) {
  if (!/^[a-z0-9_-]{8,64}$/.test(tenantId)) {
    throw new Error("refusing unbound tenant id");
  }
}

function pinnedCartKey({ tenantId, cartId, catalogVersion }) {
  assertTenantId(tenantId);
  if (!cartId || !catalogVersion) {
    throw new Error("refusing incomplete cache identity");
  }
  const material = [
    KEY_SCHEMA,
    tenantId,
    "cart",
    cartId,
    `cat:${catalogVersion}`,
  ].join("|");
  const digest = crypto
    .createHash("sha256")
    .update(material, "utf8")
    .digest("hex")
    .slice(0, 32);
  return `${KEY_SCHEMA}:${tenantId}:${digest}`;
}

function refuseModelKey(text) {
  const looksPrompted =
    /\s/.test(text) ||
    text.length > 80 ||
    /summar(y|ise|ize)/i.test(text);
  if (looksPrompted) {
    throw new Error("refusing model-shaped cache key");
  }
  return text;
}

module.exports = { pinnedCartKey, refuseModelKey, KEY_SCHEMA };

The digest is only a compact fingerprint. The tenant id stays in the clear prefix. Operators can still grep one tenant in Redis. The model never sees the key material. Whitespace in a key is treated as a defect. That defect is cheaper than a silent join.

A regression test that must stay red

This test encodes the when-not-to rule. Keep it in CI even if a later agent suggests otherwise. Do not soften the assertions to please a generator.

// cacheKey.test.js
const assert = require("node:assert/strict");
const { pinnedCartKey, refuseModelKey } = require("./cacheKey");

const tenantA = "tenant_north_01";
const tenantB = "tenant_south_09";

const inputA = {
  tenantId: tenantA,
  cartId: "cart_42",
  catalogVersion: "2026-09-20",
};

const inputB = { ...inputA, tenantId: tenantB };

const keyA1 = pinnedCartKey(inputA);
const keyA2 = pinnedCartKey(inputA);
const keyB = pinnedCartKey(inputB);

assert.equal(keyA1, keyA2);
assert.notEqual(keyA1, keyB);
assert.match(keyA1, /^cart-v3:tenant_north_01:/);
assert.match(keyB, /^cart-v3:tenant_south_09:/);

assert.throws(() =>
  refuseModelKey("shared grocery cart for returning user")
);
assert.throws(() =>
  pinnedCartKey({ tenantId: "t", cartId: "x", catalogVersion: "1" })
);

console.log("pinned cache key invariants held");

Run it with a local Node binary.

node --test cacheKey.test.js
# or:
node cacheKey.test.js

A green run means equality and isolation still hold. A red run means a later refactor tried to get clever. Do not fix a red run by asking a model for a nicer key. Restore the pinned function instead.

Decision table

Use this table during design review, not during an incident.

Signal	Free-model key	Pinned function
Same tenant, identical entity ids	Unstable	Required
Cross-tenant similarity	Collision risk	Isolated
Retry on another replica	May diverge	Stable
Need semantic grouping	Use a search index	Do not cache-merge
ETag or If-None-Match	Forbidden	Hash of bytes
Feature still in a spike	Sandbox only	Still pinned
Memoized agent tool result	Cross-user leak risk	User-prefixed hash

Semantic grouping belongs in search or embeddings. It does not belong in GET response caches. Agent tool memoization follows the same rule. A tool result is still tenant data. A caption of the tool call is not an identity.

Where a free model may still help

MonkeyCode offers free model access and a free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Those lanes can draft the helper and the tests. They can run the suite on a disposable box. They must not answer live cache-key RPCs. They must not sit on the Redis write path. A useful spike copies the test file, not the request path.

# proposed local spike, not a production runbook
mkdir -p /tmp/cache-key-spike && cd /tmp/cache-key-spike
# paste cacheKey.js and cacheKey.test.js
node cacheKey.test.js
# review the helper; do not wire it to a model client

Keep the free server off durable cache origins. A preemptible box can vanish mid-write. That is acceptable for generating tests. That is not acceptable for tenant cache partitions. Drafting code is not the same as minting keys.

Better alternatives

When someone proposes a smart key, pick one of these.

1. Hash canonical JSON of already-validated fields.
2. Use the primary key plus a schema version prefix.
3. For HTTP, hash the response bytes for the ETag.
4. For grouping, write a separate search document.
5. For experiments, use an explicit experiment id.
6. For agent memoization, hash tool name, args, and user id.

Each alternative is dull on purpose. Dull keys page people less often. Dull keys also survive a model outage. A cache fill must not wait on a prompt.

Canonical JSON needs a stable field order. Sort keys before the hash. Drop undefined fields instead of stringifying them. Never hash a raw request body with extra headers. Extra headers are not identity.

Exit criteria

Leave the free-lane key idea on the floor when any item is true.

• A wrong hit can leak another tenant's payload.
• Two replicas can mint different keys for one object.
• The key string contains natural-language tokens.
• On-call cannot grep a tenant prefix in the cache.
• A model timeout would block a cache fill.
• Rollback cannot rebuild keys without model history.
• The same string is reused as a lock, topic, or ETag.

If three items fire, treat the design as blocked. Do not A/B test isolation bugs on live tenants. Isolation bugs are not conversion experiments. They are incident reports waiting for traffic.

Who should not use a model-minted key

This approach is not a gray maybe. These teams should refuse it outright.

• Multi-tenant SaaS with shared Redis or CDN layers.
• Payments, identity, and medical record services.
• Edge caches that key on URL plus generated slugs.
• Agent backends that memoize tool results per user.
• Anyone using a free model without pinned outputs.

Solo prototypes on fake data are the exception. Even then, switch to a pinned function before the first user. A demo with one tenant hides the collision. The second tenant is the real test.

Limitations

The helper above does not encrypt cache values. It does not replace IAM or object-level authz. It does not stop a buggy caller from passing the wrong tenant id. Callers must still authenticate and authorize first.

The refuseModelKey heuristic is incomplete on purpose. A determined prompt can still look like a hash. The real control is architectural. No model client belongs in the key module. Lint for SDK imports in that folder.

This article also does not benchmark any vendor. It does not claim quotas, uptime, or model names. Time-sensitive product limits should be checked on the vendor site before a spike. Example catalog versions in the tests are fixtures only.

A short review checklist

Paste this into the pull request template.

- [ ] Cache key module imports no model SDK
- [ ] Tenant id is a prefix, not a prompt
- [ ] Schema version is explicit
- [ ] CI asserts cross-tenant inequality
- [ ] CI rejects whitespace key material
- [ ] Free-lane boxes are not cache origins
- [ ] Search/similarity is a separate store
- [ ] ETags hash bytes, not captions

A reviewer can apply this in a few minutes. The checklist is the actual control plane. Prompt output is useful for drafts and tests. It is the wrong substance for a cache key. Keep the key boring, pinned, and tenant-prefixed. Let models stay off that path.