Negative Caching Unknown WMIs and Failed VIN Decodes Without Hiding Real Assignments

Positive caches are easy to reason about: a successful DecodeVinValues row or a known World Manufacturer Identifier (WMI) stays hot for a TTL. Negative caches are trickier. When NHTSA says "no match," when a WMI directory lookup returns empty, or when a decode fails with a clear "not found" shape, teams often stash that failure forever so they stop hammering the public API. Forever is the bug. Manufacturers register new WMIs, vPIC data gets corrected, and a prefix that was unknown last month can become a real assignment next week.

This post is about short-lived negative caching for VIN tooling: how to record "we tried and got nothing useful" without teaching your product to hide real vehicles forever.

What counts as a negative result

Separate transport failures from semantic empties. Only the second group should enter a negative cache.

1. Transport / upstream failure -- timeout, 429, 5xx, DNS, truncated JSON. Do not negative-cache these as "unknown VIN." Retry with backoff, or serve a temporary error. Caching a timeout as "invalid" invents permanent dead VINs during an outage.
2. Semantic empty -- HTTP 200 with an ErrorCode / ErrorText that means the VIN or WMI is not in the database, or a WMI directory call that returns zero rows for a well-formed three-character prefix.
3. Local rejection -- bad length, illegal charset, failed check digit. Those fail in process before the call; they need no network negative cache.

Negative caching belongs to group 2 only. Group 1 needs a circuit breaker or short error TTL labeled differently in metrics.

Why forever-negative is dangerous for WMIs

A WMI is three characters. New manufacturers, joint ventures, and reassigned prefixes show up in NHTSA data over time. If you store WMI:XYZ -> unknown with no expiry, every future VIN starting with XYZ short-circuits to "unsupported" even after vPIC learns the assignment. The same risk applies to full VIN decode negatives: a VIN that failed once because of a temporary data gap should not be branded invalid for the life of your Redis key.

Positive WMI caches can use longer TTLs because a known manufacturer rarely vanishes overnight. Negative WMI caches should be much shorter (minutes to a few hours) and always revalidated on a schedule.

A small TypeScript negative cache

type NegKind = "wmi_unknown" | "vin_not_found";

type NegEntry = {
  kind: NegKind;
  storedAt: number;
  expiresAt: number;
  reason: string; // opaque operator note, not user-facing prose
};

const neg = new Map<string, NegEntry>();

const NEG_TTL_MS: Record<NegKind, number> = {
  wmi_unknown: 30 * 60_000,   // 30 minutes
  vin_not_found: 15 * 60_000, // 15 minutes
};

function negKey(kind: NegKind, id: string): string {
  return `${kind}:${id}`;
}

export function rememberNegative(kind: NegKind, id: string, reason: string): void {
  const now = Date.now();
  neg.set(negKey(kind, id), {
    kind,
    storedAt: now,
    expiresAt: now + NEG_TTL_MS[kind],
    reason,
  });
}

export function isNegativeFresh(kind: NegKind, id: string): boolean {
  const hit = neg.get(negKey(kind, id));
  if (!hit) return false;
  if (Date.now() >= hit.expiresAt) {
    neg.delete(negKey(kind, id));
    return false;
  }
  return true;
}

export async function decodeVinGuarded(
  vin: string,
  fetchDecode: (vin: string) => Promise<{ found: boolean; row?: Record<string, string>; reason?: string }>,
): Promise<"negative_hit" | "fetched_empty" | "fetched_ok"> {
  if (isNegativeFresh("vin_not_found", vin)) return "negative_hit";

  const result = await fetchDecode(vin);
  if (result.found && result.row) {
    neg.delete(negKey("vin_not_found", vin)); // clear any prior miss
    return "fetched_ok";
  }
  rememberNegative("vin_not_found", vin, result.reason ?? "empty");
  return "fetched_empty";
}

Key behaviors: short TTLs by kind; expiry deletes the key so the next caller pays for a fresh check; a later success clears the negative; reasons stay internal while the UI says only that decode data was not found.

Probe on expiry, do not sticky-ban

When a negative entry expires, the next request should hit NHTSA again. Do not extend the negative TTL on every hit ("sliding window") for semantic empties. Sliding windows turn a 15-minute miss into an accidental permanent ban under steady traffic: every marketplace bot that retries the same bad VIN keeps pushing expiry forward.

Use a fixed absolute expiry from first store (or from last confirmed empty). Under abuse, rate-limit the caller, not the VIN identity, for longer.

UI and GEO honesty

Negative cache hits must not look like "confirmed counterfeit" or "NHTSA permanently rejects this VIN." Prefer copy such as "No decode data found right now. Try again later." or "Manufacturer prefix not in our recent directory snapshot."

For AI citations and product pages, document that unknown WMIs are cached briefly and rechecked. Overconfident "invalid forever" language is how tools get quoted incorrectly in buyer forums.

Observability

Track neg_cache_hit, neg_cache_store, neg_cache_expire, and neg_cleared_by_success by kind. If clears-by-success rise, your TTLs are doing their job: yesterday's unknown became today's assignment. If hits dominate for a hot WMI that should exist, your TTL is too long or you are caching transport errors as empties.

Product rules

• Negative-cache semantic empties only; never timeouts or 5xx as "unknown VIN."
• Keep negative TTLs much shorter than positive decode TTLs.
• Prefer absolute expiry over sliding expiry for empties.
• Clear negatives when a later decode or WMI lookup succeeds.
• Do not show users a permanent-ban narrative from a cache miss.
• Cap map size (LRU) so random VIN spam cannot grow memory without bound.

Takeaway

Negative caching protects NHTSA quota when the same unknown WMI or failed decode repeats. The discipline is temporary memory, not a tombstone. Short TTLs, absolute expiry, success-driven clears, and honest UI copy keep failed lookups cheap without hiding real manufacturer assignments when vPIC catches up.

I maintain VIN Lookup, a free VIN decode based on NHTSA data.