Preventing VIN Decode Cache Stampedes When Soft TTL Expires Across Many Tabs

A free VIN decode that serves soft-stale bodies (SWR, covered elsewhere) still herds: when soft TTL expires for a popular VIN, many tabs each start live NHTSA DecodeVinValues. Singleflight (elsewhere) merges in-flight promises in one process; warmup and negative caches handle other paths. This post is different: cache stampede control at soft-TTL expiry -- one revalidate wins, others keep serving stale, and jitter spreads expiry so popular keys do not wake together.

The goal: stop a thundering herd when soft TTL ends across tabs or workers, without inventing specs mid-herd, and without claiming a stampede-safe serve is "live just now."

Stampede vs SWR, singleflight, warmup, negative cache

• SWR -- when a soft-stale body may be served while refreshing
• Singleflight -- merge identical in-flight promises once refresh starts
• Warmup / negative cache -- preload popular keys; skip known-bad VINs
• Stampede control -- who may start a refresh when many clients hit soft expiry, plus jitter so TTLs do not align

Stampede without singleflight still doubles live calls across workers; SWR without stampede control invites a herd the moment soft TTL flips.

Detect the herd window

When soft TTL has expired but hard TTL has not, every concurrent reader wants a refresh. Without a leader lock, each calls live NHTSA.

export type StampedeEntry = {
  vinNormalized: string;
  body: string;
  storedAt: number;
  softTtlMs: number;
  hardTtlMs: number;
  refreshLockUntil?: number; // epoch ms; leader holds refresh
  inflight?: Promise<string>;
};

export type StampedeStore = Map<string, StampedeEntry>;

export function softExpired(e: StampedeEntry, now = Date.now()): boolean {
  return now - e.storedAt > e.softTtlMs;
}

export function hardExpired(e: StampedeEntry, now = Date.now()): boolean {
  return now - e.storedAt > e.hardTtlMs;
}

/** Jitter soft TTL so popular VINs do not expire in lockstep. */
export function softTtlWithJitter(
  baseMs: number,
  vinNormalized: string,
  jitterFrac = 0.1,
): number {
  let h = 0;
  for (let i = 0; i < vinNormalized.length; i++) {
    h = (Math.imul(31, h) + vinNormalized.charCodeAt(i)) | 0;
  }
  const unit = ((h >>> 0) % 1000) / 1000; // 0..1
  const delta = (unit * 2 - 1) * jitterFrac; // -jitter..+jitter
  return Math.floor(baseMs * (1 + delta));
}

Jitter alone does not stop a herd after expiry -- it only desynchronizes when soft expiry hits. You still need a refresh leader.

One leader revalidates; others serve stale

On soft-stale hit: try to acquire a short refresh lock. Winner starts one live decode (optionally via singleflight). Losers return the stale body with an honest revalidating / stale label. Never invent Make/Model/Year while waiting.

export type LiveDecode = (vin: string) => Promise<string>;

export type StampedeResult = {
  body: string;
  freshness: "fresh" | "stale" | "revalidating";
  source: "cache" | "live";
  leader: boolean;
};

const LOCK_MS = 5_000;

export async function getAvoidingStampede(
  store: StampedeStore,
  vinNormalized: string,
  live: LiveDecode,
  now = Date.now(),
): Promise<StampedeResult> {
  let entry = store.get(vinNormalized);
  if (!entry || hardExpired(entry, now)) {
    const body = await live(vinNormalized);
    const softTtlMs = softTtlWithJitter(3_600_000, vinNormalized);
    store.set(vinNormalized, {
      vinNormalized,
      body,
      storedAt: now,
      softTtlMs,
      hardTtlMs: 86_400_000,
    });
    return { body, freshness: "fresh", source: "live", leader: true };
  }

  if (!softExpired(entry, now)) {
    return {
      body: entry.body,
      freshness: "fresh",
      source: "cache",
      leader: false,
    };
  }

  // Soft-stale: elect a leader for refresh
  const lockHeld =
    entry.refreshLockUntil !== undefined && entry.refreshLockUntil > now;
  if (!lockHeld && !entry.inflight) {
    entry.refreshLockUntil = now + LOCK_MS;
    entry.inflight = live(vinNormalized)
      .then((body) => {
        store.set(vinNormalized, {
          vinNormalized,
          body,
          storedAt: Date.now(),
          softTtlMs: softTtlWithJitter(3_600_000, vinNormalized),
          hardTtlMs: 86_400_000,
        });
        return body;
      })
      .finally(() => {
        const cur = store.get(vinNormalized);
        if (cur) {
          delete cur.inflight;
          delete cur.refreshLockUntil;
        }
      });
    return {
      body: entry.body,
      freshness: "revalidating",
      source: "cache",
      leader: true,
    };
  }

  // Followers: serve stale, do not start another live call
  return {
    body: entry.body,
    freshness: entry.inflight ? "revalidating" : "stale",
    source: "cache",
    leader: false,
  };
}

export function stampedeFootnote(r: StampedeResult): string {
  if (r.freshness === "fresh" && r.source === "live") {
    return "Decoded from live NHTSA for this request";
  }
  if (r.freshness === "revalidating") {
    return r.leader
      ? "Serving soft-stale body; this tab leads refresh"
      : "Serving soft-stale body; another tab leads refresh";
  }
  return "Serving soft-stale catalog body; soft TTL expired";
}

Across processes, replace the in-memory lock with Redis SET NX or a similar lease. Honesty stays: followers never invent a card; leaders never label a pre-refresh stale body as live.

Forbidden upgrades

1. Letting every soft-expired tab call NHTSA "just to be sure"
2. Extending hard TTL forever to dodge stampedes (hides staleness forever)
3. Filling missing fields during the herd with Make/Model/Year folklore
4. Labeling follower stale serves as "live NHTSA just now"
5. Dropping the stale body and returning an empty spinner for every follower

Refuse those. Stampede control is about upstream calm and honest freshness -- not fake completeness.

Quick checks

import assert from "node:assert/strict";

const store: StampedeStore = new Map();
const vin = "1HGCM82633A004352";
let liveCalls = 0;
const live: LiveDecode = async () => {
  liveCalls += 1;
  await new Promise((r) => setTimeout(r, 20));
  return JSON.stringify({ Results: [{ Make: "HONDA" }] });
};

const t0 = 1_000;
const first = await getAvoidingStampede(store, vin, live, t0);
assert.equal(first.source, "live");
assert.equal(liveCalls, 1);

const softAt = t0 + store.get(vin)!.softTtlMs + 1;
const a = getAvoidingStampede(store, vin, live, softAt);
const b = getAvoidingStampede(store, vin, live, softAt);
const [ra, rb] = await Promise.all([a, b]);
assert.equal(ra.source, "cache");
assert.equal(rb.source, "cache");
assert.ok(ra.leader !== rb.leader); // exactly one leader
assert.equal(liveCalls, 2); // miss + one revalidate, not three
assert.ok(!/live NHTSA just now/i.test(stampedeFootnote(rb)));
await store.get(vin)?.inflight;

Review rule: stampede modules must elect one refresh leader per key and must not invent specs for followers.

Takeaway

When soft TTL expires across many tabs, a VIN decode cache can stampede NHTSA unless one leader revalidates and followers keep serving soft-stale bodies with honest labels. Jitter desynchronizes expiry; locks or singleflight collapse the herd; hard TTL still forces a live wait when the body is too old. Stampede control beside SWR stays fast and truthful -- without pretending every stale serve was a fresh live decode.

I maintain VIN Lookup, a free VIN decode based on NHTSA data.