Adapting VIN Decode Timeouts from Recent NHTSA Latency Percentiles

A free VIN edge that hard-codes one upstream timeout forever either cuts off healthy DecodeVinValues calls during a quiet hour or waits too long when NHTSA p95 has already climbed. Static budget splits (client / edge / upstream slices of one UX deadline) are covered elsewhere. Circuit half-open probes and deadline propagation are separate tools. This post is about adapting the Next upstream wait from recent observed latency percentiles -- not inventing a new flat constant every deploy.

The goal is honest timing: raise or lower the DecodeVinValues timeout from a rolling p50/p95 window, clamp to safe floors and ceilings, and never treat a percentile guess as proof that NHTSA is "fast today" for marketing copy.

Adaptive timeout vs static budget split

Concern	Static budget (#143-style)	Adaptive percentile timeout
Input	One UX deadline minted at submit	Rolling latency samples from recent upstream calls
Output	Nested client / edge / upstream slices	A single next-upstream timeoutMs (then nest budgets as usual)
Changes when	Rarely (config / deploy)	After enough samples in the window
Failure if misused	Orphaned hops when slices drift	Thrashing timeouts from tiny sample sets

Use adaptive timeouts when you already record successful and timed-out DecodeVinValues durations. Keep the static nest for how much of the UX deadline each hop may consume. Adaptation only proposes the upstream slice (or a multiplier on it) -- it does not replace AbortSignal propagation or half-open probes.

Record samples, compute percentiles, clamp

Keep a ring buffer of recent upstream durations (success and timeout). Require a minimum sample count before adapting. Compute p50 and p95; set the next timeout to something like p95 * factor clamped between a floor (never starve healthy calls) and a ceiling (never hang the edge forever).

export type LatencySample = {
  durationMs: number;
  outcome: "ok" | "timeout" | "error";
  at: number; // epoch ms
};

export type AdaptiveTimeoutConfig = {
  windowSize: number; // max samples retained
  minSamples: number; // refuse adapt below this
  p95Factor: number; // e.g. 1.25
  floorMs: number; // e.g. 1_500
  ceilingMs: number; // e.g. 8_000
  defaultMs: number; // used until minSamples
};

export function pushSample(
  buf: LatencySample[],
  sample: LatencySample,
  windowSize: number,
): LatencySample[] {
  const next = [...buf, sample];
  if (next.length <= windowSize) return next;
  return next.slice(next.length - windowSize);
}

function percentile(sorted: number[], p: number): number {
  if (sorted.length === 0) return 0;
  const idx = Math.min(
    sorted.length - 1,
    Math.max(0, Math.ceil((p / 100) * sorted.length) - 1),
  );
  return sorted[idx]!;
}

export function suggestUpstreamTimeoutMs(
  samples: LatencySample[],
  cfg: AdaptiveTimeoutConfig,
): { timeoutMs: number; source: "default" | "p95"; p50: number | null; p95: number | null } {
  const usable = samples.filter((s) => s.outcome === "ok" || s.outcome === "timeout");
  if (usable.length < cfg.minSamples) {
    return {
      timeoutMs: cfg.defaultMs,
      source: "default",
      p50: null,
      p95: null,
    };
  }
  const durations = usable.map((s) => s.durationMs).sort((a, b) => a - b);
  const p50 = percentile(durations, 50);
  const p95 = percentile(durations, 95);
  const raw = Math.round(p95 * cfg.p95Factor);
  const timeoutMs = Math.min(cfg.ceilingMs, Math.max(cfg.floorMs, raw));
  return { timeoutMs, source: "p95", p50, p95 };
}

Do not adapt from a single slow call. Do not lower the floor so far that a normal DecodeVinValues always races the abort. Do not raise the ceiling past your edge platform's remaining time.

Wire into the next decode -- keep budgets nested

Adaptation proposes timeoutMs. Still nest it under the end-to-end UX budget: upstreamMs = min(suggested, remainingBudgetUpstreamSlice). Record the actual duration when the call settles so the next suggestion stays honest.

export async function decodeWithAdaptiveTimeout(
  vin: string,
  samples: LatencySample[],
  cfg: AdaptiveTimeoutConfig,
  remainingUpstreamBudgetMs: number,
  fetchDecode: (vin: string, timeoutMs: number) => Promise<unknown>,
): Promise<unknown> {
  const suggestion = suggestUpstreamTimeoutMs(samples, cfg);
  const timeoutMs = Math.min(suggestion.timeoutMs, remainingUpstreamBudgetMs);
  const started = Date.now();
  try {
    const body = await fetchDecode(vin, timeoutMs);
    samples.splice(
      0,
      samples.length,
      ...pushSample(
        samples,
        { durationMs: Date.now() - started, outcome: "ok", at: Date.now() },
        cfg.windowSize,
      ),
    );
    return body;
  } catch (err) {
    const durationMs = Date.now() - started;
    const outcome =
      err instanceof Error && /timeout|aborted/i.test(err.message)
        ? "timeout"
        : "error";
    samples.splice(
      0,
      samples.length,
      ...pushSample(
        samples,
        { durationMs, outcome, at: Date.now() },
        cfg.windowSize,
      ),
    );
    throw err;
  }
}

Never advertise "NHTSA is fast today" from a low p95 in product copy. Percentiles are operational inputs, not safety or catalog claims.

Forbidden upgrades

1. Setting timeout from one sample or from error-only samples (skews high)
2. Removing the floor/ceiling so adaptation can starve or hang the edge
3. Replacing static budget nesting with "just use p95" as the whole UX deadline
4. Claiming adaptive timeout proves DecodeVinValues health for users
5. Sharing one global adapter across unrelated upstreams without separate windows

Quick checks

import assert from "node:assert/strict";

const cfg: AdaptiveTimeoutConfig = {
  windowSize: 20,
  minSamples: 5,
  p95Factor: 1.25,
  floorMs: 1_500,
  ceilingMs: 8_000,
  defaultMs: 4_000,
};

assert.equal(suggestUpstreamTimeoutMs([], cfg).source, "default");
assert.equal(suggestUpstreamTimeoutMs([], cfg).timeoutMs, 4_000);

const samples: LatencySample[] = Array.from({ length: 8 }, (_, i) => ({
  durationMs: 1_000 + i * 100,
  outcome: "ok" as const,
  at: Date.now(),
}));
const suggested = suggestUpstreamTimeoutMs(samples, cfg);
assert.equal(suggested.source, "p95");
assert.ok(suggested.timeoutMs >= cfg.floorMs);
assert.ok(suggested.timeoutMs <= cfg.ceilingMs);
assert.ok(suggested.p95 != null && suggested.p50 != null);

const clamped = suggestUpstreamTimeoutMs(
  Array.from({ length: 8 }, () => ({
    durationMs: 20_000,
    outcome: "timeout" as const,
    at: Date.now(),
  })),
  cfg,
);
assert.equal(clamped.timeoutMs, cfg.ceilingMs);

Review rule: adaptive-timeout modules must require min samples, clamp floors/ceilings, and stay nested under the UX budget -- never replace half-open probes or static slice math with a bare percentile.

Takeaway

Adapt the Next DecodeVinValues wait from rolling latency percentiles, clamp hard, and keep static budget nesting for how slices share one deadline. Your free VIN API stays kind to users and to NHTSA when timeouts follow recent reality instead of a forgotten constant -- without inventing health grades from p95 alone.

I maintain VIN Lookup, a free VIN decode based on NHTSA data.

Comments

[SRKDAN]

Requiring a minimum sample count and clamping between a floor and ceiling is the part that keeps adaptive timeouts from thrashing. One practical addition is to count timeouts as samples at the timeout value itself, otherwise the p95 only sees calls that finished and the timeout tends to drift lower than it should. I would also add a little hysteresis so it only changes when the new value differs by more than, say, 10 to 15 percent. Do you reset the window after a long idle period, when old samples may not reflect the current upstream?