Limiting Concurrent VIN Decode Upstream Calls So Browser Tabs Do Not Exhaust NHTSA

A free VIN decode edge that forwards every browser tab's paste straight to live NHTSA DecodeVinValues can open dozens of simultaneous upstream sockets. Token buckets (covered elsewhere) shape request rate over time. Quota-header handlers (elsewhere) back off when NHTSA says so. Singleflight (elsewhere) merges identical in-flight VINs. Stampede control (elsewhere) elects one soft-TTL refresher. This post is different: a concurrency limit -- a semaphore / max-in-flight pool so only N DecodeVinValues calls run at once, with clear queue-wait vs reject behavior and no invented specs while waiting.

The goal is narrow: cap simultaneous upstream calls from your edge (or BFF), queue excess work briefly or reject with an honest busy signal, and never fabricate a decode card to "keep the UI full" under load.

Concurrency limit vs rate limit, singleflight, stampede

• Token bucket / rate limit -- how many starts per second over a window
• Quota headers -- honor upstream Retry-After / remaining when told
• Singleflight -- one shared promise per identical VIN key
• Stampede control -- one soft-TTL refresh leader per key
• Concurrency limit -- how many live upstream calls may be in flight at once, regardless of VIN identity

A rate limit of 10/s can still open 50 sockets if each call takes 5s. A concurrency cap of 8 keeps peak sockets honest even when many distinct VINs arrive together.

Semaphore shape

Track inFlight and a FIFO wait queue. Acquire before calling live NHTSA; release in finally. Bound queue length and wait time so tabs do not hang forever.

export type LiveDecode = (vin: string) => Promise<string>;

export type LimitResult =
  | { ok: true; body: string; waitedMs: number }
  | { ok: false; reason: "queue-full" | "wait-timeout"; waitedMs: number };

export type ConcurrencyLimit = {
  maxInFlight: number;
  maxQueue: number;
  maxWaitMs: number;
  inFlight: number;
  waiters: Array<{
    enqueuedAt: number;
    resolve: (granted: boolean) => void;
    timer: ReturnType<typeof setTimeout>;
  }>;
};

export function createLimit(
  maxInFlight = 8,
  maxQueue = 32,
  maxWaitMs = 3_000,
): ConcurrencyLimit {
  return { maxInFlight, maxQueue, maxWaitMs, inFlight: 0, waiters: [] };
}

function grant(limit: ConcurrencyLimit): void {
  limit.inFlight += 1;
}

function release(limit: ConcurrencyLimit): void {
  limit.inFlight -= 1;
  const next = limit.waiters.shift();
  if (!next) return;
  clearTimeout(next.timer);
  next.resolve(true);
}

async function acquire(
  limit: ConcurrencyLimit,
  now = Date.now(),
): Promise<{ granted: boolean; waitedMs: number }> {
  if (limit.inFlight < limit.maxInFlight) {
    grant(limit);
    return { granted: true, waitedMs: 0 };
  }
  if (limit.waiters.length >= limit.maxQueue) {
    return { granted: false, waitedMs: 0 };
  }
  const enqueuedAt = now;
  const granted = await new Promise<boolean>((resolve) => {
    const timer = setTimeout(() => {
      const idx = limit.waiters.findIndex((w) => w.resolve === resolve);
      if (idx >= 0) limit.waiters.splice(idx, 1);
      resolve(false);
    }, limit.maxWaitMs);
    limit.waiters.push({ enqueuedAt, resolve, timer });
  });
  const waitedMs = Date.now() - enqueuedAt;
  if (!granted) return { granted: false, waitedMs };
  grant(limit);
  return { granted: true, waitedMs };
}

export async function decodeWithLimit(
  limit: ConcurrencyLimit,
  vinNormalized: string,
  live: LiveDecode,
): Promise<LimitResult> {
  const { granted, waitedMs } = await acquire(limit);
  if (!granted) {
    return {
      ok: false,
      reason: waitedMs > 0 ? "wait-timeout" : "queue-full",
      waitedMs,
    };
  }
  try {
    const body = await live(vinNormalized);
    return { ok: true, body, waitedMs };
  } finally {
    release(limit);
  }
}

export function busyMessage(r: Extract<LimitResult, { ok: false }>): string {
  return r.reason === "queue-full"
    ? "Decode busy: upstream concurrency queue full -- try again shortly"
    : "Decode busy: waited too long for an upstream slot -- try again shortly";
}

Never map a reject into a fake Make/Model/Year card. Show the busy message; optionally retry client-side with backoff after the user confirms.

Queue wait vs reject

Product pressure often wants infinite queues so "nobody sees an error." Prefer bounded wait:

1. Short queue + timeout -- absorb bursts, then reject honestly
2. Reject fast when full -- protects NHTSA and your edge memory
3. Do not invent specs while a tab waits for a slot
4. Combine with singleflight after acquire so identical VINs still share one live body
5. Combine with token bucket so even under the concurrency cap you do not exceed sustained rate

Concurrency limits are about peak sockets; rate limits are about sustained starts. Use both.

Forbidden upgrades

1. Raising maxInFlight to "unlimited" under load so every tab opens a socket
2. Returning folklore catalog rows when queue-full or wait-timeout fires
3. Labeling a queued wait as "live NHTSA completed" before acquire succeeds
4. Dropping the semaphore because singleflight "already dedupes" (it does not cap distinct VINs)
5. Ignoring NHTSA quota headers because the concurrency cap "should be enough"

Refuse those. An honest busy state beats a stampeded upstream and a lying card.

Quick checks

import assert from "node:assert/strict";

const limit = createLimit(2, 2, 50);
let liveCalls = 0;
const live: LiveDecode = async (vin) => {
  liveCalls += 1;
  await new Promise((r) => setTimeout(r, 80));
  return JSON.stringify({ Results: [{ VIN: vin }] });
};

const jobs = ["VINAAAA", "VINBBBB", "VINCCCC", "VINDDDD", "VINEEEE"].map(
  (v) => decodeWithLimit(limit, v, live),
);
const results = await Promise.all(jobs);
const oks = results.filter((r) => r.ok);
const fails = results.filter((r) => !r.ok);
assert.ok(oks.length >= 2);
assert.ok(fails.length >= 1);
assert.ok(liveCalls <= 4); // cap + short queue, not five parallel forever
assert.ok(
  fails.every((f) => !f.ok && /busy|queue|waited/i.test(busyMessage(f))),
);
assert.ok(
  !results.some(
    (r) => r.ok && /folklore|invented/i.test(r.body),
  ),
);

Review rule: concurrency modules must bound in-flight upstream calls and must not invent decode bodies on reject.

Takeaway

A concurrency limit keeps browser tabs from opening unbounded DecodeVinValues sockets against NHTSA. Cap in-flight calls, bound the wait queue, reject with an honest busy signal, and leave rate limits, quota headers, singleflight, and stampede control in their own lanes. Your free VIN edge stays calm when peak concurrency is explicit -- and never fills a busy moment with invented catalog marketing.

I maintain VIN Lookup, a free VIN decode based on NHTSA data.