Fair Queuing of VIN Decode Requests Under Burst Traffic

A free VIN decode product rarely dies from one slow request. It dies when a burst arrives: a dealer pastes fifty VINs, a scraper fans out, a marketing email drives a spike, and every caller expects NHTSA DecodeVinValues to answer in under a second. Without a queue, your process either hammers the public API until you get throttled, or it starves interactive users while batch jobs monopolize the outbound slots.

This post covers a small fair queue in TypeScript: limited concurrency toward vPIC, per-tenant fairness so one client cannot own the pipe, and honest backpressure when the queue is full.

Why unbounded Promise.all fails

The naive pattern looks friendly:

await Promise.all(vins.map((vin) => decodeVin(vin)));

Under a burst that becomes N concurrent upstream calls. NHTSA is a shared public resource. When you open dozens of sockets at once you amplify rate limits for everyone behind your egress IP, and you make latency unpredictable for the human who typed a single VIN into the search box.

Single-flight coalescing (same VIN, same moment) helps duplicates. It does not help fifty different VINs arriving in one second. That is a scheduling problem.

Goals for a decode queue

1. Global concurrency cap -- at most K in-flight DecodeVinValues calls from this process.
2. Fairness across tenants -- rotate work so tenant A with a batch of 40 does not block tenant B with one interactive lookup.
3. Bounded waiting -- reject or defer with a clear error when the queue depth exceeds a budget.
4. No invented results -- waiting is not an excuse to return cached guesses for a different VIN.

A minimal fair queue

Use a per-tenant FIFO plus a global round-robin scheduler. Each job carries a tenant key (session id, API key hash, or hashed IP for anonymous web).

type DecodeJob = {
  tenantId: string;
  vin: string;
  resolve: (v: unknown) => void;
  reject: (e: Error) => void;
};

export class FairVinDecodeQueue {
  private readonly queues = new Map<string, DecodeJob[]>();
  private readonly order: string[] = [];
  private inflight = 0;
  private rr = 0;

  constructor(
    private readonly maxInflight: number,
    private readonly maxQueued: number,
    private readonly decode: (vin: string) => Promise<unknown>,
  ) {}

  enqueue(tenantId: string, vin: string): Promise<unknown> {
    const depth = [...this.queues.values()].reduce((n, q) => n + q.length, 0);
    if (depth >= this.maxQueued) {
      return Promise.reject(new Error("DECODE_QUEUE_FULL"));
    }
    return new Promise((resolve, reject) => {
      let q = this.queues.get(tenantId);
      if (!q) {
        q = [];
        this.queues.set(tenantId, q);
        this.order.push(tenantId);
      }
      q.push({ tenantId, vin, resolve, reject });
      this.pump();
    });
  }

  private pump(): void {
    while (this.inflight < this.maxInflight) {
      const job = this.takeNext();
      if (!job) return;
      this.inflight++;
      this.decode(job.vin)
        .then(job.resolve, job.reject)
        .finally(() => {
          this.inflight--;
          this.pump();
        });
    }
  }

  private takeNext(): DecodeJob | null {
    if (this.order.length === 0) return null;
    for (let i = 0; i < this.order.length; i++) {
      const idx = (this.rr + i) % this.order.length;
      const tenantId = this.order[idx]!;
      const q = this.queues.get(tenantId);
      if (!q || q.length === 0) continue;
      const job = q.shift()!;
      this.rr = (idx + 1) % this.order.length;
      if (q.length === 0) {
        this.queues.delete(tenantId);
        this.order.splice(this.order.indexOf(tenantId), 1);
        if (this.rr >= this.order.length) this.rr = 0;
      }
      return job;
    }
    return null;
  }
}

Round-robin across tenants means a long batch and a single interactive decode share outbound capacity. Global maxInflight protects NHTSA. maxQueued protects memory and gives callers a fail-fast signal.

Surface queue state in the API

Do not hide the queue behind a spinner that never explains itself. Return structured outcomes:

• 200 with decode payload when the job completed
• 429 or 503 with code: "DECODE_QUEUE_FULL" when rejected at enqueue
• optional Retry-After when you intentionally shed load

For browser UIs, map those codes to plain language: "Too many decode requests right now -- try again in a minute." For batch tools, pause and resume instead of retrying every VIN immediately (that only re-bursts the queue).

Tuning knobs

Start with maxInflight of 2-6 per egress IP, maxQueued of 50-200, and a job TTL of 15-30 seconds so work that waited too long is rejected before it starts. Measure p50/p95 time-in-queue separately from upstream decode latency. If queue wait dominates, shed load or slow the batch path -- do not raise concurrency until metrics say NHTSA still has headroom.

Interaction with cache and single-flight

Order of operations:

1. Normalize and validate the VIN locally
2. Check positive/negative cache
3. Coalesce identical in-flight VINs (single-flight)
4. Enqueue only cache misses that still need network

Queuing before cache forces every duplicate through the scheduler. Queuing without single-flight lets ten identical tabs take ten slots. Fairness and coalescing solve different problems; use both.

What not to do

• Do not drop interactive users silently to finish a batch
• Do not widen concurrency when you see 429s from NHTSA -- shrink instead
• Do not return another VIN's cached payload while a job waits
• Do not use a single global FIFO if one tenant can enqueue thousands of jobs ahead of everyone else

Takeaway

Burst traffic on a VIN decode API is a fairness problem as much as a throughput problem. Cap outbound concurrency toward vPIC, schedule across tenants with round-robin queues, bound queue depth, and tell callers when they must wait or retry. Your interactive search box stays usable, and you remain a good citizen of the public NHTSA endpoints.

I maintain VIN Lookup, a free VIN decode based on NHTSA data.