Pooling HTTP Connections to NHTSA Without Starving VIN Decode Workers

A free VIN decode edge that opens a fresh TCP/TLS session for every DecodeVinValues call pays handshake latency on every paste. One giant shared agent with unlimited sockets can starve other workers when NHTSA slows. Concurrency semaphores (covered elsewhere) cap logical in-flight decodes. Token buckets (elsewhere) shape start rate. Circuit breakers (elsewhere) stop calling an unhealthy dependency. This post is different: HTTP connection pooling / keep-alive limits -- reuse warm connections to NHTSA, bound pool size and wait time, and fail honestly when the pool is exhausted instead of inventing decode specs while sockets pile up.

The goal is narrow: configure a keep-alive pool for upstream DecodeVinValues, size it so workers are not starved, and surface pool-wait timeouts without fake catalog cards. Pooling is a transport concern: it does not replace app-level concurrency caps, and it must not become an unbounded socket factory under load.

Connection pool vs concurrency, token bucket, circuit breaker

• Concurrency semaphore -- how many decode operations may run at once (app logic)
• Token bucket -- how many starts per second you allow
• Circuit breaker -- whether to refuse upstream calls after error spikes
• Connection pool -- how many TCP/TLS connections may be open/idle/waiting toward NHTSA

You can have free semaphore slots and still block on a saturated pool -- or a huge pool that opens more sockets than NHTSA or your process can handle. Size the pool to match (or sit slightly under) your concurrency cap.

Shape the pool

Use a shared undici/fetch Agent (or equivalent) with bounded connections, bounded pending, and keep-alive. Acquire implicitly on each upstream call; treat pool timeouts as retryable infrastructure errors, not as successful empty decodes.

import { Agent, request } from "undici";

export type PoolConfig = {
  connections: number; // max open sockets toward NHTSA
  pipelining: number;
  keepAliveTimeout: number; // ms
  keepAliveMaxTimeout: number;
  connectTimeout: number;
  bodyTimeout: number;
  headersTimeout: number;
};

export function createNhtsaAgent(cfg: PoolConfig): Agent {
  return new Agent({
    connections: cfg.connections,
    pipelining: cfg.pipelining,
    keepAliveTimeout: cfg.keepAliveTimeout,
    keepAliveMaxTimeout: cfg.keepAliveMaxTimeout,
    connect: { timeout: cfg.connectTimeout },
    bodyTimeout: cfg.bodyTimeout,
    headersTimeout: cfg.headersTimeout,
  });
}

export type UpstreamResult =
  | { ok: true; status: number; body: string }
  | { ok: false; reason: "pool-timeout" | "upstream-error"; detail: string };

export async function decodeVinValuesPooled(
  agent: Agent,
  vinNormalized: string,
  baseUrl = "https://vpic.nhtsa.dot.gov/api/vehicles/DecodeVinValues",
): Promise<UpstreamResult> {
  const url = `${baseUrl}/${encodeURIComponent(vinNormalized)}?format=json`;
  try {
    const res = await request(url, {
      method: "GET",
      dispatcher: agent,
      headersTimeout: 5_000,
      bodyTimeout: 10_000,
    });
    const body = await res.body.text();
    if (res.statusCode >= 500) {
      return {
        ok: false,
        reason: "upstream-error",
        detail: `status ${res.statusCode}`,
      };
    }
    return { ok: true, status: res.statusCode, body };
  } catch (err) {
    const msg = err instanceof Error ? err.message : String(err);
    // undici surfaces connect/pool waits as errors -- treat as infra, not specs
    if (/timeout|connect|pool|network/i.test(msg)) {
      return { ok: false, reason: "pool-timeout", detail: msg };
    }
    return { ok: false, reason: "upstream-error", detail: msg };
  }
}

export function poolFailureMessage(r: Extract<UpstreamResult, { ok: false }>): string {
  return r.reason === "pool-timeout"
    ? "Decode busy: connection pool wait timed out -- try again shortly"
    : "Decode failed talking to NHTSA -- try again later";
}

Pair connections with your concurrency semaphore (e.g. both 8). If semaphore is 32 and pool is 2, workers queue on sockets and look "stuck." If pool is 100 and semaphore is 8, you waste file descriptors. Watch pending-acquire counts: a rising pending queue with flat successful decodes usually means the pool is undersized relative to concurrency -- or NHTSA is slow enough that warm sockets never return in time. Prefer failing a single request with pool-timeout over letting every worker block indefinitely on socket checkout.

Sizing without starvation

Prefer:

1. Pool size ≈ concurrency cap for the NHTSA host
2. Short connect/headers timeouts so wedged sockets free workers
3. Keep-alive so warm sockets serve repeat pastes cheaply
4. Honest pool-timeout messages -- never invent Make/Model/Year
5. Metrics: active, pending, timeouts -- tune from data

Avoid unlimited agents "for speed" and avoid serializing all traffic onto one socket when concurrency expects parallelism.

Forbidden upgrades

1. Opening a new TLS session per VIN while claiming you "pool"
2. Returning folklore catalog rows when pool-timeout fires
3. Setting connections far above the concurrency semaphore so sockets starve the process
4. Disabling keep-alive to "simplify" and accepting handshake storms
5. Treating pool wait as success with an empty Results array

Refuse those. A bounded warm pool is infrastructure honesty, not a place to fake specs.

Quick checks

import assert from "node:assert/strict";

const cfg: PoolConfig = {
  connections: 2,
  pipelining: 1,
  keepAliveTimeout: 10_000,
  keepAliveMaxTimeout: 30_000,
  connectTimeout: 2_000,
  bodyTimeout: 5_000,
  headersTimeout: 5_000,
};
assert.equal(cfg.connections, 2);
assert.ok(cfg.keepAliveTimeout > 0);

const fail: UpstreamResult = {
  ok: false,
  reason: "pool-timeout",
  detail: "Connect Timeout Error",
};
assert.ok(/connection pool wait/i.test(poolFailureMessage(fail)));
assert.ok(!/Make|ModelYear|invented/i.test(poolFailureMessage(fail)));

// Documented invariant: pool connections should not greatly exceed app concurrency
const appConcurrency = 8;
assert.ok(cfg.connections <= appConcurrency);

Review rule: pool modules must bound sockets toward NHTSA and must not invent decode bodies on pool timeout.

Takeaway

HTTP connection pooling keeps DecodeVinValues warm without handshake storms, but an oversized or undersized pool can starve workers just as badly as no pool at all. Match pool size to concurrency, keep timeouts tight, reuse keep-alive sockets, and fail with honest busy messages when the pool cannot grant a connection. Tune from metrics, not folklore. Your free VIN edge stays calm when sockets are bounded infrastructure -- never a backdoor for invented catalog marketing.

I maintain VIN Lookup, a free VIN decode based on NHTSA data.