A Small Node.js Wrapper for LLM API Retries, Timeouts, and Logging

Most LLM API integrations start with a direct SDK call.

That is fine for a demo.

But once the call is inside a real product, I usually want three things around it:

• a timeout
• retry rules
• useful logs when something fails

Not a giant framework. Not a full observability platform. Just a small wrapper that makes the boring production stuff harder to forget.

Here is the Node.js wrapper pattern I usually reach for.

The problem

A normal LLM call often starts like this:

const response = await client.chat.completions.create({
  model: "gpt-4.1-mini",
  messages: [
    { role: "user", content: "Summarize this support ticket." }
  ]
});

That is readable, but it hides a few production questions:

• How long can this request run before we give up?
• Which errors should be retried?
• How many times?
• Are retries safe for this workflow?
• What do we log when the request fails?
• Can someone debug this later without reproducing the whole request?

The wrapper below is intentionally small. It does not try to solve every LLM reliability problem.

It just gives each request a basic failure policy.

A minimal wrapper

This version uses fetch, so it works on modern Node.js without extra dependencies.

const DEFAULT_TIMEOUT_MS = 30_000;

function sleep(ms) {
  return new Promise((resolve) => setTimeout(resolve, ms));
}

function createTimeoutSignal(timeoutMs) {
  const controller = new AbortController();
  const timeout = setTimeout(() => controller.abort(), timeoutMs);

  return {
    signal: controller.signal,
    clear: () => clearTimeout(timeout)
  };
}

function shouldRetry({ status, error, attempt, maxRetries }) {
  if (attempt >= maxRetries) return false;

  if (error?.name === "AbortError") {
    return true;
  }

  if (!status) {
    return true;
  }

  return status === 429 || status === 500 || status === 502 || status === 503 || status === 504;
}

function retryDelayMs(attempt, retryAfterHeader) {
  if (retryAfterHeader) {
    const seconds = Number(retryAfterHeader);
    if (Number.isFinite(seconds)) {
      return seconds * 1000;
    }
  }

  const base = 500 * 2 ** attempt;
  const jitter = Math.floor(Math.random() * 250);

  return base + jitter;
}

export async function callLlmWithPolicy({
  url,
  apiKey,
  body,
  timeoutMs = DEFAULT_TIMEOUT_MS,
  maxRetries = 2,
  requestId = crypto.randomUUID(),
  logger = console
}) {
  let lastError;

  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    const startedAt = Date.now();
    const timeout = createTimeoutSignal(timeoutMs);

    try {
      logger.info({
        event: "llm_request_started",
        requestId,
        attempt,
        model: body.model,
        timeoutMs
      });

      const response = await fetch(url, {
        method: "POST",
        signal: timeout.signal,
        headers: {
          "Authorization": `Bearer ${apiKey}`,
          "Content-Type": "application/json"
        },
        body: JSON.stringify(body)
      });

      const durationMs = Date.now() - startedAt;
      const text = await response.text();

      if (!response.ok) {
        const retryAfter = response.headers.get("retry-after");

        logger.warn({
          event: "llm_request_failed",
          requestId,
          attempt,
          status: response.status,
          durationMs,
          retryAfter,
          bodyPreview: text.slice(0, 500)
        });

        if (shouldRetry({
          status: response.status,
          attempt,
          maxRetries
        })) {
          await sleep(retryDelayMs(attempt, retryAfter));
          continue;
        }

        throw new Error(`LLM request failed with status ${response.status}: ${text}`);
      }

      logger.info({
        event: "llm_request_succeeded",
        requestId,
        attempt,
        status: response.status,
        durationMs
      });

      return JSON.parse(text);
    } catch (error) {
      const durationMs = Date.now() - startedAt;
      lastError = error;

      logger.warn({
        event: "llm_request_error",
        requestId,
        attempt,
        errorName: error.name,
        errorMessage: error.message,
        durationMs
      });

      if (shouldRetry({
        error,
        attempt,
        maxRetries
      })) {
        await sleep(retryDelayMs(attempt));
        continue;
      }

      throw error;
    } finally {
      timeout.clear();
    }
  }

  throw lastError;
}

Using it

Here is a basic chat completion request:

import { callLlmWithPolicy } from "./llm-wrapper.js";

const result = await callLlmWithPolicy({
  url: "https://api.openai.com/v1/chat/completions",
  apiKey: process.env.OPENAI_API_KEY,
  body: {
    model: "gpt-4.1-mini",
    messages: [
      {
        role: "user",
        content: "Write a one-paragraph summary of this incident report."
      }
    ]
  },
  timeoutMs: 20_000,
  maxRetries: 2
});

console.log(result.choices[0].message.content);

If you are using an OpenAI-compatible provider, the shape stays almost the same. The main change is usually the base URL and model name.

That is one reason I like keeping this wrapper close to the HTTP boundary instead of burying all of the logic inside application code.

What this wrapper logs

The logs are deliberately simple.

On start:

{
  event: "llm_request_started",
  requestId: "01...",
  attempt: 0,
  model: "gpt-4.1-mini",
  timeoutMs: 20000
}

On failure:

{
  event: "llm_request_failed",
  requestId: "01...",
  attempt: 0,
  status: 429,
  durationMs: 813,
  retryAfter: "2"
}

On success:

{
  event: "llm_request_succeeded",
  requestId: "01...",
  attempt: 1,
  status: 200,
  durationMs: 1420
}

That is enough to answer a few important questions later:

• Did this fail once and recover?
• Did it burn all retries?
• Was it a timeout or provider response?
• Which model was being called?
• How long did each attempt take?
• Did the provider send Retry-After?

For a lot of production debugging, that is already much better than “the AI call failed.”

The part I would not hide in the wrapper

I would be careful about making retries automatic for every LLM call.

Some calls are safe to repeat.

For example:

• summarizing text
• classifying a ticket
• extracting fields from a document
• generating a draft that has not been shown or saved yet

Other calls may not be safe to repeat.

For example:

• sending an email
• creating a ticket
• charging a customer
• calling a webhook
• letting an agent execute a tool

The wrapper cannot know that from the HTTP status code.

So in real apps, I usually add one more input:

retryMode: "safe" | "unsafe"

Then the retry rule becomes stricter:

function shouldRetry({ status, error, attempt, maxRetries, retryMode }) {
  if (attempt >= maxRetries) return false;

  if (retryMode === "unsafe") {
    return false;
  }

  if (error?.name === "AbortError") {
    return true;
  }

  if (!status) {
    return true;
  }

  return status === 429 || status === 500 || status === 502 || status === 503 || status === 504;
}

That small distinction matters.

A retry on a read-only summarization call is boring.

A retry after an agent may have already triggered an external action is a completely different failure mode.

Timeouts need product judgment

I usually avoid one global timeout for every LLM call.

Different workflows have different tolerance.

For example:

const timeoutByWorkflow = {
  autocomplete: 5_000,
  support_summary: 20_000,
  background_analysis: 90_000
};

A user-facing autocomplete should probably fail fast.

A background analysis job can wait longer.

A support summary might sit somewhere in the middle.

The point is not to find the perfect timeout. The point is to make timeout behavior intentional instead of accidental.

Keep the wrapper boring

It is tempting to keep adding features:

• model fallback
• schema validation
• streaming recovery
• circuit breakers
• provider routing
• prompt redaction
• token accounting

Those can all be useful.

But I like starting with the boring layer first:

• timeout each request
• retry only known transient failures
• honor Retry-After
• log every attempt
• avoid retrying unsafe operations
• attach a request ID

That is the part you want in place before the first incident, not after.

Final thought

A small LLM wrapper will not make your app production-ready by itself.

But it does create a place for production policy to live.

Without that layer, retry behavior gets scattered across route handlers, background jobs, SDK calls, and agent tools. Eventually, nobody knows which LLM calls are safe to retry and which ones are quietly dangerous.

My rule now is simple:

If an LLM call matters to the product, it should not be a naked SDK call.

Put a small policy around it.

Your future debugging session will be shorter.

---

I work on TokenBay, so I spend a lot of time thinking about OpenAI-compatible APIs, multi-model routing, retries, and failure handling. The wrapper above is intentionally provider-agnostic: the same basic shape works whether you call one provider directly or route through a compatible API gateway.