One Timeout Became Two Model Calls. My Retry Logic Was the Bug.

Three ordinary things lined up against me last week: a free model endpoint running slow, a free server with a hard 8-second timeout, and a retry I had added "just in case." On their own, none of them is a bug; together, they turned one user click into two model calls, two database rows, and a quiet doubling of my free-tier quota. The worst part is that the logs made it look like the model provider had failed me. The provider had not done anything wrong.

The symptom: two identical completions, seven seconds apart

Here is what I saw when I opened the logs: two completions with the exact same prompt, timestamped about seven seconds apart, both returning status 200. In the database, two summary rows for a single click. My first instinct was to blame the model API — maybe it retried internally, or maybe the free tier duplicated the request. Both instincts were wrong, and the evidence was in the request IDs.

Two details saved me from chasing ghosts. First, the two calls carried different request IDs, which meant the provider had not retried anything; something on my side had sent two requests. Second, the gap between them was almost exactly the timeout I had configured. That gap was the fingerprint of my own retry logic.

Reconstructing the timeline

1. t=0: the user clicks "Summarize," and the server receives the request.
2. t=0.2s: the server calls the model endpoint and waits for a response.
3. t=8s: my timeout wrapper fires, logs "upstream timed out, retrying," and calls the endpoint again.
4. t=11s: the first call finally completes — it was slow, not dead.
5. t=11.2s: the first completion handler writes a row to the database.
6. t=12s: the second call completes and writes another row.

The pattern is embarrassing in hindsight. My timeout had abandoned the first request, but it had never actually cancelled it. The first call was still alive, still generating tokens, and still perfectly capable of running its completion handler when the response arrived.

The code that caused it

async function callModel(prompt: string) {
  try {
    return await withTimeout(modelApi(prompt), 8000);
  } catch {
    console.warn("Timed out. Retrying once.");
    return await modelApi(prompt); // The first call is still running!
  }
}

This is the classic "timeout without cancellation" bug. withTimeout rejects the promise when the timer fires, but the underlying fetch keeps going; Node's event loop still processes the response when it finally arrives, and every await continuation attached to that original promise still runs. The retry did not replace the first call. It joined it as a second, equally expensive request.

The fix, layer by layer

Layer 1: cancel the request for real

async function callModel(prompt: string, requestId: string) {
  const controller = new AbortController();
  const timer = setTimeout(() => controller.abort(), 8000);
  try {
    return await modelApi(prompt, { signal: controller.signal });
  } catch (err) {
    if (err.name === "AbortError") {
      console.warn(`Call ${requestId} aborted after 8s. Retrying once.`);
      return await modelApi(prompt, { signal: AbortSignal.timeout(8000) });
    }
    throw err;
  } finally {
    clearTimeout(timer);
  }
}

Now the timeout actually aborts the HTTP request, and the retry is the only survivor. This alone fixed my duplicate completions, because the first call could no longer finish and write its result. Notice that the retry gets its own timeout, so a slow second attempt cannot hang the server forever.

Layer 2: idempotency keys for the network

Cancellation fixes the common case, but it does not fix every case; what if the first request already reached the model before the abort arrived? Some providers support an Idempotency-Key header, where sending the same key on the original and the retry makes the provider return the cached response instead of generating a second one. Not every provider supports this, so check the documentation before you rely on it. When it exists, it turns "retry safely" from a hope into a guarantee.

Layer 3: make the database the source of truth

Network idempotency is nice, but the database is the only layer you fully control. A unique constraint on the request ID means that even if two calls complete, only one row survives:

CREATE TABLE summaries (
  id SERIAL PRIMARY KEY,
  request_id TEXT UNIQUE NOT NULL,
  prompt TEXT NOT NULL,
  result TEXT NOT NULL,
  created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);

await db.query(
  `INSERT INTO summaries (request_id, prompt, result)
   VALUES ($1, $2, $3)
   ON CONFLICT (request_id) DO NOTHING`,
  [requestId, prompt, result]
);

Now the worst case is a wasted model call instead of a corrupted dataset. The user still gets their summary, the duplicate write silently no-ops, and the database stays clean. That is a trade I will take every time.

The reusable debugging checklist

• Reconstruct the timeline before blaming anyone. Request IDs tell you whether the provider retried or your client did.
• Ask whether the timeout actually cancelled anything. Look for abort events and connection-close logs; if they are missing, your timeout was decorative.
• Add idempotency at the narrowest layer that works. A unique constraint beats an in-memory cache, which beats a hope.
• Test the retry path deliberately. A unit test that mocks a slow response will catch this bug in seconds.

Where this broke, and why it was inevitable

I was testing a summarizer on a MonkeyCode free server, calling the free model endpoint, when the failure showed up. Free infrastructure is where this bug surfaces first, because slow responses are normal there; my 8-second timeout was a guess, and free model endpoints sometimes take longer than a guess. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The fix was not a product feature. It was understanding that a timeout and a cancellation are different things, and that a retry without cancellation is just a duplicate request wearing a costume.

Who should not use this approach

• If your provider has no idempotency support and you cannot add a database constraint, a single-flight lock — one in-flight request per prompt — is a partial alternative, but it does not survive process restarts.
• If your prompt triggers side effects like sending emails or charging cards, deduplication at the response layer is not enough; you need exactly-once semantics at the side-effect layer itself.
• Do not add unbounded retries. On free infrastructure, one retry with a longer timeout is usually the right amount; the second retry is how you end up with three model calls.

The takeaway

Have you ever seen duplicate rows from a single click? If yes, you already know the feeling of watching your own code do twice the work for the same answer. The boring fix — cancel properly, deduplicate at the database, and treat retries as a last resort — is the one that holds up when the model is slow and the server is free.