A Pairing Hour on Retries: The Failure Taxonomy We Kept and the Three Loops We Deleted

A review bot retried a single file 41 times overnight. The model was fine. The loop was the bug. A senior engineer joined for one hour, and the hour turned into an argument about what deserves a retry rather than how long the backoff should be.

What follows is the questions asked, the three dead ends, and the one decision that survived. The 41-attempt story is the motivating scenario; every number in the tables below comes from a local fixture you can run in a few seconds, not from production telemetry.

The question that reframed the hour

The senior did not open with backoff math. The first question was narrower: which of these failures are actually the same failure?

A retry loop is a claim about failure equivalence. Every for loop with a catch inside says, implicitly, that these errors deserve the same response. Most of them lump a 429, a schema mismatch, and a model refusal into one bucket labelled "transient" — and then spend budget proving otherwise.

The taxonomy we kept

Class	Signal	Retry?	Attempt cap	Handling
transport	ECONNRESET, ENOTFOUND, 5xx	yes	4	exponential backoff + jitter
throttle	429 with Retry-After	yes	6	honour the header, no jitter guessing
schema	200 with invalid body shape	yes, once	2	stricter prompt on the second attempt
refusal	model declines the request	no	1	re-scope the input
budget	quota or limit exhausted	no	1	degrade and alert

Two rules fell out of the table. Rate limiting is a scheduling signal, not an error, so Retry-After is obeyed rather than jittered around (see RFC 9110 on 429). And anything unclassified fails closed: no policy entry means no retry.

Dead end 1: one wrapper to retry them all

The first suggestion was the obvious one, and it is the one most codebases ship.

// The loop we deleted
async function withRetry(fn, attempts = 5) {
  for (let i = 1; i <= attempts; i++) {
    try {
      return await fn();
    } catch (err) {
      if (i === attempts) throw err;
      await sleep(2 ** i * 100);
    }
  }
}

This wrapper retried the whole job, diff parsing included. A malformed diff coming from our own parser looked exactly like a flaky network call, so a deterministic bug consumed the entire attempt budget.

Forty-one attempts produced zero new information. That is the signature of a loop that should have been a crash: the error repeated, but the state never advanced.

Dead end 2: retry at the HTTP client

Moving the policy down into the SDK felt cleaner. Except an HTTP client only sees status codes.

A 200 whose body fails your schema is not a transport failure and never will be. The client returns it happily, the job fails later, and the retry policy has already been bypassed.

// A 200 is not a success until the shape is right
const parsed = CommentSchema.safeParse(body);
if (!parsed.success) {
  throw Object.assign(new Error('bad shape'), {
    name: 'SchemaError',
    issues: parsed.error.issues,
  });
}

The lesson is not "wrap your SDK". It is that the failure class has to be decided by the layer that can actually recognise it.

Dead end 3: dedupe on a payload hash

Hashing the request body looks like idempotency and is not. Two different prompt versions produce different hashes, so the same file gets reviewed twice. The same file re-uploaded with identical bytes produces the same hash, so a genuinely new run gets silently dropped.

Worse, a hash gives you nowhere to write what happened. There is no attempt history, no last known outcome, and no way to answer "did attempt 3 succeed" without grepping logs.

The senior's summary of this branch: a hash identifies the request, not the work.

The decision: retry the job, not the call

The thing that survived the hour was a single idempotency tuple plus an append-only ledger. Identity is the tuple; history is the ledger; the retry policy reads the ledger before it reads any error.

Key and ledger

// ledger.js
import { appendFileSync, readFileSync, existsSync } from 'node:fs';

export function idempotencyKey(job) {
  return [
    job.repo,
    job.commit,
    job.path,
    job.promptVersion,
    job.modelAlias,
  ].join('|');
}

export function attemptsFor(ledgerPath, key) {
  if (!existsSync(ledgerPath)) return [];
  return readFileSync(ledgerPath, 'utf8')
    .split('\n')
    .filter(Boolean)
    .map((line) => JSON.parse(line))
    .filter((row) => row.key === key);
}

export function record(ledgerPath, row) {
  appendFileSync(ledgerPath, JSON.stringify(row) + '\n');
}

Five fields, no more. promptVersion and modelAlias belong in the key because changing either one changes the work, and reviewers who forget that end up deduping across a prompt rewrite.

Classifier and policy

// retry.js
export const POLICY = {
  transport: { maxAttempts: 4, retryable: true },
  throttle:  { maxAttempts: 6, retryable: true },
  schema:    { maxAttempts: 2, retryable: true },
  refusal:   { maxAttempts: 1, retryable: false },
  budget:    { maxAttempts: 1, retryable: false },
};

export function classifyAttempt(err) {
  if (err?.code === 'ECONNRESET' || err?.code === 'ENOTFOUND') return 'transport';
  if (err?.status >= 500) return 'transport';
  if (err?.status === 429) return 'throttle';
  if (err?.name === 'SchemaError') return 'schema';
  if (err?.name === 'RefusalError') return 'refusal';
  if (err?.name === 'BudgetError') return 'budget';
  return 'unknown';
}

export function shouldRetry(cls, attemptsSoFar) {
  const policy = POLICY[cls];
  if (!policy || !policy.retryable) return false;
  return attemptsSoFar < policy.maxAttempts;
}

unknown has no policy entry, so it returns false. Failing closed here is deliberate: an unrecognised error is a missing branch in your own code, not a network hiccup.

The runner

// runJob.js
import { attemptsFor, idempotencyKey, record } from './ledger.js';
import { classifyAttempt, shouldRetry } from './retry.js';

export async function runJob({ job, call, ledgerPath, sleep }) {
  const key = idempotencyKey(job);
  const prior = attemptsFor(ledgerPath, key);
  const done = prior.find((row) => row.outcome === 'ok');
  if (done) return done.result;

  let attempts = prior.length;

  while (true) {
    attempts += 1;
    try {
      const result = await call(job);
      record(ledgerPath, { key, attempt: attempts, outcome: 'ok', result });
      return result;
    } catch (err) {
      const cls = classifyAttempt(err);
      record(ledgerPath, {
        key,
        attempt: attempts,
        cls,
        outcome: 'error',
        at: new Date().toISOString(),
      });
      if (!shouldRetry(cls, attempts)) throw err;
      await sleep(cls === 'throttle' ? (err.retryAfterMs ?? 1000) : 2 ** attempts * 100);
    }
  }
}

Restarting after a crash now resumes from the ledger instead of from attempt zero. That single property is why the process can be recycled at any time without burning budget.

Tests you can run now

// retry.test.js
import test from 'node:test';
import assert from 'node:assert/strict';
import { classifyAttempt, shouldRetry } from './retry.js';

test('transport retries to the cap, refusal never retries', () => {
  const reset = Object.assign(new Error('reset'), { code: 'ECONNRESET' });
  assert.equal(classifyAttempt(reset), 'transport');
  assert.equal(shouldRetry('transport', 3), true);
  assert.equal(shouldRetry('transport', 4), false);
  assert.equal(shouldRetry('refusal', 0), false);
});

test('unknown errors fail closed', () => {
  assert.equal(classifyAttempt(new Error('???')), 'unknown');
  assert.equal(shouldRetry('unknown', 0), false);
});

node --test retry.test.js

The Node test runner is enough; no framework required.

Fault-injection matrix

Injected fault	Expected attempts	Final state
3× ECONNRESET, then 200	4	ok
6× 429 with Retry-After	6	thrown throttle
200 with broken schema, twice	2	thrown schema
model refusal	1	thrown refusal
unrecognised error	1	thrown unknown

Run the matrix against a stub call that reads a scripted fault list. If any row produces more attempts than listed, the policy has a hole.

Where the free tier actually helped

The harness above needs iteration, not scale: rerun the matrix, bump promptVersion, watch which rows change. That is exactly the workload where MonkeyCode's free model access and free server option were used in this session — the fault matrix ran against the hosted model while the job runner sat on the free server, with the ledger written to disk rather than held in memory.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The operative assumption is the honest one: on any free-tier host, treat the process as recyclable and keep state outside it. That assumption is what made the ledger decision load-bearing instead of decorative.

Limitations and who should skip this

• The ledger is a local file. Two runners on the same key will race; move to a real queue with a uniqueness constraint before you scale out.
• Attempt caps are guesses until you have real failure distributions. Four and six are starting points, not constants.
• Jobs that must complete inside a tight latency SLO will find one attempt too many here. This design trades latency for budget discipline.
• Code that cannot leave your network, or that needs an audit trail beyond your own storage, should not use hosted inference at all.

The closing checklist

1. Classify every failure your job can produce, including your own parser bugs.
2. Put identity in one tuple and write it down before the first call.
3. Fail closed on unknown.
4. Run the fault matrix before you trust the retry cap.

If you want to reproduce the fixture, start with node --test retry.test.js and the matrix above; the free tier is enough to run it end to end.