Your Review Bot's Retry Loop Is a Token Furnace. Test the Amplifier.

AI turned developers into reviewers. The review pipeline now runs a free model behind a webhook. One flaky network can burn more tokens than a thousand working requests.

This post is an architecture review of that pipeline. I follow the constraints, the data flow, and the failure domains. At the end, I name the change I would make next.

The setup sounds harmless. A pull request arrives. The webhook forwards it to a free server. The server calls a model and then posts a review comment.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. The project offers free model access and a free server option. I treat the marketing page as secondary, because the architecture is the part you can verify.

The data flow has three stages. Input validation, model inference, and comment creation. Every stage can fail, and every failure looks retryable from the client.

That is the trap. A client-side retry loop multiplies every transient failure by a constant. You asked for one review. Your code attempts five.

Failing responses still consume tokens. The free tier does not meter attempts; it meters tokens burned. Measure the burn before production.

// Run with Node.js. Same seed, same output, fair comparison.
function mulberry32(seed) {
  let value = seed;
  return function () {
    value |= 0;
    value = (value + 0x6D2B79F5) | 0;
    let t = Math.imul(value ^ (value >>> 15), 1 | value);
    t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t;
    return ((t ^ (t >>> 14)) >>> 0) / 4294967296;
  };
}

function simulateReviewPipeline(policy, failureRate, tokensPerCall, totalRequests) {
  const rand = mulberry32(42);
  let consumed = 0;
  let succeeded = 0;
  for (let i = 0; i < totalRequests; i++) {
    let ok = false;
    for (let attempt = 0; attempt < policy.maxAttempts; attempt++) {
      consumed += tokensPerCall;
      if (rand() > failureRate) {
        ok = true;
        break;
      }
    }
    if (ok) succeeded++;
  }
  return {
    policy: policy.name,
    consumed,
    succeeded,
    wasted: consumed - succeeded * tokensPerCall,
  };
}

const scenarios = [
  simulateReviewPipeline({ name: "no-retry", maxAttempts: 1 }, 0.3, 1200, 1000),
  simulateReviewPipeline({ name: "retry-3", maxAttempts: 3 }, 0.3, 1200, 1000),
  simulateReviewPipeline({ name: "retry-5", maxAttempts: 5 }, 0.3, 1200, 1000),
];

console.table(scenarios);

Look at the consumed column when you run it. The succeeded column also grows. A bot looks productive while it devours the free tier, and the rising number of comments makes the waste harder to see.

The cost is not only tokens. Every comment also spends human attention. A bot that posts duplicated or noisy reviews burns reviewers, and that resource does not reset.

The protection belongs before the model call, not after it. A sliding-window gate caps reviews per minute and fails fast when the queue is hot.

function slidingWindowGate(limit, windowMs) {
  const stamps = [];
  return function () {
    const now = Date.now();
    while (stamps.length && stamps[0] <= now - windowMs) stamps.shift();
    if (stamps.length >= limit) return false;
    stamps.push(now);
    return true;
  };
}

const gate = slidingWindowGate(4, 60_000);
if (gate()) {
  enqueueReview(payload); // pseudo: one model call
} else {
  console.log("queue hot; skip and review manually");
}

This gate protects the model endpoint, not the humans. When the gate rejects, the pull request waits for a manual review. Bots do not outrank people.

What would I change next? Remove automatic retries entirely. Failed reviews would go to a dead-letter list that only a human can rerun. The same pull request would never reach the model twice.

I would also validate before inference. A malformed webhook payload should die before it consumes tokens, not after several failed attempts.

The failure domains are four: delivery, validation, inference, and creation. Each has a different recovery time. Delivery fails in seconds; creation fails after the tokens are already gone. One retry policy for all four is the structural mistake, and that is the part I would rewrite before the model choice.

Where does this stop working? Teams that promise a review SLA. If clients rely on response time, put a paid API behind a formal queue and store dead-letter records. A free tier is a smoke test, not a contract.

Also do not let a free-tier model find secrets or abusive code. Those checks belong in deterministic CI rules. A probabilistic comment belongs on a pull request, not in your security boundary.

If you want to try the pattern, MonkeyCode's repository is open source and its free server option keeps the smoke test local. Start with the simulation script. Measure your retry policy before you measure the model.

Run the script once. Then give every review a gate, a cap, and a way to die quietly.