The Pairing Hour That Replaced a Webhook Queue With 60 Lines of Admission Control

A pull-request review bot was ready to ship. The plan on the whiteboard was the usual one: webhook in, job into a queue, worker out, model call, comment back on the PR. It had been drawn in ten minutes and it looked finished.

The pairing hour that followed did not add features. It removed a queue, added a deadline, and produced a small module that the team could reason about at 3 a.m. What follows is the questions the senior asked, the two designs that died, and the one decision that survived contact with a free-tier endpoint.

The constraint nobody wrote down

The bot was going to run against a free model endpoint on a free server option. Both were operator-supplied availability claims, not guarantees: no SLA, no reserved capacity, no promise that a slow minute stays a slow minute.

That single sentence changes the shape of the system. A queue is a promise that every accepted job eventually runs. On a free tier, that promise is the thing you cannot keep.

Question 1: what happens when forty PRs land in ninety seconds?

The senior did not ask about model quality. The first question was about arrival rate versus service rate.

A monorepo morning looks like this: a dependency bump opens forty PRs, each firing opened, synchronize, and a couple of labeled events. The handler accepts all of them because accepting is cheap. Then the worker discovers that one review takes eight to forty seconds.

• Queue depth grows faster than it drains.
• Comments arrive on code that has already been amended twice.
• The slowest PRs get reviews last, which is exactly backwards.

Dead end A: a priority queue. Sort by PR age so the oldest gets served first. It solves ordering and solves nothing else — depth still grows without bound, and the bot still comments on stale diffs. Priority is a scheduling answer to a capacity question.

Dead end B: per-PR concurrency of four. Let each PR have up to four parallel model calls so a big diff can be chunked. On a free endpoint this multiplies the failure surface by four and makes the timeout story unreadable. When a review half-succeeds, nobody can say which chunk was dropped.

Question 2: what does the endpoint do when it is slow rather than down?

Silence is the hard failure mode. A 500 is easy: log it, retry once, move on. A request that hangs for ninety seconds and then returns a truncated body is the one that corrupts the review.

So the design needs a deadline that is enforced on the caller's side, not hoped for on the server's side. That means an AbortController with a timer, and a policy for what to do when the timer wins.

Question 3: can you reconstruct a failure from the log alone?

This one produced the artifact. The senior wanted three fields on every decision: the idempotency key, the outcome, and the reason. Not a full trace — three fields, written at the moment the decision is made.

The artifact: an admission gate, not a queue

The kept design is roughly sixty lines. It takes a job only if a token bucket allows it, refuses duplicates for the same commit, and aborts at the deadline. The excerpt below is written for this article and has not been benchmarked against any particular endpoint; treat the constants as knobs, not measurements.

// gate.js — admission control in front of a free-tier model call
const log = [];
const record = (entry) => log.push({ t: Date.now(), ...entry });

export function snapshot() {
  return { log: [...log], inFlight: inFlight.size, tokens: bucket.tokens };
}

const bucket = { tokens: 8, capacity: 8, refillPerSec: 0.25, last: Date.now() };

function take() {
  const now = Date.now();
  bucket.tokens = Math.min(
    bucket.capacity,
    bucket.tokens + ((now - bucket.last) / 1000) * bucket.refillPerSec
  );
  bucket.last = now;
  if (bucket.tokens < 1) return false;
  bucket.tokens -= 1;
  return true;
}

const inFlight = new Map(); // key -> AbortController

export function submit({ key, run, deadlineMs = 45_000, onDrop = () => {} }) {
  if (inFlight.has(key)) {
    record({ key, outcome: "deduped", reason: "same-key-in-flight" });
    return { status: "deduped", key };
  }

  if (!take()) {
    record({ key, outcome: "dropped", reason: "bucket-empty" });
    onDrop(key);
    return { status: "dropped", key };
  }

  const ac = new AbortController();
  inFlight.set(key, ac);
  const timer = setTimeout(() => ac.abort(new Error("deadline")), deadlineMs);
  const done = (outcome, detail) => record({ key, outcome, detail });

  run(ac.signal)
    .then((result) => done("ok", { bytes: String(result ?? "").length }))
    .catch((err) =>
      done(err?.name === "AbortError" ? "timeout" : "error", { message: err?.message })
    )
    .finally(() => {
      clearTimeout(timer);
      inFlight.delete(key);
    });

  return { status: "accepted", key };
}

The key is repo#pr#headSha. That choice is the whole trick: a force-push creates a new key, so the new commit is admitted while the old run is already gone. A redelivered webhook creates the same key, so GitHub's at-least-once delivery costs nothing.

Reproducing the burst without a network

The point of the probe is not to measure someone else's endpoint. It is to prove the gate's own behaviour — dedupe, drop, timeout — under a burst you can replay.

// probe.mjs — burst test for the gate, no network required
import { submit, snapshot } from "./gate.js";

// Placeholder latency profile. Replace with p50/p95 you measured yourself.
const latency = () => 4000 + Math.random() * 26000;
const sleep = (ms, signal) =>
  new Promise((res, rej) => {
    const id = setTimeout(res, ms);
    signal?.addEventListener("abort", () => {
      clearTimeout(id);
      rej(Object.assign(new Error("aborted"), { name: "AbortError" }));
    });
  });

const totals = { accepted: 0, deduped: 0, dropped: 0 };

for (let i = 0; i < 90; i++) {
  // 40 PRs, some force-pushed while a review is still running
  const pr = i % 40;
  const sha = `s${Math.floor(i / 40)}`;
  const key = `repo#${pr}#${sha}`;
  const r = submit({
    key,
    deadlineMs: 45_000,
    run: (signal) => sleep(latency(), signal),
  });
  totals[r.status]++;
  await sleep(33); // ~90 events over 3s
}

await sleep(50_000);
const s = snapshot();
const byOutcome = s.log.reduce((a, e) => ((a[e.outcome] = (a[e.outcome] ?? 0) + 1), a), {});
console.log({ totals, byOutcome, stillInFlight: s.inFlight });

Run it with node probe.mjs. What you are looking for is boring: no entry in byOutcome that you cannot explain, and stillInFlight back to zero. If a burst leaves work in flight past the deadline plus a margin, the timer is not the only thing holding the job.

The decision table we kept

Observed on the probe	Decision
Duplicate keys dominate the burst	Keep dedupe; it is cheaper than retry logic
Drops cluster at burst start	Raise capacity, keep refillPerSec low
Drops spread evenly across the burst	The service rate is the problem; reduce chunk size
Timeouts above ~5%	Lower deadlineMs and comment "partial review" instead
Timeouts with tiny payloads	Suspect the endpoint, not the diff
stillInFlight never returns to zero	You have a leak, not a capacity issue

A drop is not free. Each one is a PR without a review, so onDrop posts a single comment with the exact command to re-run. That converts an invisible failure into a human decision, which is the only kind of failure a small team can act on that week.

Where the free tier actually enters the design

The reason this bot never grew a queue is that a free model endpoint plus a free server option makes capacity an external, unknown variable. MonkeyCode is the project this bot was being wired to — its operator describes free model access and a free server option, which is exactly the setting that punishes unbounded queues. I am not repeating quotas, hardware, or limits here: those numbers move, and the project's current documentation is the only place they should be read before you size anything.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Two consequences shaped the code. First, the deadline had to be ours. Second, the token bucket had to be small enough that a bad minute degrades into drops rather than a five-hour backlog. Neither is a product feature; both are what a free tier forces you to write.

Limitations and who should not use this

• No guaranteed latency. The gate bounds queue growth, not response time. If a comment must land within thirty seconds, this design will disappoint you.
• Drops are real loss. Any team that needs every commit reviewed needs a durable queue and a paid capacity plan, not a token bucket.
• Single process only. The inFlight map is in memory. Two replicas double both the bucket and the dedupe window.
• Not for compliance work. If reviews are an audit artifact, you need storage you control and retention rules; a log array in RAM is not that.
• Not for monorepos with thousand-file diffs. Chunking multiplies calls, and this gate is deliberately hostile to parallelism.

What the senior kept

The queue was four hundred lines: retries, backoff, dead-letter storage, a dashboard nobody opened. The gate is sixty, plus a probe you can run in a minute and a log you can read without tooling.

That was the pairing hour's real output — not a better review bot, but a system that fails in one visible way at a time. If you want to try the same gate against a free endpoint, start with the project's docs, then run the probe before you run the bot.