Build a JavaScript Concurrency Limiter, Then Explain It in an Interview

TL;DR: A concurrency limiter is not “make async code faster.” It is a pressure-control tool: choose a small number of active operations, queue the rest, and make the next task start only when one finishes. In interviews, the strongest answer explains the limit, the queue, failure behavior, cancellation boundaries, and how you would measure whether the limit is right.

A surprisingly common interview prompt sounds harmless:

“We need to fetch metadata for 500 URLs. How would you avoid overwhelming the upstream service?”

The weak answer is “use Promise.all.” That starts all 500 operations immediately. It can spike sockets, memory, file descriptors, database connections, or an API’s rate limit before the first response comes back.

The other weak answer is “do it sequentially.” That is safe, but it throws away useful parallelism when each request spends most of its life waiting on I/O.

A concurrency limiter gives you the middle ground: bounded parallelism. This post builds one from scratch, tests its important behavior, then turns the implementation into an interview explanation.

What contract should a limiter have?

Before writing code, make the contract explicit. A small limiter should:

1. never run more than limit tasks at once;
2. preserve queue order for tasks waiting to start;
3. return each task’s result or rejection to its own caller;
4. start the next queued task after either success or failure;
5. reject invalid limits early.

That fourth point is easy to miss. If a rejected task leaves the scheduler stuck, one transient 503 can stop all later work. Good interview answers name this failure mode before they show an implementation.

For a batch of 500 independent HTTP calls, “three or ten at a time” is a concurrency policy. It is different from a rate limit such as “100 requests per minute.” Real systems often need both.

Can we build it without a dependency?

Here is a dependency-free JavaScript implementation. It accepts a function, rather than an already-created promise, because the limiter must decide when work begins.

function createLimiter(limit) {
  if (!Number.isInteger(limit) || limit < 1) {
    throw new RangeError("limit must be a positive integer");
  }

  let active = 0;
  const queue = [];

  function drain() {
    while (active < limit && queue.length > 0) {
      const { task, resolve, reject } = queue.shift();
      active += 1;

      Promise.resolve()
        .then(task)
        .then(resolve, reject)
        .finally(() => {
          active -= 1;
          drain();
        });
    }
  }

  return function run(task) {
    if (typeof task !== "function") {
      return Promise.reject(new TypeError("task must be a function"));
    }

    return new Promise((resolve, reject) => {
      queue.push({ task, resolve, reject });
      drain();
    });
  };
}

There are two details worth narrating while you code:

• Promise.resolve().then(task) converts both a synchronous throw and an asynchronous rejection into the returned promise’s rejection path.
• .finally(...) frees the slot regardless of the outcome, then calls drain() to pull from the queue.

That makes the state machine small: a task is either queued, active, or settled. Only the transition from active to settled changes active.

How do you prove the bound is real?

A timer-based test is useful because it records the highest observed number of simultaneous tasks. It does not depend on a network call, so it is deterministic enough to run in an interview editor.

import assert from "node:assert/strict";

const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

async function demo() {
  const limit = 3;
  const run = createLimiter(limit);
  let active = 0;
  let peak = 0;

  const work = Array.from({ length: 8 }, (_, id) =>
    run(async () => {
      active += 1;
      peak = Math.max(peak, active);
      await sleep(20 + (id % 3) * 10);
      active -= 1;
      return id * id;
    }),
  );

  const results = await Promise.all(work);

  assert.equal(peak, limit);
  assert.deepEqual(results, [0, 1, 4, 9, 16, 25, 36, 49]);
  console.log({ peak, results });
}

demo();

Run it with Node 18+:

node limiter-demo.mjs
# { peak: 3, results: [ 0, 1, 4, 9, 16, 25, 36, 49 ] }

The key assertion is not that the tasks finish in a particular order. Different task durations can reorder completion. The promise returned by each run() still maps to the right input, and the observed peak never exceeds three.

What should happen when one task fails?

A limiter should not silently turn failures into successes. Callers still need the original rejection, while the scheduler needs to release the slot.

Add this probe after the first test:

async function failureDoesNotStall() {
  const run = createLimiter(1);
  const first = run(async () => {
    throw new Error("upstream returned 503");
  });
  const second = run(async () => "still ran");

  await assert.rejects(first, /503/);
  assert.equal(await second, "still ran");
}

failureDoesNotStall();

With a limit of one, the second task cannot begin until the first settles. If this assertion passes, a rejection did not strand the queue. That is a much more meaningful test than merely checking that an error was printed.

Which trade-offs will an interviewer probe?

The code is only the opening. A senior interviewer will usually ask for boundaries:

Question	Strong answer
Why not Promise.all?	It creates unbounded pressure. I would bound in-flight work based on the narrowest downstream resource.
Why not a loop with await?	It caps concurrency at one, often wasting I/O wait time.
Is this a rate limiter?	No. It bounds simultaneous work; a rate limiter controls starts per time window. They solve different overload modes.
What about cancellation?	The basic version cannot cancel a promise already running. I would accept an AbortSignal, remove aborted queued tasks, and pass the signal to cooperative I/O.
What about retries?	Retry only idempotent operations, with backoff and jitter. Retries must also pass through the limiter or they can recreate the overload.
Is FIFO always fair?	It is fair for one queue, but a long low-priority batch can delay urgent work. Production code may need separate queues or weighted scheduling.

That last answer matters in a real design review too. “FIFO” is a policy, not a law of nature.

How would I choose the initial limit?

Do not pretend there is a universal number. Start with the tightest known constraint:

• API documentation may specify concurrent-request or connection limits.
• A database pool size bounds useful query concurrency.
• CPU-heavy tasks generally need a smaller limit than network waits.
• p95 latency, error rate, queue depth, and saturation tell you when to tune.

For example, a worker that can safely run 20 outbound requests might start at 5 or 10, then increase only after observing stable latency and no 429/5xx rise. The right operating metric is not “requests completed fastest in a local test”; it is sustained useful throughput without degrading the dependency.

Where does AI-assisted practice fit?

This is a good prompt to rehearse as a conversation, not a memorized snippet. After you can build the limiter, ask someone—or a practice tool—to interrupt with “What changes for priority traffic?” or “How would you expose queue depth?” The goal is to make the invariants audible under pressure.

aceround.app — AI interview assistant can be useful for that follow-up style of practice, but the code and the trade-offs need to be yours.

FAQ

Does a concurrency limiter make every batch faster?

No. It can improve throughput compared with sequential work, but it intentionally limits starts. If the downstream system is saturated, a smaller limit can make the whole system healthier even when one batch takes longer.

Can I use this for CPU-bound work?

Not as a substitute for worker threads or separate processes. The limiter can bound how many CPU-heavy jobs you schedule, but JavaScript running on one event loop still cannot execute CPU work in parallel.

Should a limiter retain every result?

Not necessarily. For a very large stream, consume each settled result and release it rather than creating one giant Promise.all array. Bounded active work does not automatically bound total input or output memory.

Sources

• MDN: Promise.prototype.finally()
• Node.js: timers/promises and async patterns

Disclosure: I used an AI writing assistant for editorial help. The implementation, test cases, links, and technical claims were reviewed before publication.