Short answer: when an API says 429, wait for the time it tells you in the Retry-After header. If it does not send one, back off on your own. Better still, stay under the limit before you ask. I wrote a small rate limited server and four clients in Node.js, and I counted the calls each client needed to get 8 answers. The code needs no packages.
The test
The server allows 3 requests in each 100 millisecond window. Any more gets a 429. The client wants 8 good answers.
To keep the demo fast, the Retry-After value is scaled: 1 second counts as 100 milliseconds in this code. In real life it is 1000. The server's counting is a simple fixed window, which is only one of the ways real APIs do it.
The server
// To keep the demo fast, 1 "second" in Retry-After counts as 100 ms here.
// In real life this number is 1000.
const UNIT_MS = 100;
const WINDOW_MS = 100; // the server allows LIMIT requests in each window
const LIMIT = 3;
// ---------- A small server with a fixed window limit ----------
function makeServer({ sendRetryAfter }) {
let windowStart = 0;
let count = 0;
return http.createServer((req, res) => {
const now = Date.now();
if (now - windowStart >= WINDOW_MS) {
windowStart = now;
count = 0;
}
count++;
if (count > LIMIT) {
const headers = sendRetryAfter ? { "retry-after": "1" } : {};
return res.writeHead(429, headers).end("slow down");
}
res.writeHead(200).end("ok");
});
}
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
Client 1: no care at all
// ---------- Client 1: no care at all ----------
async function naive(url, n, stats) {
for (let i = 0; i < n; i++) {
const r = await fetch(url);
stats.calls++;
if (r.status === 429) stats.tooMany++; else stats.ok++;
}
}
It sends all 8 requests and counts what comes back. It never waits, and it never retries.
Client 2: retry after a 429
// ---------- Client 2: retry after a 429, honor Retry-After, else back off ----------
function waitTimeMs(response, attempt) {
const header = Number(response.headers.get("retry-after"));
if (Number.isFinite(header) && header > 0) return header * UNIT_MS; // the server told us
return 25 * 2 ** attempt; // 25, 50, 100, 200 ...
}
async function withRetry(url, n, stats, maxRetries = 5) {
for (let i = 0; i < n; i++) {
for (let attempt = 0; ; attempt++) {
const r = await fetch(url);
stats.calls++;
if (r.status !== 429) { stats.ok++; break; }
stats.tooMany++;
if (attempt === maxRetries) { stats.failed++; break; }
await sleep(waitTimeMs(r, attempt));
}
}
}
After a 429, this client reads Retry-After and waits that long. If the header is missing or not a number, it waits 25, 50, 100, 200 and so on, which is exponential backoff. After 5 retries it gives up on that request.
Client 3: stay under the limit
// ---------- Client 3: stay under the limit before you ask ----------
function makeLimiter(maxPerWindow, windowMs, safetyMs = 10) {
const sent = []; // times of recent requests
return async function wait() {
for (;;) {
const now = Date.now();
while (sent.length && now - sent[0] >= windowMs + safetyMs) sent.shift();
if (sent.length < maxPerWindow) { sent.push(now); return; }
await sleep(sent[0] + windowMs + safetyMs - now);
}
};
}
async function withLimiter(url, n, stats) {
const wait = makeLimiter(2, WINDOW_MS); // 2 per window, the server allows 3
for (let i = 0; i < n; i++) {
await wait();
const r = await fetch(url);
stats.calls++;
if (r.status === 429) stats.tooMany++; else stats.ok++;
}
}
This client keeps a list of when it sent its last requests. It sends at most 2 per window, while the server allows 3, and it waits a little extra (10 ms) so that clocks that differ by a few milliseconds do not push it over. If it has to wait, it sleeps until the oldest request leaves the window.
The real output
I ran this on Node.js 22. I ran it three times and got the same counts each time.
Server limit: 3 requests per 100 ms. The client wants 8 responses.
1. No care at all wanted 8, got 3 ok | HTTP calls 8, 429s 5, gave up 0
2. Retry, server sends Retry-After wanted 8, got 8 ok | HTTP calls 10, 429s 2, gave up 0
3. Retry, no Retry-After (own backoff) wanted 8, got 8 ok | HTTP calls 14, 429s 6, gave up 0
4. Limiter first (2 per window) wanted 8, got 8 ok | HTTP calls 8, 429s 0, gave up 0
What it shows:
- The client with no care got 3 answers out of 8. The other 5 calls were rejected, and it did not notice.
- The client that reads Retry-After got all 8 answers with 10 calls. Two calls were rejected, one at the start of each new window.
- The client with its own backoff also got all 8 answers, but it needed 14 calls and saw six 429 errors. It guessed the wait, so it asked too early a few times.
- The client with a limiter got all 8 answers with 8 calls and no 429 at all. The price is that it is slower than it could be, because it stays under the limit on purpose.
Things to watch in a real system
- Retry-After can also be a date instead of a number of seconds. This demo reads only the number.
- Many clients that retry at the same time will all come back at the same time. Add some random delay to the backoff. I did not add it here, so I cannot show a result for it.
- This limiter lives in one process. If you run several workers with the same API key, they need to share a limiter, or each one needs a smaller part of the limit.
- Retrying is only safe for a request that is safe to repeat. A 429 means the server did not do the work, but a timeout does not tell you that. For a POST that creates something, use an idempotency key if the API offers one.
- Check how your provider counts rejected calls. Some APIs may count them against your limit. I did not test this.
The full script
Save this as ratelimit_demo.mjs and run node ratelimit_demo.mjs. Global fetch needs Node 18 or newer.
import http from "node:http";
// To keep the demo fast, 1 "second" in Retry-After counts as 100 ms here.
// In real life this number is 1000.
const UNIT_MS = 100;
const WINDOW_MS = 100; // the server allows LIMIT requests in each window
const LIMIT = 3;
// ---------- A small server with a fixed window limit ----------
function makeServer({ sendRetryAfter }) {
let windowStart = 0;
let count = 0;
return http.createServer((req, res) => {
const now = Date.now();
if (now - windowStart >= WINDOW_MS) {
windowStart = now;
count = 0;
}
count++;
if (count > LIMIT) {
const headers = sendRetryAfter ? { "retry-after": "1" } : {};
return res.writeHead(429, headers).end("slow down");
}
res.writeHead(200).end("ok");
});
}
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
// ---------- Client 1: no care at all ----------
async function naive(url, n, stats) {
for (let i = 0; i < n; i++) {
const r = await fetch(url);
stats.calls++;
if (r.status === 429) stats.tooMany++; else stats.ok++;
}
}
// ---------- Client 2: retry after a 429, honor Retry-After, else back off ----------
function waitTimeMs(response, attempt) {
const header = Number(response.headers.get("retry-after"));
if (Number.isFinite(header) && header > 0) return header * UNIT_MS; // the server told us
return 25 * 2 ** attempt; // 25, 50, 100, 200 ...
}
async function withRetry(url, n, stats, maxRetries = 5) {
for (let i = 0; i < n; i++) {
for (let attempt = 0; ; attempt++) {
const r = await fetch(url);
stats.calls++;
if (r.status !== 429) { stats.ok++; break; }
stats.tooMany++;
if (attempt === maxRetries) { stats.failed++; break; }
await sleep(waitTimeMs(r, attempt));
}
}
}
// ---------- Client 3: stay under the limit before you ask ----------
function makeLimiter(maxPerWindow, windowMs, safetyMs = 10) {
const sent = []; // times of recent requests
return async function wait() {
for (;;) {
const now = Date.now();
while (sent.length && now - sent[0] >= windowMs + safetyMs) sent.shift();
if (sent.length < maxPerWindow) { sent.push(now); return; }
await sleep(sent[0] + windowMs + safetyMs - now);
}
};
}
async function withLimiter(url, n, stats) {
const wait = makeLimiter(2, WINDOW_MS); // 2 per window, the server allows 3
for (let i = 0; i < n; i++) {
await wait();
const r = await fetch(url);
stats.calls++;
if (r.status === 429) stats.tooMany++; else stats.ok++;
}
}
// ---------- Run ----------
async function run(label, client, serverOptions, n = 8) {
const server = makeServer(serverOptions);
await new Promise((ok) => server.listen(0, "127.0.0.1", ok));
const url = `http://127.0.0.1:${server.address().port}/`;
const stats = { calls: 0, ok: 0, tooMany: 0, failed: 0 };
await client(url, n, stats);
server.close();
console.log(`${label.padEnd(44)} wanted ${n}, got ${stats.ok} ok | HTTP calls ${stats.calls}, 429s ${stats.tooMany}, gave up ${stats.failed}`);
}
console.log(`Server limit: ${LIMIT} requests per ${WINDOW_MS} ms. The client wants 8 responses.\n`);
await run("1. No care at all", naive, { sendRetryAfter: true });
await run("2. Retry, server sends Retry-After", withRetry, { sendRetryAfter: true });
await run("3. Retry, no Retry-After (own backoff)", withRetry, { sendRetryAfter: false });
await run("4. Limiter first (2 per window)", withLimiter, { sendRetryAfter: true });
Disclosure: I build Libraryminds, an API for turning video into text. This post has nothing to do with it, and the demo calls no real API.
If your provider does something different with 429, tell me in the comments.
Top comments (0)