Downstream started rate-limiting your Node BFF? Here's the retry layer that fixes it.

#webdev #javascript #node #api

Your Node BFF proxies to a downstream API — a payment provider, a recommendations service, a partner's data feed. One Tuesday the partner starts returning 429 and the occasional 503. Suddenly your users see errors, and the partner's status page lists you as their top traffic source. Something has to change on your side.

Here's the outbound call most BFFs start with:

import express from 'express';
import axios from 'axios';

const app = express();
const downstream = axios.create({ baseURL: 'https://api.partner.com' });

app.get('/recommendations/:userId', async (req, res) => {
  try {
    const { data } = await downstream.get(`/recs/${req.params.userId}`);
    res.json(data);
  } catch (e) {
    res.status(502).json({ error: 'bad gateway' });
  }
});

It does the happy path. It does nothing for the bad one. A transient 503 from the partner becomes a hard error for your user. And nothing stops a misbehaving client from calling this endpoint in a loop — which means you loop against the partner, exactly when they're struggling.

The usual fix, and where it breaks

Most of us reach for the obvious patch: a retry helper, and a manual counter for rate limiting.

async function withRetry(fn, n = 3) {
  for (let i = 0; i < n; i++) {
    try { return await fn(); }
    catch (e) { if (i === n - 1) throw e; await sleep(1000); }
  }
}

const hits = new Map(); // ip -> count, reset by... a timer? manually?

This compiles. It also has three holes you only find in production:

Retry storm. Ten concurrent requests each retry three times → thirty calls to a partner that's already down. You turned a blip into an outage.
No jitter. Every client retries after the same one second, so they all fire together, a thundering herd on recovery.
Per-process counters. That Map lives in one process. Behind a cluster or multiple instances, each worker counts on its own, so your "limit" is really limit × instances, and it resets wrong on restart.

You can fix all three by hand. People do, with varying success. The interesting part is that these are solved problems if the retry and the limit live with the request definition instead of bolted on after it.

alova/server: retry and rate limiting as request wrappers

alova isn't only a browser library. alova/server ships two server hooks — retry and createRateLimiter — that wrap a method and run when you send it. They use whatever alova instance you've set up, and on the server that instance uses the axios adapter just like on the client.

import { createAlova } from 'alova';
import { axiosRequestAdapter } from '@alova/adapter-axios';
import { retry, createRateLimiter } from 'alova/server';

const alova = createAlova({
  requestAdapter: axiosRequestAdapter()
  // server-side method use doesn't need a statesHook
});

const getRecs = userId =>
  alova.Get(`https://api.partner.com/recs/${userId}`);

Retry with backoff and jitter

retry wraps the method. Defaults are three attempts, one second apart; you can grow the delay and add jitter so a fleet of BFF instances doesn't retry in lockstep.

const data = await retry(getRecs(req.params.userId), {
  retry: 5,
  backoff: {
    delay: 1000,
    multiplier: 2,
    startQuiver: 0.1,
    endQuiver: 0.3
  }
});

First retry at ~1s, next at ~2s, next at ~4s, each with 10–30% random jitter. The downstream gets breathing room instead of a wall of retries.

Rate limiting inbound, so you don't hammer the partner

createRateLimiter wraps a method and tracks usage by a key — an IP, a user id, a tenant. Hit the limit and it throws (it does not queue), so you return 429 and stop there.

const rateLimit = createRateLimiter({
  duration: 1000,
  points: 5,
  keyPrefix: 'recs'
});

// before proxying, check the caller's budget
try {
  await rateLimit(alova.Get('/recommendations'), { key: req.ip });
} catch {
  return res.status(429).json({ error: 'too many requests' });
}

keyPrefix namespaces the counter so it won't collide with other limiters. duration and points set the window — here, five requests per second per IP.

Putting it in a middleware

Combine the two around the outbound call. Rate-limit the inbound caller first; if they're under budget, make the outbound call with retry.

app.get('/recommendations/:userId', async (req, res) => {
  try {
    await rateLimit(alova.Get('/recommendations'), { key: req.ip });
  } catch {
    return res.status(429).json({ error: 'too many requests' });
  }

  try {
    const data = await retry(getRecs(req.params.userId), {
      retry: 5,
      backoff: { delay: 1000, multiplier: 2 }
    });
    res.json(data);
  } catch {
    res.status(502).json({ error: 'partner unavailable' });
  }
});

The retry only runs on the outbound partner call, so a rate-limited caller is rejected before any retry logic kicks in — you're not retrying your own 429s.

For multi-instance deployments, point the limiter at a shared store so the count is consistent across workers:

import { createPSCAdapter, NodeSyncAdapter } from '@alova/psc';

const rateLimit = createRateLimiter({
  duration: 1000,
  points: 5,
  store: createPSCAdapter(NodeSyncAdapter())
});

createRateLimiter is built on node-rate-limiter-flexible, so cluster, multi-thread, and redis-backed stores are all supported through the storage adapter.

Canonical source: alova server rate limiting — https://alova.js.org/tutorial/server/strategy/rate-limit

Related reading