Two keys per endpoint, and the decision to fail open is made route by route

Munchable's API is small. A product lookup, a search, an OCR read of a label photo, a contribution, a support ticket, a couple of webhooks. Every one of them is rate limited, and the interesting thing about the file that does it is not the limits. It is that four separate decisions had to be made per route, and getting any of them wrong produces a system that looks protected and is not.

Two keys, always

Every guarded route checks two limiters: one keyed on the authenticated account, one keyed on the client IP.

Keying only on the account is the obvious choice and it fails immediately, because accounts are free. Somebody who wants to run a route a thousand times just creates twenty accounts. Keying only on the IP fails differently: a coffee shop, an office, or a mobile carrier's NAT puts hundreds of unrelated people behind one address, and the tenth person to open the app gets a 429 for no reason they can see.

Two keys with different budgets cover each other. The per-account limit is sized for what one person plausibly does. The per-IP limit is sized well above that, and exists so account cycling does not make the first limit decorative.

The IP you key on is not the one in the header

This is the part I would most want someone to take away, because it is easy to get wrong and impossible to notice:

export function getClientIdentifier(request: NextRequest): string {
  const realIp = request.headers.get('x-real-ip');
  if (realIp?.trim()) return realIp.trim();
  const fwd = request.headers.get('x-forwarded-for');
  if (fwd?.trim()) {
    const parts = fwd.split(',').map((s) => s.trim()).filter(Boolean);
    if (parts.length) return parts[parts.length - 1];
  }
  return 'unknown';
}

x-forwarded-for is a list, and almost every example you will find online takes the first entry. The first entry is the one the client sent. Anyone can put anything there, which means a per-IP rate limit keyed on it is bypassed by adding a header and incrementing a number.

The trustworthy entries are the ones your own infrastructure appended, which are at the right-hand end. Better still is a header your platform sets and strips from inbound requests, which is what x-real-ip is here. The fallback to the closest hop is there for the case where it is missing.

A per-IP limiter keyed on a spoofable value is worse than no limiter, because it shows up in the code review as protection.

Failing open is a per-route decision

When Redis is unreachable, the limiter cannot answer. Every route has to decide what that means, and there is no global right answer, so the choice is made at each call site and written down next to it.

Fail open where the request is cheap and reveals nothing:

try {
  decision = await enforce([
    { limiter: lim.taxonomyDev, key: user.id },
    { limiter: lim.taxonomyIp, key: getClientIdentifier(request) },
  ]);
} catch {
  decision = { success: true };
}

That is the endpoint serving our own ingredient vocabulary. It is a cached read, it is the same for everybody, and a device that cannot fetch it stops learning about new ingredients. Refusing that during a Redis blip protects nothing and degrades the product.

Webhooks fail open for a sharper reason: a dropped real webhook is worse than a duplicate one. The handler is idempotent, so a duplicate costs nothing, whereas a payment event silently 429'd is a subscription that never activated.

Fail closed where the request spends money or writes data. The label capture path, the contribution path, and the model budget all refuse when they cannot check. "We could not verify the limit" is not permission.

The shape I would recommend copying is not the specific choices, it is that the catch block is never shared. A helper that swallows limiter errors for every caller hides exactly the decision you most want visible.

The loop that is deliberately not parallel

Several routes chain a per-minute limiter in front of a per-day one. enforce runs the checks in order and returns on the first failure, which means a request already rejected by the minute limiter never touches the day counter. That is not an optimisation, it is the semantics: otherwise a client hammering a route burns through its daily allowance while every single request is being rejected anyway, and a person who comes back tomorrow finds their day already spent.

That sequencing costs a Redis round trip per check, and on a hot path where the two checks are genuinely independent it is pure latency. The temptation is to swap the loop for Promise.all and move on.

Instead there is a second function:

export async function enforceParallel(checks) {
  const results = await Promise.all(checks.map(({ limiter, key }) => limiter.limit(key)));
  for (const r of results) if (!r.success) return { success: false, ... };
  return { success: true };
}

Same contract, same first-by-check-order result, different cost model, and a comment above it saying precisely who may use it: callers whose checks are independent and for whom always incrementing every counter is acceptable. Do not repoint an existing caller at it without checking whether that caller relies on the short circuit.

Changing a shared primitive's semantics to make one caller faster is how you end up with a bug in a route nobody touched. Adding a sibling with a documented trade-off is slower to write and much easier to be right about.

One budget, two routes, and two copies of the cap

Two routes can call a language model during curation, and they share a daily spending cap. The first version held the cap as a constant inside each route, and both routes incremented the same Redis counter.

That is a bug with a long fuse. Raising the cap in one file left the other route rejecting at the old number against a counter that was already past it, so the budget behaved like whichever literal was smaller, and the file you edited was not necessarily that one.

The budget now lives in the same module as the limiters, as one function both routes call. A shared counter needs a shared threshold, and the only reliable way to guarantee that is for there to be exactly one place the threshold is written.

A background job is not a user

The same budget is spent by a background job that resolves ingredients from captures. Initially the job spent it through the per-account key, with a made-up account id, which meant every capture in the entire system shared one person's daily allowance. The job exhausted it well before the global budget was touched.

The fix is not to raise the per-account cap, because that would loosen the ceiling for real accounts at the same time. It is to make the principal explicit:

export type BudgetPrincipal = { kind: 'user'; id: string } | { kind: 'job'; name: string };

A job gets its own generous cap, a user gets a much smaller one, and both still sit under the global ceiling. The two numbers protect different things: one is abuse prevention, the other is a cost circuit breaker.

One more detail in that function. The per-principal counter is checked before the global one, so a caller that is already over its own limit cannot drain the shared budget with calls it is going to be refused anyway.

Telling the client the truth

A 429 carries Retry-After and the RateLimit-Limit, RateLimit-Remaining and RateLimit-Reset headers, and those header names are listed in the API's CORS Access-Control-Expose-Headers so a browser client can actually read them. A rate limit that a client cannot see is a rate limit a client will treat as a random failure and retry into.

The limiters are also constructed with analytics disabled, because the analytics feature of the rate limiting library reports request events to a third party, and this product's promise is that it runs no third-party tracking of any kind. A dependency's convenience feature is still data leaving your building.

Product limits are a different thing entirely

None of the above is the limit users actually feel. The free tier is five scans a month, and that number lives nowhere near this file. It is a product promise, it is enforced against entitlements, and it is the same for everybody.

The limits in this post are invisible when the product is working. They exist so that one person with a script cannot make the product worse for everybody else, and the best evidence that they are sized correctly is that nobody has ever mentioned them.

If you want to watch the headers, open the web app with the network tab open, or send a message through the support page, which goes through one of these routes. The response headers are there on every call.