We shipped a daily cap that followed last week's traffic, then deleted it seven days later

On 3 October I published a post here called Our daily API cap is three times last week's busiest day, because a fixed cap is a growth ceiling. The argument was that a constant daily cap on an outside source is a ceiling on your own growth, so the cap should follow recent use: today's limit is a multiple of the busiest of the last seven days, with a floor.

On 10 October I deleted the whole mechanism. The type, the function, the lookback constant, the extra query, all of it.

The argument in that post was not wrong about constants. It was wrong about the thing one level up, which is whether a limit shared by every account should exist at all.

What the guard actually did when it fired

Here is the deleted code:

// Spike guards: caps that follow recent use instead of a fixed number.
// Today's cap is `surge` times the busiest of the last week's days, and never
// below `floor`. Steady growth raises it on its own, so more users never hit
// it; only a sudden jump (a job stuck in a loop, a wave of bot sign-ups) is
// held until tomorrow.
export type SpikeGuard = { floor: number; surge: number };

export function spikeCap(guard: SpikeGuard, busiest: number): number {
  return Math.max(guard.floor, Math.ceil(guard.surge * busiest));
}

Twelve lines, a clean idea, and tested. The problem is one sentence in the comment: "is held until tomorrow".

Held for whom? The counter is one row per source per day for the whole product. When it trips, the next account to ask for work gets refused. That account has no relationship to the one that caused the spike. It is simply whoever arrived next.

So the failure mode of the guard, in the case it was built for, is: something goes wrong in account A, and account B's campaign stops finding creators for the rest of the day. B is paying us. B did nothing. B gets no explanation that means anything, because the true explanation is about somebody else's account.

A 3x multiplier on last week does not fix that. It makes it rarer, which is worse in one specific way: a thing that fails rarely and for reasons invisible to the person it happens to is harder to diagnose than one that fails often.

The replacement is boring, which is the point

Two rules now, and they are the only two shapes a limit in this codebase is allowed to have:

1. Per plan. Every account carries its own daily numbers, straight out of the plan definition in code. What it may search, read and send in a day is a property of what it pays for, listed on the pricing page where a customer can see it before they hit it.
2. Per campaign. Anything whose blast radius is one campaign is counted on that campaign. The clearest example is the bounce breaker: when too many of one campaign's first emails bounce, that campaign's new conversations wait while its follow-ups and replies carry on, under a counter key of its own, and no other campaign notices.

Both have the same property, which the shared guard did not: the account that runs into the limit is the account that used it. There is always something true to say in the UI, and it is always about work the customer can see.

The caps that remain are the ones that were never ours to choose:

const CAPS = {
  // The 10,000-unit pool every method but search draws on, and search.list's
  // own 100 a day. Stopped short of both, since failed requests cost quota too.
  youtube: 9_500,
  youtube_search: 95,
  ...
} satisfies Record<string, number | null>;

Those are Google's numbers, not ours. A provider imposed limit is genuinely shared, because the API key is shared, and that is a fact about the world rather than a policy decision. The honest thing is to say so in the product rather than to pretend it is a feature.

Three counters that count and never refuse

The type is now Record<string, number | null>, and three entries are null. They are the ones that used to carry spike guards: two paid sources and the number of new conversations started across the whole product.

null means the row is still written, the number is still visible in the admin run log, and nothing is ever refused because of it. The comment on each one names what bounds it instead, and in every case it is something better suited to the job: a provider's own monthly spending limit, a prepaid balance, or the per plan daily numbers that pace each account.

The upsert absorbed this without much ceremony:

.onConflictDoUpdate({
  target: [providerDays.provider, providerDays.day],
  set: { requestsUsed: sql`${providerDays.requestsUsed} + ${n}` },
  setWhere:
    limit === null
      ? sql`${providerDays.exhaustedAt} is null`
      : sql`${providerDays.requestsUsed} + ${n} <= ${limit} and ${providerDays.exhaustedAt} is null`,
})
.returning({ used: providerDays.requestsUsed });

One statement, still. setWhere is what makes a take atomic: concurrent callers cannot both read 9,499 and both write 9,500, because the check and the increment are the same row update, and a conflicting row that fails the condition updates nothing and returns nothing.

With a null limit the condition does not disappear entirely, which is the part worth noticing. exhaustedAt is null stays. A counter with no cap of ours can still be switched off for the day by the source itself.

The one thing a source is always allowed to do

export async function markExhausted(counter: Counter): Promise<boolean> {
  const rows = await db
    .insert(providerDays)
    .values({ provider: counter, day: dayOf(counter), exhaustedAt: new Date() })
    .onConflictDoUpdate({
      target: [providerDays.provider, providerDays.day],
      set: { exhaustedAt: new Date() },
      setWhere: sql`${providerDays.exhaustedAt} is null`,
    })
    .returning({ day: providerDays.day });
  return rows.length > 0;
}

When a source answers "you are out of credit" or "you are rate limited for today", we stop asking until tomorrow. Note the return value: rows come back only if exhaustedAt was null before, so the first caller to mark it gets true and everyone after gets false. That is how the operator is told once rather than once per job.

Refusal is not an error, either. It raises a SourceLimitError carrying the time the day rolls over, and the pipeline defers the stage's jobs to then instead of failing them. The difference matters because a failed job looks like a bug in a log and a deferred one looks like a schedule.

And work that is reserved but not done is given back:

export async function giveBack(counter: Counter, n = 1): Promise<void> {
  await giveBackTo(counter, dayOf(counter), n);
}

with greatest(0, used - n) in the update, because the one thing worse than a cap is a cap that leaks downward all day.

Two midnights

A detail that outlived the rewrite, and the only piece of this I have never been tempted to simplify:

// YouTube's quota day starts at midnight Pacific time; the others' at
// midnight UTC.
export const YOUTUBE_TIME_ZONE = "America/Los_Angeles";
const dayOf = (counter: Counter, at = new Date()) =>
  YOUTUBE_COUNTERS.includes(counter) ? dateIn(YOUTUBE_TIME_ZONE, at) : utcDate(at);

The key of the counter row is the day string, so "which day is it" has to be answered per counter, by whoever defines the quota. Getting this wrong does not throw. It gives you two partial days every time the clocks change in California, and a quota that appears to reset at a different hour twice a year.

What I would write in the first version now

The thing I had backwards in the original post is which question comes first. I started from "what number should this cap be", and the follow up "how do we keep the number fresh" produced a perfectly good mechanism for a cap that should not have existed.

The question that comes first is: when this limit fires, whose work stops, and can we tell them something true about it? If the answer involves explaining somebody else's account to them, the limit is in the wrong place, and no amount of adaptive arithmetic moves it to the right one.

The product facing version of all this is on the pricing page, where every per plan number is in one table, and the search section of how Nakodo works describes the daily cycle those numbers pace.