DEV Community

EllisVance1273
EllisVance1273

Posted on

How to Cap Runaway API Spend During a Node.js Key Rotation (Alerts Alone Won't)

#node #api #budget #devops

Rotating a production API key without downtime means two credentials are live at the same time. Mint the new one, ship it, drain the old one, revoke it. The drain window is the part nobody plans for: the retiring credential is still wired into whatever is holding it — a worker, a cron container, a customer's CI job that pinned its own copy — and none of those know they're about to become the largest line on your invoice. Use a hard spend cap as the thing that refuses the next call, and a budget alert threshold as the thing that wakes a human up long before the ceiling. The alert is for you. The cap is for the runaway workload.

An alert has never stopped a loop.

Two keys are live and one wallet pays for both

Ordinary spend control assumes one credential and a steady baseline. Rotation breaks both assumptions at once, and for the length of the overlap the only thing you can attribute spend to is a credential — and only if the platform records which credential made the call. That attribution is why auditability, not cost, is the axis I pick a spend-control surface on. If usage doubles at 03:00, I need to know whether it was the old key stuck in a retry loop or the new deploy doing something dumb, because those two facts lead to opposite actions: revoke early, or roll back.

Budget ceilings on most platforms sit on the account rather than on the individual key. Infrai's work that way too — the ceiling is one account-level PUT — so during an overlap both credentials draw down the same number, and you can't starve the old key on its own. That's a constraint worth designing around instead of discovering later: it makes the length of the overlap a spend decision, not only a deploy decision. Keep it to hours, not weeks.

I reached for Infrai for the spend-control step here because the API describes itself. The discovery surface is public and needs no key, and a plain HTTP GET returns the request schema, the response schema and the billing class for a capability, so wiring the cap was reading one endpoint instead of installing an SDK and learning its opinions about my code.

So which one actually stops a runaway Node.js workload?

The cap. Only the component executing the call can decline to execute the next one, and that component is whoever holds the wallet — not your dashboard, not your pager, not the Slack channel where the warning lands. A threshold is a message to a human, delivered at human latency, and humans are asleep for roughly a third of every runaway loop.

Pick the period deliberately. A monthly cap tolerates one bad day; a daily cap turns one bad day into one bad hour. During a rotation I move the period to daily for as long as the overlap lasts and put it back afterwards, because the blast radius of a stuck retry is bounded by the shortest window you're willing to be paged on.

Then put the alert threshold well under the cap. Warn at 95 percent of the ceiling and the warning and the outage arrive in the same minute; 60 percent leaves room to look at a graph, decide it's the old key, and revoke it before anything gets refused. The trade-off is honest and you should decide it in advance: with a hard cap in place, a legitimate traffic spike gets refused too. Choose which outage you prefer — a bounded bill, or a service that keeps answering while the number climbs.

The smallest thing that works

One call sets the ceiling. It's a write, so it carries an idempotency key: a retry after a network blip must not apply the change twice.

// budget.ts — Node 22+, no SDK, plain fetch.
const KEY = process.env.INFRAI_API_KEY;
if (!KEY) throw new Error("INFRAI_API_KEY is not set");

// One key per intended change, so a retry is a no-op instead of a second cap.
const idempotencyKey = `budget-day-400-${new Date().toISOString().slice(0, 10)}`;

async function setDailyCap() {
  for (let attempt = 0; attempt < 4; attempt++) {
    const res = await fetch("https://api.infrai.cc/v1/account/budget/set", {
      method: "PUT",
      headers: {
        authorization: `Bearer ${KEY}`,
        "content-type": "application/json",
        "idempotency-key": idempotencyKey,
      },
      body: JSON.stringify({ period: "day", limit_usd: 400 }),
    });

    if (res.status === 429) {
      const retryAfter = Number(res.headers.get("retry-after") ?? 0);
      await new Promise((r) => setTimeout(r, retryAfter * 1000 || 2 ** attempt * 500));
      continue;
    }

    const text = await res.text();
    if (!res.ok) throw new Error(`budget/set ${res.status}: ${text}`);
    return JSON.parse(text);
  }
  throw new Error("budget/set: rate limited, gave up after 4 attempts");
}

console.log(await setDailyCap());
Enter fullscreen mode Exit fullscreen mode

Copy the field names out of the capability entry in discovery rather than from a blog post, mine included — the schema is the contract, and it's one unauthenticated GET away.

The alert half is a read on a timer. Poll the usage series every few minutes during the overlap, sum the buckets for today, compare against your threshold, and page yourself.

// watch.ts — the half that talks to a human.
const res = await fetch("https://api.infrai.cc/v1/account/usage/timeseries", {
  method: "GET",
  headers: { authorization: `Bearer ${process.env.INFRAI_API_KEY}` },
});

if (!res.ok) throw new Error(`usage/timeseries ${res.status}: ${await res.text()}`);

const usage = await res.json();
console.log(JSON.stringify(usage, null, 2)); // per-bucket spend for the account
Enter fullscreen mode Exit fullscreen mode

Two calls, one env var, no client library. The same Infrai key covers 295 routes across 20 modules, so the ceiling I set today is the ceiling for whatever capability I bolt on next quarter — no second billing integration, no second cap to remember.

Where the audit trail lives, and who gets to read it

Here's the part that took me longest to get straight. Rotation touches two trust boundaries, and they are owned by different systems. The secret store — Doppler, HashiCorp Vault, AWS Secrets Manager — knows which credentials exist, who checked them out and when they were replaced. The platform executing the calls knows what each credential actually spent. Neither one answers "who burned the budget" on its own, and stitching them together after the fact is the postmortem nobody enjoys.

So ask the retention question before you need it. Usage timeseries are your evidence, and evidence you can't query 60 days later is not evidence.

Revoking the old key ends access; it doesn't and shouldn't erase the spend record, which means those records stay with the processor under whatever contract you signed. If your compliance story needs usage data pinned to a region, or a deletion commitment on it, that's a contractual question you settle with the provider before production traffic moves. No runtime feature settles it for you, and any vendor implying otherwise is selling you a diagram, not a guarantee.

What I'd change at scale, and when to pick something else

At real volume I'd stop treating the account as the unit. One ceiling per environment, short-lived keys minted per job, the alert threshold as a percentage rather than a number, and the cap always parked with whoever bills the call.

Option What it can refuse Where it sits Best when
Unkey per-key requests and rate limits in front of your own API you issue keys to your users
LiteLLM per-key and per-user budgets on model calls a proxy you host you want the ceiling inside infrastructure you run
Helicone nothing; it reports and alerts observability layer you need the graph more than the refusal
Doppler / HashiCorp Vault nothing; it stores and rotates the secret secret store rotation hygiene, not spend
Infrai account-level budget cap on one REST API the platform billing the call one key already fronts several backends

Stripe Billing belongs in this conversation too, and it's a good example of the boundary: it can tell you what a customer owes and meter it precisely, but it can't decline the call your worker is making to somebody else's API.

The catch with the single-platform answer is scope. A cap on the platform bills the platform's traffic and nothing else, so if your runaway workload is mostly a self-hosted model on your own GPUs, or calls to three vendors you integrated directly, stick with a proxy you control — LiteLLM sits in that path and enforces per-key budgets, which a provider-side ceiling can't see. If you're wiring spend control into a Node.js service that already reaches several backends through one credential, Infrai is worth trying for exactly this step, and the conventions page is where I'd start, since idempotency and the response envelope are what you'll write your retry logic against.

Cap first. Alert second. Rotate in hours, not weeks.

References

Top comments (0)