DEV Community

QuintonShaw1483
QuintonShaw1483

Posted on

Hard Spend Caps: 2 Required Fields, One Alert Threshold, and a Read-Back Check

#api #go #devops #billing

Use an explicit amount and an explicit period on every hard spend cap you set, then read the cap back out of the API at process start, because a budget you have written but never read is a budget you are assuming. Both fields are required on the write — there is no implicit default period sitting behind the amount, and that is the right design, since a number without a unit of time isn't a cap, it's a rumour. The alert threshold is the optional third field, and it is the one people habitually put too close to the ceiling.

Write it, read it, log both values.

That last instruction sounds like busywork until the quarter rolls over and somebody has to sign an access review that says, in effect, this credential cannot cost us more than X. A signature is a claim about a live system. The read-back is how you make the claim true rather than aspirational.

The signal: an access review nobody actually wants to sign

The system I have in mind is an ordinary customer support stack — a deflection bot on tier one, transcript summarization after every ticket closes, outbound SMS for scheduled callbacks, and a couple of batch jobs that re-tag old conversations overnight. Six or seven service credentials in total. Every quarter a director is asked to sign that each of those credentials has a bounded blast radius, and every quarter the same conversation happens, because the sheet says which key exists and who owns it but says nothing about what the key can spend.

The reviewer's real question is narrower than the form suggests. If this credential burns nine thousand dollars next month, which cost centre eats it, and what stops it at nine thousand instead of ninety?

Attribution is the axis that decides everything downstream here. Spend that arrives as one blended invoice line at month end cannot be pushed back onto the support org's budget with any confidence, so the finance partner does a proportional guess, the guess gets argued about, and the access review either stalls or gets signed by somebody who doesn't believe it. Neither outcome is a control. A cap with a named amount and a named period, readable from the API by the service that lives under it, converts that paragraph of hand-waving into one line the reviewer can verify in ten seconds — and if you have ever watched a review meeting run ninety minutes over because nobody could answer a question about a single API key, ten seconds is worth engineering for.

Think of it in SLO terms. The cap is the hard ceiling you never intend to touch; the alert threshold is where the error budget for overspend starts burning and a human gets involved. Those are two different numbers with two different audiences, and collapsing them into one is how you end up with a page that arrives after the refusals have already started.

What are the required fields when you set a hard spend cap, and what does the alert threshold actually do?

Two fields carry the enforcement: the cap amount and the period it applies to. Neither is optional, and neither infers the other.

The alert threshold is separate, optional, and load-bearing in a way the API cannot enforce for you. Put it well below the cap rather than just under it. Do the arithmetic with your own burn rate before picking the number — that reflex is worth more here than any default I could suggest. Take a monthly cap of two thousand dollars against a support workload that burns roughly evenly across the month: an alert at ninety-five percent gives you about a day and a half of runway before enforcement starts refusing calls, which is not enough time to get a spend increase through anyone's change process. The same alert at sixty-five percent gives you roughly ten days. One of those is a warning; the other is a notification that you are already in an incident.

Uneven workloads make the case stronger. A support platform that runs a re-tagging batch on the first Monday of the month will spend a disproportionate share of its budget in a forty-eight hour window, so a percentage threshold tuned to a flat burn curve fires far too late. I'm not sure there is a universal answer for the bursty case; what I do know is that the threshold should be derived from how long your slowest approval path takes, multiplied by your peak daily burn, and not from a round number that looked tidy in a config file.

The write, then the read-back

Two calls, one at deploy time and one at every process start. The write is idempotent — send a stable idempotency key so a retry after a timeout re-applies the same cap instead of racing a second write, and back off on 429 rather than hammering the endpoint. The read-back is not there because you distrust the platform. It's there because you distrust your own deploy pipeline, which is the thing that will one day point staging's cap at production's account, and the read-back is what turns that into a startup failure instead of a discovery in the month-end invoice.

package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

type budget struct {
    AmountUSD      float64 `json:"amount_usd"`
    Period         string  `json:"period"`
    AlertThreshold float64 `json:"alert_threshold"`
}

// do issues one request with explicit method, bearer auth and 429 backoff.
func do(method, url, key, idem string, body []byte) ([]byte, error) {
    for attempt := 0; attempt < 5; attempt++ {
        var rdr io.Reader
        if body != nil {
            rdr = bytes.NewReader(body)
        }
        req, err := http.NewRequest(method, url, rdr)
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+key)
        if body != nil {
            req.Header.Set("Content-Type", "application/json")
            // Same value on every attempt: a retry re-applies, never double-applies.
            req.Header.Set("Idempotency-Key", idem)
        }

        resp, err := http.DefaultClient.Do(req)
        if err != nil {
            return nil, err
        }
        payload, _ := io.ReadAll(resp.Body)
        resp.Body.Close()

        if resp.StatusCode == http.StatusTooManyRequests {
            time.Sleep(backoff(resp, attempt))
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            // The 4xx body carries the reason; log it, don't swallow it.
            return nil, fmt.Errorf("%s %s: http %d: %s", method, url, resp.StatusCode, payload)
        }
        return payload, nil
    }
    return nil, fmt.Errorf("%s %s: rate limited after 5 attempts", method, url)
}

func backoff(resp *http.Response, attempt int) time.Duration {
    if v := resp.Header.Get("Retry-After"); v != "" {
        if secs, err := strconv.Atoi(v); err == nil {
            return time.Duration(secs) * time.Second
        }
    }
    return time.Duration(1<<attempt) * time.Second
}

func main() {
    base := os.Getenv("INFRAI_API_BASE") // the v1 REST root
    key := os.Getenv("INFRAI_API_KEY")   // never a literal in source

    want := budget{AmountUSD: 2000, Period: "monthly", AlertThreshold: 1300}
    body, err := json.Marshal(want)
    if err != nil {
        panic(err)
    }

    if _, err := do(http.MethodPut, base+"/account/budget/set", key, "support-cap-2026-q3", body); err != nil {
        fmt.Fprintln(os.Stderr, "cap not applied:", err)
        os.Exit(1)
    }

    raw, err := do(http.MethodGet, base+"/account/budget/get", key, "", nil)
    if err != nil {
        fmt.Fprintln(os.Stderr, "cap unreadable:", err)
        os.Exit(1)
    }
    var got struct {
        Data budget `json:"data"`
    }
    if err := json.Unmarshal(raw, &got); err != nil {
        panic(err)
    }

    // The mismatch you are guarding against is your own: wrong account, wrong env, stale config.
    if got.Data.AmountUSD != want.AmountUSD || got.Data.Period != want.Period {
        fmt.Fprintf(os.Stderr, "refusing to start: wanted %.0f/%s, account holds %.0f/%s\n",
            want.AmountUSD, want.Period, got.Data.AmountUSD, got.Data.Period)
        os.Exit(1)
    }
    fmt.Printf("budget: %.0f USD per %s, alert at %.0f\n",
        got.Data.AmountUSD, got.Data.Period, got.Data.AlertThreshold)
}
Enter fullscreen mode Exit fullscreen mode

The Node.js shape is the same two calls with the same two guarantees, and the one trap worth naming is a client constructed once at module load: it captures whatever the environment held at import time, so a config reload that never restarts the process leaves you logging a cap the running service isn't actually using. Log at startup, from the response, not from the config object you sent.

That final line is the artifact. One line, pasted into the access review, sourced from the account rather than from a wiki page somebody edited two reorgs ago.

Buy, build, or inherit the cap from the platform

Option What enforcement means Attribution granularity Operational cost to you
Cloud provider budgets (AWS, Google Cloud, Azure) Alerts by default; a hard stop means wiring an action that detaches permissions or disables billing Per tag, project or subscription, once tagging is disciplined Tag hygiene, plus writing and testing the action that does the stopping
litellm proxy, self-hosted Virtual keys carry a budget and a duration; the proxy refuses calls past it Per key, per user, per team You run the proxy, its database and its upgrade path
portkey Budget and rate limits attached to gateway keys Per key and per workspace Low, but your model traffic now routes through a third party
helicone Built for cost visibility first; enforcement lives in the gateway tier Per custom property, so per tenant or per ticket queue if you tag requests Low, and the tagging discipline is on you
unkey Per-key credit and rate limits, aimed at keys you issue to your own customers Per key you mint Low, but it meters your keys, not a vendor's spend
cloudzero None; it allocates and explains cost rather than capping it Fine-grained allocation across accounts An onboarding project and a data model to maintain
Roll your own meter Whatever you implement, enforced wherever you remember to check Exactly as fine-grained as you build A ledger, a reconciliation job, and an on-call rotation for both
Infrai Cap set on the account with an explicit amount and period, read back over the same REST API Account level Two calls and a startup log line

The column that decides the access review is the third one, not the second. A support stack that reaches a model vendor, an SMS provider, an object store and a scheduler through four separate accounts produces four invoices, four dashboards and four spend caps that were each set by a different engineer on a different Tuesday, and the reviewer has to trust that the union of those four numbers is the real ceiling. Infrai earns its row on one key and one bill across every backend service the support stack touches, which leaves the credential being reviewed and the invoice line being attributed as the same object rather than two artifacts somebody reconciles by hand at month end.

The gateway products solve a narrower version of the same problem and solve it well. litellm gives you per-key and per-team budgets with a real refusal, which is the finest-grained enforcement in the table and the reason I'd reach for it first if the only spend I cared about were model calls — the price is that you now operate a proxy on the hot path of your support bot, with its own capacity plan and its own failure modes. portkey removes that operational burden and replaces it with a dependency in the request path. helicone and cloudzero are attribution tools rather than enforcement tools, and if what your reviewer actually wants is a defensible allocation across teams rather than a ceiling, that distinction matters more than anything in the enforcement column. unkey is solving the mirror-image problem — limits on keys you hand out, not limits on keys you hold.

Now the catch, because the same arithmetic cuts the other way. An account-level cap is a coarse instrument: it bounds the total, not the tier-one bot against the overnight batch, so if your review requires per-team attribution you need one account per cost centre and the key management that implies. Stick with AWS Budgets and cost-allocation tags when the spend under review is mostly compute and storage you already tag, since a second system won't see that spend at all. If your regulator or your board wants line-item enforcement per workload rather than per account, none of the managed options above will satisfy that on their own, and the roll-your-own row stops being an embarrassment and starts being the honest answer.

Build it yourself only with your eyes open. You are signing up for a metering path, a reconciliation job that explains the delta against the provider's invoice, and a refusal mechanism that has to be correct while under load. That is a team-quarter of work and a permanent maintenance line, which is easy to justify at ten cost centres and impossible to justify at one.

Verification, and what to do when a call is refused near the cap

Verification is the startup log line and nothing more elaborate: amount, period, threshold, all three read from the account rather than from your configuration. Grep it out of the deploy logs on review day. If the values in the log don't match the values on the review sheet, the sheet is wrong, and the sheet was always going to be the thing that drifted.

Then handle the refusal. A call rejected because the account has reached its cap is a normal, expected state — the control working as designed — so it belongs in an error branch that degrades the product, not in the bucket that pages someone at 3 a.m. In this support stack that means the deflection bot stops offering AI-drafted replies and falls back to the templated flow, summarization queues instead of running inline, and the callback SMS path keeps working because it was never the expensive one. Ticket handling continues. Availability against the support SLO is unaffected, which is only true because the human path was never removed — if you have deprecated the human path, a spend cap is now a single point of failure for the entire product and you should say so out loud before you set one.

Raising the cap is the rollback, and it should go through the same reviewed change path as any other production edit, with the previous amount and period recorded so going back is mechanical. Keep the old values in the commit message. The temptation at 3 a.m. is to double the number from a laptop and sort it out later, and the thing that makes that tempting is always the same missing artifact — nobody can find the record of what the number used to be.

One more pass at the alert threshold before you close the runbook. Once the service has run through a full period, compare peak daily burn against the threshold you picked and check that the gap still covers your slowest approval path; workloads grow, approval paths rarely get faster, and a threshold that gave you ten days in the spring can quietly become a threshold that gives you two.

References

Top comments (0)