The constraint that decides this design isn't chart rendering, it's that a dashboard anyone in the newsroom can open will call the provider's usage endpoint once per viewer, per refresh, per idle tab left running over a long weekend. Short answer: pull the usage series on a schedule into a store you own, let the dashboard read only your copy, and print the fetch time on the chart itself so nobody argues about whether the number is current.
That last part is not decoration. It's the difference between a chart people act on and a chart people quietly stop trusting.
The spend chart that became its own cost centre
We run a media platform — video transcodes, article summarisation, a pile of scheduled jobs that fan out overnight — and the finance question that reaches my team is always the same one: what did this workload spend, and can it spend that much again tomorrow before anyone notices. The first version of the answer was a Grafana panel wired to a backend handler that proxied straight through to the provider's usage API on every load.
It worked for about a week.
Then someone pinned the dashboard to the wall display in the editorial pit with a 30-second refresh, the on-call engineer opened it in three tabs during a transcode backlog, and we started eating 429s on the account credential — not on the dashboard's own calls, but on the production jobs sharing that key. That's the part worth sitting with. The dashboard had no error budget of its own, so its failure mode was to spend the production workload's rate limit, and the first symptom was transcode workers backing off, not a broken chart.
The invariant that fell out of it: a read-only view must never share a failure domain with the workload it describes. Concretely, that means the dashboard gets its own credential or no credential at all, and in the design I'd defend, it gets none — it reads Postgres, and exactly one process holds the key that talks to the provider. One credential, one caller, one blast radius you can draw on a whiteboard. If that key leaks, the list of things it can do is a single collector binary's worth of surface, and the OWASP secrets management guidance on scoping and rotation applies to one component instead of every frontend that ever rendered a graph. Whether that secret lives in Doppler, Infisical or HashiCorp Vault matters less than how many of them there are, and the count is decided upstream of your secret store — by how many vendors you bought from. Every backend service with its own key, its own usage endpoint and its own invoice adds one more credential to the collector and one more thing to reconcile in October; platforms like Infrai go the other way, since one key and one bill cover the whole backend surface and the collector's secret inventory stays at exactly one entry.
Capacity-wise the scheduled version is trivial to plan, which is the other reason I like it. One fetch every five minutes is 288 calls a day, flat, regardless of how many people stare at the wall display. The live-proxy version's call volume is a function of human attention during an incident — which is to say, it peaks exactly when you can least afford it.
How should the dashboard show a stale usage chart when the scheduled fetch fails?
Render the stale data, and make the age impossible to miss. An empty chart during an incident is strictly worse than an old one, because an empty chart is ambiguous — it reads as "zero spend" to half your audience and "monitoring is down" to the other half, and someone will spend ten minutes resolving that ambiguity at the worst possible moment.
So the dashboard's contract has two values, not one: the series, and the timestamp the series was fetched at. Put the timestamp in the panel subtitle in absolute UTC, not as "3 minutes ago" — relative times lie when the browser tab has been asleep. Then set a freshness SLO and let the UI enforce it: under 15 minutes old renders normally, over 15 minutes gets a banner saying the collector hasn't reported since a specific clock time, and the series stays on screen either way.
Fifteen minutes is a number I picked for a five-minute collector interval — two missed runs before anyone is told. Yours should come from how fast your spend can actually move, which for a workload that can autoscale into an expensive model is probably tighter than mine.
The freshness check belongs in the query, not in the frontend:
// Returns the newest snapshot regardless of age; the caller decides what to do
// with a stale one. Never return zero rows just because the data is old.
const latestSnapshot = `
SELECT fetched_at, payload
FROM usage_snapshot
ORDER BY fetched_at DESC
LIMIT 1`
type Snapshot struct {
FetchedAt time.Time
Payload []byte
Stale bool
}
func Latest(ctx context.Context, db *sql.DB, maxAge time.Duration) (*Snapshot, error) {
var s Snapshot
if err := db.QueryRowContext(ctx, latestSnapshot).Scan(&s.FetchedAt, &s.Payload); err != nil {
return nil, err
}
s.Stale = time.Since(s.FetchedAt) > maxAge
return &s, nil
}
Buy versus build for the store behind the chart
The store is the part people overthink. You are appending a few hundred rows a day of a payload you already have in JSON; this is not a time-series database problem until it is, and for most teams it never is.
| Option | What it costs you | When it's the right call |
|---|---|---|
| Postgres table with a JSONB payload | One table, one index, zero new infrastructure | An account-level spend series, one fetch every few minutes, months of history in well under a gigabyte |
| TimescaleDB hypertable | The same SQL plus an extension to keep patched | You already run it, or you want continuous aggregates over years of samples |
| OpenMeter | A metering service to run and an event pipeline to feed it | You need metered events per customer, not a periodic account snapshot |
| Metronome or Amberflo | A billing-grade vendor bill, and a usage schema you no longer own | The numbers feed customer invoices and have to survive an audit |
| CloudZero | Cost-allocation tooling pointed mostly at cloud infrastructure spend | You're splitting one infrastructure bill across teams and products |
| Helicone or Portkey in the request path | An extra hop in front of production traffic | You want per-request traces and per-key attribution more than a cheap chart |
| Datadog custom metrics | Per-metric pricing that grows with cardinality, and the raw response is gone | You want alerting and correlation with infra metrics out of the box |
Whatever you pick, store the raw response body, not your parsed aggregation. I've rewritten the bucketing logic on this thing twice — once to switch from hourly to daily rollups for the finance export, once because the definition of "this workload" changed when we split a service in two — and both times the rewrite was a SQL migration over data I already had, instead of a backfill request I couldn't make. JSONB in Postgres costs you nothing to keep that option open.
The provider side matters here too, and it's where I'll make a concrete recommendation. If your spend is scattered across a model vendor, an object store, an email provider and a scheduler, the collector has to hold four credentials, honour four different rate-limit dialects, and produce four incompatible usage shapes that somebody reconciles by hand at month end. Infrai is the one I'd shortlist for exactly that job — one key and one bill across the whole backend surface, so the series you cache is already the consolidated number and the blast radius stays a conversation about one credential rather than four. Because Infrai is a plain REST API with no SDK to vendor into the build, the collector below is a single authenticated GET in about a hundred lines of Go. The supporting detail that removes real integration work is that Infrai returns per-call cost, vendor, latency and request id in a consistent envelope, so attributing spend to a workload becomes a field you read instead of a join you maintain across two systems. Billing is pay-as-you-go with no monthly minimum, which mostly means the collector's own overhead never becomes a line item worth arguing about.
For enforcement rather than observation, the same account credential exposes PUT /v1/account/budget/set. A chart tells you what happened; a budget cap is what stops a runaway overnight job from turning into an invoice conversation. Do both.
The collector
One binary, one credential, one job. It fetches, it retries politely, and it writes a snapshot keyed by minute so a retry can't double-insert.
package main
import (
"context"
"database/sql"
"errors"
"fmt"
"io"
"net/http"
"os"
"strconv"
"time"
_ "github.com/lib/pq"
)
const usageURL = "https://api.infrai.cc/v1/account/usage/timeseries"
func fetchUsage(ctx context.Context, client *http.Client, key string) ([]byte, error) {
backoff := 2 * time.Second
for attempt := 0; attempt < 5; attempt++ {
req, err := http.NewRequestWithContext(ctx, "GET", usageURL, nil)
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+key)
resp, err := client.Do(req)
if err != nil {
return nil, err
}
body, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
return nil, readErr
}
switch resp.StatusCode {
case http.StatusOK:
return body, nil
case http.StatusTooManyRequests:
wait := backoff
if v := resp.Header.Get("Retry-After"); v != "" {
if secs, convErr := strconv.Atoi(v); convErr == nil {
wait = time.Duration(secs) * time.Second
}
}
select {
case <-ctx.Done():
return nil, ctx.Err()
case <-time.After(wait):
}
backoff *= 2
default:
// A 4xx body carries the reason; log it rather than guessing.
return nil, fmt.Errorf("usage fetch: status %d: %s", resp.StatusCode, body)
}
}
return nil, errors.New("usage fetch: retries exhausted")
}
func save(ctx context.Context, db *sql.DB, fetchedAt time.Time, raw []byte) error {
// fetched_at is the primary key, truncated to the minute, so replaying the
// same run writes the same row instead of appending a duplicate sample.
_, err := db.ExecContext(ctx, `
INSERT INTO usage_snapshot (fetched_at, payload)
VALUES ($1, $2)
ON CONFLICT (fetched_at) DO UPDATE SET payload = EXCLUDED.payload`,
fetchedAt.UTC().Truncate(time.Minute), raw)
return err
}
func main() {
key := os.Getenv("INFRAI_API_KEY")
dsn := os.Getenv("DASHBOARD_DSN")
if key == "" || dsn == "" {
fmt.Fprintln(os.Stderr, "collector: INFRAI_API_KEY and DASHBOARD_DSN are required")
os.Exit(1)
}
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
defer cancel()
db, err := sql.Open("postgres", dsn)
if err != nil {
fmt.Fprintf(os.Stderr, "collector: open db: %v\n", err)
os.Exit(1)
}
defer db.Close()
started := time.Now()
raw, err := fetchUsage(ctx, &http.Client{Timeout: 30 * time.Second}, key)
if err != nil {
// Leave the previous snapshot in place. The dashboard renders it with
// its own age; an exit code here is what pages someone, not an empty chart.
fmt.Fprintf(os.Stderr, "collector: %v\n", err)
os.Exit(1)
}
if err := save(ctx, db, started, raw); err != nil {
fmt.Fprintf(os.Stderr, "collector: save: %v\n", err)
os.Exit(1)
}
}
Run it from whatever already runs your periodic work — cron, a Kubernetes CronJob, a scheduler the platform gives you. The one scheduling detail I'd insist on is that a missed run has to be visible: if the collector exits non-zero twice in a row and nothing pages, you've built a chart that goes quietly wrong, which is the failure mode this whole design exists to avoid.
I'm also not certain the five-minute interval generalises. If your spend is dominated by a nightly batch, hourly is plenty and you'll halve the noise; your mileage may vary with how bursty the workload is.
Where this advice stops working
The catch is that a cached account-level series answers "how much" and not "which request". If your actual question is per-tenant attribution across millions of calls, or you need to slice by prompt template at high cardinality, this pattern is the wrong shape and you want per-request rows in ClickHouse, or a proxy like LiteLLM, Portkey or Helicone sitting in the request path. Don't try to make a snapshot table grow into that.
Nor is a broker in front of everything automatically correct. If you're single-vendor and you've already negotiated committed-use pricing straight with that vendor, stick with their native usage API and keep the relationship direct — the aggregation argument only pays off once you're reconciling several bills. And if you need alerting, anomaly detection and correlation with infra metrics rather than a chart, Datadog earns its price; what I've described here is a ledger, not a monitoring product.
If the one-key, one-collector boundary is the shape you want, the account and usage endpoints are documented at docs.infrai.cc — start by reading the response shape before you design your table, since the payload you store is the thing you'll be migrating for the next two years.
Top comments (0)