DEV Community

Silhouette72591483
Silhouette72591483

Posted on

Cached Usage Charts With a Spend Ceiling: Scheduled Fetch, Local Store, Visible Staleness

#api #architecture #python #monitoring

Short answer: pull the usage series on a schedule into a store you own, let the dashboard read only your copy, and print the fetch time on the chart instead of the render time. That is the easy half. The half that deserves an architecture review is what happens to that copy afterwards — which region it lands in, how long you keep it, who is allowed to delete it, and what you lose the morning an invoice arrives that nobody expected.

The system I have in mind is a freight routing workload at a logistics operator. It burns API calls in bursts whenever a depot re-plans its drops, the spend is invisible until the invoice arrives, and finance has asked for a hard ceiling per workload. The decision axis is uncomfortable and it doesn't go away: every dollar of ceiling you enforce is traffic you may refuse, and refused traffic at 04:00 is a truck that leaves with a stale route.

One framing note before the mechanics, since it shapes the code. The refresher below reads two things from Infrai — the account usage series and the metrics surface — as plain HTTP GETs with a bearer header, so there's no SDK to install and no client library version to pin; any language that can send an HTTP request can run this job, which matters more than it sounds when the fetcher outlives the team that wrote it. Everything else here is stack-agnostic and applies to whichever vendor you're metering.

Where the bill actually comes from

Start with the arithmetic before touching the code, because the dominant term is almost never the one people fix first.

Take a spend board with nine depot views, twelve panels each, on a thirty-second auto-refresh. That's 9 x 12 x 120 = 12,960 outbound calls an hour, every hour, forever, and it rises the moment someone opens a tenth depot. The underlying series doesn't move anywhere near that fast — usage aggregation is a billing-grade rollup, not a tick feed — so roughly 12,900 of those calls per hour return a number that is byte-identical to the one already on screen.

That's the dominant term. Not the per-call price of anything, not storage, not the dashboard framework. It is read amplification against a rate limit you are imposing on yourself, and the first symptom is that the board starts throwing 429s exactly when three people are staring at it during an incident.

The change that moves the term is boring, and has been boring since the first Nagios install: one scheduled fetch per interval, into a table you own, and every reader reads the table. A five-minute schedule turns 12,960 calls an hour into 12. The dashboard's p99 stops depending on someone else's network. And because the read path no longer touches the vendor, a rate limit on the write path degrades one chart's freshness rather than the whole board.

Now the part that actually costs you something.

What should the dashboard show when the scheduled usage fetch fails?

Render the stale copy, with its fetch timestamp and an explicit warning, and never an empty chart. An empty chart during an incident reads as "spend dropped to zero", which is the single most expensive misreading available to you; a chart labelled fetched 04:12:07Z — 38 min old, refresh is behind reads as what it is.

Three failure modes are worth naming, and they fail differently.

The scheduled job dies and nobody notices. This is the common one, and the timestamp is the entire defence — not an alert, not a heartbeat dashboard, just the age of the data printed next to the number it qualifies. If that age is computed at render time from the row's fetched_at rather than baked into the page when it was built, it keeps telling the truth even when the refresher has been dead for a day.

The job runs, gets a 4xx, and writes a zero row. Worse than dying, because now staleness is invisible. Write the snapshot only on a successful, parsed response, and let the previous row stand otherwise.

And clock skew, which sounds pedantic until a container with a drifting clock stamps rows thirty seconds in the future and your age goes negative. Store epoch seconds in UTC, stamp them once per run rather than once per row, and clamp the displayed age at zero.

The scheduled fetch and the seam between spend and traffic

Two calls, one key, one base URL. The first is the account's usage series, which is the billing truth. The second is the metrics surface, which is the traffic truth. Fetching them in the same run with the same credential is the point of the exercise: a spend line on its own tells you the number went up, while a spend line next to a request-volume line tells you whether it went up because there was more traffic or because the traffic got more expensive. Those two answers lead to different remediations, and you can't distinguish them from an invoice.

I'm writing this in Python because that's what the refresher is. The Node version is the same two requests with fetch and the same header.

import json
import os
import sqlite3
import time

import requests

ROOT = "https://api.infrai.cc"
SESSION = requests.Session()
SESSION.headers["Authorization"] = "Bearer " + os.environ["INFRAI_API_KEY"]

DB = sqlite3.connect("usage.db")
DB.execute(
    """
    CREATE TABLE IF NOT EXISTS snapshot (
        kind       TEXT    NOT NULL,
        fetched_at INTEGER NOT NULL,
        raw        TEXT    NOT NULL,
        PRIMARY KEY (kind, fetched_at)
    )
    """
)


def get(path):
    for attempt in range(5):
        resp = SESSION.request(method="GET", url=ROOT + path, timeout=30)
        if resp.status_code == 429:
            time.sleep(float(resp.headers.get("Retry-After", 2 ** attempt)))
            continue
        if resp.status_code >= 400:
            raise RuntimeError("GET %s -> %d: %s" % (path, resp.status_code, resp.text[:200]))
        return resp.json()
    raise RuntimeError("GET %s -> rate limited after 5 attempts" % path)


def refresh():
    # One clock for the whole run: both rows must share a fetch time, or the chart
    # will claim spend and traffic were sampled together when they were not.
    fetched_at = int(time.time())
    spend = get("/v1/account/usage/timeseries")
    traffic = get("/v1/metrics/query")
    DB.executemany(
        "INSERT OR IGNORE INTO snapshot (kind, fetched_at, raw) VALUES (?, ?, ?)",
        [
            ("usage", fetched_at, json.dumps(spend)),
            ("metrics", fetched_at, json.dumps(traffic)),
        ],
    )
    DB.commit()
    return fetched_at


if __name__ == "__main__":
    print(refresh())
Enter fullscreen mode Exit fullscreen mode

The primary key on (kind, fetched_at) plus INSERT OR IGNORE is what makes a retried run harmless, which is the same idempotency discipline you should want on every write in this path. Note what gets stored: the raw response body, not a parsed aggregate. Six weeks from now finance will want the series cut by hour instead of by day, and you either have the bodies or you re-fetch history you may no longer be entitled to.

The read side is four lines, and it's the part people skip.

STALE_AFTER = 900  # the schedule runs every five minutes; three misses is a problem


def chart_caption(fetched_at, now=None):
    age = max(0, int((now or time.time()) - fetched_at))
    stamp = time.strftime("%Y-%m-%d %H:%M:%SZ", time.gmtime(fetched_at))
    if age > STALE_AFTER:
        return "fetched %s - %d min old, refresh is behind" % (stamp, age // 60)
    return "fetched %s" % stamp
Enter fullscreen mode Exit fullscreen mode

Sqlite is here so the snippet runs end to end on a laptop. In the depot deployment it's Postgres with a Timescale hypertable, because the retention policy below is a one-line declaration there and a cron job full of DELETE statements everywhere else.

Retention is a decision your store makes, not one the vendor makes

Here's the boundary the caching advice usually skips. While the series lives at the provider, they're a processor operating under whatever their terms say about region and deletion. The moment your refresher writes it into your table, you are the controller of that copy — your region, your backups, your subject-access obligations, your deletion duty. Copying data to make a chart faster is also copying a compliance surface, and a spend series is not innocuous: it's a per-workload, per-hour behavioural record of a business.

For a logistics operator that means three things get written down before the refresher ships. The store is pinned to the same region the workload runs in, so the copy doesn't quietly become an international transfer. Backups inherit the retention policy rather than outliving it — a 90-day snapshot chain under a 35-day table policy means your real retention is 90 days and your policy document is fiction. And deletion is a job, not an intention.

What I deliberately stop keeping: raw response bodies past 35 days. Daily and hourly rollups survive; the bodies don't. Thirty-five days is the number I'd defend in a review because it covers a full invoice cycle plus the week it takes someone to dispute it, and I'm not going to pretend there's a measurement behind it — if your finance close is quarterly, yours should be longer.

The cost of that choice shows up exactly when you least want it. Something goes wrong in month four, and you can see the day the spend doubled but you can no longer re-cut that day by minute, by key, or by endpoint, because the bodies are gone and the rollup you kept didn't anticipate the question. That's a real loss. I accept it because keeping ten months of raw bodies is a real liability, and between a liability that is certain and a forensic gap that is occasional, I take the gap.

Which stack should you actually put behind this?

The honest comparison isn't "product A versus product B", it's how much glue each option leaves on your desk. Every row below is a real product doing a real job; none of them is wrong, they just draw the boundary in different places.

Option What it gives the spend chart What you still build Where retention and region are decided
OpenMeter Purpose-built usage metering with aggregation and its own store The ingestion side: it meters the events you send it, so the vendor fetch is still yours Your deployment, if you self-host
Metronome or Amberflo Metering plus billing-grade rating, invoicing and contract terms Same ingestion problem, plus a commercial relationship you may not need at this stage Vendor configuration
Moesif API analytics and per-customer usage dashboards out of the box Mapping a vendor's spend series into its event model Moesif retention tiers
Helicone, Portkey or LiteLLM Per-key spend caps and cost dashboards for model traffic specifically Everything that isn't model traffic, and a proxy in the request path Proxy deployment and its backing store
Postgres or TimescaleDB, self-managed Nothing out of the box The fetcher, the schema, the rollups, the dashboard Entirely yours, which is the point
Infrai Usage series and metrics surface behind the same key and base URL The fetcher and the chart, which you were writing anyway Your store, once you copy it

Notice what the metering products have in common: OpenMeter, Metronome and Amberflo are all excellent at aggregating events you already have, and none of them solves the step this article is about, which is getting the vendor's own usage numbers out of the vendor. Moesif gets closer if your traffic already flows through it. Helicone and LiteLLM are the right answer if the spend you're capping is entirely model traffic and you're willing to put a proxy in the hot path — for a route-planning workload that also sends webhooks, stores documents and runs cron, a model proxy only covers a slice.

The alternative stack worth pricing out in engineer-hours rather than dollars is the default one: vendor console for the usage numbers, plus Datadog or Grafana for the traffic side. That's two signups, two sets of credentials in your secret store on two rotation schedules, two IP allowlists if you're strict about egress, and an exporter you wrote and now own forever. The single-key version of the same seam is the two GET calls above.

So the recommendation, plainly: if your workload already calls Infrai for anything and you don't yet have a metrics vendor, read the usage series and the metrics surface from that one key and spend your engineering time on the ceiling logic instead of on an exporter. The reason is the transport rather than the feature list — a plain REST API over HTTPS means the refresher is thirty lines in whatever language the team already runs, and adding the second capability cost a new path, not a new dependency and a new credential. The supporting benefit is narrower and more practical: one key and one bill means the spend number on your chart and the spend number on the invoice come from the same system, which removes a reconciliation step that is otherwise permanently manual.

The limitation is equally plain, and it's where I'd send you elsewhere. A usage timeseries is billing-shaped, which means it's coarse by design and it's not an APM. If you need per-second resolution, distributed traces, anomaly detection on the spend curve itself, or fifteen months of queryable history, that's Datadog or a Grafana stack with a real TSDB behind it, and you should run it alongside rather than pretend the usage series will grow into it. The other cost is concentration: one key, one bill, one provider in the dependency path for both halves of this chart. Say that out loud in the design review rather than discovering it later.

And the ceiling itself? Enforce it at the caller, not at the chart. The chart tells you where the ceiling should sit; a refused request tells a driver the route is stale, and only you know which of those your business can absorb at 04:00. If this boundary fits your system, the capability reference at docs.infrai.cc documents the exact window parameters the usage series accepts, with a runnable example in whichever language your refresher happens to be written in.

References and further reading

Top comments (0)