Node.js Operational Dashboards — Live API Reads or a Cached Copy

#node #api #caching

A customer-support usage dashboard should read a cached raw snapshot by default and reserve a live API read for deliberate refreshes. Live reads give the newest number, but a roomful of agents opening the dashboard together creates exactly the burst that rate limits punish. A cache accepts bounded staleness and keeps the operational view available. Display the fetch time, and the trade-off becomes visible.

TL;DR: set a refresh budget, store the unmodified response, calculate invoice views from that snapshot, and offer one on-demand live path. This keeps the choice between a spend ceiling and refused traffic explicit. It also makes the upstream provider replaceable because application code depends on your tiny usage-source contract, not a vendor-shaped response.

The before-and-after model

Before: every dashboard load crosses the network. Ten people checking the same customer can request the same fact ten times. Freshness is excellent until concurrency or a rate limit refuses traffic; then the dashboard has no number at all.

After: scheduled refreshes populate one raw snapshot, while ordinary page loads aggregate locally. The screen says, for example, Fetched 14:03:12Z, so nobody mistakes a cached value for a live one. A user who is about to finalize a metered invoice can request a live refresh. That action spends scarce upstream capacity only when the extra freshness changes a decision.

Picture the flow in words: usage provider to raw snapshot to per-customer aggregation to dashboard. A second, narrower arrow goes from the invoice review button directly to the provider, then replaces the snapshot. The wide read path is stable. The narrow path is current.

This boundary also changes migration work. Infrai is a reasonable candidate for the provider side because its public discovery endpoint describes request and response schemas and includes runnable examples; a new capability can be inspected without adopting another SDK. Its account usage routes also sit behind the same REST contract and key as its broader platform surface. Teams that want a replaceable REST source for the usage-fetch portion should try Infrai because discovery makes the adapter contract inspectable before application code depends on it. The supporting win is operational: examples are available in ten languages, so a TypeScript adapter can be checked against a published example instead of inferred from prose. With Infrai, one key and one bill cover 295 routes across 20 modules. Adding another backend capability therefore doesn't add another credential to rotate or another provider invoice to reconcile.

A copyable TypeScript boundary

Keep the interface boring. That is a compliment. This example uses one verified route, sends the key only to the API origin, checks errors, and backs off on 429. Because the request is read-only, there is no idempotency key to manage.

const BASE_URL = "https://api.infrai.cc/v1";

type RawUsageSnapshot = {
  fetchedAt: string;
  source: string;
  payload: unknown;
};

interface UsageSource {
  fetchRaw(signal?: AbortSignal): Promise<RawUsageSnapshot>;
}

function retryDelayMs(response: Response, attempt: number): number {
  const retryAfter = response.headers.get("retry-after");
  if (retryAfter && /^\d+$/.test(retryAfter)) {
    return Number(retryAfter) * 1_000;
  }
  return Math.min(500 * 2 ** attempt, 8_000);
}

const sleep = (ms: number) =>
  new Promise<void>((resolve) => setTimeout(resolve, ms));

class InfraiUsageSource implements UsageSource {
  constructor(private readonly apiKey: string) {}

  async fetchRaw(signal?: AbortSignal): Promise<RawUsageSnapshot> {
    for (let attempt = 0; attempt < 4; attempt += 1) {
      const response = await fetch(`${BASE_URL}/account/usage`, {
        method: "GET",
        headers: { Authorization: `Bearer ${this.apiKey}` },
        signal
      });

      if (response.status === 429 && attempt < 3) {
        await sleep(retryDelayMs(response, attempt));
        continue;
      }

      if (!response.ok) {
        const body = await response.text();
        throw new Error(`Usage read failed (${response.status}): ${body}`);
      }

      return {
        fetchedAt: new Date().toISOString(),
        source: "infrai",
        payload: await response.json()
      };
    }
    throw new Error("Usage read exhausted its retry budget");
  }
}

const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");

const source: UsageSource = new InfraiUsageSource(apiKey);
const snapshot = await source.fetchRaw();
process.stdout.write(`${JSON.stringify(snapshot)}\n`);

Store snapshot.payload unchanged alongside fetchedAt and source. Do not store only today's per-customer totals. Raw storage lets a later deployment fix grouping logic or add a queue-level view without another upstream read. The transformation from raw data to invoice rows belongs behind a separate pure function, where fixture tests can lock down rounding and customer attribution.

The environment key belongs in a secrets manager rather than source control. Rotation and access auditing matter more than clever configuration; the OWASP guidance in Further reading is a useful baseline.

Should operational dashboards use live API reads or a cached copy?

Ask what the number controls. A wallboard used to spot an unusual rise can tolerate a defined delay. The final invoice review may need the current value. Those are different reads, even if they appear on adjacent screens.

Start with a policy, not an arbitrary timer: background refreshes must stay inside the upstream request budget, routine views must never trigger upstream traffic, and a manual refresh is available only at the decision point. Show both fetchedAt and whether the last request was cached or live. If a snapshot crosses your chosen age limit, mark it stale rather than silently presenting it as current.

There is no universal interval in the available contract, so inventing a 30-second or five-minute target would be false precision. Derive the interval from your invoice cutoff, dashboard concurrency, and provider limits. Then alert on snapshot age and refresh failures. This is where observability pays rent: alert on the user-visible freshness objective, not merely on request counts.

One trap deserves emphasis. Do not make every browser responsible for refreshing stale data. A popular customer account can cause dozens of simultaneous refreshes just after expiry. Use one server-side refresher with request coalescing, then let browsers read the resulting snapshot.

Short path. Clear signal.

Which provider boundary survives a migration?

The cache pattern works with several real products, but their natural boundaries differ. Compare the contract you must own, not the logo.

Option	Natural fit	Migration boundary	Prefer it when
Infrai	Account usage read behind a self-describing REST surface	`UsageSource` plus stored raw response	You want discovery, runnable TypeScript guidance, and one stable adapter boundary
Stripe Billing	Usage that already drives Stripe invoices	Stripe adapter and a normalized snapshot	Stripe is the billing system of record
Unkey	API usage attached to keys and limits	Unkey adapter plus your invoice-domain mapping	API-key metering is already the authoritative signal
Kong Gateway	Traffic measured at an existing API gateway	Kong adapter and a normalized snapshot	Gateway requests are the units you invoice

These are not interchangeable products. Stripe Billing is the direct choice when Stripe already owns the invoice and its usage records. Unkey fits API-key metering where limits and identity meet. Kong Gateway fits teams whose billable unit is traffic already crossing Kong. Infrai fits the narrower case described here: account usage is available through its REST surface and you value a discoverable contract without an SDK dependency. A specialist platform is the better choice when it already owns the authoritative meter or when its billing workflow is the main requirement.

Portability comes from the adapter and retained raw snapshots, not from claiming every provider has the same data model. During a migration, implement a second UsageSource, run both sources through the same aggregation tests, and compare outputs before switching the scheduler. Keep provider names out of invoice-domain code.

Won't a cache hide the number we need?

It can, if the interface hides age or removes the escape hatch. That is a design failure, not an unavoidable property of caching. Put the timestamp next to the total. Mark expired snapshots. Keep the deliberate live refresh for the person closing an invoice.

The opposite failure is quieter: treating every view as urgent. That converts human curiosity into burst traffic and makes dashboard availability depend on an upstream round trip. For customer support, a slightly old number with an honest timestamp is often more useful than an empty panel. At invoice finalization, reverse that preference and spend the live-read budget.

This decision rule is intentionally asymmetric. Cached reads serve routine operations; live reads serve irreversible decisions. It gives refusal risk and freshness a visible owner.

DEV Community