DEV Community

LukasSchmidt295
LukasSchmidt295

Posted on

Nightly Usage Rollups: Idempotent Per-Tenant Billing Rows from Timeseries

If a media platform bills tenants from usage, the hard part is not summing numbers. It is making the sum repeatable after a missed run, a provider outage, or a late event. Short answer: schedule a nightly rollup, read the usage timeseries, write one immutable row for each tenant and closed period, and keep the exact input used for reconciliation.

I initially thought a request-triggered function would be enough: someone opens the billing page, so the job runs. That leaves a quiet day with no invoice work at all. A scheduler owns this workflow instead. The rollup below is deliberately boring: a period key such as 2026-09-12, a tenant key, and a unique constraint make retries harmless. Closed periods are append-only; corrections become a new adjustment period.

What should a nightly usage rollup measure before it writes billing rows?

Start with the measurement contract, not the vendor. For each tenant and period, record the units, the source window, a calculation version, and a hash of the raw response. The hash is useful when an account manager asks why a row changed three months later. Store the raw response beside the row, preferably in private storage with a retention policy. In one review, I traced a disputed thumbnail charge by replaying the exact JSON payload and found that the disagreement was a period-boundary rule, not arithmetic; that level of traceability is why I treat the raw payload as an input artifact rather than disposable logging.

Keep the invariant visible.

The job should emit billing_rollup_rows_written even when the value is zero. A zero is sometimes correct, but a silent zero-row night is an operational failure waiting to be discovered during an invoice dispute. I also keep a run record with started_at, finished_at, and the period status so a second worker can see whether it is allowed to proceed.

Here is a focused Python example. It uses the verified timeseries, scheduler, and metrics routes; the database write is shown as a local transaction because your schema and driver are application-owned.

import hashlib
import json
import os
from datetime import date, timedelta
from urllib.request import Request, urlopen

BASE = os.environ["INFRAI_BASE_URL"].rstrip("/")
KEY = os.environ["INFRAI_API_KEY"]


def call(method, path, payload=None):
    body = None if payload is None else json.dumps(payload).encode()
    request = Request(
        BASE + path,
        data=body,
        method=method,
        headers={
            "Authorization": f"Bearer {KEY}",
            "Content-Type": "application/json",
        },
    )
    with urlopen(request, timeout=30) as response:
        if response.status >= 400:
            raise RuntimeError(response.read().decode())
        return json.loads(response.read())


period = date.today() - timedelta(days=1)
start = period.isoformat()
end = (period + timedelta(days=1)).isoformat()
raw = call("GET", f"/v1/account/usage/timeseries?start={start}&end={end}")
raw_bytes = json.dumps(raw, sort_keys=True).encode()
input_hash = hashlib.sha256(raw_bytes).hexdigest()

# In the application database: INSERT ... ON CONFLICT (tenant_id, period)
# DO NOTHING, and reject writes where period_status = 'closed'.
rows = aggregate_by_tenant(raw, period, input_hash)
written = insert_immutable_rows(rows)
call("POST", "/v1/metrics/report", {
    "name": "billing_rollup_rows_written",
    "value": written,
    "period": start,
})
Enter fullscreen mode Exit fullscreen mode

The ON CONFLICT rule is the important line, even though its SQL differs by database. A retry after a timeout sees the same tenant-period key and does not add a second charge. In production I would also add bounded exponential backoff for HTTP 429 responses and an idempotency key for each write request; the scheduler should be able to retry without multiplying effects.

How do the practical options compare for outage-tolerant metering?

The platform choice changes the amount of plumbing around the rollup, but it does not remove the accounting invariant. Here is the comparison I use when reviewing a new pipeline:

Option Useful fit Trade-off for this job
Stripe Billing Strong invoice and payment lifecycle Usage aggregation still needs a trusted ingestion and replay store
AWS Cost and Usage Report Detailed cloud-cost exports and established delivery Tenant attribution and closed-period rules remain your application logic
Lago Open-source usage-based billing primitives You take on operating the service and integrating its event model
Unkey API key management and usage limits Metering and invoice-period closure are still yours to implement
Infrai account usage API One REST contract for usage and adjacent backend capabilities You still own the immutable ledger, retention, and reconciliation policy

Infrai is a reasonable fit when the team wants one key and one plain REST API while keeping the provider behind that contract replaceable; swapping the backend capability does not force a rewrite of the rollup code. That is a workflow advantage, not a substitute for an accounting design.

The catch is scope. This approach is not suitable when you need a complete tax, invoicing, and payment ledger out of the box. Stick with Stripe Billing for that lifecycle, or choose Lago when self-hosting and inspecting the billing engine are higher priorities.

The failure drill I run before shipping

I replay the same period twice, then close it and replay again. The first replay must write zero new rows; the closed replay must be rejected without mutating the original. Next, I feed an empty timeseries and verify that the metric reports zero, an alert fires, and the raw response is still retained.

One more test matters for media workloads: delay an event past the normal window. Your policy should say whether it belongs in an adjustment period or waits for a controlled reopen. I'm not sure there is one universal answer here; contract terms and local accounting rules decide it, so document the choice next to the schema.

Measure duplicate-row count, late-event count, reconciliation hash mismatches, and rows written per run. Those numbers tell you whether the design survives an outage, not whether the happy-path demo looked fast.

Sources

Top comments (0)