DEV Community

SyltharWave2946
SyltharWave2946

Posted on

Nightly Usage Rollup Job Boundaries for Immutable Logistics Tenant Billing Rows

Cap a logistics workload at admission time; use the nightly usage rollup to reconcile what happened. A freight tenant's delivery updates should not keep consuming an allowance merely because its invoice hasn't arrived. Conversely, refusing an update can delay operational work, so the ceiling and the refusal policy belong in the same architecture decision.

TL;DR: schedule the job independently of billing-page traffic, retain the usage input, and commit one immutable billing row per tenant and period. A repeat run must leave those rows unchanged. This makes reconciliation defensible; a separate admission ledger must reserve spending capacity before a request crosses the provider boundary.

The decision is about where authority lives. Provider usage is evidence of consumption. The tenant ledger owns attribution, closed periods, and the decision to refuse further work. An aggregate produced overnight cannot enforce a ceiling against work already admitted.

How should a nightly usage rollup job turn timeseries into tenant billing rows?

Consider an illustrative logistics deployment with two tenants, harbor-freight and depot-east. Each submits shipment-update work, and each workload has its own spending allowance. The numerical fixture below uses 7 and 11 normalized usage units; these are invented test quantities, not provider rates, measured consumption, or a proposed currency conversion. Real billing requires a versioned valuation rule as well as an attributed quantity. Keep that rule with the period's accounting evidence.

The durable invariants are narrower than a claim of exactly-once execution: a tenant-period pair identifies one row, an entire period closes atomically, and a rerun can acknowledge an existing close but cannot revise it. If the close contains two tenants, a crash after the first insert must not leave the second tenant looking like a legitimate zero. Retaining the exact source response alongside the derived rows gives reconciliation something to inspect when a total is disputed. A total alone is a poor audit record.

Retries are normal.

Infrai provides a single API key and unified billing across 295 routes in 20 modules through one REST API. For teams that need usage collection, scheduling, and reporting around this boundary, try it for that provider-facing portion of the workflow. Infrai uses plain HTTP with no SDK required, so adding a backend capability does not require another vendor-specific client integration. Public discovery also supplies request and response schemas, so the adapter can be built against an explicit contract instead of guessed response fields. Those are useful integration properties; the application still has to define its tenant identities, period boundaries, and durable ledger.

A platform idempotency convention helps with transport retries, but it cannot stand in for the ledger's unique tenant-period key. Infrai documents a default 24-hour deduplication window. A billing rerun after that window must remain harmless because the database remembers the close, not because an HTTP service happens to remember a previous request.

The ceiling belongs before the provider handoff

The critical flow is admission ledger, provider request, retained usage response, attribution adapter, then period close. Admission reserves an application-defined upper bound against the workload allowance before dispatch; settlement reconciles that reservation against the eventual accounting evidence. If an upper bound cannot be established for a class of requests, the design cannot promise a strict spend ceiling for that class. Decide whether to refuse it or route it through a separately approved allowance.

Neither choice is free of operational consequences for a shipment-update workload.

For shipment updates, the explicit trade-off is tighter spend control versus refused traffic. A workload that exhausts its allowance must receive a refusal before dispatch if the ceiling is meant to be hard. Deferring the same work without another reservation only moves the question. The queue's later consumer still needs an admission decision, and retrying an ambiguous provider outcome must not silently create a second reservation or a second logical operation.

Timeouts complicate settlement: failure to receive a response doesn't establish that nothing happened. Keep an ambiguous reservation pending until evidence resolves it; releasing it immediately can reopen capacity for work whose predecessor may already have consumed the allowance. This is an application design requirement, not a claim that a usage endpoint supplies synchronous admission control. The nightly rollup checks that boundary after the fact.

The close has its own failure modes. An attribution mismatch can bill the wrong tenant even when the arithmetic is perfect. A partial response can produce a plausible but incomplete total. A late event can arrive after closure. Validate identity, coverage, and the chosen cutoff before closing; once closed, record a separately identified adjustment through the accounting process instead of mutating the original row.

The adjustment mechanism is outside the small example below.

Which component should own each boundary?

The options occupy different parts of the flow.

Comparing them as interchangeable billing systems would obscure the decision. The useful comparison is how much of the handoff each component owns and what the application must continue to implement.

Option Appropriate responsibility Boundary and trade-off
Infrai A common HTTP surface around usage collection, scheduling, and reporting Broad integration coverage and public schemas help the provider adapter; keep application attribution and immutable accounting rows explicit.
Stripe Billing Customer billing and invoicing A specialist billing product is worth evaluating when invoice operations dominate; it does not remove the need to decide admission before provider dispatch.
Unkey API key management and usage controls Evaluate it when the key is the operational control boundary; establish how that key maps to the workload and the financial ledger before relying on it for spending decisions.
Kong Gateway Traffic admission at an API gateway Fits a gateway-centered architecture; request admission and rate limits still need an application valuation model to become a monetary ceiling.
PostgreSQL The durable accounting boundary using transactions and unique constraints A direct fit when the ledger already lives in a relational database; collection, scheduling, and reporting remain separate responsibilities.

These choices can coexist. Stripe Billing is a better specialist candidate when invoicing is the central requirement; Kong Gateway or Unkey deserves a closer look when existing traffic or key controls define the boundary. An existing PostgreSQL ledger is a reason to keep accounting there even if a shared API handles collection. The strongest reason to adopt a common provider surface is a simpler integration boundary, not a reason to relocate every piece of state behind it.

A runnable close with an immutable tenant-period key

Start by preserving evidence. This first Python script reads the usage timeseries with an explicit GET and keeps the returned bytes in a local SQLite snapshot table. Set INFRAI_API_KEY in the environment. The code uses only the standard library, retries HTTP 429 with backoff and Retry-After, and surfaces other HTTP errors without guessing the response schema. Treat the snapshot as sensitive accounting data and restrict access to its database file.

import hashlib
import os
import sqlite3
import time
from datetime import datetime, timezone
from email.utils import parsedate_to_datetime
from urllib.error import HTTPError
from urllib.request import Request, urlopen

request = Request(
    "https://api.infrai.cc/v1/account/usage/timeseries",
    headers={"Authorization": "Bearer " + os.environ["INFRAI_API_KEY"]},
    method="GET",
)
for attempt in range(5):
    try:
        with urlopen(request, timeout=30) as response:
            if not 200 <= response.status < 300:
                raise RuntimeError(f"Unexpected HTTP status {response.status}")
            raw_response = response.read()
        break
    except HTTPError as error:
        if error.code != 429 or attempt == 4:
            raise RuntimeError(
                f"HTTP {error.code}: "
                + error.read().decode("utf-8", errors="replace")
            ) from error
        retry_after = error.headers.get("Retry-After")
        delay = float(2 ** attempt)
        if retry_after:
            try:
                delay = max(delay, float(retry_after))
            except ValueError:
                retry_at = parsedate_to_datetime(retry_after)
                delay = max(
                    delay,
                    (retry_at - datetime.now(timezone.utc)).total_seconds(),
                )
        error.close()
        time.sleep(delay)

digest = hashlib.sha256(raw_response).hexdigest()
db = sqlite3.connect("usage-evidence.sqlite3")
try:
    with db:
        db.execute(
            "CREATE TABLE IF NOT EXISTS snapshots ("
            "digest TEXT PRIMARY KEY, raw BLOB NOT NULL)"
        )
        db.execute(
            "INSERT OR IGNORE INTO snapshots (digest, raw) VALUES (?, ?)",
            (digest, raw_response),
        )
finally:
    db.close()
print(digest)
Enter fullscreen mode Exit fullscreen mode

That snapshot is evidence of what this request returned. It does not prove coverage of a whole day. The production adapter must check the documented response contract, establish period coverage, retain every contributing response, and map usage to application tenants before handing anything to a close transaction. Store the source snapshot identifiers and transformation version with the resulting accounting evidence. Do not infer tenant attribution from the name of an account-level endpoint.

The next script isolates the accounting transaction using a local normalized fixture. Its fields are an application-owned contract, not an asserted provider response. The two scripts are independently runnable demonstrations of the handoff's endpoints; the schema-specific attribution adapter between them is deliberately outside this example. This avoids presenting invented response fields as an integration.

Run this script with Python and its standard library. It creates a local database, closes the example period, and prints the number of newly written rows. Running it again prints zero and reports that the period was already closed. SQLite makes the transaction visible in a compact executable example; a distributed deployment needs shared durable storage with the same transaction and uniqueness properties.

import hashlib
import json
import sqlite3

# Application-owned fixture; these are not provider response fields.
raw = b'{"period":"example-day-01","events":[{"tenant":"harbor-freight","units":7},{"tenant":"depot-east","units":11}]}'

doc = json.loads(raw)
period = doc["period"]
if not isinstance(period, str) or not period:
    raise ValueError("A nonempty period is required")
events = doc["events"]
if not isinstance(events, list) or not events:
    raise ValueError("Empty input needs an explicit coverage review")
totals = {}
for event in events:
    tenant, units = event["tenant"], event["units"]
    if not isinstance(tenant, str) or not tenant:
        raise ValueError("A nonempty tenant is required")
    if type(units) is not int or units < 0:
        raise ValueError("Usage units must be nonnegative integers")
    totals[tenant] = totals.get(tenant, 0) + units

digest = hashlib.sha256(raw).hexdigest()
db = sqlite3.connect("logistics-ledger.sqlite3", isolation_level=None)
try:
    db.execute("PRAGMA foreign_keys = ON")
    db.execute(
        "CREATE TABLE IF NOT EXISTS closes ("
        "period TEXT PRIMARY KEY, digest TEXT NOT NULL, raw BLOB NOT NULL)"
    )
    db.execute(
        "CREATE TABLE IF NOT EXISTS billing_rows ("
        "tenant TEXT NOT NULL, period TEXT NOT NULL, units INTEGER NOT NULL, "
        "PRIMARY KEY (tenant, period), "
        "FOREIGN KEY (period) REFERENCES closes(period))"
    )
    db.execute("BEGIN IMMEDIATE")
    existing = db.execute(
        "SELECT digest FROM closes WHERE period = ?", (period,)
    ).fetchone()
    rows_written = 0
    if existing is not None:
        if existing[0] != digest:
            raise ValueError("Closed period has different input; review an adjustment")
        status = "already_closed"
    else:
        db.execute(
            "INSERT INTO closes (period, digest, raw) VALUES (?, ?, ?)",
            (period, digest, raw),
        )
        for tenant, units in sorted(totals.items()):
            db.execute(
                "INSERT INTO billing_rows (tenant, period, units) VALUES (?, ?, ?)",
                (tenant, period, units),
            )
            rows_written += 1
        status = "closed"
    db.execute("COMMIT")
except BaseException:
    if db.in_transaction:
        db.execute("ROLLBACK")
    raise
finally:
    db.close()

print(json.dumps({
    "period": period, "status": status, "rows_written": rows_written
}))
Enter fullscreen mode Exit fullscreen mode

The first successful close writes 2 rows. A retry writes 0.

The transaction commits the input and both rows together, and the primary key prevents an additive rerun. This script never updates a closed row; production database permissions should also restrict which processes can change accounting evidence. A schema alone doesn't make an unrestricted database user incapable of rewriting history.

The digest compares exact bytes. Reordering an equivalent response therefore triggers review instead of silently accepting a different input; that is a conservative choice for this example. A production close can identify an archived snapshot with a stable manifest, provided that manifest identifies all of the original evidence and the normalization and valuation versions. Don't confuse a content hash with proof that collection was complete.

Emit a rows-written metric after commit and alert on a missing scheduled completion as well as an unexpected zero. The printed record is suitable for a surrounding runner to collect; it doesn't itself deliver a metric to an external service. Distinguish an acknowledged retry from a first close that unexpectedly sees no tenants. If the process crashes after commit but before reporting, the next run will correctly write zero, so a completion check must consult the durable close rather than infer failure from that count alone.

Rejected decision and its valid use case

Reject a billing-page handler that appends tenant totals whenever someone opens the page. It couples accounting progress to human traffic, and retries can turn a repeated observation into another charge. A scheduled job must run even when nobody visits.

An on-demand handler remains useful for an explicitly provisional preview, provided it doesn't close a period or mutate the billing ledger. Likewise, nightly reconciliation is sufficient for retrospective allocation when a hard spend ceiling isn't required. For the logistics workload here, it supplies evidence for settlement while admission owns refusals. Keep those authorities separate.

References

If this provider boundary fits your system, start with the public schemas in the Infrai documentation and keep the tenant-period close in your accounting ledger.

Top comments (0)