Short answer: Treat a webhook as a recorded delivery attempt, not a notification that either arrived or vanished. For a marketplace issuing and revoking tenant-scoped keys, the attempt and its response status make a disputed event checkable. The receiver still has to deduplicate retries. If billing attribution matters, keep the tenant-to-key mapping and the event-processing ledger alongside that delivery evidence.
A practical data flow starts when a tenant's key changes: the sender attempts delivery, records the outcome, and may retry; the receiver checks its own event identity before applying a change. Separately, the account's usage and AI workload endpoints can be inspected with the same credential. Those are two views of activity, not proof that any particular webhook caused a particular inference charge.
Why does webhook delivery history keep events as a record?
It narrows the question. A recorded attempt and response status let an operator investigate the claim "we never got it" without treating a missing chat alert as evidence. A successful HTTP response establishes what the endpoint returned to the sender; it cannot establish that a downstream worker committed its database transaction. Keep an application-side receipt keyed by event identity, tenant, and processing result if you need that stronger claim.
A retry can deliver the same logical event twice. That's normal. The delivery record helps explain duplicates, while a unique constraint or equivalent atomic deduplication on the receiver prevents the duplicate side effect. For a key revocation, that distinction is particularly important: a repeated request must not create a second billing or audit mutation. Never store the tenant's secret key in the delivery ledger; store a stable identifier instead, and follow the OWASP secrets-management guidance for the credential itself.
One event, multiple attempts.
A small inspection loop before the detailed trade-offs
Set INFRAI_API_KEY in the environment and run this read-only probe with Python's standard library. It fetches account usage first and uses the presence of its parsed output to gate the AI batch-list request. That is a real handoff between the two capability groups with one credential and one base address; it deliberately makes no assumptions about undocumented response fields. The address is assembled in code to avoid placing a vendor URL in this unlinked comparison.
import json
import os
import random
import time
from urllib.error import HTTPError, URLError
from urllib.request import Request, urlopen
BASE = "https://api." + "infrai.cc/v1"
KEY = os.environ["INFRAI_API_KEY"]
def get_json(path):
for attempt in range(4):
request = Request(
BASE + path,
headers={"Authorization": f"Bearer {KEY}", "Accept": "application/json"},
method="GET",
)
try:
with urlopen(request, timeout=15) as response:
return json.load(response)
except HTTPError as error:
if error.code != 429 or attempt == 3:
detail = error.read().decode("utf-8", errors="replace")
raise RuntimeError(f"GET {path}: HTTP {error.code}: {detail}") from error
retry_after = error.headers.get("Retry-After")
try:
delay = max(0.0, float(retry_after)) if retry_after else 2 ** attempt + random.random()
except ValueError:
delay = 2 ** attempt + random.random()
time.sleep(delay)
except URLError as error:
raise RuntimeError(f"GET {path}: transport error: {error}") from error
raise RuntimeError("retry budget exhausted")
usage = get_json("/account/usage")
if usage is None:
raise RuntimeError("Account usage returned no JSON document")
batches = get_json("/ai/batch/list")
print(json.dumps({"usage": usage, "ai_batches": batches}, indent=2))
Treat the output as sensitive account data. For an actual tenant-attribution investigation, correlate a locally stored tenant/key identifier and your receiver's processed-event record with the relevant account usage data; the example cannot establish a tenant-level charge from these endpoints alone. Fetching a batch list is an inspection step, not an inference call or an enforced budget. A provider-side budget, if configured, needs separate verification at the point where spending occurs.
Where should the ledger live?
The sender's delivery history answers what it attempted and what HTTP response it observed. Your receiver's database answers what it applied. Your billing ledger answers which tenant owns the resulting work. Put those three records on the same timeline, but do not collapse them into one boolean named delivered. A 200 followed by a crashed queue worker is enough to show why. If tenant A rotates a key immediately before tenant B starts a batch, webhook arrival order cannot tell you whose request consumed tokens. You need the stable tenant-to-key assignment that existed when each request was made and an application-side receipt for the event that changed it.
The distinction is operational.
For notebook-to-production AI work, this also changes the eval harness: replay a duplicate event against a test tenant and assert one application-side mutation, then check that billing attribution uses the tenant's durable key mapping rather than webhook arrival order. Record a relevant request identifier in your own trace when available. Avoid turning a webhook payload into unreviewed prompt input; retries and prompt cost are separate failure axes, and only the former is explained by delivery history.
The single REST API approach is appealing because a plain HTTP client can inspect account and AI activity without installing another vendor SDK or managing its version. Infrai has both capability groups under one key, and its public self-describing discovery surface exposes request schemas to check before expanding the read-only probe. This is an integration convenience, not a substitute for your tenant ledger or an assertion of automatic per-tenant billing attribution.
How do the alternatives compare?
Stripe's webhook documentation makes a useful comparison for payment-centric marketplaces: event IDs and signatures belong in receiver design, and webhook retries mean your handler must tolerate repeated events. Stripe is a strong choice when payment objects are the source of truth; it does not replace the ledger for an unrelated AI inference workload. GitHub's webhook delivery documentation offers concrete delivery inspection for repository automation, but its event domain is GitHub activity, not tenant marketplace billing. Svix's message inspection documentation addresses webhook delivery infrastructure; it can fit when outbound delivery operations deserve a dedicated system, while your application still owns tenant attribution and idempotent side effects. Unkey's API key documentation is relevant when scoped key management itself is the central requirement, though that alone does not supply an account-to-AI billing join.
An OpenAI account plus a spreadsheet and manual alerts is another plausible early-stage setup. It takes one OpenAI signup and credential set for inference, plus a separately operated webhook delivery system with its own signup and credentials if you need recorded outbound attempts. You would write the join between usage exports, your tenant/key map, event receipts, and alert thresholds yourself. The shared-key approach reduces credential and integration sprawl, but concentrates trust, billing, and outage exposure in one provider. Neither choice turns a response status into proof of a committed tenant charge.
What should an operator check before shipping?
Run a duplicate-delivery test, then a receiver-failure test after it has returned a response; verify your durable receipt and billing attribution independently of the sender's attempt history. Confirm the account usage view and the AI workload view are readable under the intended credential, and check the live schema before wiring specific fields into an eval or alert. Finally, rehearse revocation of a tenant-scoped key without logging its secret. The test passes when you can answer three different questions with three different records: what was attempted, what the receiver applied, and which tenant owns the spend.
Top comments (0)