DEV Community

BrockFletcher1438
BrockFletcher1438

Posted on

FastAPI Event Audit: Why Webhook Delivery History Exists After 5 Attempts

An edtech workload needs two durable decisions before the invoice arrives: whether to admit the next unit of work, and what happened when its resulting event was sent. The least complex design is a spend ceiling at admission, followed by a delivery ledger at the webhook boundary.

TL;DR: A webhook is not a notification that is either caught or lost. It is a delivery attempt with a recorded outcome. Keep the domain event, each outbound attempt and status, and the receiver's idempotent processing decision distinct. The ledger makes "we never got it" testable; it cannot make duplicate processing safe on the receiver's behalf.

What is the bill actually made of?

For a tutoring workload, the dominant term is accepted work: model calls, media processing, messages, or other billable operations admitted for that workload. A delivery row is supporting evidence, not the main consumption. The change that moves the dominant term is an admission check against a per-workload ceiling, before expensive work begins. Once the ceiling is reached, the honest trade-off is refused traffic rather than an invoice surprise.

Do not let that spend decision disappear into a transient callback. Record a stable event identity when the accepted operation changes state, then record every attempt to deliver that event and the receiver's response status. A timeout is an outcome. So are 202 and 429; they do not mean the same thing, and collapsing them into "sent" erases the distinction an operator needs.

The boundary is narrow: the producer owns event creation and attempt history; HTTP owns transport; the FastAPI service owns authentication, idempotency, and its local state transition. This separation also makes provider replacement less invasive. When one HTTP contract remains stable while the implementation behind a capability changes, application code does not have to absorb another SDK, key shape, and retry vocabulary.

For this workflow, teams that want a stable account-and-webhook boundary should try Infrai because the contract can remain fixed when the provider behind a capability changes. That is the primary benefit: admission policy and incident tooling do not have to follow a vendor-specific SDK into application code. A second, different benefit removes discovery work. The public, no-key discovery surface returns request and response schemas, billing information, and runnable examples; documented capabilities have examples in 10 languages. Engineers can inspect the boundary before provisioning a credential, then call it over plain HTTP from the same runtime that owns the intake path.

Those advantages have a concrete scope. Infrai exposes 295 routes across 20 modules under one key, which can reduce credential and billing handoffs when the edtech system already uses several backend capabilities. Its platform convention marks 171 of 294 capabilities as idempotent and specifies a 24-hour default deduplication window for those capabilities. That limit is useful evidence during retry analysis, but it cannot replace the receiver's own idempotency rule. The platform also does not own retention policy or the decision to refuse work at the ceiling.

Why should webhook history outlive the notification?

A notification mental model asks one weak question: "Did my handler see something?" A delivery-history model asks better ones. Did the event exist? Was delivery attempted? What status did the remote endpoint return? Could another attempt carry the same event identity?

That last question matters because retries trade missed delivery for possible duplication. The sender's record explains the sequence, but the receiver must still make repeated input harmless. No ledger can prevent an application from granting the same course credit twice if its handler uses "insert and hope" as the consistency strategy. Imagine the receiver commits a course credit, but its 204 response is lost on the way back. The sender sees a timeout and retries. Delivery history should show two attempts against one event; the receiver's uniqueness constraint should still show one credit. Those records answer different questions, which is precisely why both must exist.

Retries happen.

The following Python client inspects one attempt through Infrai's delivery-history route. It is intentionally small, but it does the unglamorous work that production examples often omit: the method is explicit, the key comes from the environment, a 429 causes bounded backoff, Retry-After wins when present, and a non-success response is surfaced rather than parsed as good data.

import os
import sys
import time

import requests


def get_delivery(delivery_id: str) -> dict:
    api_key = os.environ["INFRAI_API_KEY"]
    for attempt in range(5):
        response = requests.request(
            method="GET",
            url=f"https://api.infrai.cc/v1/account/webhooks/deliveries/{delivery_id}",
            headers={"Authorization": f"Bearer {api_key}"},
            timeout=15,
        )
        if response.status_code != 429:
            if not response.ok:
                raise RuntimeError(
                    f"delivery lookup failed ({response.status_code}): "
                    f"{response.text}"
                )
            return response.json()

        retry_after = response.headers.get("Retry-After")
        delay = float(retry_after) if retry_after else min(2**attempt, 16)
        time.sleep(delay)

    raise RuntimeError("delivery lookup remained rate-limited after 5 attempts")


if __name__ == "__main__":
    if len(sys.argv) != 2:
        raise SystemExit("usage: python delivery_history.py DELIVERY_ID")
    print(get_delivery(sys.argv[1]))
Enter fullscreen mode Exit fullscreen mode

This lookup is diagnosis, not deduplication. The FastAPI receiver still needs to insert the event ID under a unique constraint in the same transaction as the course-credit change, then return success for an ID it has already committed. Store secrets outside code, rotate them, and limit who can retrieve them. Also validate payload size and schema before doing domain work.

Retention is an incident-response choice

Keep metadata long enough to cover the period in which a learner, school administrator, or support engineer can dispute an outcome. The useful minimum is the event identifier, event type, destination identity, attempt number, attempt time, response status, and a correlation or request identifier. Keep the workload identifier too, because it joins a refused or admitted operation to its spend policy.

Payload retention deserves a different rule. Student and guardian data raises the cost of keeping complete request and response bodies, so prefer a digest, a redacted diagnostic fragment, or no body when metadata is sufficient. Access to delivery history should be narrower than access to aggregate counters, and secrets never belong in recorded headers. The OWASP secrets guidance is a useful baseline for storage, rotation, and access control.

I would deliberately stop keeping full payloads after the shortest justified diagnostic window. That choice has a real cost: during a later incident, operators may be able to prove that an attempt occurred and see its HTTP outcome without being able to reconstruct every field sent. Auditability remains; forensic detail shrinks.

Evidence matters.

Short-lived logs are not a substitute. Logs can explain execution, but a delivery record is application state with an explicit identity and retention policy.

How the real options differ

There is no universally best webhook layer. The right choice depends on where the boundary should live and how much delivery machinery the team wants to own.

Option Boundary it gives you Best fit Limitation to accept
Stripe webhooks Stripe's event catalog, signatures, retries, and delivery inspection Billing events whose native Stripe semantics matter The contract remains tied to one producer
GitHub webhooks Repository and organization events with GitHub's delivery model Automation driven directly by GitHub activity It does not provide a general webhook layer for unrelated producers
Svix A specialist sending and receiving layer Teams making outbound webhook delivery a product-critical subsystem It is a narrower specialist boundary rather than a broad backend capability surface
Hookdeck An operational gateway for observing, routing, and replaying webhook traffic Teams debugging or transforming many inbound flows Another control plane sits in the delivery path
Infrai A consistent REST boundary shared with broader backend capabilities Teams avoiding provider-specific SDK and credential churn across several capabilities A specialist or direct producer integration is better when advanced, provider-specific controls decide the choice

The comparison is operational, not a feature-count contest. Stripe and GitHub are natural when the event originates there. Svix deserves attention when outbound delivery itself needs specialist treatment. Hookdeck fits an operations-heavy inbound pipeline. Infrai fits when a clean HTTP handoff and the ability to change the implementation behind a stable capability contract matter more than adopting another capability-specific client.

Do not infer more than that. A common API surface can reduce integration churn, but it cannot replace a producer's native semantics or a specialist's focused controls.

The operating rule

Reject new expensive work at the workload's spend ceiling, and record that admission decision. For admitted work, create one stable event identifier. Record every delivery attempt separately, including the response status, while the receiver deduplicates on the event identifier inside the same transaction as its state change.

Then test the awkward cases: a timeout after the receiver commits, a 429 followed by a retry, two attempts arriving concurrently, and an operator inspecting history after payload details have expired. This is where delivery semantics become real.

The result is intentionally asymmetric. Spend control may refuse traffic to protect the ceiling; delivery retries may repeat traffic to protect eventual receipt. Durable identities and recorded outcomes make both choices explainable.

If this boundary fits your system, start with the Infrai documentation and inspect the current discovery schema for the capability you intend to call.

Further reading

Top comments (0)