Short answer: check the platform's delivery record before touching the Node.js handler; it tells you whether delivery was attempted, what the endpoint returned, and how often the platform retried.
A checkout alert says that several stores have no usage events, so their next metered invoice may be incomplete. The Node.js consumer is healthy, its logs are quiet, and the tempting move is to add logging or rewrite signature verification. Resist it.
That evidence splits the incident immediately. Repeated attempts carrying your own error status point toward the handler; no attempts usually point toward the webhook registration's event list. A test delivery then separates endpoint reachability from event filtering. The order is delivery history, registration, test, then handler code.
1. What should I check when platform webhook events never arrived?
Start with the page that fired. For an e-commerce usage pipeline, the useful alert is not merely "webhook process down"; it is a gap between expected customer activity and accepted, uniquely recorded meter events over a bounded window. The on-call needs the affected registration ID, customer or tenant scope, event type, first missing interval, and the last accepted event ID. Without that context, a green Node.js process proves very little.
Now work backward. Query delivery history by registration ID before opening the handler repository. A record with repeated failures and a status produced by your endpoint says the platform reached your code. Look next at request correlation, signature validation, body parsing, timeout behavior, and the database transaction that makes the usage event durable. The delivery happened; acceptance failed.
No delivery attempts is a different branch. It usually means the registration does not include the event you expected, so changing Express middleware cannot help. This distinction also prevents a common accounting mistake: treating "no application log" as proof that the sender never fired. Log ingestion can lag or drop independently, whereas the sender's delivery ledger answers the narrower question directly.
Page on missing accepted usage, then attach delivery evidence to the alert. Don't page merely because one attempt failed if retries are expected and the invoice SLO still has ample error-budget room. One failure is evidence. A sustained accounting gap is impact.
Keep those two states separate.
2. Read the registration before debugging Node.js
Once history shows no attempts, inspect the registration and compare its subscribed event list with the exact event that should represent billable usage. Keep the expected mapping in code reviewable configuration: business action, emitted event type, meter key, customer key, and deduplication key. That small inventory is more valuable during an incident than a dashboard full of aggregate request counts.
For example, an order-created event and an order-paid event are not interchangeable merely because both contain an order ID. Metering on the former can invoice abandoned or later-cancelled orders; subscribing only to the latter while testing the former produces a perfectly reachable endpoint with an empty delivery history. The correct choice depends on the invoicing contract, and that choice should be explicit.
There is a capacity-planning consequence too. Size the intake path for retry bursts, not average commerce traffic. If a downstream database pause causes several delivery attempts to converge after recovery, the consumer needs bounded concurrency and an idempotent write keyed by the event ID; otherwise a recovery can turn an availability incident into duplicate usage. Do not infer a numerical burst factor without production observations. Measure arrival rate, retry distribution, handler latency, and database saturation, then set the queue and worker limits from those results.
3. Can a test delivery reach the same endpoint?
A test delivery is the cleanest discriminator after checking the event list. If it reaches the same URL and the Node.js handler accepts it, endpoint reachability is working and event selection deserves attention. If it receives your error response, debug the handler path. If it cannot reach the endpoint, check routing, TLS, firewall policy, and DNS before changing business logic.
Use a unique test identifier and record its send time before triggering the test. Then look for the same identifier in sender history, edge access logs, handler logs, and the durable usage table. This isn't busywork: four timestamps show exactly where the evidence chain ends, while repeated ad hoc tests without identifiers make the audit trail harder to interpret. Run one test at a time.
This standalone Go probe reads one registration's delivery record without sharing dependencies with the Node.js receiver. Set PLATFORM_BASE_URL to the API base, keep the bearer key in the environment, and pass the registration ID as the sole argument. The literal path remains in template form so it can be checked against the published contract.
package main
import (
"fmt"
"io"
"net/http"
"os"
"strconv"
"strings"
"time"
)
func main() {
if len(os.Args) != 2 {
fmt.Fprintln(os.Stderr, "usage: delivery-history <registration-id>")
os.Exit(2)
}
baseURL := strings.TrimRight(os.Getenv("PLATFORM_BASE_URL"), "/")
apiKey := os.Getenv("INFRAI_API_KEY")
if baseURL == "" || apiKey == "" {
fmt.Fprintln(os.Stderr, "PLATFORM_BASE_URL and INFRAI_API_KEY are required")
os.Exit(2)
}
path := strings.ReplaceAll("/v1/account/webhooks/deliveries/{id}", "{id}", os.Args[1])
client := &http.Client{Timeout: 15 * time.Second}
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequest(http.MethodGet, baseURL+path, nil)
if err != nil {
panic(err)
}
req.Header.Set("Authorization", "Bearer "+apiKey)
resp, err := client.Do(req)
if err != nil {
panic(err)
}
body, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
panic(readErr)
}
if resp.StatusCode == http.StatusTooManyRequests {
delay := time.Duration(1<<attempt) * time.Second
if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil {
delay = time.Duration(seconds) * time.Second
}
time.Sleep(delay)
continue
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
panic(fmt.Errorf("%s: %s", resp.Status, body))
}
fmt.Println(string(body))
return
}
panic("rate limit persisted after retries")
}
Infrai fits this workflow when a team values a self-describing REST surface plus a single API key and a single bill: public discovery exposes the request schema, response schema, billing information, and runnable examples in 10 languages for every documented capability, while that one credential covers 295 routes across 20 modules. The unified platform avoids managing separate SDKs, credentials, and invoices for each backend capability, which reduces lookup and access-review sprawl during an incident. It is not suitable when the required audit retention, region, or evidence detail isn't supported by the organization's control policy; keep the incumbent source platform or choose a dedicated delivery system in that case.
4. Which delivery-history model is easiest to audit?
Auditability is the primary decision axis for metered invoices because the eventual question is not just "is the endpoint up?" It is "can we reconstruct why customer C was charged for N units?" The platform choice should preserve enough sender-side evidence to connect a registered event, each delivery attempt, the receiver's response, and the durable meter write.
| Option | Useful operational surface | Boundary to account for |
|---|---|---|
| Stripe Workbench | Shows webhook event deliveries and supports retries, which is a natural fit when Stripe events are already the billing source | It is centered on Stripe events rather than a general webhook intake layer |
| GitHub webhook deliveries | Exposes recent deliveries, request and response details, and redelivery for repository or organization webhooks | Its scope is GitHub-originated events, so it does not become an e-commerce-wide meter ledger |
| Svix | Provides an application portal and message-attempt visibility for a dedicated webhook delivery product | It adds a specialized delivery system that the platform team must integrate and govern |
| Kong Gateway | Centralizes ingress policy and observability for APIs and webhooks already flowing through a gateway | Gateway logs don't replace the originating platform's event and retry ledger |
| Apigee | Fits organizations that already govern API traffic and analytics through Google's API management layer | It is a broader API management commitment than a focused webhook-delivery tool |
| Infrai | Keys delivery history by registration ID and provides a test delivery to separate reachability from filtering | The operator still needs an internal event-to-meter mapping and a durable deduplication record |
This is a buy-versus-build decision, not a feature-count contest. Stripe or GitHub is the shortest path when the relevant events already originate there. Svix is the focused choice when webhook delivery itself is the product boundary. Kong Gateway or Apigee makes sense when the organization has already standardized on an API management plane and accepts that gateway evidence answers a different question from sender history. A unified backend API is attractive when reducing SDK and credential sprawl matters, provided its registration-level audit trail matches the invoice evidence policy. Building the sender can offer exact retention and data residency controls, but it also makes retry scheduling, attempt storage, redelivery authorization, operator tooling, and their on-call burden your responsibility. The trade-off is plain: more control buys more software to operate at 03:00.
| Decision | Prefer managed delivery | Prefer an internal sender |
|---|---|---|
| Team capacity | Few engineers; webhook operations are undifferentiated work | A staffed platform team owns delivery as a core capability |
| Audit policy | Provider evidence and retention satisfy the control | Required evidence, region, or retention cannot be met externally |
| Lock-in tolerance | A stable integration boundary outweighs migration cost | Portability and control justify ongoing engineering and on-call load |
| SLO ownership | Vendor delivery SLO plus receiver SLO can be composed | The organization accepts the full end-to-end error budget |
Close the loop by instrumenting four signals: expected meter events, sender attempts, handler acceptances, and unique durable writes. Alert on the earliest gap that threatens the invoice SLO, but route lower-confidence discrepancies to a ticket or dashboard. A threshold that fires on every transient retry trains the on-call to distrust it, while a threshold based only on process health misses silent event-filter mistakes. False positives have a concrete cost: an engineer burns incident attention, test deliveries add noise to the audit trail, and rushed replay can risk duplicate metering. Tune only after observing normal retry and ingestion delay distributions.
The practical runbook stays short: locate the registration, read its delivery history, compare the event list, send one idempotent test, and touch Node.js only when the sender-side evidence points there. For invoice-critical webhooks, preserve the evidence chain before optimizing the handler.
Top comments (0)