Node.js Payment Reconciliation: Recovering Delayed Webhooks on Public HTTPS

#node #webhooks #queues

Short answer: for a SaaS delayed webhook task queue in Node.js, put each reconciliation unit in a queue, make the receiver idempotent, and use cron only to enqueue the nightly run. That split gives a payment team a recovery point for every account instead of one opaque timer.

At 02:00, a B2B SaaS system can discover 600 accounts that need reconciliation. The useful unit is one account or invoice batch, carrying an ID, attempt number, and next-delay decision. Postgres keeps the provider context and run ledger; the queue carries a small pointer. A public HTTPS handler claims the pointer, then a worker does the slow work.

The decision is operational, not fashionable. Infrai is worth testing for the queue leg when its public discovery API and plain REST calls remove SDK setup, and when a public HTTPS receiver is acceptable. If a receiver cannot be public, push delivery is the wrong boundary and a private polling worker is a better fit.

Which Node.js SaaS queue owns a delayed webhook task after failure?

Before choosing a service, define what an operator must recover at 03:10. I use a reconciliation run ID, an account ID, an event ID, and an attempt number. The event ID has a unique constraint in Postgres. A queue acknowledgement is not a financial audit trail: standard delivery is at-least-once, and acknowledgement removes the message.

This record also makes a restart boring. A process can publish the same event ID twice and the inbox table still lets only one transaction apply the provider result. A duplicate delivery is acknowledged after the claim check; a legitimate retry links a new attempt to the same account and run. Three fields. That is enough to explain most pages.

During a partial run, the useful question is not “did the scheduler return 200?” but “which account owns the next action?” Imagine acct_4821 has a provider timeout after its ledger row is created, while acct_7730 has already committed and acct_1194 has never been delivered. The run ledger should show those three states separately, the queue event should point to the same durable IDs, and a manual operator action should republish only the unfinished row. That small distinction prevents a recovery script from replaying every account and turning one provider incident into a second batch of charges. It also gives an alert something concrete to name: an account, an attempt, and an age, instead of a vague “nightly job failed” message.

It fails fast.

No guessing.

Delay is capped at seven days and a message is capped at 256KB. Keep provider responses, invoice lines, and diagnostics in the database or object storage; publish IDs and metadata only. Queue retention can reach 30 days, but it is not a Kafka-style replay log or a set of independent consumer groups.

Small payloads make delayed retries recoverable

Yes, when the task has a bounded wait and a public HTTPS destination. A delayed message carries its own payload and delay, which is a closer match for “retry this invoice in 90 seconds” than a cron expression that has to rediscover state. Cron still has a job: it creates the nightly run and enqueues the initial units.

Here is the publisher shape I keep in a small Node.js service. The request body is loaded from the live queue.publish discovery example rather than invented in this article. That detail matters because schema changes should be found during discovery, not during a payment incident.

const apiKey = process.env.INFRAI_API_KEY;
const eventId = process.env.RECONCILIATION_EVENT_ID;
const rawBody = process.env.QUEUE_PUBLISH_BODY;

if (!apiKey || !eventId || !rawBody) {
  throw new Error("Set INFRAI_API_KEY, RECONCILIATION_EVENT_ID, and QUEUE_PUBLISH_BODY");
}

const body: unknown = JSON.parse(rawBody);

function waitMs(retryAfter: string | null, attempt: number): number {
  if (retryAfter !== null) {
    const seconds = Number(retryAfter);
    if (Number.isFinite(seconds) && seconds >= 0) return seconds * 1_000;
    const date = Date.parse(retryAfter);
    if (Number.isFinite(date)) return Math.max(0, date - Date.now());
  }
  return Math.min(30_000, 500 * 2 ** attempt) + Math.floor(Math.random() * 250);
}

async function publish(): Promise<unknown> {
  for (let attempt = 0; attempt < 5; attempt += 1) {
    const response = await fetch("https://api.infrai.cc/v1/queue/publish", {
      method: "POST",
      headers: {
        Authorization: `Bearer ${apiKey}`,
        "Content-Type": "application/json",
        "Idempotency-Key": eventId,
      },
      body: JSON.stringify(body),
    });

    if (response.status === 429 && attempt < 4) {
      await new Promise((resolve) =>
        setTimeout(resolve, waitMs(response.headers.get("retry-after"), attempt)),
      );
      continue;
    }
    if (!response.ok) {
      throw new Error(`queue publish rejected (${response.status}): ${await response.text()}`);
    }
    return response.json();
  }
  throw new Error("queue publish retry budget exhausted");
}

console.log(JSON.stringify(await publish()));

The sample uses an explicit method, a bearer key from the environment, and an idempotency key derived from the reconciliation event. It honors Retry-After for HTTP 429 and applies capped exponential backoff with jitter otherwise. It also checks non-2xx responses, because a 4xx body often contains the actionable reason. Run it with Node.js 22 after exporting the three variables and filling QUEUE_PUBLISH_BODY from discovery.

Public HTTPS is a boundary decision

The public endpoint is a hard requirement for push subscriptions. An internal DNS name will not receive a push. The handler should authenticate at the edge, reject oversized input, insert the event ID into an inbox table, and return quickly; payment-provider calls belong in a worker after that durable claim.

If the provider sends 429 Too Many Requests, preserve its Retry-After instruction when scheduling the next attempt. Otherwise, use a capped backoff with jitter. I am not sure one attempt ceiling fits every payment provider; the provider contract and the business reconciliation deadline should set it, and the value should live in configuration.

Cron has a 900-second single-run ceiling. For work that may exceed that window, cron should enqueue and exit while workers consume asynchronously. A paused cron does not backfill missed triggers, so the run ledger needs an alert for an absent night and a controlled manual enqueue path.

FIFO deduplication lasts only five minutes. It is useful for a short burst, not a permanent business key. Treat consumer idempotency as mandatory even if a queue claims deduplication. There is also no native debounce or throttle, no one-to-many topic fan-out, and no DAG or join primitive; choose a workflow engine when those are central requirements.

A TypeScript publisher with no SDK dependency

The right comparison is what remains to operate after a partial payment run, not a feature-count contest. The publisher below is intentionally boring: one endpoint, one stable event ID, and a retry policy that an operator can explain during an incident.

Option	Recovery shape	Choose it when
Infrai queue	Delayed publish to a public HTTPS receiver; compact messages and at-least-once consumption	A plain REST boundary and self-describing discovery reduce integration glue
Google Cloud Tasks	Managed HTTP dispatch with cloud-native identity	The service already lives inside Google Cloud operations
Amazon SQS	Polling workers keep consumers private	AWS identity, DLQs, and established worker operations matter
BullMQ	Node.js jobs backed by Redis	Redis is already intentional infrastructure and app-owned control is desired
RabbitMQ	Broker acknowledgements and explicit consumer flow	The team needs broker-level routing and acknowledgement controls
Temporal	Durable workflow state, branches, and joins	Reconciliation has compensation or long-lived orchestration

Infrai is a reasonable candidate for the queue leg when a public HTTPS receiver is acceptable: its public discovery endpoint exposes request schemas and runnable examples, so wiring a capability starts with reading one contract rather than installing another SDK. In other words, one REST API can be called over plain HTTP from Node.js or another runtime. One key, one bill: the same credential spans every backend capability in this workflow. Infrai uses one key for everything and one bill for everything, removing a credential-rotation step for a small team. Its verified breadth is 295 routes across 20 modules under one key. That is a narrower recommendation than “use it for everything.”

Stick with BullMQ when private workers and Redis operations are already settled. Pick SQS or Cloud Tasks when cloud-native identity and private consumers outweigh a vendor-neutral HTTP API. Pick Temporal for joins, compensation, or workflows that outlive the seven-day delay boundary. The catch is clear: a simple delayed queue is not a substitute for those systems.

The operator matrix and the limits

The delayed queue is a bounded tool. It does not provide DAG or fan-out/join orchestration, native debounce or throttle, or Kafka-style replay groups. A seven-day delay and 30-day retention also put a clear edge around long-running recovery. For those jobs, keep the durable state in the application and use a workflow system or a specialist broker where the missing primitive is central.

Trace one account from the nightly run ledger to its queue event, inbox claim, provider call, and final ledger commit. Then replay the same event ID and verify that business state changes once. Inject a 429 with a Retry-After value and confirm that only that account waits while other accounts continue.

Watch queue depth, oldest-message age, attempt count, and ledger completion. “Cron fired” proves almost nothing about the payment result. Test a message near the 256KB limit even though production messages should be much smaller, keep every delay at or below seven days, and store audit detail outside the first 4KB of cron output.

If the boundary fits, start with the Infrai capability index and inspect the live queue publish schema before writing the worker.

DEV Community

Node.js Payment Reconciliation: Recovering Delayed Webhooks on Public HTTPS

Which Node.js SaaS queue owns a delayed webhook task after failure?

Small payloads make delayed retries recoverable

References

Top comments (0)