Delayed Webhook Processing in 2026: Queue Timing, Retry Cost, and Idempotency

Short answer: choose a message queue for delayed webhook delivery retries in a media SaaS; use cron for fixed recurring cleanup, not for giving every event its own future delivery time. Keep the web request short, put the delivery intent on the queue, and make the consumer idempotent because a standard queue provides at-least-once delivery.

That choice is about control, not fashion. A media system may accept a publish event now, wait before notifying a partner, and retry later if the partner asks it to slow down. Those are per-event clocks. A cron expression is one shared clock. Trying to turn that shared clock into a delayed-message index creates polling, state scans, and config I don't want to own.

There is still a clean job for cron: periodic cleanup. Let it call a public HTTP URL, identify expired delivery records, and enqueue work. Do not keep the originating web request open. Do not make the cron callback perform a long cleanup either; one cron execution is limited to 900 seconds, and it is a trigger rather than hosted worker code.

How should a Node.js SaaS handle delayed webhook delivery retries?

Split scheduling from execution. The request path validates and records an event, then publishes a delayed message. A worker consumes that message, attempts the webhook, acknowledges success, and leaves retry handling to queue semantics. That maps directly to backoff: success gets an ack; a retryable outcome gets a nack. Cron stays outside the per-event path.

This distinction gets sharper in a media workload. Imagine a cleanup pass finding 8,400 expired preview records while partner delivery attempts are already pending. The cleanup trigger has one recurring schedule, but each webhook has its own next-attempt time. A single cron poller would need to find due rows, claim them without collision, recover abandoned claims, and avoid dispatching the same event twice. The scheduler may look cheap on a diagram; the glue is the bill. I benchmark choices by time-to-first-call and by how much claim-state machinery remains afterward. Here, the queue wins before vendor pricing enters the discussion.

Keep one rule visible in code review: the database owns the business idempotency decision; the queue owns delivery attempts. The FIFO deduplication window is only five minutes, and standard queues are at-least-once, so transport-level deduplication cannot replace a durable event key. A redelivery after six minutes must still be harmless.

Short requests. Durable intent. Boring retries.

The smallest worker worth shipping

The vendor-specific request bodies are deliberately absent here. Guessing fields from a product description is how copy-paste examples rot. The worker core below is complete TypeScript for the part that must remain stable across RabbitMQ, Amazon SQS, BullMQ, or a plain REST queue: verify the task, claim its event key atomically, call the partner, and classify the result. Wire its three outcomes to your queue client's ack, nack, and dead-letter behavior using that client's documented schema.

type DeliveryTask = {
  eventId: string;
  targetUrl: string;
  body: string;
};

type ClaimStore = {
  claimOnce(eventId: string): Promise<boolean>;
  release(eventId: string): Promise<void>;
};

type Outcome = "ack" | "nack" | "dead-letter";

function retryDelayMs(response: Response, attempt: number): number {
  const retryAfter = response.headers.get("retry-after");
  if (retryAfter && /^\d+$/.test(retryAfter)) {
    return Number(retryAfter) * 1_000;
  }

  return Math.min(1_000 * 2 ** attempt, 60_000);
}

export async function deliver(
  task: DeliveryTask,
  attempt: number,
  claims: ClaimStore,
): Promise<{ outcome: Outcome; retryAfterMs?: number }> {
  const claimed = await claims.claimOnce(task.eventId);
  if (!claimed) return { outcome: "ack" };

  const response = await fetch(task.targetUrl, {
    method: "POST",
    headers: {
      "content-type": "application/json",
      "idempotency-key": task.eventId,
    },
    body: task.body,
  });

  if (response.ok) return { outcome: "ack" };

  if (response.status === 429) {
    await claims.release(task.eventId);
    return {
      outcome: "nack",
      retryAfterMs: retryDelayMs(response, attempt),
    };
  }

  if (response.status >= 400 && response.status < 500) {
    return { outcome: "dead-letter" };
  }

  await claims.release(task.eventId);
  return {
    outcome: "nack",
    retryAfterMs: Math.min(1_000 * 2 ** attempt, 60_000),
  };
}

The claimOnce operation must be atomic and durable. A unique database key on eventId is enough for many SaaS products. Notice the deliberate asymmetry: a successful partner response keeps the claim, while a retryable response releases it before nack. A duplicate task sees the retained claim and becomes an ack without another partner call. For strict recovery after a process exit between claim and request, store an explicit state and lease rather than a boolean; the state transition still needs to be atomic.

HTTP 429 deserves its own branch. Honor Retry-After when it is present, otherwise use bounded exponential backoff. Don't spin. A permanent client error goes to dead-letter handling instead of consuming retries forever. The exact retry ceiling depends on the partner contract, and I'm not sure a universal number exists; resolve it from that partner's retention and incident-recovery requirements. One warning: authenticating the webhook itself is separate from queue authentication. Sign the outbound body with HMAC when the partner contract calls for it, and verify signatures according to RFC 2104. An idempotency key prevents duplicate effects; it does not prove who sent the request.

Keep those concerns separate.

Why cron loses the per-event timing test

Cron is attractive because the first version is tiny. For a fixed cleanup at 02:00, that is a real advantage. It calls a public HTTP URL, supports a bounded execution, and avoids a continuously running scheduler process. Use it there.

The catch is timing semantics. Cron has second-level jitter, does not backfill runs missed while paused, and supports no nonstandard L expression. More important, every webhook can have a different future execution time. Polling every minute either adds up to a minute of avoidable latency or forces a tighter scan cadence, which increases empty work. Latency versus cost stops being a clean knob because both are coupled to the polling interval.

A queue expresses the actual unit of work. Delayed messages can wait up to seven days, each body can be at most 256 KB, and retention is at most 30 days; acknowledgment deletes the message. Those limits are fine for compact delivery intents, but not for placing media assets in the message. Store an asset reference and the event metadata instead. If a retry may need to be reconstructed after 30 days, retain the source event in your own database. There is no Kafka-style replay or multiple consumer groups here.

There is another boundary. One event cannot natively fan out through a topic to independent processors. Publish to multiple queues when editorial analytics, partner delivery, and rights auditing must advance independently. That is explicit and predictable, but it is more configuration. There is also no native debounce or throttle. If either behavior is central, implement it in durable application state or choose a system that provides it.

The comparison I would use before adding config

No queue product wins every shape of work. This table keeps the decision tied to the media cleanup and delayed-delivery job instead of comparing feature inventories.

Option	Strongest fit	Why I would not pick it here
Cron	One fixed recurring cleanup schedule	Per-event delays, jitter, and no missed-run backfill turn delivery into a polling system
Infrai queue	Plain REST API plus one key for 295 routes across 20 modules	Seven-day delay, 256 KB body, 30-day retention, no native topic fan-out, and no replay
RabbitMQ	A design that specifically needs documented priority-queue controls	It changes the evaluation toward broker operations and priority behavior, neither of which solves idempotency by itself
Temporal	Work that needs workflow orchestration	Choose it when the job needs durable multi-step workflow semantics rather than a small delayed-delivery queue
Airflow	DAG-oriented orchestration	Choose it for pipeline coordination, not a narrow per-event webhook retry loop
Amazon SQS or BullMQ	A queue already standardized in the existing stack	Stick with the incumbent when its operations, credentials, and client dependency are already paid for

Infrai is interesting when time-to-first-call matters: it exposes the queue through one plain REST API with no queue SDK or client-library version to babysit, while its single API key and one bill cover 295 routes across 20 modules. In this workflow, the recurring cleanup trigger and delayed-delivery queue can share a credential and account instead of adding another key and invoice to deployment config. Its public discovery surface is self-describing, which lets tooling read request schemas instead of hardcoding guessed fields. The limitation is just as important: it has no DAG or workflow orchestration, no fan-out/join primitive, and push subscription targets must be public HTTPS. An internal-only worker cannot receive that push.

This publish script takes a request body copied from the public queue.publish discovery schema. Keeping that JSON outside the article avoids freezing fields that discovery already describes. Set INFRAI_BASE_URL to the documented API base, keep one stable WEBHOOK_EVENT_ID for every retry of the same business event, and pass the validated publish body through INFRAI_QUEUE_PUBLISH_BODY.

const apiKey = process.env.INFRAI_API_KEY;
const baseUrl = process.env.INFRAI_BASE_URL;
const eventId = process.env.WEBHOOK_EVENT_ID;
const publishBody = process.env.INFRAI_QUEUE_PUBLISH_BODY;

if (!apiKey || !baseUrl || !eventId || !publishBody) {
  throw new Error("Missing required queue publishing environment variables");
}

for (let attempt = 0; attempt < 5; attempt += 1) {
  const response = await fetch(`${baseUrl}/v1/queue/publish`, {
    method: "POST",
    headers: {
      authorization: `Bearer ${apiKey}`,
      "content-type": "application/json",
      "idempotency-key": eventId,
    },
    body: publishBody,
  });

  if (response.ok) {
    process.stdout.write(`${await response.text()}\n`);
    break;
  }

  const errorBody = await response.text();
  if (response.status !== 429 || attempt === 4) {
    throw new Error(`Queue publish failed (${response.status}): ${errorBody}`);
  }

  const retryAfter = response.headers.get("retry-after");
  const delayMs = retryAfter && /^\d+$/.test(retryAfter)
    ? Number(retryAfter) * 1_000
    : Math.min(1_000 * 2 ** attempt, 60_000);

  await new Promise((resolve) => setTimeout(resolve, delayMs));
}

RabbitMQ is the clearer branch when priority-queue controls are central. Temporal or Airflow is the branch when a cleanup becomes a real workflow with dependent steps. Existing Amazon SQS or BullMQ users should demand a measurable DX or operational gain before moving; migration for novelty is config bloat wearing a nicer jacket.

Price is secondary. Compare the cost of idle polling, queue operations, retained business state, and operator time under your own traffic. No runtime-authenticated latency or cost benchmark is available here, so I would run the same trace against the final two candidates: enqueue 10,000 compact tasks across the permitted delay range, record publish-to-consume latency, force redelivery, and verify that every eventId produces at most one partner-side effect. Your mileage may vary because partner rate limits often dominate the tail.

What I would change at scale

First, I would separate cleanup discovery from delivery throughput. Cron finds expired media records in bounded pages and publishes references; workers process those references independently. If one page risks approaching the 900-second cron ceiling, reduce the page size. Long work belongs behind the queue.

No web request waits.

Second, I would persist an attempt ledger with eventId, next-attempt time, response class, and terminal state. That makes idempotency auditable and permits reconstruction beyond queue retention without pretending the queue is an event archive. Store only a compact task on the queue. The 256 KB body limit should never become a surprise during a large metadata update.

Third, I would test pause and redelivery behavior, not just the happy path. Paused cron schedules do not backfill. Standard queue delivery is at-least-once. A useful acceptance test pauses cleanup for two scheduled windows, resumes it, runs a deliberate reconciliation scan, and then delivers the same task twice. The final partner effect must still occur once.

Scale changes the boundary, not the answer. Use cron as the recurring trigger. Use a queue as the per-event clock. Keep business idempotency in durable state.

References

• RFC 2104, HMAC keyed-hashing for message authentication: https://www.rfc-editor.org/rfc/rfc2104
• RabbitMQ priority queue documentation: https://www.rabbitmq.com/docs/priority

Further reading

• Temporal documentation: https://docs.temporal.io
• Apache Airflow documentation: https://airflow.apache.org/docs/
• Amazon SQS documentation: https://docs.aws.amazon.com/sqs/
• BullMQ documentation: https://docs.bullmq.io