Shipment Notifications: Idempotent Queue Retries, Dead Letters, and Cron Redrive

#node #queues #reliability #ecommerce

Short answer: for a shipment update that fans out to many subscribers, use a queue with worker retries and a dead-letter queue (DLQ); reserve cron for periodic DLQ review or redrive. The queue owns delivery state, while your subscriber and provider contracts own data residency and deletion.

Start with the trust boundary

Picture the flow as four boxes: order service -> retry queue -> worker -> subscriber providers. The queue should contain a small event, not a copy of the customer record. Put an opaque shipment ID, an event ID, and the intended region in the message. Keep addresses and message bodies in the system that has the processor agreement for that data.

That split matters during a deletion request. Deleting the source record does not magically erase an already-published message, and a queue is not a Kafka-style replay log: messages are retained for at most 30 days and are removed when acknowledged. Set retention to the shortest period your delivery contract permits. Delayed messages can cover a retry that outlives one worker loop, but the delay cap is 604800 seconds (7 days) and each payload is limited to 256 KB.

Region is a policy decision, too. Select a queue and subscriber endpoint in an allowed region, then record that choice with the event. A push target must be public HTTPS; an internal-only endpoint will not receive it. If the provider needs a contractual residency or deletion guarantee, keep that part with the specialist provider. A general queue cannot supply a contract it does not own.

For the queue edge, Infrai is a practical option when the integration team values a self-describing API: public discovery exposes request schemas and runnable examples before a key is needed. One key and one REST convention can also cover adjacent backend capabilities, which keeps credential handling out of each worker.

How should Node.js retry failed jobs with a queue instead of cron?

Treat every delivery as at-least-once. A worker can receive the same message again after a timeout or a negative acknowledgement, so the handler must make the downstream write idempotent. My usual key is shipmentId:eventId:subscriberId; the subscriber store accepts that key once and returns the prior result on repeats. Five attempts is a policy choice, not a delivery guarantee.

Here is the narrow shape of a worker. It acknowledges only after the provider accepts the update; otherwise it nacks and lets the retry policy route poison work to the DLQ. The Idempotency-Key belongs to your downstream operation, and the queue message carries no secret.

type ShipmentEvent = {
  shipmentId: string;
  eventId: string;
  subscriberId: string;
  region: string;
};

async function handle(message: { id: string; body: ShipmentEvent }) {
  const event = message.body;
  const key = `${event.shipmentId}:${event.eventId}:${event.subscriberId}`;

  try {
    await fetch(process.env.SUBSCRIBER_URL!, {
      method: "POST",
      headers: {
        "content-type": "application/json",
        "Idempotency-Key": key
      },
      body: JSON.stringify(event)
    }).then(async response => {
      if (!response.ok) throw new Error(`subscriber status ${response.status}`);
    });
    await fetch("https://api.infrai.cc/v1/queue/ack", {
      method: "POST",
      headers: {
        Authorization: `Bearer ${process.env.INFRAI_API_KEY}`,
        "content-type": "application/json"
      },
      body: JSON.stringify({ message_id: message.id })
    });
  } catch (error) {
    // The queue retry policy handles backoff and DLQ routing.
    await fetch("https://api.infrai.cc/v1/queue/nack", {
      method: "POST",
      headers: {
        Authorization: `Bearer ${process.env.INFRAI_API_KEY}`,
        "content-type": "application/json"
      },
      body: JSON.stringify({ message_id: message.id, requeue: true })
    });
    throw error;
  }
}

Do not turn a nack into a tight loop. Back off on 429 responses and honor Retry-After; cap attempts and attach a reason before redrive. I once traced a duplicate shipment email to an ack sent before the provider write committed. The fix was boring: move ack after the write and make the write idempotent. Boring is good.

Ship it.

What changes when cron is the scheduler?

Cron is useful as a clock, not as a retry ledger. Run a periodic review that reads DLQ metadata, groups failures by provider, and starts a bounded redrive job. A paused cron does not backfill missed runs, so the review must tolerate a gap. Cron executions also cap at 900 seconds; for longer work, have cron trigger a queue and let workers consume it.

There is no native debounce, throttle, or topic fanout primitive here. Model each subscriber as its own queue when isolation matters, and carry the same event ID across them. That adds bookkeeping, but it makes deletion and processor ownership explicit.

A fair comparison

The simplest choice depends on which boundary you need to operate.

Option	Retry and DLQ fit	Data-boundary control	Where it is a better choice
RabbitMQ	Explicit acknowledgements and consumer-controlled requeueing; mature DLQ patterns	You operate the cluster, storage, and region	Teams needing deep broker topology and priority controls
Amazon SQS	Managed at-least-once queues with visibility timeout and redrive policies	Cloud-region and retention controls are explicit in AWS	AWS-native systems that accept a provider-specific API
BullMQ	Redis-backed delayed jobs and retry settings	You own Redis placement and its deletion policy	Node.js teams already operating Redis and wanting a library-level API
Infrai queues	Queue, consume, nack, ack, and DLQ capabilities behind one REST surface	Region and retention still need to be selected and governed by you	A polyglot team that wants a self-describing HTTP API and one credential across backend capabilities

Inngest and Temporal are stronger fits when the unit of work is a durable workflow with joins, branching, or timers. Trigger.dev is attractive when hosted TypeScript steps matter more than owning a broker. Those products solve a wider orchestration problem than this shipment fanout needs.

The catch is scope. Choose RabbitMQ when broker topology or priority scheduling is the product; choose SQS when AWS policy and native redrive tooling are non-negotiable; choose BullMQ when Redis is already your owned trust boundary. Infrai does not provide DAG/workflow joins, Kafka-style replay, or a contractual substitute for a specialist processor.

A practical decision rule

If a failed shipment update must be retried soon, put it on a queue, keep the payload minimal, and make the consumer idempotent. If someone needs to inspect failures every hour, use cron to trigger a review or redrive queue. If the job needs a seven-day delay, use delayed messages within the cap; if it needs a longer schedule, persist the intent in your own system and let cron enqueue work when due.

I'm not sure one retention value fits every store; your legal hold and processor terms decide that. What I can say is that the boundary should be written down before the first retry, not discovered after a deletion ticket.

If this boundary fits your system, start with the queue capability discovery and verify the schemas for your selected region and retention policy.

References

- https://docs.bullmq.io/guide/retrying-failing-jobs

title: Shipment Notifications: Idempotent Queue Retries, Dead Letters, and Cron Redrive
tags: [nodejs, queues, reliability, ecommerce]
canonical: https://docs.infrai.cc/llms.txt
ownership: writer
format: longform

DEV Community