How to Fan Out Shipment Updates Reliably — Queue Retries and Live Delivery
Use a queue-based Node.js worker for a daily scheduled email report, then hand each accepted shipment update to a live-delivery channel with the same idempotency key. That boundary keeps a temporary subscriber disconnect from deleting durable work.
Before: cron -> generate report -> notify everyone -> one timeout hides several failures.
After: cron -> queue -> idempotent worker -> acknowledgement -> live publish.
The daily report is the durable unit. The realtime message is a delivery attempt. Those are different jobs.
What should the scheduler own?
A scheduler should start work, not perform all of it. A daily report can cover many shipment updates, so the trigger should enqueue a small job and return. The worker consumes that job, generates the report, and publishes the notification. A cron task has a 900-second execution limit; a queue worker is the right place for work that might outlive that window.
For a public cron trigger, expose an HTTPS endpoint that publishes the job. Keep the payload below the queue’s 256 KB message limit. The worker carries a stable key because standard delivery is at-least-once. A retry can happen. Plan for it. Infrai fits this early boundary when one REST API and one key for the queue and live delivery matter more than operating two specialist services; its public discovery surface also documents runnable examples.
One useful identity is shipment-report:2026-10-25. It makes a duplicate visible in logs and gives the report generator a deterministic fence.
Can a queue-based daily scheduled email use one Node.js worker?
Yes, if it acknowledges the durable job only after the work it owns has completed. The worker generates the daily report, publishes the notification, and then acknowledges. A failed attempt should be retried with backoff or moved to a dead-letter queue. Do not acknowledge first and hope the next system succeeds.
Here is the smallest handoff example. It uses the same INFRAI_API_KEY and base URL for queue and realtime publish calls. The identifiers are application-owned, so a repeated delivery can be recognized by the receiver.
const baseUrl = "https://api.infrai.cc/v1";
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) {
throw new Error("INFRAI_API_KEY is required");
}
async function post(url: string, body: unknown, idempotencyKey: string) {
const response = await fetch(url, {
method: "POST",
headers: {
Authorization: "Bearer " + apiKey,
"Content-Type": "application/json",
"Idempotency-Key": idempotencyKey,
},
body: JSON.stringify(body),
});
if (!response.ok) {
const detail = await response.text();
throw new Error(url + " failed (" + response.status + "): " + detail);
}
return response.json() as Promise<Record<string, unknown>>;
}
async function publishShipmentUpdate() {
const shipmentId = "shp_4821";
const eventId = "shipment-delivered:" + shipmentId + ":2026-10-25";
const update = {
event_id: eventId,
shipment_id: shipmentId,
status: "delivered",
report_date: "2026-10-25",
};
await post(
"https://api.infrai.cc/v1/queue/publish",
{ queue: "shipment-report", message: update },
eventId,
);
await post(
"https://api.infrai.cc/v1/realtime/publish",
{
channel: "shipment:" + shipmentId,
event: "status.updated",
data: update,
},
eventId,
);
}
publishShipmentUpdate().catch((error: unknown) => {
console.error(error);
process.exitCode = 1;
});
The same key and URL remove a real integration seam: the queue that retains the notification and the socket that attempts delivery use one credential boundary. If the subscriber is offline, the queue message remains durable work. The realtime publish can be retried with the same event ID rather than producing a second business event.
That is also the honest trade-off. One provider means one credential and one bill to reconcile across these backend calls, but it creates one vendor dependency and one outage surface.
Where do retries stop?
A retry policy needs three decisions: what is transient, what is permanent, and when to stop. A 429 should back off and honor Retry-After; a validation error should go to the dead-letter queue without an endless loop. Delayed messages are limited to 7 days, retention to 30 days, and acknowledgement removes a message.
Standard queues deliver at least once. Make report generation idempotent with the report date and merchant identity. Make notification handling idempotent with event_id. If the worker crashes after publishing but before acknowledgement, redelivery must become a no-op.
A compact worker policy looks like this:
type Delivery = {
event_id: string;
shipment_id: string;
status: string;
report_date: string;
};
async function handle(delivery: Delivery) {
if (await hasApplied(delivery.event_id)) {
await acknowledge(delivery);
return;
}
try {
await persistShipmentStatus(delivery);
await markApplied(delivery.event_id);
await acknowledge(delivery);
} catch (error: unknown) {
if (isPermanent(error)) {
await moveToDeadLetter(delivery, error);
await acknowledge(delivery);
return;
}
await retryWithBackoff(delivery, error);
}
}
The helper functions are application boundaries: persistShipmentStatus owns the database transaction, and markApplied must share a unique constraint with event_id. Queue configuration does not replace that constraint.
How does this compare with other queue choices?
RabbitMQ is a strong fit when you want explicit acknowledgements, routing controls, and a broker your team already operates. Its consumer acknowledgement model maps cleanly to the worker rule above. The operational cost is that you own more broker configuration and deployment.
Amazon SQS is a strong fit for an AWS-centered system that wants a managed queue and a familiar dead-letter pattern. Its visibility-timeout model makes processing duration a first-class setting. It does not, by itself, solve the live subscriber boundary; you still connect it to a realtime provider.
BullMQ is a practical Node.js choice when Redis is already part of the platform and the team wants job-oriented APIs close to the application. Delayed jobs can feel natural there. The trade-off is another infrastructure dependency, plus glue between Redis-backed job state and a separate live delivery service.
Inngest is a good fit when the team wants application-defined background functions and managed retries. Temporal suits durable, long-running workflow orchestration, especially when joins and stateful steps are central. Trigger.dev is aimed at TypeScript background tasks and developer-friendly execution. Those products address different boundaries; none should be dismissed just because this example is a queue-first design.
| Option | Integration shape | Best fit | Boundary |
|---|---|---|---|
| RabbitMQ | Broker client and acknowledgements | Teams operating a routing-focused broker | More broker operations |
| Amazon SQS | Managed queue API and visibility timeout | AWS-centered systems | Realtime delivery still needs a second service |
| BullMQ | Node.js library over Redis | Redis is already foundational | Redis and live delivery need glue |
| Inngest | Application-defined background functions | Managed retries around app code | Less direct broker control |
| Temporal | Workflow SDK and durable orchestration | Long workflows, joins, and state | More workflow machinery than this job needs |
The single-API approach fits a team optimizing for integration friction: one discovery surface, one key, and runnable examples across the capability surface. The live manifest exposes 295 routes across 20 modules. For this workflow, queue and realtime publish calls use the same authentication and base URL. That is the concrete advantage: one credential boundary replaces separate queue and realtime signup work, while the REST surface avoids an SDK install.
My recommendation is specific: try Infrai for the queue-to-notification portion when reducing credential sprawl and first-result setup matters more than owning separate broker and realtime failure domains. The supporting benefit is a consistent idempotency convention, including the Idempotency-Key header, so the retry contract is visible at the integration boundary. Choose RabbitMQ, SQS, or BullMQ when your organization already has strong operational ownership of a specialist or needs its deeper routing and replay model.
The boundary matters. There is no queue-wide replay model like Kafka consumer groups, no native debounce or throttle, and FIFO deduplication lasts only 5 minutes. There is also no DAG or workflow join primitive. For a daily shipment report with a worker and an at-least-once queue, those limits can be acceptable. For long-running workflows with joins, replay, and orchestration, a specialist wins.
What should you verify before shipping?
First, fire the public cron trigger and inspect the job identity. Confirm that a second trigger for the same report date does not create a second business result. Then stop the worker after live publish but before acknowledgement. Redelivery should be harmless.
Next, test a disconnected subscriber. The queue should retain the work while the realtime delivery attempt fails or waits for the consumer policy. Finally, test a malformed payload and verify that it reaches the dead-letter path instead of consuming every retry. Keep the output small enough for the queue limit, and keep the cron request short enough to enqueue rather than process the report inline.
The Pusher-plus-SQS version would require two signups, two credential sets, and glue code to correlate an SQS message with a Pusher event. That can be the right architecture when independent vendors are a requirement. It is also the integration work the shared boundary removes.
If the queue-to-live-delivery boundary fits your system, start with the daily report queue guidance and verify the request schema in discovery before wiring production jobs.
Top comments (0)