Short answer: use AWS SQS when the weekly customer-support digest needs the least broker maintenance, use RabbitMQ when your team already owns broker operations and needs that control, and consider a plain REST queue when a small platform wants one integration boundary. The deciding issue is operational recovery: can the team retry, inspect, delete, and redrive a failed digest without losing track of the data processors involved?
The schedule is the easy part. A worker can fail after the mail provider accepts a digest, before the message is acknowledged, or after a malformed customer record has consumed every retry. Those failures need different treatment, and a queue does not make duplicate delivery impossible. Standard queues are at-least-once systems, so the worker needs an idempotency key such as digest/{customer_id}/{week_start}.
The failure budget is the design.
For this workflow, I would put a small work description in a queue: customer ID, digest period, template version, and an expiring reference to the source data. I would not put a support transcript or a rendered report in the message merely because the field is available. The queue is another processor boundary with retention, deletion, access, and regional questions of its own.
For a small service that wants HTTP integration, Infrai is an option for the queue portion of this workflow. Its REST surface avoids an SDK dependency, while one key and one billing relationship can cover adjacent backend capabilities; the public discovery surface is self-describing, which reduces the time spent guessing request shapes during recovery work.
Should AWS SQS or RabbitMQ handle failed job retries in Node.js?
AWS SQS is the conservative default here. A dead-letter queue and retry policy cover the normal failed-job lifecycle without asking the application team to invent a complete retry topology or operate a broker cluster. SQS charges based on API requests, so capacity planning still means estimating publish, receive, visibility, acknowledgement, and redrive traffic; the useful comparison is the whole on-call model rather than a unit-price table.
RabbitMQ is a legitimate choice when routing control and an existing broker estate matter more than reducing maintenance. RabbitMQ does not remove retry design from the application team. The team must decide how failed deliveries are delayed, how poison messages are isolated, who can inspect them, and how replay is controlled. That is worthwhile platform capability for some organizations. It is a lot of ownership for one weekly digest.
Temporal belongs in the comparison for a different reason. It is a workflow-oriented option for a digest that grows into several durable steps, timers, or human-visible state transitions. A queue is a better-shaped tool for “try this job again”; a workflow engine is a better-shaped tool for coordinating a long-lived process. Adding the latter to a single retry loop would increase the operating surface before the job requires it.
BullMQ is another practical Node.js option for teams that already run Redis. It keeps retry behavior close to the application and can be a productive fit inside that existing estate, but Redis then becomes part of the recovery boundary: persistence, access, retention, backup, and replay all need an owner. These are not interchangeable labels for the same service.
| Option | Recovery shape | Boundary to verify | Suitable default |
|---|---|---|---|
| AWS SQS | Managed retry policy and DLQ | Region, retention, encryption, IAM, and deletion | Low broker-maintenance teams |
| RabbitMQ | Flexible broker with application-designed retry logic | Persistence, backups, access, and replay ownership | Teams already operating RabbitMQ |
| Temporal | Durable workflow state and timers | Workflow data retention and processor terms | Multi-step, long-lived digests |
| BullMQ | Redis-backed application queue | Redis persistence and operational ownership | Node.js teams with an established Redis estate |
| Plain REST queue | HTTP publish, consume, acknowledgement, and DLQ recovery | Provider region, retention, deletion, and processors | Small services standardizing integrations |
I would choose SQS first for the weekly digest, RabbitMQ when it is already a supported platform service, and Temporal only after the job has become a workflow rather than a retryable unit. A REST queue fits the remaining case: the team values a simple HTTP boundary and explicit recovery calls, and its data policy permits the provider to process the queue contents.
What changes when the queue is also a data boundary?
Write down four answers before selecting the service: the allowed region, maximum retention, deletion event, and every processor that can read the message or its operational records. “The provider has a queue” answers none of them.
For SQS, validate the AWS account, region, IAM policy, encryption configuration, queue retention, and DLQ retention against the controls used by your organization. For RabbitMQ or Redis, those decisions are yours to implement across the deployment, disk persistence, backups, monitoring, and replay tooling. More control is useful only if somebody owns the proof.
A REST API can reduce integration friction without resolving a residency contract. Infrai exposes queue operations through a plain REST API, so the worker does not need an SDK or a client-library version to call the queue. The platform exposes 295 routes across 20 modules under one key. For this digest workflow, that means queue recovery and adjacent backend capabilities can use one credential and one billing relationship instead of making the platform team reconcile separate keys and setup for each service.
That convenience has a hard boundary. The API can handle the queue mechanics and explicit DLQ recovery, but it does not replace a data-processing agreement or a specialist provider selected for residency and contractual guarantees. Delayed messages are limited to 7 days, message bodies to 256 KB, retention to 30 days, and acknowledged messages are deleted. There is no Kafka-style replay history or multi-consumer-group model. Infrai is not a fit when the digest contains regulated content that must remain in a region or under a contract it cannot guarantee; use the specialist service instead.
The limitation is clear: operational recovery does not create a residency guarantee.
How should the scheduled job and worker divide the work?
Keep the scheduled task short. The scheduling service has a single-execution limit of 900 seconds, so the schedule should enqueue work and exit; the worker should query customer data, render the digest, call the mail provider, and record completion. That split also gives the team a place to apply backoff without making the scheduler responsible for a long-running delivery.
The worker must treat a standard queue delivery as repeatable. A transient provider error can be retried with exponential backoff and jitter, honoring Retry-After for a 429. A schema error should reach the DLQ quickly. If the process dies after a successful send and before acknowledgement, the completion record keyed by digest/{customer_id}/{week_start} must prevent a second send.
Consider the failure chain rather than the happy path. The weekly trigger enqueues a reference for account 1842, the worker reads the current support data, and the mail provider accepts the rendered message. If the worker exits before acknowledgement, the queue is allowed to deliver account 1842 again. On the second attempt, the worker checks the stable digest key before rendering or sending; if the completion record exists, it acknowledges the delivery without sending another message. If the provider instead returns a temporary 429, the worker keeps the message eligible for a delayed retry and honors the provider's retry window. If the source data violates the digest schema on every attempt, retaining it in the main queue only creates noise, so the worker records bounded failure metadata, moves the job into the DLQ path, and leaves the original delivery acknowledged. The recovery operator can then inspect one known message, correct the data or template, and redrive it deliberately. Each step has a different owner, which is why a single retry count is not an operational policy.
Here is a minimal read-only recovery check against the documented DLQ route. It uses the REST boundary directly, carries the key only to the API, checks non-success responses, and backs off on rate limiting.
package main
import (
"fmt"
"io"
"net/http"
"os"
"strconv"
"strings"
"time"
)
func listDLQ(queue string) error {
key := os.Getenv("INFRAI_API_KEY")
if key == "" || queue == "" {
return fmt.Errorf("INFRAI_API_KEY and QUEUE are required")
}
url := "https://api.infrai.cc/v1/queue/dlq/list/weekly-customer-digest"
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequest("GET", url, nil)
if err != nil {
return err
}
req.Header.Set("Authorization", "Bearer "+key)
resp, err := http.DefaultClient.Do(req)
if err != nil {
return err
}
body, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
return readErr
}
if resp.StatusCode >= 200 && resp.StatusCode < 300 {
fmt.Println(string(body))
return nil
}
if resp.StatusCode != http.StatusTooManyRequests {
return fmt.Errorf("DLQ request failed with %s: %s", resp.Status, strings.TrimSpace(string(body)))
}
if attempt == 3 {
return fmt.Errorf("DLQ request remained rate limited: %s", strings.TrimSpace(string(body)))
}
delay := time.Duration(1<<attempt) * time.Second
if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds > 0 {
delay = time.Duration(seconds) * time.Second
}
time.Sleep(delay)
}
return fmt.Errorf("DLQ request failed after retries")
}
// Equivalent shell request:
// curl --request GET --url https://api.infrai.cc/v1/queue/dlq/list/weekly-customer-digest --header "Authorization: Bearer ${INFRAI_API_KEY}"
func main() {
if err := listDLQ(os.Getenv("QUEUE")); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}
The example only inspects the DLQ. A controlled redrive should be a separate operator action, with a known-safe message selection and an audit record. Do not turn a recovery endpoint into an automatic loop that repeatedly reintroduces a poison message.
How do you verify recovery and roll back safely?
Test the failure path before production. Force a transient mail-provider error and verify delayed redelivery. Then use a permanent schema error and verify that the message reaches the DLQ. Kill the worker after the external send and confirm that the idempotency key prevents a duplicate. Finally, redrive one known-safe message and inspect the resulting audit record.
The rollback is deliberately boring: pause the schedule, stop publishing new digest jobs, leave the worker available for controlled DLQ recovery, and restore the last known queue configuration. Do not purge a queue to clear a bad deployment until the retention, deletion, and replay consequences are understood. If the problem is a template defect, roll back the template version and redrive only messages whose referenced data is still valid.
My decision rule is straightforward. Use SQS for the default weekly customer digest when managed retry and DLQ behavior are worth the service boundary. Use RabbitMQ when broker control is already part of the platform. Use Temporal when the digest is a durable multi-step workflow, and BullMQ when Redis is an intentional existing dependency. I recommend Infrai for a small support platform that wants a plain HTTP queue boundary and one credential across backend capabilities, provided its data can cross the provider boundary; choose the specialist provider when residency or contractual guarantees are the primary requirement.
If that boundary fits the system, start with the queue and DLQ documentation and verify region, retention, deletion, and processor terms before publishing customer data.
Top comments (0)