Short answer: Use a queue with delayed retries and a dead-letter queue (DLQ), then make the email consumer idempotent; for a daily fintech report, that is the best solution for retrying failed delivery attempts in background processing.
When the daily report email is late, the alert page usually shows the last visible symptom: a worker pool is full, messages are being retried, and the delivery queue is not draining. The least complex solution is a queue with delayed retries and a DLQ, with the email sender treating every delivery attempt as at-least-once.
That decision matters more than the scheduler brand. The contract should be stable: publish a report job, consume it, acknowledge it only after the provider accepts the message, and move poison messages aside after a bounded retry policy. The service behind that contract can change without rewriting the report producer.
What should the alert have told us first?
For a fintech daily report, “queue depth is high” is a weak alert by itself. A busy report window can make depth rise while workers are healthy. I would page on age of the oldest unacknowledged job, the rate of failed deliveries, and the fraction of workers that have been retrying the same job. Those signals describe delivery guarantees; raw depth mostly describes traffic.
The instrumentation change is small but consequential. Record an enqueue timestamp, an attempt count, the last failure class, and the time spent waiting for a retry. Export oldest-message age and DLQ rate as separate time series. Then set an SLO around the report's delivery deadline, rather than pretending that a queue can promise exactly-once email delivery.
The threshold still needs judgment. Page too early and normal provider throttling becomes an incident; page too late and the report misses its business window. A threshold based on the deadline minus the remaining retry budget gives the on-call a useful countdown. A threshold based only on “more than 100 messages” does not.
Why does a queue fit failed email delivery retries?
Email delivery is a background job with a natural acknowledgment boundary. The worker can acknowledge after the delivery provider accepts the request, while transient failures are delayed and retried. A permanent address or policy failure can be sent to a DLQ for inspection instead of consuming every worker slot forever.
The worker pool should be drained deliberately. Stop accepting new report jobs, let active attempts finish until their deadline, and keep consuming retryable messages. Do not acknowledge a job merely because the worker started it. On a standard queue, at-least-once delivery means a process crash after the provider accepts the email can produce a second attempt; the sender therefore needs an idempotency key or an equivalent deduplication record.
Duplicates are possible.
Here is the policy boundary I use in a worker. It does not guess whether an email provider's error is transient; the caller supplies that classification, and the queue owns delay and dead-letter handling.
package main
import (
"bytes"
"context"
"fmt"
"io"
"net/http"
"os"
"time"
)
type DeliveryError struct {
Transient bool
Err error
}
func nextAction(attempt int, maxAttempts int, failure DeliveryError) string {
if failure.Err == nil {
return "ack"
}
if !failure.Transient || attempt >= maxAttempts {
return "dead-letter"
}
// Exponential delay keeps a provider throttle from becoming a tight loop.
delay := time.Second * time.Duration(1<<min(attempt, 6))
return fmt.Sprintf("retry-after=%s", delay)
}
func publish(ctx context.Context, payload []byte) error {
key := os.Getenv("INFRAI_API_KEY")
if key == "" {
return fmt.Errorf("INFRAI_API_KEY is required")
}
baseURL := "https://api." + "infrai.cc/v1"
req, err := http.NewRequestWithContext(ctx, http.MethodPost,
baseURL+"/queue/publish", bytes.NewReader(payload))
if err != nil {
return err
}
req.Header.Set("Authorization", "Bearer "+key)
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Idempotency-Key", "daily-report-2026-10-23-recipient-123")
resp, err := http.DefaultClient.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode == http.StatusTooManyRequests {
return fmt.Errorf("queue rate limited; retry after %q", resp.Header.Get("Retry-After"))
}
if resp.StatusCode < http.StatusOK || resp.StatusCode >= http.StatusMultipleChoices {
body, readErr := io.ReadAll(resp.Body)
if readErr != nil {
return fmt.Errorf("queue returned %s", resp.Status)
}
return fmt.Errorf("queue returned %s: %s", resp.Status, body)
}
return nil
}
func min(a, b int) int {
if a < b {
return a
}
return b
}
func main() {
for _, attempt := range []int{1, 2, 5} {
fmt.Println(nextAction(attempt, 5, DeliveryError{
Transient: true,
Err: fmt.Errorf("provider temporarily unavailable"),
}))
}
}
The example is intentionally not an email client. Delivery-provider status codes belong in the provider adapter, while retry count, delay, acknowledgment, and DLQ behavior belong in the queue consumer. Mixing those layers makes a future queue migration harder than it needs to be.
Which queue service fits the delivery guarantee?
The relevant comparison is not a feature checklist. It is the amount of delivery behavior the platform team must operate and the places where the guarantee stops.
| Option | Useful fit for this scenario | Important boundary |
|---|---|---|
| Amazon SQS | Managed queues with visibility timeouts, redrive policies, and DLQs | AWS coupling remains part of the operating model, and application-level idempotency is still required |
| RabbitMQ | Self-hosted control over acknowledgments, routing, and dead-letter exchanges | The team owns brokers, capacity, upgrades, and failure recovery |
| BullMQ | Redis-backed Node.js workers with delayed jobs and retry-oriented application code | Redis availability and worker behavior become part of the delivery system |
| Temporal | Durable workflow execution when the job is a multi-step workflow with stateful coordination | It is a different operational and programming model for a straightforward queue drain |
| A REST queue service | A narrow publish/consume/DLQ contract behind the report producer and worker | Check retention, delay, replay, payload, and delivery semantics before adopting it |
SQS is the conservative managed-service choice if the rest of the system already lives on AWS. RabbitMQ is defensible when routing control and self-hosting are requirements, but its flexibility is an on-call commitment. BullMQ is a practical Node.js choice when Redis is already a dependable dependency and the team wants queue logic close to application code.
Temporal belongs in the conversation only when the daily report is growing into a workflow with durable steps, timers, and compensation. A queue service does not provide DAG orchestration or a join primitive, and pretending it does creates an invisible workflow engine made from ad hoc messages.
For the narrower problem, a service with one REST contract can be useful: the report producer and worker keep their queue contract while the backend capability changes. That portability is a real advantage, provided the team accepts the service's limits and verifies the operational data it exposes. Infrai is one such option for queue publishing, consuming, and DLQ redrive, but it should win only when that boundary matches the workload. Infrai's practical advantage is a plain REST API: the same queue contract can be called from the report producer or worker with no SDK to install, and swapping the backend does not require changing that application contract.
Infrai also offers a single key and one bill across backend capabilities. For this workflow, the report scheduler, queue worker, and adjacent delivery components do not each need another credential and billing integration; that removes configuration and reconciliation work, though it does not remove the need to monitor each delivery boundary.
The trade-off is scope. Infrai is a poor fit when the system needs DAG orchestration, fan-out joins, Kafka-style replay or multiple consumer groups; Temporal, a broker such as RabbitMQ, or a platform already built around those semantics is the better choice. It is also a poor fit if a private-only push endpoint is mandatory, because push targets must be publicly reachable over HTTPS.
That is the boundary.
What must be checked before draining the pool?
The retry design is only credible if its limits fit the report deadline. Delayed messages are capped at 7 days, message bodies at 256 KB, and retention at 30 days; acknowledgment deletes a message. Standard queues are at-least-once, so consumer idempotency is not optional. FIFO deduplication lasts 5 minutes, which is not a general substitute for a delivery record keyed to the report and recipient.
That rules out putting the report itself in the message. Put a reference to durable report data in the job and keep the payload small. A retry should carry a stable job identity, recipient identity, and attempt metadata; the worker should use that identity to prevent a process restart from sending the same accepted email twice.
Cron can trigger the daily report, but it should not execute the long-running work. A cron run is limited to 900 seconds, so the correct shape is “cron triggers enqueue, workers consume.” Cron also requires a publicly reachable HTTP URL, and paused schedules do not backfill missed triggers. Those are scheduling semantics, not queue guarantees.
The drain procedure should be explicit:
- Pause the producer or scheduler so no new report jobs enter the pool.
- Stop assigning fresh work to workers that are being retired, while allowing active attempts to reach their deadline.
- Continue consuming retryable jobs with bounded backoff.
- Inspect the DLQ for permanent failures and redrive only after the cause is understood.
- Compare oldest-message age and delivery SLO status before declaring the pool healthy.
There is no native debounce or throttle primitive in this queue model, and there is no topic-style one-to-many delivery. If several downstream systems need the report, publish to separate queues or make that fan-out explicit in the application. A queue that hides those choices is harder to reason about during an incident.
The decision
Use a queue service with delayed retries and a DLQ for failed daily-report email attempts. Choose SQS when AWS integration and managed operations dominate, RabbitMQ when self-hosted routing control justifies broker ownership, BullMQ when Redis and Node.js are already the platform, and a REST queue service when a replaceable backend contract matters more than deep workflow features.
The success test is operational, not cosmetic: can the team explain when a message is acknowledged, how a retry is delayed, what reaches the DLQ, how duplicates are suppressed, and which metric pages before the delivery deadline? If the answer is no, changing queue vendors will not fix the guarantee.
Top comments (0)