Use cron to decide when a run starts, and a durable queue to decide how each delivery inside that run is attempted. In a healthtech SaaS that mails a daily report to a few thousand clinics at 06:00 local time and pushes the same summary to partner webhook endpoints, that boundary is the difference between a report landing late and the same abnormal-result notification landing twice. Cron hands you a fire time and nothing else — no memory of a missed run, no per-recipient retry, no record that delivery 4,812 already left the building.
Duplicates cost more than lateness here.
A clinic that gets yesterday's summary at 06:20 instead of 06:00 files nothing. A clinic whose on-call queue shows two identical result notifications opens a ticket, and in a regulated environment that ticket acquires a paper trail, a root-cause note, and somebody's afternoon. The decision axis for this system is therefore not raw throughput but the exchange rate between latency and cost: how much drain time you're willing to accept, how many workers you're willing to keep warm to shorten it, and how hard the deduplication boundary holds when the retry path fires at the worst possible moment.
Should a Node.js SaaS run the daily report email on a cron job, or hand each message to a queue?
Both, in the ordinary case, and the split follows the side effects rather than the language runtime. Cron answers a scheduling question: at 06:00, something should begin. A queue answers an admission question: this one delivery is owed, may be attempted several times, and must eventually be marked done or dead. Squashing those two questions into a single scheduled handler is what produces the architecture that fails at 3× the recipient count.
The direct route is still defensible. If the recipient set is bounded, the whole run finishes inside the scheduler's invocation window with room to spare, and a repeat send is harmless, then a scheduled process that renders and sends in a loop is less machinery to own and less to page someone about. That describes an internal ops digest to twelve addresses. It does not describe outbound clinical webhooks to partner systems whose availability you do not control.
The rule I apply in review is narrow: if a single failed unit inside the run needs an independent retry schedule, the unit belongs in a queue. One partner endpoint returning 503 for eleven minutes should not delay the other 2,900 deliveries, and it should not force a re-send to recipients who already received the message. Cron cannot express that. It was never asked to — crontab(5) specifies when a command runs, and the scheduler's contract ends the moment the command is invoked.
The failure mode: at-least-once delivery meeting a non-idempotent side effect
Message brokers give you at-least-once semantics by design, not by accident. RabbitMQ's acknowledgement model is explicit about the consequence: if a consumer's connection drops before it acknowledges, the message is requeued and delivered again, so consumers have to tolerate seeing the same message twice. That is the correct trade — a broker that guaranteed exactly-once across a network partition would have to lie about something.
So the duplicate arrives. The question is what your worker does with it.
The pattern that survives contact with production keeps the delivery ledger in your own database rather than in the broker. Each row is one recipient's copy of one report for one date, carrying a stable key such as report:2026-08-10:clinic-7714, a state column, an attempt counter, and a next-attempt timestamp. The queue message carries the row ID and nothing else. Workers claim, act, then commit the outcome. When the same message is redelivered, the claim query finds a row already marked delivered and the worker does nothing — the second delivery becomes a metric rather than an email. This also solves the harder version of the problem, which is not broker redelivery at all but partial success: the HTTP request to the partner endpoint timed out client-side after the partner had already accepted the payload. The network told you nothing useful. The only thing that resolves that ambiguity is an idempotency key the receiver honors, which is why the outbound request carries one and why HTTP semantics distinguish idempotent methods from safe ones in the first place.
Two failure modes, one ledger. Neither is fixed by picking a different broker.
Claim the row, then send: the worker code that survives a redelivery
The worker below is Go rather than Node.js on purpose: the process that drains a delivery table has different memory and lifecycle characteristics than the API that serves clinician dashboards, and separating them is often cheaper than tuning one runtime for both jobs. The API can stay Node.js. The claim query is the load-bearing part, and it is the same in any language.
package main
import (
"bytes"
"context"
"database/sql"
"fmt"
"net/http"
"os"
"time"
)
// One pending row per call. SKIP LOCKED lets N workers drain the same table
// without serializing on each other, and without two of them claiming one row.
const claimSQL = `
UPDATE deliveries
SET state = 'sending', attempts = attempts + 1, claimed_at = now()
WHERE id = (SELECT id FROM deliveries
WHERE state = 'pending' AND next_attempt_at <= now()
ORDER BY next_attempt_at
FOR UPDATE SKIP LOCKED
LIMIT 1)
RETURNING id, endpoint, payload, idempotency_key, attempts`
type delivery struct {
ID string
URL string
Payload []byte
Key string
Attempts int
}
func drainOne(ctx context.Context, db *sql.DB, client *http.Client) error {
var d delivery
err := db.QueryRowContext(ctx, claimSQL).
Scan(&d.ID, &d.URL, &d.Payload, &d.Key, &d.Attempts)
if err == sql.ErrNoRows {
return nil // run is drained
}
if err != nil {
return err
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, d.URL, bytes.NewReader(d.Payload))
if err != nil {
return err
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Idempotency-Key", d.Key) // identical on every attempt
req.Header.Set("Authorization", "Bearer "+os.Getenv("PARTNER_TOKEN"))
resp, err := client.Do(req)
if err != nil {
return retryLater(ctx, db, d) // transport gave no verdict: assume it may have landed
}
defer resp.Body.Close()
switch {
case resp.StatusCode >= 200 && resp.StatusCode < 300:
_, err = db.ExecContext(ctx,
`UPDATE deliveries SET state='delivered', delivered_at=now() WHERE id=$1`, d.ID)
return err
case resp.StatusCode == http.StatusTooManyRequests, resp.StatusCode >= 500:
return retryLater(ctx, db, d)
default:
_, err = db.ExecContext(ctx,
`UPDATE deliveries SET state='dead', last_status=$2 WHERE id=$1`, d.ID, resp.StatusCode)
return err
}
}
func retryLater(ctx context.Context, db *sql.DB, d delivery) error {
backoff := time.Duration(1<<min(d.Attempts, 6)) * time.Second // 1s .. 64s
_, err := db.ExecContext(ctx,
`UPDATE deliveries SET state='pending', next_attempt_at = now() + $2::interval WHERE id=$1`,
d.ID, fmt.Sprintf("%d seconds", int(backoff.Seconds())))
return err
}
Three details carry the weight. The idempotency key is derived from business identity, not generated per attempt, so a retry is recognizable as a retry by the receiver. A transport error is treated as possibly-delivered rather than not-delivered, which is the conservative reading and the one that keeps the receiver's dedup window doing real work. And a 4xx that is not 429 terminates the row instead of looping forever, because a payload the partner rejects at 06:00 will still be rejected at 06:40.
The cron entry that starts all this stays boring: it enqueues, records a run ID, and exits.
Sizing the drain: latency budget against idle worker cost
Capacity planning for this shape is arithmetic, not intuition. With N deliveries, W workers, and T seconds of realistic per-delivery service time including tail latency and the occasional retry, the floor on drain time is roughly N × T / W. Ten thousand deliveries at 400 ms with eight workers is about eight minutes; the same run with two workers is half an hour. Pick the target from the promise you actually made — "reports arrive by 06:30" is an SLO you can size against, "reports arrive at 06:00" is a wish — then add headroom for the day a partner endpoint is slow and every attempt sits near timeout.
| Where the run lives | Latency profile | Cost profile | Where it fails first |
|---|---|---|---|
| In-process timer inside the API | Fastest to build, no hop | Free until it isn't | Deploys and restarts silently skip the run |
| Cron → direct send loop | Serial; drain time grows with the recipient list | One process, minimal idle spend | One slow recipient consumes the whole invocation window |
| Cron → queue → pool of workers | Tunable; scale W against the SLO | Pay for concurrency you keep warm | Needs an idempotent worker and a delivery ledger |
| Managed workflow orchestrator | Good for dependent multi-step runs | Platform fee plus a new operational concept | Overweight for a single fan-out of independent sends |
The buy-versus-build question underneath that table is mostly about on-call load rather than licence cost. Running your own broker means you own its capacity, its upgrades, and its disk. Renting one means you own an integration and someone else's incident page. Both are defensible; what isn't defensible is choosing the sophisticated option and still leaving the ledger in the broker, because then a queue purge during an incident erases the only record of what was already sent.
Test the duplicate path before you widen the pipe
Test the duplicate path deliberately, in staging, before you raise worker concurrency in production. Reset one row to pending, replay it, and assert that the second attempt produces no second delivery.
psql "$DATABASE_URL" -c "update deliveries set state='pending' where id='dlv_2f9c'"
curl -s -o /dev/null -w '%{http_code}\n' -X POST "$PARTNER_ENDPOINT" \
-H 'Idempotency-Key: report:2026-08-10:clinic-7714' \
-H 'Content-Type: application/json' \
--data @payload.json
psql "$DATABASE_URL" -c "select state, attempts, last_status from deliveries where id='dlv_2f9c'"
Then watch three numbers during the real run: scheduler lag between intended and actual fire time, age of the oldest unclaimed row, and the completion ratio per run ID. Those three separate a slow scheduler from a slow drain from a slow partner, which is the distinction every incident review asks for and few systems can answer.
Rollback has one rule — stop admission first. Pause the cron entry, leave the ledger intact, and decide per run ID what resumes, what gets dead-lettered, and what a human replays by hand. Resuming a paused schedule does not replay the fires you missed, so any backfill has to be an explicit, idempotent action in your own code.
The catch is that this design buys reliability with moving parts. A queue, a worker pool, a ledger table, and a dead-letter policy are four things to monitor where you previously had one scheduled script, and for a low-volume internal digest that trade is not a good fit; stick with the direct loop until a single recipient's failure starts affecting the others. I'm also not convinced every team needs the separate worker runtime — your mileage may vary with how much your platform already runs. The ledger, though, I'd argue for regardless of the message queue you land on. It's the part that lets you answer, at 07:15 on a bad morning, exactly which clinics got their daily report email and which ones didn't.
Top comments (0)