Short answer: put each event into a durable notification outbox, let a queue worker claim small batches, and use cron polling only as a recovery trigger; make every delivery attempt idempotent before tuning email or SMS throughput.
That decision keeps the business transaction separate from an external provider call without pretending the two systems can commit atomically. It also puts the hardest question in the right place: not "How fast can this loop send?" but "What happens when the process dies after the provider accepts a message and before the worker records success?"
The answer is at-least-once processing with explicit deduplication. Exactly-once delivery is not a property a normal application can promise across its database, a queue, an email service, mobile networks, and a recipient's handset. Keep the promise narrower: one durable intent, controlled retries, and one stable idempotency identity per channel delivery.
How should a Node.js queue worker batch email and SMS event notifications?
Write the event and its notification intents in the same database transaction. One event may create several intents because email and SMS have different payloads, consent rules, retry policies, and terminal outcomes. The outbox row should carry an immutable event ID, recipient ID, channel, template version, locale, scheduled time, and a deduplication key. Don't store a pre-rendered message unless audit or legal requirements demand it; templates change, but silently changing the content of an already queued transactional message can be just as dangerous. Pick one rule and record the template version.
A dispatcher reads due outbox rows and publishes only their IDs to the work queue. Workers then claim deliveries using a lease. A lease is important because process termination is ordinary, not exceptional: another worker must be able to recover the row after the lease expires. The worker loads the current row, checks consent and suppression state at send time, renders the pinned template, calls the channel adapter, and records the outcome.
Keep channel concurrency separate. Email acceptance and SMS acceptance have different rate limits, payload rules, and feedback loops, so a single shared semaphore creates accidental coupling. A burst of SMS retries should not starve password-reset email. Likewise, a slow email campaign shouldn't consume the capacity reserved for urgent SMS alerts. Separate queues are optional; separate concurrency budgets are not.
Batching belongs at two boundaries. Claim database rows in bounded groups to reduce lock overhead, then let each channel adapter decide whether a provider-side batch call is appropriate. Those are not the same batch. If a provider accepts only part of a request, every item still needs its own outcome and retry clock.
Small batches are boring.
That is useful. Start with a batch size that fits comfortably inside the queue visibility or lease interval, measure the oldest due item rather than just average throughput, and increase it only after the p95 processing time leaves a wide renewal margin. I'm not sure what margin is right for your provider because rate-limit behavior and latency distributions are deployment facts, not standards; production telemetry should settle it.
Invariants and failure boundaries
An architecture decision record is only helpful if its invariants can be tested. For this system, the first invariant is that committing a business event commits its notification intent. The second is that a worker never sends an intent whose consent or suppression check fails. The third is that a retry reuses the same deduplication key and never invents a second logical delivery. The fourth is that terminal outcomes remain queryable for support and compliance review.
The ambiguous boundary is the provider call. Suppose a worker sends an email, loses its connection before reading the response, and its lease later expires. The application cannot infer acceptance from the timeout. Marking the row successful risks losing the notification; immediately creating a fresh message risks a duplicate. Reusing the original delivery identity gives a provider that supports idempotent submission a chance to collapse the retry. Without that capability, the system remains at-least-once and must tolerate an occasional duplicate. Be honest about it.
Trace that case all the way through before shipping: worker A owns the lease and submits delivery evt-42:recipient-7:email; the remote side may accept it, but A receives no conclusive response and terminates without changing the row. After the lease expires, worker B claims the same row. B must reuse evt-42:recipient-7:email, keep the original template version, and increment the existing attempt rather than insert another delivery. If the channel can deduplicate that identity, B can learn or recreate the accepted result without producing a second logical message. If it cannot, B follows the documented ambiguous-outcome policy and the audit trail retains both attempts under one delivery. This is why a random request ID generated inside send() is useless: it changes at exactly the moment stability matters. Test the sequence by terminating the process on each side of the network call, then inspect stored state rather than trusting worker logs.
Provider acceptance is not recipient delivery. Email can be accepted and later bounce, while SMS can be accepted upstream and later receive a delivery-status update. Model accepted, delivered, temporary_failure, permanent_failure, and suppressed as distinct states. Do not turn a delayed receipt into an immediate retry: that is a good way to send the same OTP twice and train users to trust the wrong code.
Duplicates hurt.
Email authentication is another boundary. DKIM, defined by RFC 6376, lets a signing domain attach a cryptographic signature to selected message headers and the body; a verifier retrieves the public key through DNS and validates the signature. It helps establish responsibility for a signed message, but it does not make content wanted, guarantee inbox placement, or replace suppression handling. Sign consistently, preserve the signed content in transit, and monitor authentication results alongside bounces and complaints.
SMS payload size affects both operations and user experience. A GSM-7 message fits 160 characters as one segment and 153 characters per segment when concatenated. UCS-2 allows 70 characters in one segment and 67 per concatenated segment. One emoji or unsupported character can change the encoding and segment count — a tiny copy edit with a large fan-out. Validate rendered text before enqueueing, record encoding and segment count, and keep OTP text short enough that carrier-added material or localization does not surprise the system. I've learned to inspect encoding before blaming a queue for an apparent delivery gap; queue latency and handset delivery are different clocks.
The failure policy should be explicit:
- Retry timeouts, connection failures, and documented temporary provider outcomes with exponential backoff plus jitter.
- Stop on invalid destinations, revoked consent, suppression matches, and other permanent outcomes.
- Cap attempts and move exhausted deliveries to a reviewable dead-letter state; never spin forever.
- Rate-limit by channel and destination where abuse or OTP flooding is possible.
- Expose queue age, claim latency, attempt count, acceptance rate, final delivery rate, bounce or failure class, and suppression count without putting message bodies or OTP values in logs.
Decision table
The trigger is less important than the durable state behind it. These options differ mainly in latency, operational complexity, and how they recover from missed work.
| Option | Normal trigger | Recovery behavior | Main trade-off | Best fit |
|---|---|---|---|---|
| Transactional outbox plus queue | Commit produces an outbox row; dispatcher publishes its ID | Poller republishes due, unclaimed rows | More components and state transitions | High-volume or latency-sensitive notifications |
| Transactional outbox plus cron polling | Scheduled worker claims due rows directly | The next run sees rows left unclaimed after lease expiry | Latency follows the schedule; overlapping runs need leases | Moderate traffic where minute-scale delay is acceptable |
| Direct send after commit | Request handler calls the channel | Application-specific retry, often disconnected from the original intent | Simple path, weak crash recovery and request latency coupling | Low-stakes internal notices where loss or duplication is acceptable |
For most customer-facing event notifications, choose the first option and retain a low-frequency polling reconciler. The queue supplies prompt work distribution. The database remains the source of truth. Cron is the seat belt — it looks for due rows that were never published or whose lease expired, rather than becoming a second independent sending path.
The catch is operational weight. A queue, a dispatcher, leases, replay tooling, and delivery-state metrics are not suitable when a team sends a handful of noncritical internal messages and can manually resend them. In that case, stick with a database-backed cron worker. It preserves durable intent and clear retries without requiring a separate broker. If the database cannot support short indexed claims without contention, a dedicated queue becomes attractive earlier.
Critical path in code
The following Python expresses the worker contract even if the production service is Node.js. The important part is the state transition, not the language: claim IDs atomically, handle each delivery independently, and acknowledge queue work only after durable outcome recording. store and channel are deliberately generic interfaces rather than hidden vendor clients.
from dataclasses import dataclass
from datetime import datetime, timezone
from enum import Enum
from typing import Protocol
class ResultKind(Enum):
ACCEPTED = "accepted"
TEMPORARY_FAILURE = "temporary_failure"
PERMANENT_FAILURE = "permanent_failure"
@dataclass(frozen=True)
class Delivery:
delivery_id: str
event_id: str
recipient_id: str
channel: str
template_version: str
deduplication_key: str
attempt: int
@dataclass(frozen=True)
class SendResult:
kind: ResultKind
provider_message_id: str | None = None
reason_code: str | None = None
class DeliveryStore(Protocol):
def load_claimed(self, delivery_id: str) -> Delivery | None: ...
def is_allowed(self, delivery: Delivery) -> bool: ...
def mark_suppressed(self, delivery_id: str) -> None: ...
def mark_accepted(self, delivery_id: str, external_id: str | None) -> None: ...
def schedule_retry(self, delivery_id: str, attempt: int) -> None: ...
def mark_permanent_failure(self, delivery_id: str, reason: str | None) -> None: ...
class Channel(Protocol):
def send(self, delivery: Delivery, idempotency_key: str) -> SendResult: ...
def process_delivery(
delivery_id: str,
store: DeliveryStore,
channel: Channel,
max_attempts: int = 8,
) -> None:
delivery = store.load_claimed(delivery_id)
if delivery is None:
return
if not store.is_allowed(delivery):
store.mark_suppressed(delivery.delivery_id)
return
result = channel.send(delivery, delivery.deduplication_key)
if result.kind is ResultKind.ACCEPTED:
store.mark_accepted(delivery.delivery_id, result.provider_message_id)
elif result.kind is ResultKind.PERMANENT_FAILURE:
store.mark_permanent_failure(delivery.delivery_id, result.reason_code)
elif delivery.attempt >= max_attempts:
store.mark_permanent_failure(delivery.delivery_id, "attempts_exhausted")
else:
store.schedule_retry(delivery.delivery_id, delivery.attempt + 1)
def utc_now() -> datetime:
return datetime.now(timezone.utc)
In a real worker, schedule_retry should calculate backoff in one place, and the claim query should use an indexed due time plus a lease owner and expiry. Keep the database transaction around claim or outcome updates short. Never hold it open across the network call. The delivery record's stable key crosses that gap.
Node.js workers should also bound promise concurrency rather than pass an entire batch to Promise.all. A batch of 500 is a database transport choice, not permission to open 500 outbound connections. Process a fixed number concurrently, refresh leases for genuinely long jobs, and let backpressure leave the remaining IDs queued.
Deployment deserves the same care as code. Stop claiming new work on shutdown, allow active sends a bounded drain period, and leave unfinished leases to expire. During a template rollout, pin new intents to the new version while old intents retain their original version. During a channel incident, pause that channel's claims rather than repeatedly consuming attempt budgets. Logs should correlate event ID, delivery ID, attempt, and external message ID, but avoid addresses, phone numbers, message bodies, and OTPs.
Rejected option: cron as the sender of record
A cron-only loop that selects pending rows, sends them, and then marks them sent looks sufficient. The crash window makes it incomplete unless rows are claimed with leases and retries preserve identity. Overlapping scheduler runs can otherwise select the same rows, while a long provider call can exceed the interval and amplify the overlap. Adding a global lock reduces overlap but also turns one stuck run into delayed work.
This option still has a valid use case. Keep it for moderate, delay-tolerant workloads when the database can claim rows atomically, every row has a lease, and operators can see queue age and dead letters. It is also a reasonable first implementation when introducing a broker would exceed the team's operating capacity. The standard should not be architectural fashion; it should be whether the simpler design preserves the invariants under termination, retry, and concurrent execution.
Do not use cron frequency as a throughput control. Concurrency and rate limits should be explicit, because making the schedule run every few seconds can create overlapping workers without increasing safe capacity. Conversely, a five-minute schedule may be entirely acceptable for a daily digest and unacceptable for an OTP. Different event classes deserve different service-level objectives and queues, even if they share storage and adapters.
The final acceptance test is failure-oriented: terminate a worker immediately before and after submission, run two pollers at once, revoke consent while an item waits, inject a temporary channel failure, deliver a late status callback, and render localized SMS containing non-GSM characters. Confirm that state converges, attempts stop, sensitive content stays out of telemetry, and the oldest due notification remains visible. Happy-path throughput comes later.
References
- RFC 6376, DomainKeys Identified Mail (DKIM): https://datatracker.ietf.org/doc/html/rfc6376
- SMS character limits and segmentation (GSM-7/UCS-2): https://www.twilio.com/docs/glossary/what-sms-character-limit
Top comments (0)