DEV Community

dawn li
dawn li

Posted on

Queue-Based Daily Scheduled Email: A Node.js Example for Fintech Deadlines

#queuebased #daily #scheduled #email

Short answer: for a queue-based daily scheduled email in Node.js, schedule a small, durable due-record for each business deadline, then let a worker publish an email job to a queue. Keep the scheduler's transaction separate from email delivery, acknowledge a job only after the provider accepts it, and make the send operation idempotent. That arrangement is less clever than putting a timer in a web process, which is exactly why it survives restarts.

A renewal reminder is a delivery-guarantee problem disguised as a daily report email. The useful question is not whether a job ran at 09:00. It is whether the business received one reminder for the intended deadline, and whether an operator can explain a late or duplicated message.

What does the deadline actually mean?

Start with the bill. Retaining every scheduler tick, rendered email, queue delivery, and provider response forever creates a large operational history, but most of that history does not improve delivery. The dominant record is the durable business fact: account 1842 has a renewal date, a reminder policy, a timezone, and a delivery state. Store that first.

For a daily sweep, the query can be as plain as next_due_at <= now with a lease or claim that prevents two schedulers from selecting the same row. A daily cadence is an implementation detail; the deadline and timezone are business data. A job due at 09:00 in New York should not be interpreted as 09:00 UTC because a process happened to run there.

The retention decision is deliberate. Keep the renewal record and a compact delivery ledger long enough to answer a dispute, retain provider identifiers where policy permits, and discard rendered bodies and verbose polling traces after their diagnostic value expires. That saves storage, but it also means a later investigation may prove that a message was accepted without reconstructing its exact HTML.

How should Node.js run a queue-based daily scheduled email?

The scheduler should claim a due reminder and enqueue a stable message identifier. It should not wait for SMTP or an email API. The worker should load the current business record, check whether the reminder was already completed, render the message, submit it, and acknowledge the queue message only after the submission result is durable enough for the chosen guarantee.

Here is the shape of that worker in Python. The interfaces are intentionally generic; the same boundary can be implemented by a Node.js worker without changing the state transitions.

from dataclasses import dataclass


@dataclass(frozen=True)
class ReminderJob:
    reminder_id: str
    account_id: str
    due_at: str


def handle(job, reminders, mailer, message):
    reminder = reminders.get(job.reminder_id)

    if reminder.status == "sent":
        message.ack()
        return

    if reminder.account_id != job.account_id:
        message.reject(requeue=False)
        return

    result = mailer.submit(
        idempotency_key=f"renewal:{job.reminder_id}",
        recipient=reminder.recipient,
        subject=reminder.subject,
        body=reminder.body,
    )

    reminders.mark_submitted(job.reminder_id, result.provider_id)
    message.ack()
Enter fullscreen mode Exit fullscreen mode

The important line is not submit. It is the ordering around ack. If the process acknowledges first and crashes before recording submission, the queue believes the work is finished. If it records submission and crashes before acknowledgement, redelivery is expected, so the idempotency key must make the second attempt harmless. A queue gives delivery semantics, not magically exactly-once email.

One identifier.

No magic.

Which failures matter more than the timer?

I first treated the scheduler as the risky component. The more consequential boundary is the gap between an accepted email and the local ledger. A timeout can mean that the provider rejected the message, accepted it, or accepted it while the response disappeared. Retrying blindly risks duplicates; never retrying risks a missed reminder. This is where a tidy diagram usually hides the difficult state: the worker may have done the external action while still looking idle locally, and a queue redelivery cannot distinguish those outcomes without an idempotent identity and a ledger lookup.

Failure Observable state Conservative response
Worker exits before acknowledgement Message is redelivered Re-read state and reuse the same idempotency key
Mail submission times out Acceptance is unknown Retry only with provider-side deduplication or an operator review state
Scheduler crashes after claiming Lease remains active Let the lease expire and record the recovery latency
Due time crosses a daylight-saving change Local clock and UTC differ Persist timezone-aware due data and test the transition explicitly

Poison messages deserve a separate path. A malformed recipient or missing account should not consume infinite retries, and a transient provider outage should not be discarded as a permanent error. Set a bounded retry policy, preserve the reason, and route exhausted messages to a review queue whose contents are monitored rather than forgotten.

Queue acknowledgements are transport mechanics, not evidence that a human read the mail. Delivery to an inbox, filtering, and user action are later states with different evidence. State that boundary plainly in the design review.

How do you test the guarantee before production?

Test transitions, not just the happy-path clock. Freeze time around midnight and daylight-saving changes. Run two scheduler instances against the same due row. Kill a worker immediately before acknowledgement, then repeat the redelivered job. Return a timeout after the mail service has accepted the request. The expected result is one business reminder and an auditable ambiguous state when the system cannot prove more.

Metrics should describe that contract: due reminders, claimed reminders, queue age, attempts per reminder, accepted submissions, rejected submissions, expired leases, and messages awaiting review. Alert on queue age and deadline lateness, not merely on process uptime. A process can be healthy while every reminder is waiting behind a poisoned message.

The smallest reliable design is therefore a durable due record, a bounded claim, a queue message carrying an idempotent identity, and a ledger that distinguishes submitted from confirmed. Keep less history where it has no operational value, but retain enough state to explain a missed deadline. That is the trade-off I would put in front of a fintech reviewer: predictable recovery and explicit uncertainty are more valuable than a timer that looks punctual in a dashboard.

Further reading

Top comments (0)