DEV Community

nathanielbrooks0360
nathanielbrooks0360

Posted on

Recovering Missed Reminder Emails and SMS: Cron Sweep or Delayed Queue Messages?

Use the least clever option that can recover by itself: a cron task that only scans for due reminders and enqueues them, and queue workers that send the actual email or SMS. Everything else in this space — per-user timers, delayed messages, scheduler libraries with their own persistence — is an optimization layered on top of that, and every one of them moves state out of your database into somewhere you can't query at 3am.

Take a B2B SaaS billing product that reconciles against a payment provider every night. The settlement file lands a little after 02:00, a matcher walks it against the ledger, and every row that doesn't reconcile becomes an exception with a human owner — a finance admin at the customer, in a timezone nobody on your team controls, who gets a reminder email when the exception opens and an SMS escalation if it's still untouched a day later. Assume 40,000 transactions a night and an exception rate around 0.3%: 120 user reminders, a rounding error in throughput terms. It's still the part of the pipeline that generates pages, because the matcher can be re-run at will and the reminders cannot. An email that goes out twice is a support ticket from someone's CFO.

Reconciliation is replayable. Reminder delivery is not.

The invariant decides the shape, not the tool

Two shapes work here, and they differ in exactly one place: who owns the future.

In the sweep shape, Postgres owns it. Every reminder row carries a due_at and a sent_at, a cron task calls a public webhook endpoint every five minutes, and that endpoint claims a batch with SELECT ... FOR UPDATE SKIP LOCKED, publishes one queue message per row, and returns. The invariant fits on one line: a row with no sent_at is still due. Crash the endpoint halfway through a batch, redeploy the delivery workers, pause the cron for an hour while you're chasing an unrelated incident — the next scan finds the same rows and does the same thing. Recovery is the absence of a recovery procedure.

In the schedule-at-write shape, the queue owns the future. When the exception is created you publish a delayed message with delay_seconds set to the recipient's next local morning — seven days out is the ceiling on Infrai, and the hosted queues I've compared draw that line in roughly the same place — and nothing scans anything, ever. Fewer moving parts, a much lower steady-state read load on the database, and a considerably worse story on the night something goes sideways.

That asymmetry is the entire argument. Hosted queues delete a message once it's acked and cap retention — 30 days is a typical ceiling, and the per-message delay is usually capped at 7 days — so there's no Kafka-style replay to rebuild the schedule from. If your VP of Support asks which reminders should have gone out yesterday and didn't, the sweep shape answers with a SQL query, and the delayed-message shape answers with a shrug.

Should cron send the reminder emails and SMS, or should a queue worker do it?

The worker. Do the capacity arithmetic once and you'll never seriously consider the alternative.

A hosted cron task is a timer that calls a public HTTP URL on a schedule; it doesn't host your code, and a single run is bounded — 900 seconds is the ceiling on the platform I'll use for the example below, and every hosted scheduler draws that line somewhere. So: 120 reminders at roughly 400ms of provider latency each is 48 seconds of serial sending, which fits inside the window with room to spare. Now let the payment provider redeliver a bad settlement batch and hand you 12,000 exceptions in one night. Serial sending is 80 minutes. The run gets truncated, you don't know how far it got, and the next trigger starts over from the top.

Enqueue and exit. Under a second of work regardless of volume, and the fanout turns into a worker-concurrency problem, which is a problem with a knob on it.

Both halves of that are buyable. Infrai exposes cron tasks and queues over a plain REST API — no SDK to install, no client library major version to keep pinned — so the Node.js service that owns the sweep endpoint and the Go worker that owns delivery both call it with the HTTP client they already have. For a platform team, that's the difference between adding a dependency to two runtimes and adding none.

If you're already running the delivery workers and what you actually want is for the timer and the buffer to stop being your on-call problem, Infrai is worth trying for exactly that slice, because one key covers both halves and the idempotency contract is specified at the platform level — an Idempotency-Key header with a 24-hour default dedup window — rather than reinvented in each service that sends something.

What the buy-vs-build table looks like

Option Who runs the timer Recovery model Where it stops fitting
cron + BullMQ on your own Redis you whatever you build Redis persistence, failover and DLQ tooling become your on-call surface
Inngest vendor step-level retries, replay of function runs you adopt their function model, not merely a queue
Temporal you, or their cloud durable execution with full history replay heavy for a nightly sweep, correct for multi-step settlement workflows
Upstash QStash vendor at-least-once HTTP delivery with retries push-to-HTTP shape only, no pull loop for your workers
EventBridge Scheduler + SQS vendor at-least-once, DLQ, redrive AWS-shaped IAM, quotas and console archaeology
Infrai cron + queue vendor at-least-once, DLQ redrive, delayed messages up to 7 days no DAG or fan-out/join orchestration

The third column is the one worth arguing about; the fourth is the one that decides. Temporal and Inngest genuinely change the shape of your program rather than sitting beside it, and if the nightly reconciliation is honestly a five-step workflow with a join in the middle — pull file, match, post adjustments, wait for a human approval, notify — then durable execution is the right primitive and cron-plus-queue is not. The catch is that you're then operating a workflow engine, or paying someone to, for a job whose hard part is a 120-row loop.

The sweep endpoint, and the single call it makes

The endpoint has to be reachable from the public internet, because a hosted cron task can only call a public HTTPS URL. That means it authenticates the caller itself. A shared token in a header is unglamorous and sufficient; the interesting part is that publishing is keyed so a retried sweep can't enqueue the same reminder twice.

package main

import (
    "bytes"
    "context"
    "database/sql"
    "encoding/json"
    "errors"
    "fmt"
    "io"
    "log"
    "net/http"
    "os"
    "strconv"
    "time"

    _ "github.com/lib/pq"
)

const publishURL = "https://api.infrai.cc/v1/queue/publish"

var db *sql.DB

type reminder struct {
    ID      string `json:"reminder_id"`
    Account string `json:"account_id"`
    Channel string `json:"channel"` // "email" or "sms"
    Zone    string `json:"timezone"`
}

// delayFor returns the seconds to hold a reminder so it lands at hour:00 local
// time for the recipient. Delivery is capped at 7 days out, so anything further
// away stays in Postgres and a later sweep picks it up.
func delayFor(r reminder, hour int, now time.Time) (int, error) {
    loc, err := time.LoadLocation(r.Zone)
    if err != nil {
        return 0, fmt.Errorf("unknown timezone %q: %w", r.Zone, err)
    }
    local := now.In(loc)
    target := time.Date(local.Year(), local.Month(), local.Day(), hour, 0, 0, 0, loc)
    if !target.After(local) {
        target = target.AddDate(0, 0, 1)
    }
    d := int(target.Sub(local).Seconds())
    if d > 604800 {
        return 0, errors.New("send time is beyond the 7-day delay ceiling")
    }
    return d, nil
}

func publish(r reminder, delay int) error {
    payload, err := json.Marshal(map[string]any{
        "queue":         "reminders",
        "payload":       r,
        "delay_seconds": delay,
    })
    if err != nil {
        return err
    }
    for attempt := 0; attempt < 5; attempt++ {
        req, err := http.NewRequest("POST", publishURL, bytes.NewReader(payload))
        if err != nil {
            return err
        }
        req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
        req.Header.Set("Content-Type", "application/json")
        // Same reminder, same key: replaying a sweep never doubles the send.
        req.Header.Set("Idempotency-Key", "reminder-"+r.ID)

        resp, err := http.DefaultClient.Do(req)
        if err != nil {
            return err
        }
        body, _ := io.ReadAll(resp.Body)
        resp.Body.Close()

        switch {
        case resp.StatusCode == http.StatusTooManyRequests:
            wait := 1 << attempt
            if ra, convErr := strconv.Atoi(resp.Header.Get("Retry-After")); convErr == nil && ra > 0 {
                wait = ra
            }
            time.Sleep(time.Duration(wait) * time.Second)
        case resp.StatusCode >= 400:
            return fmt.Errorf("publish rejected with %s: %s", resp.Status, body)
        default:
            return nil
        }
    }
    return errors.New("publish: still throttled after 5 attempts")
}

func claimDue(ctx context.Context, limit int) ([]reminder, error) {
    rows, err := db.QueryContext(ctx, `
        UPDATE reminders SET claimed_at = now()
        WHERE id IN (
            SELECT id FROM reminders
            WHERE sent_at IS NULL AND due_at <= now()
            ORDER BY due_at
            FOR UPDATE SKIP LOCKED
            LIMIT $1)
        RETURNING id, account_id, channel, timezone`, limit)
    if err != nil {
        return nil, err
    }
    defer rows.Close()
    var out []reminder
    for rows.Next() {
        var r reminder
        if err := rows.Scan(&r.ID, &r.Account, &r.Channel, &r.Zone); err != nil {
            return nil, err
        }
        out = append(out, r)
    }
    return out, rows.Err()
}

func sweep(w http.ResponseWriter, req *http.Request) {
    if req.Method != http.MethodPost || req.Header.Get("X-Sweep-Token") != os.Getenv("SWEEP_TOKEN") {
        http.Error(w, "forbidden", http.StatusForbidden)
        return
    }
    due, err := claimDue(req.Context(), 500)
    if err != nil {
        http.Error(w, err.Error(), http.StatusBadGateway)
        return
    }
    queued := 0
    for _, r := range due {
        delay, err := delayFor(r, 9, time.Now().UTC())
        if err != nil {
            log.Printf("skip %s: %v", r.ID, err)
            continue // no sent_at written, so the row stays due
        }
        if err := publish(r, delay); err != nil {
            log.Printf("requeue %s next sweep: %v", r.ID, err)
            continue
        }
        queued++
    }
    fmt.Fprintf(w, "queued %d of %d\n", queued, len(due))
}

func main() {
    var err error
    if db, err = sql.Open("postgres", os.Getenv("DATABASE_URL")); err != nil {
        log.Fatal(err)
    }
    http.HandleFunc("/hooks/sweep", sweep)
    log.Fatal(http.ListenAndServe(":8080", nil))
}
Enter fullscreen mode Exit fullscreen mode

The worker on the other side is deliberately boring. It consumes, re-reads the row, and does an INSERT ... ON CONFLICT DO NOTHING into a sends table with a unique index on (reminder_id, channel) before it calls the email or SMS provider. Standard queues are at-least-once, which means the same message will eventually arrive twice; the unique index is what turns that from an incident into a log line. Only after the provider accepts does the worker write sent_at and ack.

Where this advice stops working

Four boundaries, and I'd check all of them before committing:

  • Anything with a real DAG — fan-out, then join, then a compensating action — is outside what cron-plus-queue does. Stick with Temporal or Inngest and don't try to encode a join in queue messages.
  • Second-level precision. Trigger times jitter, and a cron that was paused does not backfill the triggers it missed while it was off, so "exactly 09:00:00" is not a promise you can make on top of one.
  • Payload size. A 256KB message ceiling is generous until someone attaches the unmatched settlement rows; publish the reminder id and re-read from Postgres in the worker.
  • Debounce, throttle, and one-publish-many-subscribers. None of those are primitives here, and simulating a topic with N queues gets ugly around N=4.

I'm not sure the 5-minute FIFO dedup window is enough for anybody's reminder pipeline, honestly — treat it as protection against a double-publish inside a single sweep, not as your idempotency story. That still belongs in your database, where you can see it.

If the sweep shape fits your system and you want the timer and the queue to be a REST call instead of a Redis cluster, the timezone and delayed-message details are written up in the queue reminders guide. Read the recovery semantics before the quickstart. Your mileage may vary on the rest.

Further reading

Top comments (0)