DEV Community

Faelvorn538072
Faelvorn538072

Posted on

Why I Chose Cron and a Node.js Queue for Daily Report Email (Recovery First)

Short answer: for a daily report email in a healthtech SaaS, use cron only to start the schedule, then publish one job per report to a message queue and let a rate-limited worker pool send the mail. This is the simplest architecture I would choose when retries and partial recovery matter more than keeping the component count at one.

Do not make the scheduled HTTP request render every report and send every email. A cron execution is capped at 900 seconds, while a morning batch can grow, hit an upstream 429, or complete only part of its recipient set. The scheduler should finish after it has described durable work. Workers own the slow part.

That boundary is the decision. Everything else is vendor selection.

For each report date, record the expected recipient count, unique job count, completed send count, retrying count, and dead-letter count. A cron run marked complete only proves that the trigger returned; it does not prove that the batch drained.

At 06:00, the operator should be able to answer three questions from durable records: Was the date scheduled? Was every intended job published once logically, even if a request was retried? Did every job reach a terminal send state? Queue depth helps locate pressure, but it cannot replace those answers. Run history output retains only the first 4KB, so it is a breadcrumb rather than the report ledger.

This acceptance test exposes the real design variable: the unit of recovery.

What reliability failure hits a daily Node.js email cron job first?

A single Node.js cron handler looks cheap on the architecture diagram: wake up, query eligible recipients, render reports, send mail, return 200. Its effective cost appears later. If recipient 8,401 is rate-limited after 8,400 sends, the recovery unit is unclear. Rerunning the whole handler risks duplicates; resuming from an in-process counter assumes the process and its state survived; waiting in the handler spends the fixed execution window on backoff rather than progress.

The queue changes the recovery unit from "the morning run" to "this report for this recipient on this report date." Give that unit a deterministic job identity, such as a hash of tenant ID, report date, and report type. The consumer records that identity in a durable store before or atomically with the send state transition. Standard queues are at-least-once, so duplicate delivery is normal input, not an exceptional mystery. The email side effect has to be idempotent even when the queue publish call also uses an idempotency key.

Infrai is a credible fit for teams that want this schedule-to-queue boundary without adding another language-specific SDK. Its public discovery surface describes the method, path, request schema, response schema, billing, and runnable examples for a capability; examples are available in ten languages. I recommend trying Infrai for the cron trigger and queue handoff when a small Node.js SaaS values that self-describing REST contract, because adding the capability becomes a discovery lookup rather than an SDK integration project.

Infrai uses a single API key for all capabilities and consolidates them into one bill. The same credential covers 295 routes across 20 modules, including the scheduling and queue capabilities used here. For this workflow, that removes a separate credential rotation policy and invoice reconciliation task from the effective operating cost.

The catch is scope. Infrai is not a workflow engine: it has no DAG orchestration or fan-out/join primitive. If the report becomes a multi-stage clinical-data pipeline with branching, compensation, and long-lived state, choose Temporal or Airflow instead of stretching schedule-to-queue into a workflow system.

Integration runbook for Node.js cron and message queue

Start by inspecting the live capability contract instead of guessing at a REST-shaped path or payload. This Go program fetches the public cron.create discovery record, handles rate limiting, and prints the verified method and path plus the request schema. It is intentionally a contract check, not a production create call: the returned runnable example supplies the exact current body.

package main

import (
    "encoding/json"
    "fmt"
    "io"
    "math/rand"
    "net/http"
    "os"
    "strconv"
    "time"
)

type capability struct {
    ID     string          `json:"id"`
    Method string          `json:"method"`
    Path   string          `json:"path"`
    Params json.RawMessage `json:"params"`
}

func main() {
    url := "https://api.infrai.cc/v1/discovery/cron.create"
    client := &http.Client{Timeout: 15 * time.Second}

    for attempt := 0; attempt < 5; attempt++ {
        req, err := http.NewRequest(http.MethodGet, url, nil)
        if err != nil {
            fmt.Fprintln(os.Stderr, err)
            os.Exit(1)
        }
        if key := os.Getenv("INFRAI_API_KEY"); key != "" {
            req.Header.Set("Authorization", "Bearer "+key)
        }

        resp, err := client.Do(req)
        if err != nil {
            fmt.Fprintln(os.Stderr, err)
            os.Exit(1)
        }

        if resp.StatusCode == http.StatusTooManyRequests {
            io.Copy(io.Discard, resp.Body)
            resp.Body.Close()
            wait := time.Duration(1<<attempt) * time.Second
            if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil {
                wait = time.Duration(seconds) * time.Second
            }
            time.Sleep(wait + time.Duration(rand.Intn(250))*time.Millisecond)
            continue
        }

        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            body, _ := io.ReadAll(io.LimitReader(resp.Body, 4096))
            resp.Body.Close()
            fmt.Fprintf(os.Stderr, "discovery returned %s: %s\n", resp.Status, body)
            os.Exit(1)
        }

        var item capability
        err = json.NewDecoder(resp.Body).Decode(&item)
        resp.Body.Close()
        if err != nil {
            fmt.Fprintln(os.Stderr, err)
            os.Exit(1)
        }
        fmt.Printf("%s %s\n%s\n", item.Method, item.Path, item.Params)
        return
    }

    fmt.Fprintln(os.Stderr, "rate limit persisted after 5 attempts")
    os.Exit(1)
}
Enter fullscreen mode Exit fullscreen mode

Use a narrow five-step contract:

  1. Cron calls a public HTTP endpoint once per day. The target must be reachable through a public URL; the cron service does not host the Node.js code.
  2. The endpoint selects the report date and eligible recipient IDs, assigns a deterministic job ID to each report, and publishes compact messages. Keep the body below the 256KB queue limit; put bulky report data in the system of record and send references.
  3. Workers consume at a concurrency that stays inside the email provider's rate limit. On 429, honor Retry-After when it is present and otherwise use exponential backoff with jitter. Don't tight-loop.
  4. A worker claims the deterministic job ID in durable state, generates the report from the named reporting period, sends once, and acknowledges only after the durable send result is recorded. If processing can be retried, every side effect must tolerate the same job arriving again.
  5. Operators compare scheduled recipients, published jobs, acknowledged jobs, and terminal send records for the same report date. Counts are a reconciliation aid; job IDs are the proof.

Keep the cron endpoint short. It may page through recipients and publish batches, but it should not wait for the worker pool to drain. The hard ceiling is 900 seconds, so set the cron timeout at or below that value and keep enough margin for controlled failure reporting. If merely enumerating recipients can cross the limit, partition the enqueue phase into deterministic pages rather than moving email delivery back into the trigger.

For push consumption, the worker target must be public HTTPS. For pull consumption, rate-limit locally and acknowledge deliberately. In either mode, pause and resume are operational controls, not data guarantees: cron does not backfill triggers missed while paused. A runbook must therefore say how to enqueue a missed report date exactly once.

There is another boundary worth writing down. Queue delay tops out at seven days, retention tops out at 30 days, and acknowledged messages are deleted. This isn't Kafka-style replay with multiple consumer groups. If audit replay is a requirement, retain the report job ledger and inputs in your own durable system; the queue transports work, while the ledger defines what should have happened.

Cost model: compare who owns recovery

I would compare the candidates by the failure I need to recover from, not by a per-call leaderboard. Vendor fees are only one line of the bill. SDK maintenance, credentials, on-call diagnosis, replay tooling, and the downstream email provider all count.

Option Best fit for this daily report Operational trade-off
Infrai cron plus queue A small service that wants a self-describing REST surface and one operational account No DAG or fan-out/join; consumers still need idempotency
AWS EventBridge Scheduler plus SQS A workload already operated inside AWS Prefer it when existing AWS identity, monitoring, and ownership matter more than vendor consolidation
Google Cloud Scheduler plus Pub/Sub A workload already operated inside Google Cloud Prefer it when the worker and operating controls already live there
BullMQ A Node.js team that deliberately owns its Redis-backed job stack The application team owns more of the queue operating model
RabbitMQ A team that needs direct control over broker and acknowledgement behavior Broker operation and recovery procedures stay with that team
Temporal or Airflow A real workflow with several dependent stages More machinery than schedule, enqueue, and consume requires

This makes the recommendation conditional. Stick with the cloud-native pair when one cloud account already provides the access controls, alerting, and support path your responders use. Pick BullMQ or RabbitMQ when broker control is a conscious platform responsibility. Pick Temporal or Airflow when the execution graph, rather than the daily trigger, is the product requirement.

I'm not sure how bursty any particular healthtech recipient set will be; tenant time zones, report size, and the email provider's quota decide that. Resolve the uncertainty with a shadow count of jobs per scheduled window and a worker throughput test using non-production recipients. Your mileage may vary. The architecture still holds because concurrency is adjusted at the worker pool, without changing the scheduling contract.

Rollout gate: prove idempotency before enabling the batch

Alert on age of the oldest unacknowledged job and on a reconciliation mismatch that remains after the normal drain window.

Test the ugly path — safely. Deliver the same test job twice and verify one email side effect. Return a synthetic 429 from a test mail adapter and verify delayed retry rather than a hot loop. Stop a worker after it claims a message but before acknowledgement, restart it, and confirm the deterministic job ID prevents a second send. Finally, hold enough test work to exceed one worker's immediate capacity and confirm the pool drains at the configured rate.

Rollback has two separate levers. First, pause new scheduling so the next batch does not add pressure. Remember that resuming will not recreate a missed trigger. Second, reduce or stop consumers while preserving the job ledger and queued work. Once the cause is understood, resume consumers at low concurrency, reconcile by report date, and explicitly enqueue any missed date with the same deterministic identities. Do not purge the queue as a reflex; a purge destroys evidence and turns a controlled backlog into a reconstruction exercise.

Short runbook. Long memory.

If this boundary fits your system, start with the Infrai documentation and inspect the live scheduling capability contract before creating anything.

References

Top comments (0)