DEV Community

MitchellCross2134
MitchellCross2134

Posted on

Marketplace Fan-Out: Node.js Cron and Queue Patterns for Scheduled Email Reports

Short answer: use cron to create one durable reporting job, and use a queue when that job fans a shipment update out to many subscribers or needs independent retries. The decision is about the failure boundary, not which tool has the shorter configuration.

I care about this distinction because I have been paged for missed jobs and duplicate deliveries. A daily report email looks small on a diagram. In a marketplace, the report may also carry a shipment-status change to a large subscriber list, and a single slow recipient can turn a tidy timer into a long-running delivery operation.

The report is a business record, not a timer event

Cron is the clock. A queue is the work ledger. The durable report record is the evidence. Keep those responsibilities separate even if the first implementation lives in one Node.js process.

The simplest design is a scheduled command that computes the reporting period, writes a run record, and hands a bounded job to a worker. If report generation and the initial email handoff are predictably short, the scheduled command can perform both steps. When the job fans out shipment updates, however, the command should create independently retryable units: one tenant and reporting period, or one stable subscriber group, depending on the product's delivery contract.

That gives the scheduler a small, inspectable responsibility. It starts work once for a period. The worker owns delivery attempts. The database records what the business considers complete. A queue does not make a message exactly once; it gives the worker a place to retry work whose acknowledgement and side effect may finish at different times. I want the runbook to answer four questions without asking an engineer to infer intent from process logs: which period was due, which shipment update was selected, which subscribers were targeted, and which delivery result was final. The same record also gives support a stable explanation when a subscriber reports a missing email and gives an operator a precise item to replay instead of rerunning every tenant's work.

Use a queue for duration, retry isolation, or fan-out. Don't add one merely because queues sound more production-ready.

What does a marketplace Node.js SaaS need from cron, queue, and a scheduled daily email report?

The bounded failure I keep in the runbook is straightforward. A scheduled run creates a shipment-update job, the worker hands the email to the provider, and the worker loses its acknowledgement before it records success. The broker delivers the job again. A second send follows unless the application has a stable delivery key.

It failed quietly.

The reverse failure is just as important. The scheduler may run late, fail before creating a job, or create a run that has no completed delivery record. A timer log cannot answer the customer question, “Did my report for this period arrive?” The application needs a durable record keyed by the tenant, reporting period, shipment event or digest version, and subscriber.

This is where a small postmortem becomes an architecture rule: schedule intent and delivery completion are different facts. Alert on a missing business record after the agreed deadline, then replay the same period with the same idempotency key. Recovery should converge on one report, not create a new edition every time an operator presses retry. In the shipment fan-out case, the key must be derived from the business identity rather than from the queue message ID: a new message created during recovery still represents the same tenant, period, shipment update or digest version, and subscriber. That identity choice is the difference between a replay and an accidental second notification, so it belongs in the data model, tests, and operator command before production traffic arrives.

The practical consequence is that the queue consumer must tolerate redelivery. RabbitMQ's acknowledgement documentation describes this boundary for a standard message consumer, and the principle applies to any at-least-once delivery system. Acknowledging before the side effect succeeds loses work; acknowledging after the side effect succeeds can repeat work. Idempotency is the control that makes the second outcome safe.

The queue is a governance choice

Write down the delivery contract before choosing the component. “Send the daily report” is not specific enough. Decide whether one subscriber gets one email per reporting period, whether shipment updates may be coalesced, and how late data changes the report. Decide which clock defines the period as well: tenant timezone, marketplace timezone, or UTC.

Then define the recovery path. The operator needs to find a missing period, enqueue it without changing its identity, observe attempts and final status, and stop a retry storm. These are runbook requirements, not optional dashboard polish.

Boundary Cron-only worker Cron plus queue
Time Starts the bounded run Starts job creation
Work Performs the whole operation Performs one retryable unit
Retry Repeats the whole run Retries an individual unit
Duplicate control Application idempotency Application idempotency plus consumer acknowledgement
Operations One process and its logs Scheduler, broker, workers, and metrics

The trade-off is real. A queue improves isolation while adding a broker, worker lifecycle, visibility rules, and another failure surface. A cron-only path is easier to operate while making the whole report the retry unit. For a small report with one recipient and a harmless rerun, that simplicity can be the better engineering choice.

Make replay the normal operator action

The following code keeps the business rule independent of the queue implementation. SendOnce must use durable storage or a mail boundary that honors the key. An in-memory map is not enough; a process restart would erase the only duplicate check.

package delivery

import (
    "context"
    "crypto/sha256"
    "encoding/hex"
    "fmt"
    "time"
)

type ShipmentReport struct {
    TenantID     string
    SubscriberID string
    Period       time.Time
    Address      string
    Body         []byte
}

type Mailer interface {
    SendOnce(ctx context.Context, key, address, subject string, body []byte) error
}

func Deliver(ctx context.Context, mailer Mailer, report ShipmentReport) error {
    if report.TenantID == "" || report.SubscriberID == "" ||
        report.Address == "" || report.Period.IsZero() {
        return fmt.Errorf("invalid shipment report")
    }

    period := report.Period.UTC().Format("2006-01-02")
    key := deliveryKey(report.TenantID, report.SubscriberID, period)
    subject := "Daily shipment report for " + period

    if err := mailer.SendOnce(ctx, key, report.Address, subject, report.Body); err != nil {
        return fmt.Errorf("deliver report %s: %w", key, err)
    }
    return nil
}

func deliveryKey(tenantID, subscriberID, period string) string {
    value := tenantID + "\x00" + subscriberID + "\x00" + period
    sum := sha256.Sum256([]byte(value))
    return hex.EncodeToString(sum[:])
}
Enter fullscreen mode Exit fullscreen mode

The worker should acknowledge only after SendOnce returns success. If it returns an error, the queue policy can retry the same logical job. The mail adapter should retain the successful result for the key, and the reporting record should make reconciliation possible when the network outcome is ambiguous.

There is no universal exactly-once promise between a database and an external mail service. If the mail boundary has no idempotency contract, document the remaining duplicate risk and build a reconciliation state machine. “Exactly once” in an architecture document does not create an atomic transaction.

A Go delivery boundary for the decision

Stay with cron alone when the report is bounded, the recipient set is small, the complete operation can be retried safely, and the team can monitor the process that owns the timer. Keep a durable run record anyway. Small systems still miss work.

Choose cron plus a queue when subscriber fan-out makes one slow delivery hold up every other delivery, when a failed subscriber should retry without rebuilding the report, or when operators need to replay one period and inspect each attempt. Split work around a business-stable key, not an arbitrary batch size that changes between retries.

Your mileage may vary on the exact split. Measure generation time, delivery latency, retry volume, and the deadline promised to subscribers before adding another component. I’m not sure a recipient count alone can tell you when to introduce a queue; the data shape and recovery contract matter more.

This pattern is not suitable when the actual problem is a multi-step workflow with durable waits and joins. Use a workflow engine if the report requires several dependent stages whose state must survive deployments. It is also a poor fit when the team cannot operate a broker or has no need for isolated retries. Stick with a direct scheduled worker until the failure mode justifies the operational cost.

Keep the smallest honest system

The final rule is plain: let cron express when a reporting period becomes due, let a queue carry retryable shipment-update work, and let durable application records prove what each subscriber received. The tools may change. That evidence should not.

References

Top comments (0)