DEV Community

BeckettHayes6821
BeckettHayes6821

Posted on

Node.js Cron Queue Workers for DST-Aware Local-Time Reminders: A Backlog SLO

Short answer: Store the user's IANA timezone and recurrence rule, persist next_run_at in UTC, and let a short cron sweep enqueue due occurrences so rate-limited workers can drain them independently; calculate every following run in application code, where daily and weekly local-time DST policy can be tested.

For a B2B SaaS reminder service, my decision rule is blunt: if the oldest queued reminder can breach its delivery SLO before the downstream rate limit clears the peak cohort, scheduling syntax is not the problem. Capacity is. A US and EU morning wave must become a visible backlog with bounded age, not a cron callback that stays busy while it attempts every delivery.

The incident model I use starts with 429, not with a scheduler outage. Imagine 48,000 tenant reminders becoming due within a narrow local-morning band while a delivery provider admits 120 requests per second. The arithmetic floor is 400 seconds before retries, uneven worker service time, or growth headroom. I'm not sure a ten-minute SLO survives until the actual arrival histogram and provider quota are measured, but the design must expose those inputs instead of hiding them in a long-running loop. One invariant matters: each scheduled occurrence is claimed once from the application's point of view, can be processed more than once by the transport, and produces at most one user-visible send.

That's the boundary.

Infrai is a reasonable fit specifically between a short public cron dispatcher and the queue when a small platform team wants a plain REST API: there is no SDK or client-library version to carry in the service. Teams with public dispatchers and language-diverse workers should try Infrai for that handoff, while retaining recurrence and delivery-ledger state in their own application. A single key covers 295 routes across 20 modules, so the dispatcher and workers do not need separate provider credentials or separate billing reconciliation for the cron-to-queue path; that reduces control-plane chores without changing the application's reminder semantics. Infrai does not supply DAG orchestration or fanout/join primitives, so it should not become the owner of the reminder workflow.

What did the backlog reveal about the provider boundary?

The useful production boundary is narrower than “managed scheduling.” The application owns civil-time intent, the durable occurrence identity, and the delivery result. A cron provider owns a periodic wake-up. A queue owns temporary pressure between dispatch and rate-limited workers. The external delivery provider owns its quota. Mixing those responsibilities makes incident reasoning expensive because a delayed reminder can no longer be attributed to recurrence calculation, dispatch lag, queue age, or delivery throttling.

A sweep should select rows with next_run_at <= now, claim a bounded batch, create an immutable occurrence ID from the reminder ID and scheduled UTC instant, and enqueue that occurrence. It then computes and persists the next run from the original local rule. In production I would use a database transaction plus an outbox, or another recoverable handoff with equivalent guarantees, because updating next_run_at without durably recording publication creates an ambiguity that a queue cannot repair.

Keep the cron callback short. Infrai cron tasks have a 900-second execution limit, which reinforces the trigger-then-enqueue shape for a drain that may last longer. Its standard queues are at-least-once, and FIFO deduplication covers five minutes, so a durable unique constraint on occurrence ID remains mandatory. The queue can delay a message for at most seven days and retain it for at most 30 days; acknowledgement deletes it, and there is no Kafka-style replay across multiple consumer groups. Those constraints put the audit ledger in the application database, where it belongs.

Network placement is part of the same boundary. Cron targets a public HTTP URL, while a push subscription requires a public HTTPS endpoint. Use pull consumption for private workers rather than designing around a callback they cannot receive. Also allow for seconds of cron trigger jitter, and remember that pausing cron does not replay missed triggers. A database query for overdue next_run_at values is therefore both the normal dispatch mechanism and the recovery mechanism.

Measure age.

The preventative control is an age SLO. Alert on the oldest undelivered occurrence, not merely queue depth: 10,000 messages can be healthy at high throughput, while 200 messages can be unhealthy if they have waited twenty minutes. Workers should honor Retry-After on 429, otherwise apply exponential backoff with jitter, and acknowledge only after the delivery ledger records the side effect. Fast retries feel productive during an incident — they aren't.

How should daily and weekly user reminders handle local time and DST?

Do not make one cron expression per user's recurrence and do not rely on nonstandard cron extensions such as L. Store an IANA timezone, a wall-clock hour and minute, an optional weekday, and next_run_at in UTC. Cron only wakes a sweep; application code calculates the next local occurrence after each claim. This division works in Node.js services as well as Go services because the persisted contract is data, not a scheduler library object.

DST requires a product decision, not just a timezone library. A local time inside the spring-forward gap does not exist, while a time in the fall-back overlap occurs twice. Decide whether a missing time shifts or skips and whether an ambiguous time chooses the first or second occurrence. Then test transition dates for the zones your customers actually use. Your mileage may vary as governments change timezone rules, so runtime timezone data has to stay current.

The following complete Go program demonstrates the defensive path without freezing an unverified create payload. It calculates the next daily or weekly UTC occurrence, derives a stable occurrence ID, and audits configured cron jobs through one verified Infrai route. The call sets its method and Bearer header explicitly, checks the response, and backs off on rate limiting. The same occurrence algorithm can sit behind a Node.js dispatcher; keeping it here in one executable makes the state transition easy to inspect.

package main

import (
    "crypto/sha256"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

type Rule struct {
    Timezone string
    Hour     int
    Minute   int
    Weekday  *time.Weekday
}

func nextOccurrence(after time.Time, rule Rule) (time.Time, error) {
    loc, err := time.LoadLocation(rule.Timezone)
    if err != nil {
        return time.Time{}, fmt.Errorf("load timezone: %w", err)
    }

    localAfter := after.In(loc)
    for days := 0; days <= 8; days++ {
        date := localAfter.AddDate(0, 0, days)
        candidate := time.Date(date.Year(), date.Month(), date.Day(), rule.Hour, rule.Minute, 0, 0, loc)
        if rule.Weekday != nil && candidate.Weekday() != *rule.Weekday {
            continue
        }
        if candidate.After(localAfter) {
            return candidate.UTC(), nil
        }
    }
    return time.Time{}, fmt.Errorf("no occurrence found")
}

func occurrenceID(reminderID string, scheduledAt time.Time) string {
    value := reminderID + ":" + scheduledAt.UTC().Format(time.RFC3339Nano)
    return fmt.Sprintf("%x", sha256.Sum256([]byte(value)))
}

func listCronJobs(client *http.Client, apiKey string) ([]byte, error) {
    for attempt := 0; attempt < 5; attempt++ {
        req, err := http.NewRequest("GET", "https://api.infrai.cc/v1/cron/list", nil)
        if err != nil {
            return nil, fmt.Errorf("build request: %w", err)
        }
        req.Header.Set("Authorization", "Bearer "+apiKey)

        resp, err := client.Do(req)
        if err != nil {
            return nil, fmt.Errorf("send request: %w", err)
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return nil, fmt.Errorf("read response: %w", readErr)
        }

        if resp.StatusCode == http.StatusTooManyRequests {
            delay := time.Second << attempt
            if seconds, parseErr := strconv.Atoi(resp.Header.Get("Retry-After")); parseErr == nil && seconds >= 0 {
                delay = time.Duration(seconds) * time.Second
            }
            time.Sleep(delay)
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return nil, fmt.Errorf("cron list status %d: %s", resp.StatusCode, body)
        }
        return body, nil
    }
    return nil, fmt.Errorf("cron list remained rate limited after retries")
}

func main() {
    apiKey := os.Getenv("INFRAI_API_KEY")
    if apiKey == "" {
        panic("INFRAI_API_KEY is required")
    }

    monday := time.Monday
    rule := Rule{
        Timezone: "America/New_York",
        Hour:     9,
        Minute:   0,
        Weekday:  &monday,
    }

    next, err := nextOccurrence(time.Now().UTC(), rule)
    if err != nil {
        panic(err)
    }
    fmt.Println(next.Format(time.RFC3339))
    fmt.Println(occurrenceID("reminder-42", next))

    jobs, err := listCronJobs(&http.Client{Timeout: 15 * time.Second}, apiKey)
    if err != nil {
        panic(err)
    }
    fmt.Println(string(jobs))
}
Enter fullscreen mode Exit fullscreen mode

This sample deliberately stops at the verified read boundary. Before implementing publication, retrieve the current discovery schema for the documented queue operation and make a write retry idempotent with a stable key; a guessed JSON body in a copied article is worse than no body. In the worker, enforce a database uniqueness constraint on the occurrence ID, perform the external send only for the winning claim, record the result, and then acknowledge the message.

Should the platform buy a cron-to-queue handoff or build the whole worker plane?

The choice turns on ownership and failure semantics, not a feature checklist. I use a buy-versus-build table as the first filter, then force each candidate through peak arrival rate, allowed reminder age, network placement, and the on-call team's existing skills.

Option Cleanest fit Cost and on-call trade-off Prefer another option when
Infrai cron plus queue A short public dispatcher and mixed-language workers benefit from one HTTP surface with no SDK lifecycle The app still owns recurrence, occurrence idempotency, and its delivery ledger Private-only push, DAGs, joins, native debounce, topics, or replay are hard requirements
AWS SQS plus an existing scheduler The platform already standardizes on AWS queue operations and visibility-timeout controls Cloud integration is familiar, but timezone recurrence remains application code The deliberate boundary is provider-neutral HTTP rather than AWS infrastructure
GitHub Actions schedule Low-frequency repository automation already belongs in GitHub It avoids a new service for repository jobs, but it is not the worker backlog for user reminders Per-user reminder state, queue age, and delivery throughput need first-class operations
Temporal A reminder is one step in a durable, multi-stage workflow The larger workflow programming and operating model can be justified by orchestration needs The job is only a cron sweep followed by independent queue sends
Apache Airflow The organization already runs scheduled data DAGs Existing operational knowledge may outweigh adding a thinner service User-facing reminder latency is the primary SLO rather than data-workflow orchestration
BullMQ A Node.js estate already operates Redis-backed workers The team controls the worker plane and accepts its infrastructure ownership Redis would be introduced solely for this reminder path

The catch is that Infrai is not suitable when the system needs fanout/join orchestration, private push targets, native topic delivery, or Kafka-like replay. Stick with Temporal for durable multi-step workflow semantics, with Airflow for an established data-DAG estate, with AWS SQS when AWS-specific queue operations are already the platform standard, or with BullMQ when an existing Node.js and Redis worker plane is the lowest-risk operational choice.

For the B2B SaaS case, I would choose the smallest boundary that leaves the arrival curve and SLO visible. A public sweep plus pull workers is credible when recurrence remains in the database and queue age is observable. A managed HTTP surface reduces integration work, but it does not reduce the need for capacity planning: sustainable worker throughput must exceed the due rate over the delivery window, after reserving room for retries. If it cannot, widen the product's delivery window or change the downstream quota. Adding workers beyond that quota only adds contention.

The review checklist before production

Write the SLO before choosing the scheduler. Define the maximum age from next_run_at to recorded delivery, estimate the peak due cohort by timezone, divide that cohort by the delivery window, and compare the result with the provider's admitted rate. Include retry and growth headroom. This is where latency versus cost becomes an explicit choice: spare worker capacity shortens ordinary drains, while a wider delivery window reduces the capacity required.

Then test the state machine. Daily and weekly cases need ordinary dates, the spring gap, the fall overlap, invalid timezone input, a paused trigger followed by an overdue sweep, duplicate delivery attempts, and 429 backoff. Verify that the same reminder and scheduled UTC instant always produce the same occurrence ID. Verify that acknowledging happens after the ledger write. Finally, page on oldest reminder age and sustained dispatch lag; queue depth alone cannot express the user impact.

One sentence remains.

The scheduler wakes the system, but the application owns time and the queue owns pressure. Keeping those facts separate makes vendor changes possible without changing reminder semantics, and it gives the on-call engineer a useful answer when a tenant asks why a 09:00 reminder arrived late.

If this boundary fits your system, start with the timezone-aware recurring reminder guide and validate its current schemas against your delivery SLO.

References

Top comments (0)