Short answer: a cheap, simple webhook task-processing design puts a durable queue behind the weekly scheduler, applies a rate limit before dispatch, and retries transient outcomes with bounded backoff; a plain cron callback is enough only when missed or duplicate sends are acceptable.
The least complex reliable design separates deciding what is due from attempting delivery. A scheduler writes one task per active customer to durable storage. Workers claim those tasks at a controlled rate, record the result, and retry transient failures without rebuilding the audience. That boundary matters more than whether the application happens to run on Node.js.
Consider a capacity-planning exercise, not a claimed production benchmark: an edtech SaaS has 24,000 active customers to receive a weekly learning digest within a six-hour window. That is an average of about 1.1 deliveries per second, which looks trivial. Yet an upstream mail or webhook destination limiting a tenant to one request per second can turn a harmless average into a burst, while a deployment halfway through audience generation can create either a gap or a duplicate. The incident lesson is simple: the weekly clock should create durable intent; it should never be the delivery mechanism.
One clock tick is not a guarantee.
The six-hour window determines queue capacity
Treat the task as a small state machine. pending becomes leased, then either delivered, retryable, or dead. Persist the customer ID, digest period, attempt count, next-attempt time, and an idempotency key derived from the customer and period. Do not store only a serialized request and hope that replay means the same thing a week later; templates, memberships, and authorization may have changed. Either freeze the intended payload when the task is created or define, document, and test that retries intentionally render from current data.
The worker should classify outcomes rather than retrying every non-success. A timeout, connection interruption, or rate-limit response can be retried. A rejected credential or invalid payload needs intervention, not another 30 attempts. Backoff needs a cap and jitter so that a recovered dependency does not receive the entire backlog at once. A maximum attempt count alone is weak: five rapid attempts can consume the budget before a ten-minute throttle clears, while five attempts spread across two days may miss the digest's usefulness window. Bound both attempts and elapsed age.
Here is a deliberately generic Go decision function. It does not depend on a queue vendor, and the exact policy values are planning inputs rather than universal constants.
package delivery
import (
"math/rand"
"time"
)
type Decision struct {
Retry bool
At time.Time
}
func NextAttempt(now, createdAt time.Time, attempt, status int) Decision {
if now.Sub(createdAt) >= 24*time.Hour || attempt >= 8 {
return Decision{}
}
retryable := status == 0 || status == 408 || status == 429 || status >= 500
if !retryable {
return Decision{}
}
delay := time.Second * time.Duration(1<<min(attempt, 10))
if delay > 15*time.Minute {
delay = 15 * time.Minute
}
jitter := time.Duration(rand.Int63n(int64(delay/4) + 1))
return Decision{Retry: true, At: now.Add(delay + jitter)}
}
Status 0 means no HTTP response was received. A real worker also needs a lease expiry, atomic acknowledgement, and a dead-letter path. The important invariant is that a crash after the destination accepts a request but before the worker acknowledges it can cause another attempt. Exactly-once delivery across that boundary is not a realistic default; an idempotency key and a deduplicating receiver make at-least-once processing tolerable.
I would make the rate limiter tenant-aware. A single global bucket is easy to operate but lets one noisy school delay every other customer; a bucket per destination protects fairness but creates more state and needs an inactive-key cleanup policy. For a weekly digest, reserve enough global capacity to drain the planned audience inside the delivery window, then enforce per-tenant limits underneath it. If the arithmetic does not include retry traffic and the slowest allowed downstream rate, it is optimism dressed as capacity planning.
How should a Node.js SaaS process webhook retries under a rate limit?
“Runs every Monday” is a schedule, not an SLO. Define a completion objective such as the proportion of eligible digests accepted by their destination before the six-hour deadline, then measure the queue stages that explain misses: audience-generation lag, oldest pending-task age, lease expirations, attempt distribution, rate-limit responses, terminal rejections, and dead-letter count. Alerting on worker CPU while the oldest task quietly ages is operational theater.
Use two identifiers. The idempotency key identifies the business action, such as customer-123:week-32; the attempt ID identifies one execution. Logs and traces should carry both. This makes a repeated attempt visible without falsely counting it as a second digest, and it lets an operator answer the question that matters during recovery: which customer-period pairs still lack a recorded acceptance?
The preventative test is a crash matrix. Stop a worker before the request, after the request but before acknowledgement, and after acknowledgement. Pause dispatch long enough for leases to expire. Feed it a rate-limit response, a permanent rejection, and a timeout. Advance the clock beyond the task-age limit. Then verify three properties: no eligible task disappears, the same business action retains the same idempotency key, and work past the usefulness deadline stops consuming delivery capacity. I don't approve a scheduler design from the happy path alone.
There is an uncomfortable ambiguity here: a destination accepting a webhook does not necessarily mean a learner saw the digest. I'm not sure any queue metric can close that semantic gap. A downstream delivery receipt or application-level acknowledgement can, if the destination offers one; otherwise the SLO must honestly stop at “accepted by destination.”
Three operating models expose different recovery boundaries
The selection is less about the cheapest advertised task and more about who owns durable state, redelivery, throttling, and the pager. A useful comparison starts with operational boundaries.
| Option | Delivery boundary | Team burden | Main limitation |
|---|---|---|---|
| Cron-only invocation | Starts audience generation on a clock | Low until a run overlaps, fails, or must be replayed | Not suitable when each customer delivery needs independent retry state |
| Managed workflow or task service | Persists steps and retries under a service contract | Lower infrastructure load; policy and lock-in still need review | Stick with another option when portability or custom queue semantics dominate |
| Self-hosted durable queue | Team owns storage, workers, leases, recovery, and upgrades | Highest on-call and capacity-planning load | A poor fit for a small team without queue operations experience |
Cloudflare Workers Cron Triggers is evidence of the cron-trigger category, while Inngest documents a managed event-driven approach. Neither name decides the architecture. Compare the documented guarantees against the same worksheet: task retention, retry controls, concurrency and rate-limit scope, deduplication support, observability, replay, regional behavior, payload limits, and what happens when the scheduler fires while the previous run is still producing work. Then test the claims with a disposable workload before making the service part of an SLO.
Price belongs in that worksheet, but it is not the first column. Model scheduled invocations, queue operations, retries during a throttle, retained history, and engineering on-call time. Your mileage may vary because traffic shape matters more than the weekly average: a service that looks inexpensive at one attempt per digest can move differently when a destination forces a long retry tail. I care more about a bounded recovery procedure than a small difference in the happy-path bill.
The catch is that the durable-queue design is unnecessary for low-value notifications where duplicates and omissions are explicitly acceptable. In that case, keep the cron callback, cap its runtime, record a run marker, and move on. At the other extreme, stick with a self-hosted queue when regulatory isolation, bespoke scheduling semantics, or existing operational expertise outweigh managed-service convenience. “Managed” transfers work; it does not transfer accountability for the delivery objective.
Recovery is a deployment property
Deployment should be boring. Roll out consumers independently from producers, keep task schemas backward compatible while old jobs remain queued, and stop producers before removing a field that an older consumer requires. During an incident, pause claims without deleting tasks, lower concurrency when a dependency asks for relief, and replay dead letters only after the classification rule has changed or the underlying input has been corrected.
For the weekly digest, retain a compact delivery ledger keyed by customer and digest period. Audience generation inserts missing rows atomically; rerunning it becomes reconciliation rather than duplication. A dispatcher claims due rows in bounded batches and updates the next-attempt time after a retryable outcome. This is a little more machinery than calling a webhook from cron, but every part answers a concrete recovery question. Who was eligible? Who was attempted? Who was accepted? Who can still be retried before the deadline?
Keep those answers queryable.
The selection rule is therefore conditional: use a cron-only trigger when best-effort execution matches the product promise; use a managed workflow or task service when durable retries are required and reducing on-call surface is worth its constraints; operate a queue yourself only when control, isolation, or existing expertise pays for the added failure modes. For an edtech weekly digest with a stated delivery window, start from the guarantee and work backward to capacity, retry age, idempotency, and recovery. The brand can come last.
References
- Cloudflare Workers Cron Triggers documentation: https://developers.cloudflare.com/workers/configuration/cron-triggers/
- Inngest documentation: https://www.inngest.com/docs
Further reading
The two references above are useful starting points for comparing a clock-triggered execution model with a managed event-driven model. Read their current documentation against your own SLO worksheet and verify the behavior with failure-injection tests before committing production delivery guarantees.
Top comments (0)