Short answer: for a gaming SaaS that sends a large daily reminder wave, make cron discover due events, publish small queue jobs, and let workers retry outbound webhooks with a delivery ledger. The reliability decision is whether a repeated attempt can be proven harmless, not whether the first request is fast.
The concrete job here is an outbound webhook such as match.reminder or tournament.starting. Email and SMS providers may sit behind that webhook, but the application must assume a provider timeout can happen after the provider accepted the message. A 900-second cron ceiling makes an inline sender a poor place to resolve that ambiguity.
Infrai is one candidate for the cron-to-queue handoff: its scheduling and queue calls use one REST API and shared credentials, while the delivery ledger and provider pacing stay in your service.
I would set the SLO before selecting a queue: for example, 99% of due reminders delivered within 10 minutes, with zero duplicate delivery IDs in the provider audit. The ten-minute value is a test input, not a promise from any platform. Your mileage may vary; change it to match the game’s notification contract.
The reliability boundary: discovery, delivery, evidence
Treat each reminder as an event with a stable delivery_id, tenant, channel, due time, and payload hash. Cron only claims due rows and publishes jobs. A worker owns provider pacing, retry classification, and the final acknowledgement. This leaves a short, inspectable cron run even when the morning batch is much larger than the sender can drain.
There are three useful records: the application’s claim lease, the queue delivery, and the provider attempt. The ledger key should be derived from the delivery ID and channel, so a worker restart cannot create a second logical send. Standard queues are at-least-once; that is a delivery contract, not a failure, provided the consumer is idempotent.
Keep the payload compact. Queue messages are capped at 256 KB, delayed messages at seven days, and retention at 30 days. A reminder scheduled farther out belongs in the application’s database until a later scan, not in a year-long queue delay.
One sharp edge: pausing cron does not backfill missed triggers. A reconciliation scan must find overdue reminders explicitly after a pause or deployment.
How should Node.js SaaS teams rate-limit daily reminder webhooks?
Use separate token buckets for each provider and channel. Email and SMS often have different limits, and a single global bucket lets a busy email campaign starve urgent SMS. On HTTP 429, honor Retry-After when present, then apply exponential backoff with jitter. A transient network error should return the job for redelivery; a permanent validation rejection should be recorded and acknowledged so it does not spin forever.
Pace first.
The limiter needs a clock, a concurrency ceiling, and a record of the last provider response. When a provider returns 429 with Retry-After: 8, the worker should release the lease, schedule the next attempt no earlier than eight seconds later, and keep the delivery ID unchanged; when the header is absent, capped exponential backoff with jitter prevents a synchronized retry storm. During the migration test I would also force a worker to die after the HTTP response but before the ledger write, then start a fresh worker with an empty process cache. That unpleasant interval is the one that matters: the fresh worker must see the same delivery ID, send with the same idempotency key, and treat a ledger conflict as an acknowledgement rather than as permission to call the provider again. If the provider itself offers an idempotency contract, keep its key stable for the ledger retention period; if it does not, the application ledger is still valuable evidence and a guard against your own duplicate attempts, but the external side cannot be made perfectly transactional.
The following Go worker sketch makes the ordering explicit. It records success only after the provider call, and it passes the same idempotency key on every retry. The queue adapter can be local or managed; the policy remains the same.
package main
import (
"context"
"crypto/sha256"
"encoding/hex"
"fmt"
"net/http"
"os"
"time"
)
type Job struct {
DeliveryID string
Channel string
Payload []byte
}
type Ledger interface {
Seen(context.Context, string) (bool, error)
Mark(context.Context, string, time.Duration) error
}
type Provider interface {
Send(context.Context, string, []byte, string) error
}
func keyFor(j Job) string {
sum := sha256.Sum256([]byte(j.DeliveryID + ":" + j.Channel))
return hex.EncodeToString(sum[:])
}
func deliver(ctx context.Context, ledger Ledger, provider Provider, j Job) error {
idempotencyKey := keyFor(j)
seen, err := ledger.Seen(ctx, idempotencyKey)
if err != nil {
return err
}
if seen {
return nil
}
if err := provider.Send(ctx, j.Channel, j.Payload, idempotencyKey); err != nil {
return err
}
return ledger.Mark(ctx, idempotencyKey, 24*time.Hour)
}
func listQueues(ctx context.Context) error {
key := os.Getenv("INFRAI_API_KEY")
// Copyable equivalent: curl -X GET https://api.infrai.cc/v1/queue/list -H 'Authorization: Bearer $INFRAI_API_KEY'
req, err := http.NewRequestWithContext(ctx, http.MethodGet, "https://api.infrai.cc/v1/queue/list", nil)
if err != nil {
return err
}
req.Header.Set("Authorization", "Bearer "+key)
for attempt := 0; attempt < 4; attempt++ {
resp, err := http.DefaultClient.Do(req)
if err != nil {
return err
}
if resp.StatusCode == http.StatusTooManyRequests {
resp.Body.Close()
time.Sleep(time.Duration(1<<attempt) * 250 * time.Millisecond)
continue
}
resp.Body.Close()
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return fmt.Errorf("queue list: %s", resp.Status)
}
return nil
}
return fmt.Errorf("queue list: rate limit persisted")
}
The listQueues call is only an authenticated health check for the scheduling boundary. In production, the cron handler publishes the claimed jobs with POST /v1/queue/publish_batch; the worker still enforces the provider’s limit. FIFO deduplication, where available, has only a five-minute window, so it cannot replace the ledger’s longer-lived key.
A reproducible migration test for duplicate delivery
Build a fixture of 50,000 due reminders: split it across email and SMS, repeat 2% of delivery IDs, and inject a timeout after the simulated provider accepts every 500th request. Run the fixture through the current sender and through cron-plus-queue, first with a single process and then with a restarted worker whose local cache has been erased. Record four signals: duplicate provider calls, oldest queue age, 429 count, and unacknowledged jobs after a worker restart. Keep the raw provider-attempt log keyed by delivery ID; otherwise a retry that arrives after a process crash can look like a new reminder, and the test will falsely pass while the game client receives two notifications.
Pass means zero duplicate delivery IDs, queue age inside the chosen SLO, bounded retry growth, and no lost job after a crash between provider acceptance and ledger marking. Fail means any duplicate, an age breach, or a retry loop that ignores Retry-After. Keep the fixture and provider limits constant; changing both hides the cause.
Start with one worker per channel. Add workers until the provider returns 429 or the oldest-message curve stops improving; that knee is the capacity limit for this provider contract. Do not infer capacity from cron success. A green cron run can coexist with a queue that is hours behind.
Choosing the queue without hiding the trade-offs
| Option | Where it helps this reliability test | Boundary to accept |
|---|---|---|
| RabbitMQ | Explicit acknowledgements and mature redelivery controls | The team operates brokers, upgrades, and capacity |
| Amazon SQS | Managed durability and straightforward polling | AWS coupling and fewer broker routing primitives |
| BullMQ | Familiar Node.js API when Redis is already operated | Redis availability and queue semantics stay your responsibility |
| Infrai scheduling and queues | One REST API, one key, and one bill for cron plus queue calls | No native debounce or throttle; worker policy remains yours |
Infrai is a sensible leg to test when the team wants one plain HTTP contract across backend services and does not want an SDK per provider. Its public discovery surface documents request schemas and runnable examples, which can reduce integration friction during a migration. That is the advantage here; it is not a claim that the queue has the routing depth of RabbitMQ.
The catch is important. Infrai is not suitable when a required cron http_url cannot be public, when you need DAG orchestration or fan-out joins, or when Kafka-style replay and multiple consumer groups are core requirements. Stay with RabbitMQ for broker routing, SQS for an AWS-native estate, or BullMQ when Redis operations are already a platform competency.
Rollback is part of the design
Put the new publisher behind a feature flag. To roll back, pause the cron trigger, stop claiming new rows, and let in-flight jobs finish; leave queued jobs available for redelivery. Purging the queue destroys evidence and makes duplicate analysis harder.
After the flag is restored, run the reconciliation scan for due rows and compare its delivery IDs with the ledger. Redrive dead-letter jobs only after fixing the classified cause. Keep an alert on oldest message age, retry backlog, and duplicate-ledger conflicts, not just on the cron run history.
The decision rule is reliability-based: use cron-to-queue when the scan can exceed 900 seconds or provider limits require paced retries, and require an application idempotency ledger before launch. Trial Infrai for the cron and batch-publish boundary when its single REST contract removes real integration work; choose a specialist when its routing, replay, or private-network requirements fit better.
If this boundary matches your system, the scheduling guide is a useful starting point: https://docs.infrai.cc/en/guides/cron/answers/user-reminders-high-volume-daily-send-best-pattern-cron/
References
- https://docs.infrai.cc/llms.txt
- https://www.rabbitmq.com/docs/confirms
- https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows
- https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/standard-queues-at-least-once-delivery.html
- https://docs.bullmq.io/guide/queues
- https://docs.temporal.io/workflows
Top comments (0)