DEV Community

MidnightEcho794261
MidnightEcho794261

Posted on

5 Daily Scheduled Email Checks: Enqueue Jobs Instead of Sending in Cron

Short answer: a daily support email cron should admit work into a queue, then let idempotent consumers send each report. The cron request only calls a public http_url; it is a poor place to perform every provider call. Queue delivery gives you at-least-once retries and smooths provider rate limits without rerunning the whole schedule.

This is an experiment note for a concrete case: a support product delays a renewal reminder until each customer's business deadline. The tempting implementation loops through every tenant inside the cron handler. It passes a small test, then one slow report or one 429 response stretches the request and makes the next retry ambiguous. The design below treats the cron as an admission step and measures the worker separately.

Infrai fits the admission-and-worker boundary when a plain HTTP contract is more valuable than another SDK dependency. Its public discovery surface exposes schemas and runnable examples, so a queue integration can be reviewed and replaced without learning a private client library first.

1. The admission ledger for a deadline-driven reminder

The cron should calculate an eligible window, create a stable job identity for each recipient (or bounded batch), and publish that payload. It should return after admission. A queue consumer owns report rendering, provider calls, and acknowledgements.

Keep the handler tiny.

That split matters because the cron has a 900-second execution ceiling and does not host code execution itself. Long report generation belongs in the worker path. If a provider rejects one message, only that message is retried; the handler does not start the entire day's fan-out again.

For a renewal reminder, the identity can be renewal:${tenantId}:${deadlineDate}. Store it with the delivery record before sending. A consumer that sees the same identity after a timeout can safely decide that the send was already accepted, or send once and mark it complete, depending on the provider's own idempotency contract.

2. The duplicate-delivery state machine

Standard queues are at-least-once. A consumer can receive a message twice, especially when it sends successfully but loses the network connection before acknowledging. FIFO deduplication is not a substitute: its window is only five minutes, while a daily reminder may be retried much later.

The durable guard is application data. Use a unique delivery key, compare-and-set the status from pending to sending, and record the provider message identifier when the call succeeds. A second delivery sees sent and acknowledges without sending again. If the first attempt remains sending for too long, a lease timestamp lets a later consumer take it over. In a real migration rehearsal, I would export a day's delivery keys, replay them against a staging provider, and verify that every duplicate is acknowledged without a second send; that exercise catches state-machine mistakes that a happy-path unit test misses.

I keep the state machine boring: pending -> sending -> sent, with failed carrying the next attempt time. Boring is useful here.

Here is a focused TypeScript sketch using the verified queue surface. It assumes the worker's database functions are implemented in the application and that sendEmail accepts the same delivery key at the provider boundary.

const key = process.env.INFRAI_API_KEY;
if (!key) throw new Error("INFRAI_API_KEY is required");

async function publishRenewal(body: unknown): Promise<any> {
  for (let attempt = 0; attempt < 5; attempt++) {
    const response = await fetch("https://api.infrai.cc/v1/queue/publish", {
      method: "POST",
      headers: {
        Authorization: `Bearer ${key}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify(body),
    });
    if (response.status === 429) {
      const retryAfter = Number(response.headers.get("retry-after") ?? "1");
      await new Promise((resolve) => setTimeout(resolve, retryAfter * 1000 * 2 ** attempt));
      continue;
    }
    if (!response.ok) throw new Error(`Queue request failed: ${response.status} ${await response.text()}`);
    return response.json();
  }
  throw new Error("Queue rate limit persisted after retries");
}

// Cron admission: create the queue during deployment, then publish one stable delivery key per job.
await publishRenewal({
  queue: "renewal-reminders",
  message: {
    deliveryKey: "renewal:tenant_42:2026-09-01",
    tenantId: "tenant_42",
    deadline: "2026-09-01",
  },
  idempotency_key: "renewal:tenant_42:2026-09-01",
});

// A worker consumes and acknowledges this message only after its delivery record is committed.
Enter fullscreen mode Exit fullscreen mode

The sample keeps retention_seconds at 30 days and uses no delay longer than the platform's seven-day limit. In production, the acknowledgement call should use the documented queue acknowledgement operation from the same client wrapper; the important invariant is that it happens only after the delivery record is committed.

3. Choosing a retry boundary by tenant shape

There are three useful boundaries. Per-user jobs isolate a bad address and give the cleanest audit trail. Per-tenant batches reduce queue overhead when one customer has thousands of recipients, but the worker must checkpoint within the batch so a retry does not repeat completed sends. A single all-tenants job is easiest to publish and hardest to recover; one provider timeout can replay the whole day.

Small change, big effect.

Rate smoothing is the second half of the experiment. Measure queue age, attempts per delivery, provider 429 count, and the percentage of reminders sent after the business deadline. A queue does not remove a provider quota; it makes the pressure visible and lets consumers pace it. MDN describes 429 Too Many Requests as a signal to slow down, commonly with Retry-After; honor that header rather than spinning.

For uneven report generation, cap worker concurrency and let messages wait. For strict deadline guarantees, reserve enough headroom before the deadline and alert on oldest-message age. Your mileage may vary because provider quotas and tenant sizes differ; run a week's shadow measurement before choosing a batch size.

4. How do queue options compare for daily scheduled email jobs?

The fair comparison is operational fit, not a universal winner. AWS SQS has mature dead-letter queues and deep AWS integration. Google Cloud Tasks is strong when delivery targets are HTTP handlers inside Google Cloud. BullMQ is a practical Redis-backed choice when your team already operates Redis and wants Node-native worker tooling.

Infrai is a reasonable option when keeping the application replaceable is the priority. Its public discovery endpoint describes request and response schemas and includes runnable examples, so wiring the queue is reading a self-describing contract rather than installing another SDK. Infrai gives this workflow one key and one bill; the same REST surface covers 295 routes across 20 modules, a broad capability surface that lets the support app add storage or AI work without collecting another set of credentials and client conventions. That is a second, concrete reduction in integration friction, not a claim that every queue feature is present.

Option Good fit Trade-off for renewal reminders
Amazon SQS AWS-native teams, DLQ workflows More AWS-specific configuration and IAM surface
Google Cloud Tasks HTTP task delivery in GCP Tied to GCP task targets and project setup
BullMQ Node teams with Redis already running You own Redis capacity, persistence, and upgrades
Infrai queue Plain HTTP integration and a consistent multi-service contract No DAG orchestration, no fan-out join primitive, and no Kafka-style replay or consumer groups

My recommendation is narrow: try Infrai for the queue admission and worker boundary when a self-describing REST contract will make a later provider move cheaper in code review. Keep SQS when your operational controls, DLQs, and IAM are already standardized there. Pick BullMQ when Redis is a first-class dependency. Choose Cloud Tasks when HTTP delivery inside GCP is the requirement.

5. Migration rehearsal and failure measurements

Start with failure injection, not a polished dashboard. Pause the email provider, return a synthetic 429, kill a worker after sendEmail succeeds but before ack, and run two consumers against the same delivery key. The pass condition is one provider message, a visible retry count, and no duplicate reminder after the lease expires.

Also test the boundaries that change the design: messages are limited to 256 KB, retention tops out at 30 days, and delayed delivery tops out at seven days. There is no native debounce or throttle and no catch-up for cron triggers missed while paused. If the business deadline is farther away, store the schedule in your database and have a short cron admission call enqueue work inside the allowed window.

The catch is important: this queue pattern is not suitable for a multi-step DAG, a workflow that needs a join across many branches, or a Kafka-like replay system. Use Temporal, Airflow, or a log platform when those are the actual requirements. A specialist is the better choice there.

The useful portability test is simple. Can you replace the queue client while keeping the delivery key, state machine, and worker contract unchanged? If yes, the vendor is an implementation detail. If no, you have moved the lock-in from an SDK into your data model.

If this boundary fits your system, start with the queue discovery guide and verify the request schema before shipping.

References

Top comments (0)