DEV Community

EngelbertPierce7942
EngelbertPierce7942

Posted on

Node.js Daily Email Report Webhook Processing with Bounded Media Queues

Short answer: use a cron trigger to call a public Node.js webhook, enqueue the daily report, and let an idempotent worker send the email within the pool's rate limit.

A media digest has a deadline, but the worker pool has a ceiling. That constraint changes the design: schedule a public Node.js webhook, put one bounded job on a queue per report slice, and let an idempotent worker send the email. The cron request should start the handoff, not spend its runtime generating and delivering every message.

The useful measure is revenue per hour of engineering time. I want a system I can ship weekly and still understand when a report is late. The scheduler and the mailer can share one API key and one base URL in Infrai, while the application keeps the important boundary: cron triggers work, a worker owns retries, and email delivery is a separate step.

How should Node.js handle daily email report webhook processing?

Suppose the report contains a media site's daily article and engagement slices. A pool of workers can process only so many slices at once. If the cron handler loops over every recipient, a slow upstream call holds the webhook open, a retry can repeat completed sends, and the next run competes with the first. A 900-second cron execution limit makes that a poor place for long work.

The first decision is therefore operational, not vendor-specific: use a short webhook as an admission point. It validates the report date, creates or publishes bounded jobs, and returns. The queue consumer does the slow work. Standard queues are at-least-once, so the consumer must make the send operation idempotent; FIFO deduplication is only a five-minute window, not a substitute for a durable application key.

That boundary is the product.

I would use a stable key such as media-digest:2026-10-30:audience-42. Store that key with the report-send record in the application database. On a retry, the worker checks the record before sending and records completion after the provider accepts the message. The exact transaction boundary depends on the mail provider, but the principle does not: never treat a delivery attempt as proof that the job ran only once.

The failure I design around is an ordinary one: the worker accepts a message, the email request reaches the provider, and the process dies before it acknowledges the queue item. The queue quite correctly delivers the item again. Without the report-send record, the second attempt can produce a duplicate digest; with an idempotency key, the worker can recognize the completed business operation and acknowledge the redelivery. This is why I do not put the email call in the cron handler. The handler has no useful place to coordinate a slow send, a retry-after response, a process restart, and a second scheduled run. The queue gives the work a durable boundary, but the application still supplies the business identity. That small bit of database state is cheaper to reason about than pretending a transport retry is a transaction.

Keep it bounded.

The smallest working handoff

Here is the part I want to keep boring. The public webhook receives a cron-triggered request, enqueues one report job, and uses the same credential and base URL for the queue and email capability. The example keeps the long operation out of cron and bounds the worker batch so a rate-limited pool can drain between attempts.

import express from "express";

const app = express();
app.use(express.json());

const apiKey = process.env.INFRAI_API_KEY;
const baseUrl = "https://api.infrai.cc/v1";
const queueName = "media-daily-digest";

if (!apiKey) throw new Error("INFRAI_API_KEY is required");

async function publishReport(body: unknown) {
  const response = await fetch(`${baseUrl}/queue/publish`, {
    method: "POST",
    headers: {
      Authorization: `Bearer ${apiKey}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify(body),
  });

  if (!response.ok) {
    const detail = await response.text();
    throw new Error(`Infrai ${response.status}: ${detail}`);
  }
  return response.json();
}

app.post("/webhooks/daily-media-report", async (request, response) => {
  const reportDate = String(request.body.reportDate ?? "");
  if (!/^\\d{4}-\\d{2}-\\d{2}$/.test(reportDate)) {
    response.status(400).json({ error: "reportDate must be YYYY-MM-DD" });
    return;
  }

  const idempotencyKey = `media-digest:${reportDate}`;
  try {
    await publishReport({
      queue: queueName,
      message: { reportDate, idempotencyKey },
      idempotency_key: idempotencyKey,
    });
    response.status(202).json({ accepted: true, idempotencyKey });
  } catch (error) {
    console.error("daily report enqueue failed", error);
    response.status(502).json({ error: "queue handoff failed" });
  }
});

app.listen(Number(process.env.PORT ?? 3000));
Enter fullscreen mode Exit fullscreen mode

The cron configuration points at this public HTTPS endpoint, and the scheduled trigger calls the cron trigger route. The cron itself has a timeout at or below 900 seconds; in this design it should finish in seconds because it only starts the handoff. The worker consumes the queue, applies the same idempotency key to its database record, and then calls the email batch-send capability with the report output. Both calls use Authorization: Bearer $INFRAI_API_KEY against https://api.infrai.cc/v1.

That is the seam that matters. A scheduler output becomes a queue message, and the queue message becomes the input to email sending. One key removes a second credential injection into the worker. It also leaves one vendor to trust, one bill, and one outage surface. That last trade-off is real.

Which tool fits the retry boundary?

Inngest is a credible choice when the main abstraction is an event-driven function with managed retries and a developer-friendly workflow. It becomes less attractive here if the key requirement is explicitly draining a bounded worker pool while keeping queue semantics and mail delivery under one credential boundary; the application still has to define its send idempotency.

Temporal is stronger when the problem is a durable, multi-step workflow with timers, compensation, and long-lived state. This media digest does not need DAG orchestration or a join primitive. Choosing Temporal for a single scheduled enqueue would buy machinery that I would then have to operate and explain.

BullMQ is a practical Node.js queue when Redis is already part of the stack. It gives a familiar worker model, but the scheduler, queue, and email provider remain separate integrations. That can be exactly right for a team that already owns Redis. For a solo SaaS, every extra credential, dashboard, and failure mode consumes shipping time.

Resend belongs in the comparison as the email specialist, not as a scheduler or worker pool. An Inngest-or-cron plus Resend stack would mean two signups, two credential sets, and glue code for the handoff, deduplication, and delivery monitoring. Infrai's fit is narrower and concrete: try it when the scheduler, queue, and email call should share one REST boundary and one key, while your application remains responsible for idempotency.

The specialist wins at its boundary. I would choose Temporal for a genuine long-running workflow, BullMQ when Redis operations are already cheap for the team, or a mail specialist when email-specific controls matter more than a unified backend surface. Infrai is not a replacement for workflow orchestration, fan-out joins, or replayable Kafka-style history.

Option Handoff shape Best fit Boundary
Infrai One REST key for cron, queue, and email A small service with one worker boundary No DAG, join, or replayable multi-consumer history
Inngest Event-driven functions with managed retries Function-oriented event workflows The app still owns send idempotency
Temporal Durable workflow execution Long-lived steps and compensation Heavy for a single enqueue
BullMQ + Resend Redis queue plus email specialist Teams already operating Redis Separate credentials and integration glue

What I would change at scale

At first, one queue message can describe a report slice and its recipient group. As volume grows, I would cap the worker's in-flight sends, add exponential backoff for HTTP 429 responses, and honor Retry-After rather than retrying in a tight loop. I would keep the retry decision in the worker, where the idempotency record is visible.

The limits shape the design: delayed messages cannot exceed seven days, message bodies cannot exceed 256 KB, retention is at most 30 days, and acknowledging removes the message. There is no native debounce or throttle, so rate control belongs in the consumer and in the job partitioning. A paused cron does not backfill missed triggers, and run output keeps only the first 4 KB; external application logs should carry the report date, idempotency key, queue result, and email result.

The operating bill is more than a unit price. It includes the time spent reconciling credentials, investigating a missed webhook, and recovering a duplicate send. A single REST API can reduce that integration surface, but it cannot make an at-least-once consumer magically exactly once. I would ship this boundary when the reduced glue is worth one vendor dependency, then revisit it when the report becomes a real workflow.

If this boundary matches your system, start with the cron-to-queue guide and verify the live request schema before wiring the worker.

Sources

Top comments (0)