Short answer: put each daily report email behind a durable queue, retry only transient failures with a bounded budget, and move exhausted jobs to a dead-letter queue (DLQ) for inspection. The queue is needed because a scheduler records intent, while an email provider can still time out, throttle, or reject a message.
I think of this as a small state machine: scheduled -> queued -> sending -> sent, with retryable and terminal exits. That picture keeps the design honest. A retry is not a second business event; it is another delivery attempt for the same report.
What do daily report email retries, failed jobs, and a dead-letter queue actually solve?
A daily report often starts at one predictable time, which tempts teams to call the mail API directly from a cron process. That couples scheduling to delivery. If the provider takes 20 seconds, the scheduler is busy. If the process dies after the provider accepted the message, a restart can send a duplicate. If the provider returns a permanent address rejection, blind retries waste capacity and hide the real problem.
A queue separates those failure domains. The scheduler writes a job with a stable jobId, recipient, report date, and template version. A worker claims it, sends the message, and records the outcome. The job is idempotent: a second delivery attempt checks the same key before creating another send request.
The DLQ is not a trash can. It is a bounded quarantine for jobs that reached the retry limit, violated a policy, or need a human decision. Keep the original error class, attempt count, and last-attempt timestamp with the payload metadata. Do not silently drop a report that a finance or operations user expects.
A field guide for queue choices
Use the simplest mechanism that preserves the state transitions you need.
| Pattern | Pick it when | Main trade-off |
|---|---|---|
| Scheduler plus direct send | A tiny internal report can tolerate occasional manual reruns | Least moving parts, weakest recovery and duplicate control |
| Durable queue with worker retries | Delivery must survive process restarts and provider throttling | More state to monitor, plus idempotency work |
| Queue plus DLQ and replay tool | Operators need an audit trail and controlled recovery | Requires retention, access control, and runbooks |
| Pull-based batch worker | Reports can arrive late and throughput matters more than exact timing | Higher latency and more worker capacity planning |
For most customer-facing daily reports, choose the second pattern and add the third when failed jobs have business consequences. A queue does not make a bad report correct; it makes the attempt observable and recoverable.
Keep the recovery path boring.
The catch is latency versus cost. A longer backoff reduces pressure on a throttled provider but delays the inbox. A short schedule and a large worker fleet reduce latency but spend more on idle capacity. Measure the age of the oldest queued job, not just average send time.
Implement one retry-safe job in TypeScript
The worker below keeps policy separate from transport. It treats 429 and temporary network failures as retryable, while an invalid recipient is terminal. The exact provider SDK is deliberately outside this boundary.
type EmailJob = {
jobId: string;
reportDate: string;
recipient: string;
templateVersion: string;
attempt: number;
};
type SendResult =
| { kind: "sent"; providerId: string }
| { kind: "retryable"; reason: string; retryAfterMs?: number }
| { kind: "terminal"; reason: string };
function nextDelay(attempt: number, retryAfterMs?: number): number {
const exponential = Math.min(15 * 60_000, 2 ** attempt * 1_000);
const jitter = Math.floor(Math.random() * 500);
return Math.max(exponential + jitter, retryAfterMs ?? 0);
}
async function handle(job: EmailJob, maxAttempts: number, send: (job: EmailJob) => Promise<SendResult>) {
const result = await send(job);
if (result.kind === "sent") return { state: "sent", providerId: result.providerId };
if (result.kind === "terminal") return { state: "dead_letter", reason: result.reason };
if (job.attempt + 1 >= maxAttempts) return { state: "dead_letter", reason: result.reason };
return { state: "retry", delayMs: nextDelay(job.attempt, result.retryAfterMs) };
}
The queue acknowledgement belongs after the durable state write, not before it. On a retry, preserve jobId; do not mint a new identity. If the provider has an idempotency key, use that same value. Otherwise, keep a send ledger and make the ledger write and job transition transactional where your storage supports it.
I once assumed that a processed = true flag was enough. It was not. Two workers can read false before either writes true. The business transition has to be conditional, such as “send only when status is queued,” so the second worker gets an explicit duplicate outcome. Small detail. Big difference.
How should beginners test queue patterns for daily report email failures?
Start with a table of failure classes, not a load test.
- A timeout before the provider response: retry with exponential backoff and jitter.
- A 429 response: honor
Retry-Afterwhen present and count throttles separately. - A malformed address: mark terminal and send the job to the DLQ without another attempt.
- A worker crash after the provider accepts mail: redeliver the same
jobIdand verify one business send. - A report-generation error: keep the delivery job distinct from the report artifact, so a fixed artifact can be replayed safely.
Record jobId, report date, attempt, outcome, queue age, and a redacted error class in structured logs. Emit counters for retry, terminal failure, DLQ entry, and successful replay. Alert on DLQ growth and oldest-job age. A green scheduler metric does not prove that an inbox received anything.
For a useful rehearsal, inject ten jobs: eight transient failures that recover on attempt two, one invalid address, and one worker crash after the send call. The expected result is eight sends, one terminal DLQ item, and one idempotent duplicate check. That is a test expectation, not a benchmark claim. Your mileage may vary with provider limits and report size.
Limits and the decision rule
This pattern is not suitable when a report may wait for hours without business impact and the team cannot operate another durable component; a direct scheduled send plus a manual rerun may be enough. It is also a poor fit when recipients require strict exactly-once semantics that the mail provider cannot offer. In that case, change the product promise or add a provider-side idempotency contract.
Do not replay every DLQ item automatically. Fix the cause, preview the report, then replay by jobId with an operator identity and an audit record. Keep retention finite, encrypt recipient data, and redact message bodies from logs.
The practical rule is simple: use retries for temporary transport problems, terminal classification for permanent input problems, and a DLQ for work that deserves a human decision. That boundary turns “the daily email failed” into a traceable workflow.
Top comments (0)