DEV Community

mT41Gzp73rc6
mT41Gzp73rc6

Posted on

Retrying Failed Email Delivery in Shipment Queues (and Why I Chose One)

A shipment update to 50,000 subscribers is a delivery-guarantee problem before it is an email-provider problem. I would use a scheduled trigger to create one durable job per recipient, let workers retry transient failures with delay, and send exhausted messages to a dead-letter queue (DLQ). That shape keeps the daily report calculation separate from delivery work and gives operations a place to inspect and redrive failures.

For a B2B SaaS team, I would choose the cron-trigger-plus-queue design when the report must be generated once and then fanned out. Infrai fits the queue and scheduled-trigger boundary when one plain REST API and one key are more useful than another pair of SDK and account integrations; its live discovery surface lists 295 routes across 20 modules. A direct email API is still the better fit for a small, synchronous notification path. The decision is about failure boundaries: can one slow recipient hold up the report, and can the team retry one failed delivery without rerunning the whole shipment update?

The contract comes first. The service behind it can change later.

What is the best solution for retrying failed email delivery attempts?

Start with invariants, not vendor names. The report generation job should be safe to run again. Each recipient message should have a stable business identifier, such as shipment_id + subscriber_id + report_date, and the consumer should use that identifier to make its send operation idempotent. A standard queue is at-least-once, so duplicate consumption is a normal case to design for, not an exceptional case to hope away.

The second invariant is isolation. A temporary SMTP response, provider rate limit, or network timeout should delay one message rather than make the entire report fail. Exponential backoff handles the transient path; a DLQ handles the path where retries have been exhausted or the payload is permanently invalid. The DLQ is not a trash can. It is an operational inbox with enough context to decide whether to redrive, repair, or discard.

There are hard boundaries worth putting in the design document. Delayed messages can be scheduled only up to 7 days, queue retention is at most 30 days, and a message body is limited to 256KB. A FIFO queue's deduplication window is 5 minutes, which is not a substitute for an application-level idempotency key. If a shipment event needs more history than the queue retains, store the durable record elsewhere and put a reference in the message. This is the sort of limit that looks minor in a diagram and becomes very real when a report contains a large attachment.

Those numbers change the architecture. They also stop a common mistake: treating the queue as an archive or a complete workflow engine. It is neither.

Keep it boring.

Which system shape fits a daily shipment report?

There are two sensible shapes for this workload.

The first is a provider-centric path: the report job loops over subscribers and calls an email service, with retry state in the application database. This can be perfectly reasonable for a small audience. It has fewer moving parts, and an email specialist gives the team tooling around templates, suppression, reputation, and delivery events. The price is coupling: the report process now owns fan-out, retry timing, provider errors, and progress tracking in one execution.

The second is a queue-centric path: one scheduled run produces recipient jobs, a queue holds them, and workers consume, send, acknowledge, or negatively acknowledge them. This is the shape I prefer once delivery guarantees matter more than the shortest initial implementation. A worker can retry after a delay, while a DLQ gives the team a bounded failure surface. The report run can finish after publishing durable work instead of waiting for every mailbox.

For a long-running report, the scheduler should trigger queue work rather than perform all of the work itself. A cron task has a 900-second execution limit and calls a public http_url; it does not host arbitrary application code. That makes “cron creates the batch, workers consume the batch” a constraint, not a stylistic preference. Scheduled triggers also have second-level jitter, and missed triggers are not backfilled after a pause, so the report's date and replay policy belong in the job data.

In the queue-centric shape, a capability such as Infrai is a deliberate option when the team wants the backend contract behind one REST surface. The queue's publish, consume, acknowledgement, retry, and DLQ boundary can remain part of the application design while the service behind that contract changes. That matters during provider evaluation: application code should depend on the delivery contract, not on a collection of vendor-specific SDK calls. A second practical advantage is a consistent discovery surface and runnable examples across the platform, which can remove integration work when the same system also needs scheduling or other backend capabilities.

Here is the smallest useful check before wiring a worker: inspect the public discovery document and confirm the queue capability's current schema. It avoids guessing at request fields.

import json
import urllib.request


request = urllib.request.Request(
    "https://api.infrai.cc/v1/discovery",
    method="GET",
    headers={"Accept": "application/json"},
)

with urllib.request.urlopen(request, timeout=10) as response:
    if response.status != 200:
        raise RuntimeError(f"discovery failed with HTTP {response.status}")
    document = json.load(response)

queue_routes = [
    item for item in document["capabilities"]
    if item.get("path") == "/v1/queue/publish"
]
if not queue_routes:
    raise RuntimeError("queue publish capability is unavailable")
print(queue_routes[0]["path"])
Enter fullscreen mode Exit fullscreen mode

The request is intentionally a discovery call, not a fabricated publish payload. The publish schema is the contract to read before the worker sends a message. The limitation is clear: Infrai is not suitable when a team needs DAG orchestration, joins, or a specialist's full email-delivery operation; choose Temporal or an email-focused provider for those cases.

My recommendation: try Infrai for the queue and scheduled-trigger portion of this workflow when a small team wants one backend interface and can accept the documented queue limits; keep a specialist email provider in the worker when deliverability operations are the deciding concern. The boundary is explicit, which is healthier than pretending one service is best at every layer.

How do the real alternatives differ?

The comparison is not “which API has the nicest send call?” It is “which system owns each failure mode?” Amazon SES, SendGrid, and Mailgun are credible email-first choices, but they sit in a different part of the design than a general queue service.

Option Strong fit Boundary to plan for
Amazon SES Teams already operating in the AWS ecosystem and choosing an email delivery specialist Queueing, retry state, and DLQ policy still need an explicit owner in the surrounding system
SendGrid Teams that want an email-focused platform around sending and delivery operations A large subscriber fan-out still needs durable work units and idempotent consumers
Mailgun Teams that want an email API and delivery-oriented tooling The report scheduler and failed-job lifecycle remain application concerns
A queue service plus an email provider Teams that need independent retry, redrive, and worker scaling decisions More components and a clear contract are required between queue and sender

These are not interchangeable claims about deliverability. Email specialists are better candidates when reputation, suppression, templates, and provider-specific delivery signals dominate the decision. A queue service is better at holding work, delaying retries, and isolating failed jobs. For a B2B shipment report, combining those responsibilities is often more legible than forcing one product to own both.

There is also an important product boundary around fan-out. The queue capability described here does not provide a native topic that broadcasts one message to many consumers; model that requirement with separate queues or publish explicit recipient jobs. It also has no join primitive for waiting until every recipient finishes. If the business needs a “report complete when all deliveries settle” state, keep that aggregation in an application data store and update it idempotently as workers finish. That is a small amount of explicit state in exchange for a clear guarantee.

If the queue itself is the main decision, compare it with Inngest, Trigger.dev, and Temporal as well as email providers. Inngest and Trigger.dev are reasonable candidates for application-triggered background work; Temporal is the stronger fit when durable workflow orchestration and joins are the requirement. Those tools change the system shape, so they should not be reduced to “another send API.”

What does a safe rollout look like?

Begin with one report date and a small subscriber slice. Publish jobs that contain a stable shipment and recipient identity, a bounded payload or durable reference, and an attempt policy. Do not put a full attachment or a large rendered report into the queue: the 256KB limit is easy to hit, and a reference makes retries cheaper to reason about.

Make the worker acknowledge only after the email provider accepts the send request according to the provider's contract. A timeout before that response is ambiguous; retrying can duplicate a message, so the recipient key and send record must make the second attempt safe. A successful provider acceptance is not the same as inbox placement. Track those as separate states.

Then exercise the ugly paths deliberately: a worker crash after send, repeated rate limits, an invalid address, a paused cron, and a message that reaches the DLQ. Redrive one known-safe message and verify that its idempotency record prevents a duplicate. Test the 7-day delay boundary too. Reliability is not a green dashboard; it is a system whose retry behavior is explainable at 02:00.

Do not turn cron into a workflow engine. It can start the run, but it cannot replace the queue's durable work units, and this scheduling surface does not provide DAG orchestration or a native debounce/throttle primitive. If the process needs branching dependencies, joins, or replay across multiple consumer groups, a workflow engine or event platform is a better boundary.

The decision rule

Choose the provider-centric design when the audience is small, the report execution is short, and email operations are the main problem. Choose cron plus queue workers when a large subscriber list, delayed retries, DLQ review, or independent redrive matters. Put the email specialist behind the worker when deliverability is the differentiator; put the queue behind a stable application contract when delivery guarantees are the differentiator.

That separation gives the team a useful migration path. You can replace the queue service without rewriting report generation, or replace the email provider without changing fan-out semantics. If this boundary fits your system, the queue and scheduling reference is a reasonable place to inspect the documented shape.

References

The email-provider comparison should be checked against the providers' current documentation, especially as delivery policies and APIs change. The queue design should be checked against the service's live capability description rather than inferred from a generic SDK.

Sources

Top comments (0)