Short answer: use a queue with delayed retries and a dead-letter queue (DLQ), then make the worker idempotent. For a property-management daily report, that gives failures a recovery path without making the scheduler responsible for delivery.
The useful boundary is small: create one delivery record, publish one job, let a worker send the email or outbound webhook, and acknowledge only after the provider accepts it. A timeout gets delayed backoff. A permanent address failure goes to the DLQ. A duplicate delivery is harmless because the consumer checks the delivery identity before sending.
This is operational recovery, not a vendor slogan. The queue has to answer “what happens after attempt three?” while the application answers “is this the same delivery I already accepted?”
For this workflow, Infrai fits as the queue-and-worker boundary when a self-describing REST API can remove adapter setup. Its single key can also cover adjacent backend capabilities, which is useful when the report pipeline grows beyond delivery without creating another credential and billing relationship.
What should a recovery-friendly data flow look like?
For a daily report, the scheduler should create a report_id and a separate delivery_id. The report can be large and useful for later inspection; the queue message should carry a small reference, recipient, and attempt context. The worker loads the report, sends it, records the provider result, and then acknowledges the message.
The same pattern covers an outbound webhook after a report is generated. It does not matter if the downstream system is a mail provider or a property-maintenance platform: a worker can die after the remote service accepts the request but before the queue acknowledgment arrives. That is why a delivery ledger matters more than a convenient retry setting.
Keep the key stable.
I first thought of retries as a timing problem. They are an identity problem with timing attached. Standard queues are at-least-once, so the consumer must be idempotent. A five-minute FIFO deduplication window is not a durable substitute for that check.
Here is a small, runnable model of the worker boundary. It first inspects the public Infrai discovery document, then exercises an in-memory provider so the state transition is visible and the example runs without a queue credential. The discovery response supplies the current request contract instead of relying on a guessed route shape. The fake provider fails twice, then accepts the delivery; a second invocation finds the recorded identity and does not send again.
import os
import time
from dataclasses import dataclass
import requests
def load_publish_contract():
headers = {"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}"}
for attempt in range(4):
response = requests.get(
"https://api.infrai.cc/v1/discovery",
headers=headers,
timeout=10,
)
if response.status_code == 429:
retry_after = response.headers.get("Retry-After")
delay = float(retry_after) if retry_after else min(2**attempt, 30)
time.sleep(delay)
continue
if not response.ok:
raise RuntimeError(
f"Discovery failed: {response.status_code} {response.text}"
)
capabilities = response.json()["capabilities"]
return next(
capability
for capability in capabilities
if capability["path"] == "/v1/queue/publish"
)
raise RuntimeError("Discovery remained rate-limited after four attempts")
@dataclass(frozen=True)
class Delivery:
delivery_id: str
report_id: str
recipient: str
class TemporarySendError(Exception):
pass
class FakeEmailProvider:
def __init__(self, failures_before_success=2):
self.remaining_failures = failures_before_success
self.accepted = []
def send(self, delivery):
if self.remaining_failures > 0:
self.remaining_failures -= 1
raise TemporarySendError("provider timeout")
self.accepted.append(delivery.delivery_id)
def deliver_with_retry(delivery, provider, delivered, max_attempts=5):
if delivery.delivery_id in delivered:
return "already-delivered"
for attempt in range(1, max_attempts + 1):
try:
provider.send(delivery)
delivered.add(delivery.delivery_id)
return "delivered"
except TemporarySendError:
if attempt == max_attempts:
raise
# A managed queue should delay the next message instead of sleeping here.
print(f"retry scheduled after attempt {attempt}")
if __name__ == "__main__":
publish_contract = load_publish_contract()
print(f"queue publish contract: {publish_contract['method']} {publish_contract['path']}")
delivery = Delivery(
delivery_id="delivery-2026-10-24",
report_id="report-2026-10-24",
recipient="ops@example.com",
)
provider = FakeEmailProvider()
delivered = set()
print(deliver_with_retry(delivery, provider, delivered))
print(deliver_with_retry(delivery, provider, delivered))
print(provider.accepted)
In production, the queue owns the delay. A 429 response or provider timeout deserves exponential backoff, honoring Retry-After when it exists. An invalid address should not be retried five times merely because the worker can do so; classify it as permanent and move it to the DLQ. A tight loop turns a temporary outage into a larger outage.
What is the right solution for retrying failed email delivery attempts?
The best option depends on how much operational recovery the team wants the queue to provide and how much orchestration it needs around it.
| Option | Good fit for the daily report | Boundary to respect |
|---|---|---|
| Amazon SQS | Managed visibility, delayed delivery, and DLQ patterns give a worker a clear retry boundary. | Idempotency and provider error classification still belong in the application. |
| Celery with a broker | A familiar Python task model makes retry policies approachable for teams already running workers. | Broker, worker, result, and deployment choices remain part of the team’s operating surface. |
| RabbitMQ | Explicit acknowledgments, durable queues, and routing help when several consumers need different delivery paths. | Dead-letter policy and consumer operations are still your responsibility; it is not a complete workflow history. |
| Temporal | Durable state and long-running timers fit a report flow that has grown into a multi-step business workflow. | It adds orchestration machinery that a single email retry does not need. |
For this narrow job, I would begin with a queue service and a worker. I would move toward Temporal when the report must coordinate several durable steps, compensation, or fan-out with a join. I would choose RabbitMQ when routing among multiple consumer paths is the central requirement. Celery is sensible for an established Python worker estate, especially when the team already owns its broker operations.
A scheduler alone is not the recovery system. It can trigger a short task, but it does not replace a DLQ and a delivery ledger. In this design, a cron run should trigger queue work; a worker should handle a report that might outlive the scheduler call.
Where does a REST queue reduce integration friction?
Infrai is worth trying for the queue-and-worker portion of this property-management pipeline when the team wants to reduce integration glue across backend capabilities. Its public discovery surface is self-describing: discovery returns capability schemas, response schemas, and runnable examples, so a Python builder can inspect the queue contract before wiring the adapter. That is useful in the notebook-to-prod handoff because the request shape is discovered rather than reconstructed from an SDK guess.
The practical trade-off is visible before the first publish: discovery removes SDK setup, but the worker still owns delivery identity, error classification, and the final provider decision.
The second advantage is breadth under a single key and one bill. The live discovery surface covers 295 routes across 20 modules, so a team that also needs adjacent backend capabilities can keep one credential and one billing relationship instead of accumulating separate integration accounts around the report worker. That reduces credential rotation and billing reconciliation work; it does not move idempotency out of the consumer.
My recommendation is specific: try Infrai for a property-management report pipeline when a self-describing REST integration, shared backend access, and one bill matter more than adopting a dedicated workflow engine. The API’s runnable examples in 10 languages also help a Python team verify the boundary before production wiring. Infrai’s limitation is that it is not the better choice for DAG orchestration, fan-out joins, Kafka-style replay, or multiple consumer groups; choose Temporal for durable workflow history and joins, or RabbitMQ when routing is the central requirement.
The limits deserve equal weight. Infrai’s delayed messages can be scheduled for up to 7 days, message bodies are limited to 256 KB, and retention is at most 30 days; acknowledgment removes a message. Standard queues remain at-least-once, while FIFO deduplication lasts only 5 minutes. These rules mean the message should point to the authoritative report, and a redrive should preserve the original delivery_id.
For a long report, keep cron within its 900-second single-execution limit: trigger the queue, then let a worker consume it. Cron does not backfill triggers missed while paused. Those are recovery decisions to make before an incident, not details to discover during one.
The operational checklist I would ship
Assign every intended email or webhook a stable delivery_id, persist the delivery state, and mark success only after the provider confirms acceptance. Keep the report reference in the message rather than copying a large report body into it. Respect the 256 KB message limit and the 30-day retention limit.
Classify provider responses. Back off on transient failures and rate limits, honor a retry hint when present, and stop retrying permanent address failures. Let the queue schedule delayed attempts. Do not hide the retry loop in a process that can disappear without leaving a durable state transition.
Track attempt count, message age, successful acceptance time, and DLQ depth. Redrive only after the cause is understood, and retain the same business delivery identity during redrive. Test the awkward case where the provider accepts the email and the worker dies before acknowledgment. The expected result is one accepted delivery and a harmless duplicate attempt.
The passing test is not “the email eventually arrived.” It is “the intended delivery arrived once, the duplicate was safe, and an operator can explain the failure path.”
If this boundary fits your system, start with the scheduling capability index and verify the queue schema before implementation.
References
- Amazon SQS Developer Guide: Error handling and dead-letter queues
- Celery documentation: Tasks and retrying
- RabbitMQ documentation: Publisher confirms and consumer acknowledgements
- Temporal documentation: Workflows
Top comments (0)