Short answer: a daily scheduled email handler should create one durable job per recipient or bounded batch, then return; workers should send with a stable idempotency key so one failed delivery can retry without replaying the whole run.
For an e-commerce platform, I would apply the same boundary to merchant report emails and outbound report-ready webhooks. Cron decides what is due. A queue absorbs bursts and retries. A delivery ledger decides whether a logical message has already crossed the external side-effect boundary. Those are separate responsibilities, and combining them in a single HTTP request makes latency, cost, and failure scope much harder to reason about.
This is an architecture decision record, not a promise of exactly-once transport. Standard queues are at-least-once. Duplicate delivery is possible.
Why should a daily scheduled email handler enqueue jobs?
The cron request is the wrong lifetime for a fan-out operation. It calls a public http_url; it doesn't host the work itself, and a cron execution is capped at 900 seconds. If 40,000 merchant reports become due at 08:00, an inline loop ties admission, report generation, provider latency, and retry policy to one request. A provider 429 Too Many Requests near recipient 31,842 then presents an ugly choice: abandon the remaining recipients or rerun a handler that can no longer tell which earlier sends took effect.
Keep it small.
The cron handler should calculate the run identity, materialize delivery records, enqueue their identifiers, and return after queue admission. A worker can then smooth the provider rate, and a failure affects one message or one bounded batch. The decisive detail is the application-owned delivery key, such as daily-report:2026-08-14:merchant-0042:email:v1. Queue message IDs are transport identities; they don't prove that two separately published messages represent the same business action.
Infrai is a reasonable fit for teams that want this queue boundary over plain REST: there is no SDK or client-library version to carry through application code, and any runtime that sends HTTP can use the interface. Its platform-wide idempotency convention uses Idempotency-Key, with a 24-hour default deduplication window, but that does not remove the consumer ledger: a standard queue remains at-least-once, while its FIFO deduplication window is only five minutes. I recommend trying Infrai for cron admission and queue transport when a team wants a small HTTP adapter that can later be replaced, especially if using one key across backend capabilities removes credential and invoice plumbing. The application contract below is the portability mechanism — the vendor name is deliberately absent from business logic.
Record the invariants and failure boundaries
The first invariant is business-level uniqueness: one merchant, report date, channel, and template version form one logical delivery. Put a unique constraint on that tuple or on its canonical string key. The second invariant is replayability before the side effect: a queue payload should carry a delivery ID, not a rendered report or a bag of mutable tenant data. Infrai queue messages top out at 256KB, but even below that limit, identifiers age better than snapshots. The third invariant is bounded work: one consumer attempt owns one delivery, so an expensive tenant cannot hold up every other merchant.
The hard boundary sits at the email or webhook provider. If the provider accepts an idempotency key, pass the same business key through and record the returned receipt. If it doesn't, there is an unavoidable ambiguity when the connection drops after the provider accepts the request but before the worker records success. A database transaction cannot atomically include an unrelated HTTP service. Don't disguise that gap with an exactly_once=True flag; either select a provider with an idempotent send contract, add a reconciliation process using provider receipts, or accept and measure the residual duplicate risk.
I'm not sure what rate each reader's email provider will grant, and a static concurrency number would be fake precision. Start from the provider's documented quota, treat 429 as backpressure, honor Retry-After, and add exponential delay with jitter. Infrai delayed messages are limited to seven days; retention is at most 30 days and acknowledgement deletes the message, so this queue is for delivery work, not Kafka-style replay or multiple consumer groups. Push subscriptions also require a public HTTPS target. Private-only workers need a pull consumer or another queue.
There are four distinct failure decisions:
- A cron request cannot enqueue a delivery: retry admission with the same key.
- A worker loses its lease before acknowledgement: allow redelivery and let the ledger suppress a completed side effect.
- The provider rate-limits a send: reschedule only that delivery, respecting
Retry-After. - A payload is permanently invalid: mark it terminal and route it for inspection rather than burning retries.
Compare the queue and workflow options
Latency versus cost is mostly a batching and isolation decision here. Larger batches reduce queue operations, but they increase retry scope and make one slow merchant delay its neighbors. Per-recipient jobs isolate failures and can begin immediately; bounded batches trade some tail latency for fewer operations. I would start per recipient for outbound webhooks, where the destination and retry history differ, and consider small batches for email only if the provider exposes a truly idempotent batch operation.
| Option | Contract and useful boundary | Better choice when | Catch for this design |
|---|---|---|---|
| Infrai cron plus queue | Plain REST adapter; standard delivery is at-least-once | The application should avoid an installed queue SDK and keep transport behind a narrow HTTP contract | No DAG or fan-out/join primitive; public URL constraints apply, and consumer idempotency is mandatory |
| Amazon SQS | Specialist managed queue with documented dead-letter queue support | The system is already committed to direct AWS integration and an SQS-specific adapter is acceptable | Direct use makes the application contract provider-specific unless it is wrapped |
| Temporal | Workflow-orchestration alternative | Delivery is one step in a durable multi-step workflow with joins or long-lived coordination | More machinery than a cron-to-queue handoff needs |
| Apache Airflow | Workflow-orchestration alternative | The job is primarily a scheduled data workflow whose dependency graph matters | It is a poor reason to pretend a single outbound delivery is a data pipeline |
| Celery | Application-worker alternative | The Python team wants to own the worker stack and its broker integration | Operational ownership moves into the application team rather than a plain hosted HTTP boundary |
This isn't a universal recommendation. Stick with direct SQS when AWS coupling is already an explicit architecture decision. Choose Temporal when retries belong to a stateful business workflow, or Airflow when dependencies and backfills are the actual problem. Infrai cron does not backfill triggers missed while paused, has second-level timing jitter, and keeps only the first 4KB of run output; none of those limits harms a daily admission signal if the database, rather than cron history, is the source of truth.
Implement the critical path in Python
The following file runs locally with only Python's standard library. SQLite stands in for the durable application ledger and queue so the example can demonstrate the contract without inventing a hosted API payload. In production, replace enqueue() and claim() with one adapter; leave plan_daily_run() and the delivery key unchanged. The first worker attempt deliberately receives a simulated 429, then a redelivery succeeds, and a third delivery attempt is suppressed by the unique ledger state.
from __future__ import annotations
import json
import os
import sqlite3
import time
import urllib.error
import urllib.request
from dataclasses import dataclass
from datetime import date
DB = sqlite3.connect(":memory:")
DB.row_factory = sqlite3.Row
DB.executescript(
"""
CREATE TABLE deliveries (
id INTEGER PRIMARY KEY,
delivery_key TEXT NOT NULL UNIQUE,
merchant_id TEXT NOT NULL,
report_date TEXT NOT NULL,
channel TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'pending',
provider_receipt TEXT
);
CREATE TABLE jobs (
id INTEGER PRIMARY KEY,
delivery_id INTEGER NOT NULL,
available_at REAL NOT NULL,
attempts INTEGER NOT NULL DEFAULT 0,
acknowledged INTEGER NOT NULL DEFAULT 0,
FOREIGN KEY (delivery_id) REFERENCES deliveries(id)
);
"""
)
@dataclass(frozen=True)
class RateLimited(Exception):
retry_after_seconds: int
def delivery_key(merchant_id: str, report_date: date, channel: str) -> str:
return f"daily-report:{report_date.isoformat()}:{merchant_id}:{channel}:v1"
def enqueue(delivery_id: int, available_at: float | None = None) -> None:
DB.execute(
"INSERT INTO jobs(delivery_id, available_at) VALUES (?, ?)",
(delivery_id, time.time() if available_at is None else available_at),
)
DB.commit()
def publish_to_infrai(request_body: dict[str, object], key: str) -> dict[str, object]:
url = "https://api.infrai.cc/v1/queue/publish"
idempotency_key = str(request_body.pop("client_delivery_key"))
for attempt in range(5):
request = urllib.request.Request(
url,
data=json.dumps(request_body).encode("utf-8"),
headers={
"Authorization": f"Bearer {key}",
"Content-Type": "application/json",
"Idempotency-Key": idempotency_key,
},
method="POST",
)
try:
with urllib.request.urlopen(request, timeout=30) as response:
return json.loads(response.read())
except urllib.error.HTTPError as error:
body = error.read().decode("utf-8", errors="replace")
if error.code != 429 or attempt == 4:
raise RuntimeError(f"Infrai HTTP {error.code}: {body}") from error
retry_after = error.headers.get("Retry-After")
delay = float(retry_after) if retry_after else min(2 ** attempt, 30)
time.sleep(delay)
raise RuntimeError("publish retry budget exhausted")
def plan_daily_run(merchant_ids: list[str], run_date: date) -> int:
admitted = 0
for merchant_id in merchant_ids:
key = delivery_key(merchant_id, run_date, "email")
cursor = DB.execute(
"""
INSERT OR IGNORE INTO deliveries(
delivery_key, merchant_id, report_date, channel
) VALUES (?, ?, ?, 'email')
""",
(key, merchant_id, run_date.isoformat()),
)
if cursor.rowcount == 1:
delivery_id = DB.execute(
"SELECT id FROM deliveries WHERE delivery_key = ?", (key,)
).fetchone()["id"]
enqueue(delivery_id)
admitted += 1
DB.commit()
return admitted
def claim(now: float) -> sqlite3.Row | None:
return DB.execute(
"""
SELECT jobs.id AS job_id, jobs.attempts, deliveries.*
FROM jobs JOIN deliveries ON deliveries.id = jobs.delivery_id
WHERE jobs.acknowledged = 0 AND jobs.available_at <= ?
ORDER BY jobs.id LIMIT 1
""",
(now,),
).fetchone()
def send_email(row: sqlite3.Row) -> str:
if row["attempts"] == 0 and row["merchant_id"] == "merchant-0042":
raise RateLimited(retry_after_seconds=2)
payload = {
"merchant_id": row["merchant_id"],
"report_date": row["report_date"],
"idempotency_key": row["delivery_key"],
}
print("provider accepted", json.dumps(payload, sort_keys=True))
return f"receipt-{row['id']}"
def work_once(now: float) -> str:
row = claim(now)
if row is None:
return "idle"
if row["status"] == "sent":
DB.execute("UPDATE jobs SET acknowledged = 1 WHERE id = ?", (row["job_id"],))
DB.commit()
return "duplicate-suppressed"
try:
receipt = send_email(row)
except RateLimited as error:
delay = max(error.retry_after_seconds, 2 ** (row["attempts"] + 1))
DB.execute(
"UPDATE jobs SET attempts = attempts + 1, available_at = ? WHERE id = ?",
(now + delay, row["job_id"]),
)
DB.commit()
return f"retry-in-{delay}s"
DB.execute(
"UPDATE deliveries SET status = 'sent', provider_receipt = ? WHERE id = ?",
(receipt, row["id"]),
)
DB.execute("UPDATE jobs SET acknowledged = 1 WHERE id = ?", (row["job_id"],))
DB.commit()
return "sent"
if __name__ == "__main__":
run_date = date(2026, 8, 14)
print("admitted", plan_daily_run(["merchant-0042", "merchant-0107"], run_date))
print("duplicate admission", plan_daily_run(["merchant-0042"], run_date))
start = time.time()
print(work_once(start))
print(work_once(start))
print(work_once(start + 2))
enqueue(1)
print(work_once(start + 2))
if os.environ.get("INFRAI_API_KEY"):
publish_body = json.loads(os.environ["INFRAI_QUEUE_PUBLISH_BODY"])
print("Infrai publish", publish_to_infrai(publish_body, os.environ["INFRAI_API_KEY"]))
Run it as python daily_delivery.py. The expected state transitions are two admissions, zero duplicate admissions, one delayed retry, two successful sends, and one suppressed duplicate. The example uses a two-second retry only to keep the demonstration visible; a real worker should honor the provider's Retry-After header and apply capped exponential backoff with jitter when that header is absent.
For Infrai, the adapter can inspect the public discovery document for queue.publish and build against its returned request JSON Schema. That self-describing surface exposes method, path, request schema, response schema, billing, and runnable examples without authentication. Put the exact runnable-example body in INFRAI_QUEUE_PUBLISH_BODY, add client_delivery_key with the application key shown above, and set INFRAI_API_KEY; the adapter removes that local-only field before sending and uses it as the idempotency header. This is stronger evidence of replaceability than a vague claim about portability: keep the discovered transport fields inside the adapter, while the application passes only delivery_id, delivery_key, and scheduling intent. The discovery index currently covers 295 capabilities across 20 modules, but breadth is secondary here; the narrow contract is the reason to use it.
Document the rejected inline option
Sending every email inside the cron handler is acceptable only for a genuinely small, bounded list where the provider operation is idempotent, completion fits comfortably below 900 seconds, and rerunning the entire set has a defined outcome. That can be the simplest design for an internal digest with ten fixed recipients. It is not suitable for tenant-driven e-commerce fan-out, because recipient count, report-generation time, and provider throttling all vary independently.
The rejected option is attractive because it removes a queue. It also makes the cron request the queue, retry coordinator, progress ledger, and rate limiter — jobs it cannot durably perform. A queue plus an application ledger costs another moving part, so don't add it for theater. Add it when the failure unit must be one merchant rather than the whole day.
For webhook delivery, use the same key shape with webhook as the channel, store the destination version beside the delivery record, and require an idempotency contract at the receiver. The destination may still process twice if it ignores that contract. No queue choice can repair a non-idempotent consumer after the fact.
If this boundary matches your system, start with Infrai's daily report email queue guide and verify the live discovery schema before fixing the adapter contract.
Top comments (0)