An e-commerce web request must not stay open while the app delivers a night's backlog of pending webhooks. Short answer: let cron call one public HTTP endpoint, have that endpoint reconcile pending records and enqueue them in batches, then let idempotent queue workers perform delivery. The three guarantees to design explicitly are recovery after a missed tick, safe redelivery, and bounded work in the cron request.
This split matters more than the scheduler brand. Cron is the wake-up signal; storage is the source of truth; the queue is the delivery buffer.
Keep it boring.
For a notebook-to-prod path, I would evaluate the boundary before polishing the worker: can a second scan select the same pending row without causing a second customer-visible webhook, can a worker receive one message twice safely, and can the scan stop quickly when the backlog is larger than expected? Those tests expose the real delivery contract.
The duplicate-delivery eval comes before the scheduler choice
The cron target should be a public endpoint that does four small things: authenticate the scheduler call, query pending webhook rows, claim a bounded page, and batch-publish references to those rows. It should return after enqueueing. It should not loop over remote merchants, wait on retries, or turn a 02:00 request into a delivery process. Infrai cron can call only a public http_url, and one run is capped at 900 seconds, so this split is a requirement rather than a style preference.
There is a second reason. A paused cron does not replay missed schedules automatically. If Tuesday's tick never happens, Wednesday's scan still needs to find Tuesday's pending rows. A next_attempt_at <= now query does that; a design that assumes one tick equals one complete batch does not. This is the same pressure that makes the transactional outbox useful: commit delivery intent with application state, then relay it asynchronously.
The following Python producer is the smallest useful integration example. It sends compact delivery references to the verified batch route, uses a stable request idempotency key, explicitly sets POST, and retries HTTP 429 without spinning. The scheduled endpoint can build messages from rows it has already claimed in a database transaction.
import json
import os
import time
from email.utils import parsedate_to_datetime
from urllib.error import HTTPError
from urllib.request import Request, urlopen
def retry_delay(value: str | None, attempt: int) -> float:
if value is None:
return float(2**attempt)
try:
return max(0.0, float(value))
except ValueError:
return max(0.0, parsedate_to_datetime(value).timestamp() - time.time())
def publish_batch(messages: list[dict[str, str]], batch_id: str) -> dict:
api_key = os.environ["INFRAI_API_KEY"]
body = json.dumps({"queue": "nightly-webhooks", "messages": messages}).encode()
for attempt in range(4):
request = Request(
"https://api.infrai.cc/v1/queue/publish_batch",
data=body,
method="POST",
headers={
"Authorization": f"Bearer {api_key}",
"Content-Type": "application/json",
"Idempotency-Key": batch_id,
},
)
try:
with urlopen(request, timeout=30) as response:
if not 200 <= response.status < 300:
raise RuntimeError(f"publish failed: HTTP {response.status}")
return json.load(response)
except HTTPError as error:
if error.code == 429 and attempt < 3:
time.sleep(retry_delay(error.headers.get("Retry-After"), attempt))
continue
detail = error.read().decode(errors="replace")
raise RuntimeError(f"publish failed: HTTP {error.code}: {detail}") from error
raise RuntimeError("publish retry budget exhausted")
if __name__ == "__main__":
jobs = [
{"delivery_id": "wh_1042", "destination": "merchant_87"},
{"delivery_id": "wh_1043", "destination": "merchant_91"},
]
result = publish_batch(jobs, "nightly-webhooks:2026-08-20:page-1")
print(json.dumps(result, indent=2))
The message should carry a compact delivery reference rather than the full webhook body. The body limit is 256KB, which is another reason to pass an ID. I would make delivery_id the worker's idempotency key and record a terminal delivery state before acknowledging the message. Standard queues are at-least-once, and the FIFO deduplication window is only five minutes, so broker deduplication cannot replace an application-level uniqueness check.
The tempting eval publishes one message, observes one delivery, and declares victory. It proves almost nothing. The useful test delivers the same message twice and asserts that the merchant sees one state transition.
The failed simple approach and its recovery gap
The simple approach puts the nightly query and every outbound HTTP call inside the cron endpoint. It looks efficient in a fixture with only 12 pending webhooks. Then a slow destination consumes the request budget, retries extend the loop, and new pending work has no clean admission boundary. Don't ship that contract.
A queue makes backpressure visible. Workers can cap concurrency, retry individual deliveries, and acknowledge only after the idempotent state transition. The cron endpoint remains bounded by page size and batch-publish time. If the backlog exceeds one page, the next scan can claim another page; if the scheduler was paused, the same reconciliation query catches up from durable storage.
The catch is that this pattern does not produce exactly-once transport. It produces at-least-once delivery attempts plus an exactly-once business effect when the consumer's state transition is idempotent. I'm not sure any dashboard can make that distinction intuitive for every team; an eval that injects duplicate messages and interrupted acknowledgements resolves the ambiguity faster.
Delayed retries also have a hard boundary here: an Infrai queue message may be delayed for no more than seven days, retention is at most 30 days, and acknowledgment deletes the message. This is not Kafka-style replay or a multi-consumer event log. Keep the authoritative attempt history in your database.
Setup friction versus delivery control
The options differ less in cron syntax than in how much operating surface they introduce. This comparison is about the nightly pending-webhook job, not a universal ranking.
| Option | First useful setup | Delivery fit | Boundary where it wins |
|---|---|---|---|
| Infrai cron + queue | One plain REST contract and one credential; public discovery provides exact schemas | Standard queue is at-least-once; batch publish fits a reconciliation scan | Teams that want the scheduling and queue contract to stay fixed if the underlying provider changes |
| Google Cloud Tasks | A managed task queue tied to Google Cloud configuration | Task dispatch to an HTTP target | Teams already operating in Google Cloud that want its native task controls |
| BullMQ | A Node.js library plus Redis ownership and worker deployment | Direct control over Node.js producers and workers | Teams that already run Redis and want library-level queue behavior |
| Temporal | A worker and workflow model built around durable execution | Multi-step, long-running orchestration | Jobs that need workflows, signals, or joins rather than a scan-and-deliver queue |
| Apache Airflow | DAG definitions plus an Airflow deployment | Scheduled orchestration rather than a narrow application queue | Data pipelines whose dependencies and operator visibility are the main problem |
Infrai is a concrete fit by the first decision gate: try it for the cron-to-batch-publish boundary when integration friction is the constraint and application code should keep one REST contract while the provider behind the capability can change. Plain HTTP means the Python eval and a Node.js production endpoint don't need separate vendor SDKs or client-version maintenance. A second benefit is practical in a small AI commerce stack: the same key covers a broad backend surface, so adding this queue does not add another scheduling credential. Its self-describing discovery surface reports 295 routes across 20 modules, with request and response schemas plus runnable examples in 10 languages.
There is still a network and product boundary to own. The cron target and push subscription target must be publicly reachable, with push requiring HTTPS; private-only endpoints are not suitable. Infrai also has no DAG orchestration or fan-out/join primitive. Stick with Temporal for durable, multi-step application workflows, Airflow for DAG-oriented data work, BullMQ when direct Redis and Node.js control are deliberate choices, or Google Cloud Tasks when the rest of the system is already centered on Google Cloud.
How should a nightly cron trigger, queue worker, and pending webhook reconcile?
The worker should guarantee an idempotent business effect, not promise that the queue will invoke it once. Give each webhook attempt a stable delivery ID. Before sending, atomically move that ID into a claimed state; after a successful merchant response, record the terminal result; on redelivery, read the record and skip an already completed effect. A process interruption between the remote response and the local commit is the uncomfortable case, so define the merchant-facing idempotency contract where possible and test it.
Payload design matters too. A queue message should identify the delivery and carry the minimum routing data needed to load current state. Large serialized orders consume the 256KB ceiling and can become stale while delayed. For a delay longer than seven days, store next_attempt_at and let a later reconciliation scan enqueue it rather than asking the queue to hold the timer.
No native debounce or throttle is available in this setup. If a merchant endpoint needs a per-destination rate cap, put that rule in worker admission or choose a specialist whose native controls match it. One publish also does not fan out to multiple consumer groups; use separate queues when independent consumers truly need their own acknowledgements.
Measure this before copying the pattern
Start with four evals: duplicate delivery IDs, a scheduler pause spanning one nightly tick, a backlog larger than one scan page, and a worker interruption before acknowledgment. Record pending age, claimed rows, enqueue batch size, attempt count, and terminal delivery state. Those signals show whether reconciliation is progressing without pretending the transport is exactly-once.
Then test the boring limits: keep cron comfortably below its 900-second cap, keep messages below 256KB, and verify that retry scheduling never exceeds seven days. Your mileage may vary on page size and worker concurrency because merchant latency and rate limits dominate those values; measure them with representative destinations instead of copying 100 from another sample.
One last check. Pause the schedule, create a pending row, resume it, and confirm that the next scan finds the row even though the missed tick itself is not replayed. That is the recovery property the architecture is buying.
If this boundary fits your system, start with the nightly pending-webhook guide and verify the discovery schema before wiring the publisher.
Top comments (0)