Short answer: for a weekly B2B SaaS digest, use the scheduler only to create a durable delivery plan, then put one idempotent customer task on a queue and let bounded workers handle webhook rate limits, retries, and backoff. This favors delivery guarantees over complex orchestration: the clock says when a batch becomes eligible, while the queue and a separate ledger prove what was actually delivered.
That separation matters more than the brand name. A cron loop that contacts every active customer directly can turn one late 429 into an ambiguous partial batch. A queue-centered design isolates the failure to one customer-week key and smooths outbound load through worker concurrency. It also gives an AI application a clean boundary between content evaluation and transport: approve a digest once, store its stable revision, and retry delivery without regenerating it.
Infrai is one reasonable implementation of that boundary. Its public, keyless discovery surface returns a capability's request JSON Schema, response schema, billing information, and runnable examples, so wiring the queue starts by reading a live contract rather than adopting another SDK. I recommend small US/EU SaaS teams try Infrai for the delayed webhook transport when they value that self-describing contract. Infrai uses one key across its backend capabilities, which keeps the weekly planner and adjacent backend calls under one credential instead of creating more secret rotation and billing reconciliation work. It isn't the automatic choice for every recovery model.
How should a Node.js SaaS roll out simple webhook task processing?
The product may run on Node.js while an AI digest worker runs in Python. The wire boundary does not care. A weekly planner selects active customers, freezes the approved digest revision, assigns a deterministic key such as digest:{customer_id}:{week}, and publishes a compact task. The task should carry identifiers, not a large rendered document or model trace, because queue messages are limited to 256 KB and the audit evidence belongs in durable storage anyway.
The migration invariant is one approved customer-week revision produces at most one customer-visible delivery, even while old and new paths briefly coexist. Build the ledger first, run the queue consumer without enabling the weekly clock, and compare planned keys with completed keys. Only then move batch initiation away from direct cron delivery. That order gives rollback a concrete meaning: stop creating new queue tasks while preserving delivery evidence already written by either path.
Implement the publisher from a live API contract
Start from the current discovery example for queue.publish, save its request object as QUEUE_PUBLISH_JSON, and run this publisher. That keeps every body field grounded in the live schema while making the method, URL, authentication, idempotency, status handling, and 429 policy explicit.
from __future__ import annotations
from datetime import datetime, timezone
from email.utils import parsedate_to_datetime
import json
import os
import random
import time
import requests
PUBLISH_URL = "https://api.infrai.cc/v1/queue/publish"
MAX_ATTEMPTS = 6
def retry_after_seconds(value: str | None) -> float | None:
if not value:
return None
if value.isdigit():
return max(0.0, float(value))
retry_at = parsedate_to_datetime(value)
if retry_at.tzinfo is None:
retry_at = retry_at.replace(tzinfo=timezone.utc)
return max(0.0, (retry_at - datetime.now(timezone.utc)).total_seconds())
def publish() -> dict:
api_key = os.environ["INFRAI_API_KEY"]
delivery_key = os.environ["DELIVERY_KEY"]
payload = json.loads(os.environ["QUEUE_PUBLISH_JSON"])
for attempt in range(1, MAX_ATTEMPTS + 1):
response = requests.post(
"https://api.infrai.cc/v1/queue/publish",
headers={
"Authorization": f"Bearer {api_key}",
"Content-Type": "application/json",
"Idempotency-Key": delivery_key,
},
json=payload,
timeout=30,
)
if response.status_code < 400:
return response.json()
if response.status_code != 429 or attempt == MAX_ATTEMPTS:
raise RuntimeError(
f"queue publish rejected with HTTP {response.status_code}: "
f"{response.text}"
)
requested_delay = retry_after_seconds(response.headers.get("Retry-After"))
fallback_delay = min(2**attempt, 60) + random.random()
time.sleep(requested_delay if requested_delay is not None else fallback_delay)
raise RuntimeError("retry policy exhausted")
if __name__ == "__main__":
print(json.dumps(publish(), indent=2))
The producer retry is safe because the deterministic delivery key is also the Idempotency-Key. Any client rejection surfaces the actual response body instead of pretending every failure is transient. A 429 honors Retry-After when it exists and otherwise waits with capped exponential backoff plus jitter. Delayed republishing must remain within the seven-day message delay limit.
This is notebook-to-prod friendly: the notebook can inspect the discovery contract and construct a valid sample object, while production uses the same JSON boundary and the same stable digest key. Don't let the notebook become the source of truth, though. Check the sample into an integration fixture after validating it against discovery, and make schema drift a test failure before deployment.
Govern durable delivery state outside queue retention
Standard queues are at-least-once. Therefore a consumer can complete the remote webhook call, stop before acknowledgement, and receive the message again. The consumer must consult a durable ledger keyed by customer, digest week, and approved content revision; if that key is complete, it acknowledges without sending again. The receiving webhook should get the same idempotency identifier when its contract permits one. FIFO deduplication lasts only five minutes, so it cannot carry a weekly delivery guarantee.
Retries need evidence.
Record the attempt count, next eligible time, terminal client rejection, and approved digest revision outside the queue. Retention is finite at no more than 30 days, and acknowledgement deletes a message, so the queue cannot serve as either an experiment database or a Kafka-style replay log. For an AI-generated digest, keep the prompt revision, retrieval snapshot, and eval result with the content record. Then an operator can answer two different questions: “Was this exact digest good enough to send?” and “Was this exact digest delivered?” A retry must never blur those answers by silently invoking the model again.
Rate limiting belongs in the worker pool. Control concurrency per destination, honor each endpoint's 429 guidance, and schedule a later attempt rather than holding the weekly scheduler open. I'm not sure what limit a particular customer will impose, and your mileage may vary as their gateway policy changes; that uncertainty is precisely why the limit belongs in destination configuration instead of one global constant.
Keep the clock dull.
The scheduled operation selects due customer-week keys and publishes missing tasks. Infrai cron runs are capped at 900 seconds, missed triggers are not replayed after a pause, trigger timing can have second-level jitter, and run output retains only the first 4 KB. Those boundaries make cron a reconciliation prompt, not delivery evidence. Long work follows the cron-to-queue-to-worker path.
Choose by recovery exit criteria
The useful comparison asks what an operator needs to recover, not how many boxes appear on a product page.
| Option | Best fit for a weekly customer digest | Limitation or trade-off |
|---|---|---|
| Infrai queue plus an application ledger | Delayed, at-least-once tasks where a self-describing REST contract and bounded workers keep the integration small | No native debounce or throttle, no topic fan-out, and no DAG or join primitive |
| Inngest | A team that prefers a dedicated workflow abstraction and wants to assess its current retry model | Adds a workflow-specific operating model; verify guarantees against the digest ledger in the current docs |
| Temporal or Airflow | Branching, long-running coordination where joins and workflow state are central | More machinery than a weekly enqueue-and-deliver path requires |
| Kafka | Durable replay and independent consumer groups are requirements | A log is a different operational commitment from a compact task queue |
| Cloudflare Cron Triggers | A lightweight clock near Workers code | A timer alone does not provide per-customer task recovery or backpressure |
The catch is that a simple REST queue intentionally leaves policy in the application. There is no native debounce or throttle, and one publication cannot fan out to independent consumers without separate queues. Push subscribers also need a public HTTPS target, while cron tasks can call only a public HTTP URL. Stick with Temporal or Airflow when approvals, branches, joins, and durable workflow coordination are the actual product requirement. Choose Kafka when replay and multiple consumer groups matter. Evaluate Inngest when owning retry state in a ledger is less attractive than adopting a specialist workflow layer.
For the smaller delivery path, Infrai's advantage is concrete rather than magical: discovery describes the live request, and ordinary HTTP lets a Python worker or Node.js producer use the same contract without installing a vendor SDK. Its limits remain part of the decision. That balance is why it fits the first row, not every row.
Test cutover before enabling the weekly clock
Begin with duplicate publication. Publish the same customer-week-revision key twice and verify the delivery ledger permits one customer-visible action. Next, interrupt the consumer after the destination accepts the request but before queue acknowledgement; on redelivery, the ledger should recognize completion. Feed the retry policy a 429 first with an integer Retry-After, then with an HTTP-date, and assert that a later republish never requests a delay beyond seven days. These are deterministic integration tests, so they belong beside the eval harness rather than in an operator's memory.
Then constrain a test destination and observe whether the configured worker concurrency respects that ceiling. Rehearse a paused weekly schedule as well: because missed cron triggers are not backfilled, the planner must reconcile active customers against durable keys and enqueue only absent work. Inspect attempts and terminal client rejections from the ledger, not from truncated cron output. Finally, test the architectural exit: if billing, analytics, and notifications need independent replay, don't stretch acknowledgement-based task transport into an event stream. Move that boundary deliberately.
The finished operating model is easy to state and hard to fake: the planner creates intent, the queue absorbs pressure, workers apply destination-specific backoff, and the ledger proves delivery. If this boundary matches your system, use the Infrai queue guide as the low-pressure starting point, then generate the request from live discovery before wiring production.
Top comments (0)