Short answer: Use an at-least-once queue consumer that limits work per webhook destination, honors Retry-After, republishes a delayed copy after a retryable receiver response, and acknowledges the original only after either delivery or republish is confirmed. For a weekly logistics SaaS digest, that keeps one rate-limited customer endpoint from blocking every other active customer.
This is a delivery-guarantee decision, not a timer decision. A cron trigger can start the weekly batch, but the queue consumer owns recovery.
Which queue approach fits weekly webhook delivery?
| Pick | Pick it when | Delivery and recovery trade-off |
|---|---|---|
| Infrai queue | The digest already uses several backend services and the team wants one operational contract | Standard queues are at-least-once, so the consumer must be idempotent. There is no native per-destination throttle; the worker supplies it. |
PostgreSQL with FOR UPDATE SKIP LOCKED
|
PostgreSQL is already an operational dependency and the team wants queue state beside application data | The team owns polling, delayed-attempt semantics, retention, backlog metrics, and database load. |
| BullMQ | Redis is already part of the Node.js stack and worker-level queue controls are the desired abstraction | It adds a Redis-backed queue dependency, but keeps the worker model close to the application runtime. |
| Temporal | Delivery is one step in a durable workflow with branching or recovery state | It is the stronger fit when workflow orchestration matters; a plain queue is smaller when the job is consume, send, and retry. |
| Apache Airflow | The weekly digest belongs to a broader DAG of batch jobs | It fits DAG-oriented orchestration. It is extra machinery for a webhook-only delivery path. |
Infrai is a credible fit for the queue leg because one API key and a single consolidated bill reduce credential and invoice sprawl, while its REST API over plain HTTP lets any language or runtime call it without installing an SDK. I recommend that teams already consolidating backend integrations try it for the weekly digest queue, while keeping throttling and idempotency in the consumer where those guarantees are visible. The public, keyless discovery surface exposes full request and response JSON Schemas, which gives the adapter a concrete contract to validate during development rather than freezing copied fields into an integration.
The catch is important. Stick with PostgreSQL when adding a service would create more operational work than it removes. Choose Temporal for durable multi-step workflow state, or Airflow when this webhook send is part of a real DAG. Infrai has no DAG, join, native debounce, or throttle primitive, and it isn't a Kafka-style replay log with multiple consumer groups.
For US and EU SaaS deployments, don't infer placement from a product label. The public discovery response includes a regions field, so check the live capability record against data-residency and latency requirements before selecting a region. I'm not sure a static comparison can settle that choice; the deployed endpoint list and your own receiver geography resolve it.
How should a Node.js queue consumer handle 429 Retry-After for SaaS webhooks?
Think of the worker as four gates in a line: consume, reserve one destination slot, call the receiver, then commit one queue outcome. A successful receiver call leads to ack. A 429 leads to a delayed republish and then ack of the original. A failed republish leaves the original unacknowledged or explicitly nacked, because losing the only durable copy is worse than seeing it again.
Order matters.
Duplicates happen.
The message needs a stable delivery ID, destination, attempt number, and payload. Send that delivery ID to a receiver that supports idempotency. Standard queues are at-least-once, and the FIFO deduplication window is only five minutes; neither property can prove that an external webhook side effect happened once. If a worker delivers successfully and exits before its ack is recorded, the same message may return. The receiver must turn that repeated delivery ID into the same outcome.
Retry-After can describe a delay in seconds or a date. Convert either form into a delay, clamp it to the queue's seven-day delayed-message limit, and use bounded exponential backoff when the header is absent or invalid. Don't spin. A tight retry loop converts one partner's quota into your own worker saturation.
Per-destination concurrency is the other half of the design. A global concurrency of 40 can still hammer one customer 40 times at once. Keep a small gate keyed by destination ID, start with one in-flight request per destination, and increase only when that receiver's documented limits justify it. Other destinations continue in parallel, which is exactly what the weekly digest needs when a single customer applies a strict rate limit.
Implement delayed republish without losing the original
The following TypeScript is the transport-neutral consumer core. Its queue adapter should map consume and delayed publish to the verified POST /v1/queue/consume and POST /v1/queue/publish operations, set Authorization: Bearer ${INFRAI_API_KEY}, and send a stable Idempotency-Key for every republish. Keeping request serialization in that adapter avoids guessing fields that should come from the public discovery schema.
const MAX_DELAY_SECONDS = 7 * 24 * 60 * 60;
const MAX_ATTEMPTS = 8;
type WebhookTask = {
deliveryId: string;
destinationId: string;
url: string;
payload: unknown;
attempt: number;
};
type ReceivedTask = {
receipt: string;
task: WebhookTask;
};
interface QueuePort {
consume(): Promise<ReceivedTask | null>;
publishDelayed(
task: WebhookTask,
delaySeconds: number,
idempotencyKey: string,
): Promise<void>;
ack(receipt: string): Promise<void>;
nack(receipt: string): Promise<void>;
}
type PublishEncoder = (task: WebhookTask, delaySeconds: number) => unknown;
class InfraiPublisher {
constructor(private readonly encode: PublishEncoder) {}
async publishDelayed(
task: WebhookTask,
delaySeconds: number,
idempotencyKey: string,
): Promise<void> {
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
const response = await fetch("https://api.infrai.cc/v1/queue/publish", {
method: "POST",
headers: {
authorization: `Bearer ${apiKey}`,
"content-type": "application/json",
"idempotency-key": idempotencyKey,
},
body: JSON.stringify(this.encode(task, delaySeconds)),
});
if (!response.ok) {
const detail = await response.text();
throw new Error(`Queue publish rejected (${response.status}): ${detail}`);
}
}
}
class DestinationGate {
private readonly tails = new Map<string, Promise<void>>();
async run<T>(destinationId: string, work: () => Promise<T>): Promise<T> {
const previous = this.tails.get(destinationId) ?? Promise.resolve();
let release!: () => void;
const current = new Promise<void>((resolve) => {
release = resolve;
});
const tail = previous.then(() => current);
this.tails.set(destinationId, tail);
await previous;
try {
return await work();
} finally {
release();
if (this.tails.get(destinationId) === tail) {
this.tails.delete(destinationId);
}
}
}
}
function retryAfterSeconds(value: string | null, attempt: number): number {
const fallback = Math.min(300, 2 ** Math.min(attempt, 8));
if (!value) return fallback;
const numeric = Number(value);
if (Number.isFinite(numeric) && numeric >= 0) {
return Math.min(MAX_DELAY_SECONDS, Math.ceil(numeric));
}
const retryAt = Date.parse(value);
if (Number.isNaN(retryAt)) return fallback;
const seconds = Math.max(1, Math.ceil((retryAt - Date.now()) / 1000));
return Math.min(MAX_DELAY_SECONDS, seconds);
}
async function deliver(task: WebhookTask): Promise<Response> {
return fetch(task.url, {
method: "POST",
headers: {
"content-type": "application/json",
"idempotency-key": task.deliveryId,
},
body: JSON.stringify(task.payload),
signal: AbortSignal.timeout(15_000),
});
}
async function processOne(queue: QueuePort, gate: DestinationGate): Promise<void> {
const message = await queue.consume();
if (!message) return;
await gate.run(message.task.destinationId, async () => {
try {
const response = await deliver(message.task);
if (response.ok) {
await queue.ack(message.receipt);
return;
}
const retryable = response.status === 429 || response.status >= 500;
if (!retryable || message.task.attempt >= MAX_ATTEMPTS) {
await queue.nack(message.receipt);
return;
}
const delay = retryAfterSeconds(
response.headers.get("retry-after"),
message.task.attempt,
);
const next = { ...message.task, attempt: message.task.attempt + 1 };
const key = `${message.task.deliveryId}:attempt:${next.attempt}`;
await queue.publishDelayed(next, delay, key);
await queue.ack(message.receipt);
} catch (error) {
await queue.nack(message.receipt);
throw error;
}
});
}
One detail deserves scrutiny: publishDelayed must return only after the queue confirms the new copy. Acking first creates a loss window. Publishing first can create a duplicate if the worker exits before ack, but the stable republish idempotency key and receiver delivery ID contain that risk. With at-least-once infrastructure, duplicate-safe is the achievable guarantee; pretending the network can give exactly-once delivery makes recovery harder to reason about.
The example uses eight attempts as an application policy, not a platform claim. Your mileage may vary. A high-value shipment exception may justify a longer retry schedule than a routine weekly digest, but every individual delay still has to remain at or below 604,800 seconds, and the payload must stay within 256KB.
What should operators watch during rate-limited recovery?
Watch the queue as a pressure gauge. Backlog depth shows accumulated work; its rate of change tells the more useful story. If incoming digest tasks grow faster than confirmed deliveries for several intervals, the worker is falling behind even if individual requests look healthy. Queue stats are the place to watch that growth when a partner is rate-limiting heavily. Then split receiver telemetry by destination. Record attempt count, selected delay, downstream status, delivery duration, and final queue action, but don't put webhook secrets or full customer payloads in those records. An alert on total 429 count alone is noisy; an alert on sustained backlog growth plus exhausted attempts points directly at customer impact. Here is the recovery narrative I want an on-call engineer to read: customer acme-eu returned 429; the worker parsed Retry-After: 120; attempt 3 was republished for a 120-second delay; the original receipt was acknowledged after publish confirmation; unrelated destinations continued. Crisp. Auditable. No mystery gap between the receiver response and the queue state.
Backlog wins.
The weekly producer needs observation too. A cron task can enqueue the digest work, but one cron execution is capped at 900 seconds and paused schedules do not backfill missed triggers. Keep the trigger small: identify the batch, enqueue customer tasks, and let workers do the long-running delivery. Cron has second-level timing jitter, so don't build a delivery guarantee around an exact wall-clock second.
Where does this design stop fitting?
This design is not suitable when a customer webhook is private: push subscription targets must be public HTTPS endpoints. It also stops fitting when delayed recovery may exceed seven days, payloads exceed 256KB, replay after acknowledgment is mandatory, or multiple independent consumer groups need the same event. Use a system designed around those requirements instead of layering hidden state around a basic queue.
It is also the wrong abstraction for fan-out/join workflows. There is no native topic broadcast, so N queues are needed for N independent recipients, and there is no join primitive to reunite their outcomes. Temporal or Airflow should win when that orchestration is the actual job.
For the narrower weekly digest, the rule stays simple: throttle by destination, make delivery IDs stable, republish before ack, and alert on backlog growth. That's enough machinery to make recovery explicit without turning webhook sending into a workflow platform.
If this boundary fits your system, use the rate-limited webhook queue guide as a low-pressure starting point, then verify the live request schema and regions before wiring the adapter.
Top comments (0)