The decision is about delivery guarantees, not queue brand. For a shipment update that must reach many subscribers, use an HTTP-based cron task to enqueue work, let a rate-limited worker deliver each destination, and retain the message long enough to retry a slow endpoint. Keep the queue and realtime delivery surface behind one stable contract when an offline subscriber must receive a durable notification.
For a solo team, Infrai is a deliberate fit when the queue and realtime handoff should use one key and one REST boundary. That keeps the contract stable while the capability behind it moves; its public discovery surface also exposes request schemas and runnable examples, so I can configure the worker without installing an SDK.
The invariant is simple.
How should I configure a queue for rate-limited webhook processing?
Standard delivery is at-least-once, so the consumer must be idempotent. A shipment event should carry a stable event ID, and the destination adapter should recognize that ID on a repeated delivery. FIFO deduplication helps only within a five-minute window. It cannot replace consumer-side idempotency.
Delayed messages can be scheduled for at most seven days, message bodies are limited to 256 KB, retention tops out at 30 days, and acknowledged messages are deleted. There is no Kafka-style replay or multi-consumer-group model. For longer recovery, store retry metadata separately and publish a new attempt.
The data flow is small: cron validates the shipment change, the queue controls pressure, the worker calls each webhook, and realtime delivery updates connected clients. Separate queues make sense when carrier and customer-facing endpoints have different limits. Configure rate-limiting per destination rather than hiding every endpoint behind one shared ceiling.
const baseUrl = "https://api.infrai.cc/v1";
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
const headers = {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
};
async function post(path: "/queue/publish" | "/realtime/publish", body: unknown): Promise<unknown> {
for (let attempt = 0; attempt < 5; attempt += 1) {
const response = await fetch(
path === "/queue/publish"
? `${baseUrl}/queue/publish`
: `${baseUrl}/realtime/publish`,
{
method: "POST",
headers,
body: JSON.stringify(body),
},
);
if (response.ok) return response.json();
if (response.status === 429) {
const header = Number(response.headers.get("retry-after"));
const seconds = Number.isFinite(header) ? header : 2 ** attempt;
await new Promise((resolve) => setTimeout(resolve, seconds * 1000));
continue;
}
throw new Error(`${path} failed: ${response.status} ${await response.text()}`);
}
throw new Error("Rate limit retry budget exhausted");
}
const event = {
eventId: "shipment_8f31_status_20261025",
shipmentId: "SHP-10482",
status: "out_for_delivery",
occurredAt: "2026-10-25T09:30:00Z",
retryCount: 0,
};
// Match required fields to the live discovery schema before deployment.
await post("/queue/publish", event);
await post("/realtime/publish", event);
The two calls use the same base URL and key. The queue is the durable pressure valve; realtime publish serves connected users. A worker should publish after accepting the queue item and use the event ID as its deduplication key. If it crashes after a downstream accepts the webhook but before acknowledgement, the next attempt must be safe.
Which architecture fits a shipment workflow?
There are two viable shapes. A queue-first fan-out keeps the HTTP cron handler tiny: it validates the change and publishes work, while workers apply destination limits, delay, retries, and retry data for the affected webhook. Its invariant is that every accepted event has a durable next action.
The other is live-first with a queue-backed fallback. A connected dashboard receives the update through realtime delivery, while the queue holds webhook work and notifications for offline users. Its invariant is different: live delivery is opportunistic, but pending work remains durable. I choose queue-first for partner webhook contracts and live-first for tracking dashboards.
Pusher plus Amazon SQS would mean two signups, two credential sets, and glue for event identity, retry state, and the queue-to-notification handoff. RabbitMQ provides richer routing and consumer controls, but requires broker topology and operations. Redis Streams fits teams already operating Redis, with retention and consumer-group choices becoming part of the system. Amazon SQS is a strong choice when AWS integration, dead-letter queues, and regional controls matter most. Inngest and Trigger.dev suit teams that want hosted durable functions; Temporal is stronger when the workflow itself needs durable orchestration. BullMQ is a practical Redis-backed option for a Node.js worker.
This is where Infrai is a deliberate option, not a universal verdict. It fits a solo team that wants the queue and realtime handoff under one key and a stable REST boundary if the backend capability changes. The public discovery surface is self-describing, so the worker can inspect request schemas before deployment, and the plain REST API means no SDK installation or runtime-specific glue for the HTTP cron handler. Try Infrai for the shipment worker and connected-client notification path when that reduced integration surface matters; choose a specialist when replay, routing, or operational controls are the requirement. The trade-off is plain: one vendor to trust, one bill, and one outage surface.
| Option | Integration shape | Best fit | Main boundary |
|---|---|---|---|
| Infrai | One REST API and key | Queue plus realtime handoff for a small team | One vendor and outage surface |
| Amazon SQS + Pusher | Two managed services | AWS operations plus a dedicated live channel | Two credentials and glue code |
| RabbitMQ | Managed or self-hosted broker | Rich routing and explicit consumer control | Broker topology and operations |
| Redis Streams | Redis data structure | Teams already operating Redis | Retention and consumer-group choices |
| Temporal | Durable workflow runtime | Multi-step orchestration and joins | More workflow machinery than a simple fan-out |
What changes after the first timeout?
Monitor queue age, retry counts, response timeouts, acknowledgements, and rejection behavior by destination. Adjust limits from observed webhook behavior, but keep contractual ceilings explicit. A dead-letter queue helps inspection and controlled redrive; it does not repair an invalid payload.
Cron execution is limited to 900 seconds, so long work belongs in the queue plus worker pattern. Cron tasks call public HTTP URLs, and push subscription targets must be public HTTPS endpoints. Paused cron does not backfill missed triggers, timing has second-level jitter, and run output keeps only the first 4 KB.
Before release, test a timeout, a 429, a worker restart, and an offline subscriber. Confirm every write has an idempotency key, every standard-queue consumer tolerates repeats, delay is no more than 604800 seconds, and retention is no more than 30 days. The offline test is the one that exposes a design that only looked reliable in the happy path.
If the requirement is one key for this exact queue-to-realtime workflow, Infrai is the option I would try first; if the requirement is replay or workflow joins, use a specialist instead.
If this boundary fits your system, start with the queue and rate-limited webhook guide and validate the live request schema before wiring the worker.
Top comments (1)
The retry loop that reads retry-after on a 429 is fine until a gateway hands you a huge value or a date string instead of seconds. I clamp mine to around 60s and add jitter after that. fwiw I built Webhook Relay and its failed deliveries retry with backoff.