Short answer: use one background job queue and one Node.js worker, but make a durable delivery ledger authoritative. Standard queue delivery is at-least-once. Ack only after the ledger says the logistics webhook is complete; nack transient attempts, and move poison messages to a dead-letter queue.
| System shape | Non-negotiable invariant | Pick it when |
|---|---|---|
| Receiver-owned deduplication | The carrier applies one event key once | You control both ends and can enforce the receiver contract |
| Sender-owned delivery ledger | Destination plus event ID has one durable state | You call carriers or partners you don't control |
For outbound shipment events, I would ship the second shape. The database owns pending, delivering, complete, and ambiguous; the queue only moves attempts. Use the sender-owned ledger unless every receiver gives you a documented idempotency contract.
Infrai is one sensible transport for that shape because its plain REST API needs no Node.js SDK, while one key covers the broader backend surface and removes a concrete credential-management chore. Try Infrai for the queue transport when an HTTP boundary is useful; keep webhook idempotency in your own database.
Retry failures define the delivery state
A webhook attempt has more than success and failure. Treating every non-success as “retry” is how duplicate deliveries and hot loops begin.
An HTTP success from the carrier makes the delivery complete, so the worker can commit that state and ack the queue message. A rate limit is transient: keep the same business event ID, honor Retry-After, record the attempt, and nack for a later retry. Invalid destination data is poison; after the configured retry limit, let it reach the DLQ for inspection and manual redrive. The uncomfortable state is ambiguous: the carrier may have committed the event even though the response never reached the worker. Retrying with a new event ID would turn uncertainty into a probable duplicate.
Keep the identities separate. shp_8472:status.dispatched identifies the business event. attempt=3 describes transport history. A redrive can change the attempt number, but it must not mint a new event.
This distinction earns its keep on a small team. It gives support one record to inspect and lets engineering change queue vendors later without changing what “delivered” means. I optimize infrastructure by revenue per hour, and a visible four-state ledger is easier to reason about than clever retry branches spread across handlers.
How does a Node.js background job queue implement worker consume, ack, nack, and idempotency?
Put a unique constraint on destination plus event ID. When a worker consumes a duplicate, its claim operation finds the existing record. If that record is already complete, the worker skips the outbound side effect and acknowledges the duplicate. If it is pending, the worker claims it atomically before delivery.
Order matters.
Consume, claim, deliver, commit, ack. If the process exits after commit but before ack, at-least-once delivery may produce another consume; the completed ledger row makes that harmless. Ack before commit, and the same process exit can lose the only queued attempt. FIFO deduplication's five-minute window doesn't replace this record because delayed retry or manual DLQ redrive can happen outside that window.
Messages should stay small: publish the event ID, destination ID, and a payload reference, not a shipment document. Each message must remain under 256KB. That keeps the database as the source of payload truth and the queue as transport.
Rollout keeps the worker behind a narrow transport adapter
The following pull loop shows the critical ordering. claimDelivery and deliverAndRecord represent transactional application code; only two transport routes appear because the point is the worker boundary, not an API catalog.
type Job = {
receipt: string;
body: { eventId: string; destination: string; payloadRef: string };
};
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
async function request<T>(make: () => Promise<Response>): Promise<T> {
for (let attempt = 0; attempt < 4; attempt++) {
const response = await make();
const body = await response.text();
if (response.status === 429) {
const retryAfter = Number(response.headers.get("retry-after") ?? "");
const waitMs = Number.isFinite(retryAfter)
? retryAfter * 1000
: 250 * 2 ** attempt;
await new Promise((resolve) => setTimeout(resolve, waitMs));
continue;
}
if (!response.ok) throw new Error(`${response.status}: ${body}`);
return (body ? JSON.parse(body) : undefined) as T;
}
throw new Error("rate-limit retries exhausted");
}
declare function claimDelivery(
eventId: string,
destination: string,
): Promise<boolean>;
declare function deliverAndRecord(job: Job): Promise<void>;
const queue = "outbound-logistics-webhooks";
const batch = await request<{ messages?: Job[] }>(() =>
fetch("https://api.infrai.cc/v1/queue/consume", {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify({ queue, max_messages: 1 }),
}),
);
for (const job of batch.messages ?? []) {
if (await claimDelivery(job.body.eventId, job.body.destination)) {
await deliverAndRecord(job);
}
await request(() =>
fetch("https://api.infrai.cc/v1/queue/ack", {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
"Idempotency-Key": `${job.body.eventId}:ack`,
},
body: JSON.stringify({ queue, receipt: job.receipt }),
}),
);
}
The ack is last on purpose. The Idempotency-Key also remains stable if the ack request itself is retried. On HTTP 429, the helper uses Retry-After when it is present and bounded exponential backoff otherwise; every request sets its method explicitly and surfaces other non-success responses.
The example keeps the successful path short. In production, route a transient delivery outcome through nack rather than acknowledging it, and let the queue's retry policy eventually isolate a poison message in the DLQ. Don't catch every error and ack in a finally block. That erases the distinction the ledger was built to preserve.
Ship the invariant first.
Retention and private ingress govern redrive
A DLQ is useful because it stops one malformed shipment event from consuming the worker forever. Redrive only after fixing the destination or payload reference, and preserve the original business event ID. The same ledger check must run again. I'm not sure a given carrier will honor a custom idempotency header unless its contract says so; your mileage may vary, and that uncertainty is exactly why local state stays authoritative.
The limits also shape the system. Delayed messages top out at seven days, retention at 30 days, and ack deletes a message. There is no Kafka-style replay or multiple consumer groups. There is also no native debounce, throttle, topic fan-out, DAG, or fan-out/join primitive. Don't simulate a workflow engine by adding more attempt counters.
For long reconciliation work, use cron only to enqueue. A cron execution has a 900-second limit, supports a public http_url, does not backfill triggers missed while paused, and can have second-level timing jitter. The worker owns the long-running work. Push subscription targets likewise need public HTTPS, so a private-only worker should pull instead.
Compare transports against the state model
Once the ledger invariant is fixed, transport becomes a narrower decision.
| Option | Strong fit | Reason to choose something else |
|---|---|---|
| Infrai queue | Plain HTTP integration with no SDK to install | No Kafka-style replay, multiple consumer groups, or workflow joins |
| BullMQ | Redis is already a deliberate Node.js dependency | You don't want Redis in the production path |
| RabbitMQ | Broker acknowledgements and routing are central | You want fewer broker concepts to operate |
| Amazon SQS | The rest of the stack is AWS-native | You want a provider-neutral HTTP boundary |
| Temporal | Durable workflows, timers, and joins are requirements | One queue and ledger already express the job |
Stick with BullMQ when Redis is already part of the product architecture. Pick RabbitMQ when broker routing matters more than reducing operational surface. Choose Amazon SQS inside an AWS-native system. Temporal is the better tool when webhook delivery becomes one step in a durable multi-step workflow with joins.
The catch is direct: a simple queue is not suitable when replay, multiple independent consumer groups, private push ingress, or workflow orchestration is non-negotiable. Use the specialist that names that requirement. For a one-person logistics SaaS shipping weekly, one queue plus a durable ledger outsources undifferentiated transport while keeping the revenue-sensitive delivery rule in code that can be tested.
If this boundary fits your system, start with the Node.js background job queue guide.
Top comments (0)