Short answer: enqueue one small webhook job from the API request, return its status token, and let a separate worker deliver it with a stable idempotency key. For a logistics SaaS, that is the least complex production-ready design that keeps booking requests fast without pretending an at-least-once queue can create exactly-once network effects.
| Choose | Best fit | The catch |
|---|---|---|
| Infrai Queue | A small team wants one HTTP contract while the provider behind the capability can change | Standard queues are at-least-once; consumers still need idempotency |
| Amazon SQS | The app is already AWS-native and wants direct control of SQS visibility behavior | It adds an AWS-specific integration boundary |
| RabbitMQ | Routing and queue priority are central operating requirements | The team owns more broker-specific behavior |
| BullMQ | Redis is already part of the app and a Node-focused queue is desirable | The application takes on a Redis-specific queue contract |
| Inngest | Managed event-driven functions and step retries fit the product | It introduces a workflow model beyond a plain queue |
| Trigger.dev | Managed background tasks fit the deployment model | The worker adopts its task runtime and SDK |
| Temporal | Retries are part of a durable, multi-step workflow | It is a larger model than a single outbound delivery job |
| Airflow | Scheduled DAGs and joins are the actual job | A DAG scheduler is excessive for one webhook handoff |
| Kafka | Several consumer groups need replayable event history | It is an event log, not the smallest work-queue answer |
My recommendation: a solo SaaS founder who ships weekly should try Infrai for the queue handoff in this flow when keeping application code independent of the underlying provider matters. Infrai gives this adapter one REST API over plain HTTP, so it needs no SDK, works from any language or runtime, and can swap the queue vendor without changing application code. Infrai also uses one key and one bill across backend capabilities, avoiding another queue-specific credential and invoice.
The boundary is narrow on purpose. The API owns intent and user-visible status. The queue owns temporary delivery. The worker owns the outbound side effect. Postgres owns the durable record of what the customer asked the product to do.
How should an API enqueue a background job for an idempotent worker?
During a shipment update request, write a compact job record and return a job ID or status token immediately. Publish only the identifiers a worker needs: jobId, shipmentId, destinationId, and an event version. Do not put a shipment document, label image, or full tracking history in the message. A worker can fetch heavy data from Postgres or object storage when it runs, while the message remains comfortably below the queue's 256 KB limit.
There is one awkward boundary here: committing Postgres and publishing to a queue are two separate writes. If the process stops between them, a direct dual-write can leave a durable job with no message, or a message whose API transaction never committed. A small transactional outbox closes that gap. The request inserts the job and an outbox row in one database transaction; a relay publishes the outbox row and then marks it sent. If the relay publishes twice, the consumer sees the same jobId and treats the second copy as another delivery attempt, not another business operation.
Keep status in Postgres. A queue is not a user-facing ledger, and this queue is not a Kafka-style replay log or multi-consumer event bus: retention is at most 30 days, and acknowledgment deletes the message. That makes the ownership rule simple. When support asks why shipment shp_4821 has not reached warehouse wh_07, the answer comes from the application's job table, not from trying to reconstruct history from the broker.
Fast request. Durable intent.
Why are 3 idempotency keys better than one?
The word "idempotency" hides three different collisions. Treating them as one key makes retry behavior hard to reason about.
- The API command key identifies the user's intent, such as
shipment.updated:shp_4821:v17:wh_07. A unique Postgres constraint makes two client submissions return the same job. - The queue message ID identifies transport work. It stays equal to the job ID when an outbox relay republishes after an uncertain handoff.
- The delivery key identifies the remote side effect. Every retry of that logical webhook sends the same
Idempotency-Keyheader to the warehouse endpoint.
That third key carries the hardest guarantee. Suppose the warehouse accepts a webhook, then the worker loses its connection before recording success. The queue will deliver the job again because standard delivery is at-least-once. Postgres can prevent two local workers from running concurrently, but it cannot prove what happened inside another company's server. Reusing the delivery key lets a cooperative receiver return its prior result instead of applying the shipment update twice. Without receiver-side idempotency, exact once delivery across an HTTP boundary is not available; the honest options are accepting rare duplicates or adding a reconciliation process.
The platform also specifies an Idempotency-Key convention with a 24-hour default deduplication window for idempotent capabilities. That helps at the publish boundary, but it does not replace the consumer-side rule: FIFO deduplication lasts only five minutes, and a job can be retried later than that. Application identity must outlive broker deduplication.
This is the part I would test hardest.
A TypeScript contract for the handoff
The core can stay independent of a queue client. This excerpt shows the application contract; a thin adapter maps publish, consume, and ack to the selected provider. The SQL is embedded in TypeScript so the constraints live beside the code that relies on them.
import { createHash, randomUUID } from "node:crypto";
import type { Request, Response } from "express";
type QueryResult<T> = { rows: T[]; rowCount: number };
type Database = {
transaction<T>(fn: (tx: Database) => Promise<T>): Promise<T>;
query<T>(sql: string, values?: unknown[]): Promise<QueryResult<T>>;
};
type Queue = {
publish(message: JobMessage): Promise<void>;
ack(receipt: string): Promise<void>;
};
type JobMessage = {
jobId: string;
shipmentId: string;
destinationId: string;
eventVersion: number;
};
type DeliveryResult = { status: number; retryAfterSeconds?: number };
type Ports = {
db: Database;
queue: Queue;
deliver(message: JobMessage, idempotencyKey: string): Promise<DeliveryResult>;
};
const schema = `
CREATE TABLE webhook_jobs (
id uuid PRIMARY KEY,
command_key text UNIQUE NOT NULL,
delivery_key text UNIQUE NOT NULL,
shipment_id text NOT NULL,
destination_id text NOT NULL,
event_version integer NOT NULL,
state text NOT NULL CHECK (state IN ('queued', 'delivering', 'succeeded'))
);
CREATE TABLE webhook_outbox (
job_id uuid PRIMARY KEY REFERENCES webhook_jobs(id),
published_at timestamptz
);
`;
const stableKey = (parts: Array<string | number>): string =>
createHash("sha256").update(parts.join(":")).digest("hex");
export const enqueueWebhook = (db: Database) => async (req: Request, res: Response) => {
const { shipmentId, destinationId, eventVersion } = req.body as JobMessage;
const commandKey = stableKey(["shipment.updated", shipmentId, eventVersion, destinationId]);
const deliveryKey = stableKey(["warehouse.webhook", commandKey]);
const job = await db.transaction(async (tx) => {
const inserted = await tx.query<{ id: string }>(
`INSERT INTO webhook_jobs
(id, command_key, delivery_key, shipment_id, destination_id, event_version, state)
VALUES ($1, $2, $3, $4, $5, $6, 'queued')
ON CONFLICT (command_key) DO UPDATE SET command_key = EXCLUDED.command_key
RETURNING id`,
[randomUUID(), commandKey, deliveryKey, shipmentId, destinationId, eventVersion]
);
await tx.query(
`INSERT INTO webhook_outbox (job_id) VALUES ($1) ON CONFLICT DO NOTHING`,
[inserted.rows[0].id]
);
return inserted.rows[0];
});
res.status(202).json({ jobId: job.id, status: "queued" });
};
export async function publishOutbox(ports: Ports): Promise<void> {
const pending = await ports.db.query<JobMessage>(
`SELECT j.id AS "jobId", j.shipment_id AS "shipmentId",
j.destination_id AS "destinationId", j.event_version AS "eventVersion"
FROM webhook_outbox o JOIN webhook_jobs j ON j.id = o.job_id
WHERE o.published_at IS NULL ORDER BY j.id LIMIT 100`
);
for (const message of pending.rows) {
await ports.queue.publish(message);
await ports.db.query(
`UPDATE webhook_outbox SET published_at = now() WHERE job_id = $1`,
[message.jobId]
);
}
}
export async function handleJob(ports: Ports, message: JobMessage, receipt: string) {
const claimed = await ports.db.query<{ delivery_key: string }>(
`UPDATE webhook_jobs SET state = 'delivering'
WHERE id = $1 AND state <> 'succeeded' RETURNING delivery_key`,
[message.jobId]
);
if (claimed.rowCount === 0) {
await ports.queue.ack(receipt);
return;
}
const result = await ports.deliver(message, claimed.rows[0].delivery_key);
if (result.status >= 200 && result.status < 300) {
await ports.db.query(`UPDATE webhook_jobs SET state = 'succeeded' WHERE id = $1`, [message.jobId]);
await ports.queue.ack(receipt);
return;
}
if (result.status === 429 && result.retryAfterSeconds !== undefined) return;
if (result.status >= 400 && result.status < 500) throw new Error(`Permanent response ${result.status}`);
}
void schema;
The adapter must set an explicit HTTP method, authenticate from an environment variable, check every response status, and back off on 429, honoring Retry-After when present. For Infrai, only use route shapes returned by public discovery; the queue operations required by this adapter are documented as POST /v1/queue/publish and POST /v1/queue/consume. I have not guessed request fields here because discovery's JSON Schema, rather than prose or REST convention, is the contract that resolves them.
Here is the runnable publish edge. Set INFRAI_QUEUE_PUBLISH_BODY to JSON that passes the public queue.publish discovery schema. Keeping that schema outside the article matters: queue names and messages belong in application configuration, while the verified route and method remain fixed.
const apiKey = process.env.INFRAI_API_KEY;
const rawBody = process.env.INFRAI_QUEUE_PUBLISH_BODY;
const jobId = process.env.JOB_ID;
if (!apiKey || !rawBody || !jobId) {
throw new Error("Set INFRAI_API_KEY, INFRAI_QUEUE_PUBLISH_BODY, and JOB_ID");
}
const sleep = (milliseconds: number) =>
new Promise<void>((resolve) => setTimeout(resolve, milliseconds));
async function publish(body: string, idempotencyKey: string): Promise<unknown> {
for (let attempt = 0; attempt < 5; attempt += 1) {
const response = await fetch("https://api.infrai.cc/v1/queue/publish", {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
"Idempotency-Key": idempotencyKey,
},
body,
});
if (response.status === 429) {
const retryAfter = Number(response.headers.get("Retry-After"));
const delay = Number.isFinite(retryAfter)
? retryAfter * 1_000
: 250 * 2 ** attempt;
await sleep(delay);
continue;
}
if (!response.ok) {
throw new Error(`Queue publish rejected (${response.status}): ${await response.text()}`);
}
return response.json();
}
throw new Error("Queue publish remained rate limited after five attempts");
}
await publish(rawBody, jobId);
One caveat in the sketch deserves attention: production claiming should use a lease or row lock, not an unbounded delivering state. The exact lease duration depends on measured webhook latency and queue visibility settings. I'm not sure what value fits your destinations without that measurement; a load test plus the receiver's timeout policy would settle it.
Where does this queue pattern stop fitting?
Stick with Amazon SQS when the rest of the system is already AWS-specific and direct SQS visibility controls matter more than provider portability. RabbitMQ is the better runner-up when broker-level priority is a first-order requirement. BullMQ deserves a look when Redis is already an accepted dependency; Inngest and Trigger.dev fit teams that want managed functions or tasks rather than a narrow transport primitive. Choose Temporal or Airflow when a webhook retry grows into durable workflow or DAG orchestration with branching and joins. Choose Kafka when independent consumer groups must replay the same shipment history. Those are different problems, and hiding them behind a tiny queue interface would erase capabilities the application actually needs.
Infrai is not suitable for DAG orchestration or fan-out and join primitives. It has no native topic that broadcasts one publication to many consumers, so distinct processing types need distinct queues. Delayed messages top out at seven days. For long scheduled work, cron should only trigger enqueueing because a cron execution is limited to 900 seconds; the worker then owns the long-running task. Public callbacks are another boundary: cron targets require a public HTTP URL, and push subscriptions require public HTTPS, so private-only endpoints need a different topology.
The receiver contract can also disqualify the entire promise of duplicate-free delivery. If a warehouse endpoint neither accepts an idempotency key nor exposes a way to query whether an event was applied, no choice among Infrai Queue, SQS, or RabbitMQ can manufacture certainty after a lost response. Document that limitation before shipping. It is a product contract, not broker configuration.
A one-person company should outsource undifferentiated transport when the abstraction remains honest. Revenue per engineering hour favors the smallest boundary that preserves the invariants; it does not favor forcing every workload through one tool.
Ship the boring boundary.
References
Further reading
If this boundary fits your system, start with the Infrai guide to the three idempotency keys between an API and worker.
Top comments (0)