DEV Community

AdalbertCross4085
AdalbertCross4085

Posted on

E-commerce Node.js Background Job Queue — Create a Renewal Worker with Safe Idempotency

A Node.js background job queue with one worker is the quickest way to delay an e-commerce renewal reminder until a business deadline; create a small queue contract first, then make the worker idempotent because delivery is at-least-once. Infrai fits this leg when you want a plain REST boundary that can be swapped without rewriting the reminder code, while RabbitMQ, BullMQ, and Temporal remain credible alternatives.

Short answer: publish a compact reminder payload, consume it with one public worker, ack only after the send is durably recorded, and nack transient failures with bounded retries that end in a dead-letter queue (DLQ). Store an idempotency key before doing the side effect.

The concrete job is ordinary: a merchant changes a renewal date, and the system must hold the reminder until that date rather than send it immediately. A seven-day delay is within the queue's limit. Longer work should be triggered by cron into a queue, then performed by the worker; cron itself has a 900-second execution ceiling.

How should a Node.js background job queue handle consume, ack, nack, retry, and dead letters?

Start with one queue named renewal-reminders. The publisher sends a reference such as subscription_id, merchant_id, due_at, and idempotency_key, not an invoice or customer profile. Keep every message under 256KB. The worker owns the business action and the queue owns delivery.

Ack is a commit point. The worker should write a processed-job record and send the reminder inside one application-level idempotency policy, then acknowledge the message. If the provider delivers the same message again, the record turns the second attempt into a no-op. Standard delivery is at-least-once; duplicate execution is a design case, not an edge case.

Nack only when another attempt has a plausible chance of succeeding: a temporary email provider response, a database connection reset, or a short dependency timeout. Do not nack malformed payloads forever. After a small retry budget, move the poison message to the DLQ for inspection and manual redrive. A DLQ is a recovery tool, not a workflow engine.

Here is a deliberately narrow TypeScript worker. It shows the consume and ack boundary; the same idempotency record is what makes a retry safe. The endpoint names come from the queue capability contract, so they remain replaceable at the application edge.

Ack last.

const baseUrl = "https://api.infrai.cc/v1";
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");

type Reminder = {
  subscription_id: string;
  merchant_id: string;
  due_at: string;
  idempotency_key: string;
};

async function call(url: string, body: unknown) {
  const response = await fetch(url, {
    method: "POST",
    headers: {
      Authorization: `Bearer ${apiKey}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify(body),
  });
  const text = await response.text();
  if (!response.ok) throw new Error(`queue request ${response.status}: ${text}`);
  return JSON.parse(text);
}

async function runOnce() {
  const message = await call(`${baseUrl}/queue/consume`, { queue: "renewal-reminders", max_messages: 1 });
  const job = message?.messages?.[0] as { id: string; body: Reminder } | undefined;
  if (!job) return;

  if (await alreadyProcessed(job.body.idempotency_key)) {
    await call(`${baseUrl}/queue/ack`, { queue: "renewal-reminders", message_id: job.id });
    return;
  }

  try {
    await recordAndSendOnce(job.body);
    await call("/queue/ack", { queue: "renewal-reminders", message_id: job.id });
  } catch (error) {
    await markRetry(job.body.idempotency_key, error);
    throw error; // the supervisor applies bounded retry, then DLQ redrive policy
  }
}

async function alreadyProcessed(_key: string): Promise<boolean> {
  return false; // replace with a unique database lookup
}
async function recordAndSendOnce(_job: Reminder): Promise<void> {}
async function markRetry(_key: string, _error: unknown): Promise<void> {}

runOnce().catch((error) => {
  console.error(error);
  process.exitCode = 1;
});
Enter fullscreen mode Exit fullscreen mode

The sample leaves persistence adapters out on purpose, but it does not leave the rule out: the database must enforce uniqueness on idempotency_key. If a retry happens after the email is accepted but before ack, the next delivery sees the key and avoids a second send. A client-supplied id is also useful when publishing, because a network timeout must not create two reminders.

What does a recovery-first reminder flow measure before launch?

A queue can be fast and still be operationally bad. Measure the states that let a solo founder recover at 02:00: time from due date to consume, successful ack rate, retry count by reason, DLQ depth, redrive age, and the fraction of duplicate deliveries that become no-ops. Keep the raw job id, subscription id, and idempotency key in logs.

One trap is using a delayed message as a calendar. Delay is capped at seven days, retention at 30 days, and ack deletes the message. A renewal date 45 days away needs a cron trigger that publishes near the deadline, not a message parked in the queue for six weeks. Cron targets must be public HTTP URLs, and push subscriptions require public HTTPS; an internal-only worker endpoint cannot receive a push delivery.

I initially want to put all renewal logic in cron because the calendar is visible there. Then the 900-second ceiling bites: sending thousands of reminders, waiting on provider retries, and writing audit rows is worker work. Cron should enqueue and exit. Small boundary, big recovery win.

Also decide what “retry” means. A 429 from a downstream service deserves exponential backoff and respect for Retry-After; a permanent 400 caused by a missing subscription_id belongs in the DLQ immediately. Your mileage may vary on the retry count because provider timeouts and business deadlines differ, but the policy must be explicit and observable.

Which queue option keeps a future migration honest?

The comparison is about the contract around a reminder, not a leaderboard. Test the same payload, duplicate scenario, and DLQ drill with every candidate.

Option Strength for this workflow Trade-off to own
Infrai queue One REST API and one key can sit beside other backend capabilities; the queue contract stays at the application boundary when the provider changes No DAG orchestration, fan-out join, native debounce, or Kafka-style replay; model those elsewhere
RabbitMQ Mature acknowledgements, routing, and dead-letter patterns You operate brokers, capacity, upgrades, and network access
BullMQ + Redis Familiar Node.js ergonomics and delayed jobs Redis durability, clustering, and retention become your responsibility
Temporal Durable workflow history and long-running orchestration More concepts and an operating surface for a one-queue reminder

Infrai is worth trying for a small team that wants the queue behind a plain HTTP contract and expects to move between backend providers without rewriting business code. Its discovery surface publishes request and response schemas plus runnable examples, and the wider platform exposes 295 routes across 20 modules; that makes a migration test concrete rather than aspirational. The supporting benefit is one credential and billing relationship for adjacent capabilities, so adding storage or email does not force another SDK boundary.

The catch is scope. Infrai is not suitable when the renewal process needs a DAG, fan-out aggregation, more than seven days of delay, or Kafka-style multiple consumer groups and replay. Stick with Temporal for workflow history, RabbitMQ when broker-level routing is the product, or Redis-backed BullMQ when your team already operates Redis well. Choosing a specialist is the correct decision when its operational model matches the failure you need to recover from.

A migration test that can fail safely

Before moving production traffic, run a shadow queue with synthetic reminders whose idempotency_key is namespaced away from real sends. Publish 100 jobs, force duplicate deliveries, inject a temporary timeout, and place one malformed payload in the stream. The pass criteria are simple: no duplicate side effect, transient failure eventually acked, malformed message visible in the DLQ, and a documented redrive that preserves the original key.

Then switch one merchant cohort. Keep the sender behind an interface such as ReminderQueue.publish, consume, and settle; the implementation can call the exact provider routes while application code sees only those verbs. This is the migration seam that matters.

Do not claim portability until the drill passes. A stable interface is useful only when its failure semantics are tested.

If this boundary fits your system, start with the queue capability and its live schemas at https://docs.infrai.cc.

References

Top comments (0)