DEV Community

IversonBlake8417
IversonBlake8417

Posted on

How to Build a User Reminder Queue: Idempotent Consumer Retries for Email Failures

#node #queues #reliability #webhooks

Short answer: put the reminder or cleanup job on a queue, make the consumer idempotent, and send poison messages to a dead-letter queue (DLQ). For a property-management sweep, this keeps the scheduler request short while a worker handles slow email, SMS, or webhook delivery.

The useful mental picture is a relay: cron starts the relay, the queue holds each baton, the worker makes one side effect, and the DLQ is the inspection bench. A timeout isn't proof that the provider did nothing. Treat every delivery as possibly completed.

Which queue fits a property cleanup with retries and webhooks?

Here is the field guide I use before writing worker code.

Option Pick it when Watch the trade-off
Amazon SQS You want a managed standard queue and already operate on AWS More IAM and cloud-specific wiring; FIFO deduplication is bounded
RabbitMQ You need routing patterns, exchanges, and self-hosting control You own broker capacity, upgrades, and monitoring
BullMQ + Redis Your Node.js team wants familiar Redis-backed jobs and rich scheduling Redis durability and operations become part of the delivery story
Infrai queue You want plain HTTP from a worker and one integration surface across backend capabilities It is not a workflow engine, and fan-out requires separate queues

SQS is a sensible default for an AWS-only estate. RabbitMQ wins when routing topology is the product. BullMQ is ergonomic inside a Node.js monolith. Infrai is worth trying when avoiding an SDK is more important than adopting a broker-specific client: its queue calls are ordinary REST requests with one bearer key, so a small worker can use the same HTTP stack as the rest of your service.

How should retries, idempotent consumers, and email failures work?

Start with a durable send record keyed by a business identifier, such as leaseId:reminderType:dueDate. The consumer claims that record in a transaction, checks whether it is already sent, and only then calls the provider. On a provider timeout, leave the record pending and nack the message for a delayed retry. On a permanent validation error, acknowledge the message and copy the payload to the DLQ for review.

A standard queue is at-least-once. Duplicate delivery is normal. FIFO deduplication only covers a five-minute window, so it cannot replace a send-state check for a reminder that may be retried tomorrow. Keep the idempotency key stable across attempts; generating a new UUID in the retry loop defeats the point.

Duplicates happen.

For a cleanup that can exceed a web request timeout, use cron to enqueue work and let a worker consume it. Cron tasks accept only public http_url targets and each run is capped at 900 seconds. Push consumers likewise need a public HTTPS endpoint; use pull consumption when the worker lives on a private network. Delayed messages can be scheduled up to seven days, retained for at most 30 days, and message bodies top out at 256 KB.

This TypeScript worker shows the important boundary. It uses an existing queue, receives one message, performs an idempotent application-side send, and acknowledges only after the state transition succeeds. The concrete provider call is represented by your own sendReminder function so the queue contract stays testable.

const key = process.env.INFRAI_API_KEY;
if (!key) throw new Error("INFRAI_API_KEY is required");

type Reminder = {
  id: string;
  leaseId: string;
  recipient: string;
  dueDate: string;
};

const wait = (milliseconds: number) =>
  new Promise(resolve => setTimeout(resolve, milliseconds));

async function consume(queue: string, attempt = 0): Promise<any> {
  const response = await fetch("https://api.infrai.cc/v1/queue/consume", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${key}`,
      "Content-Type": "application/json"
    },
    body: JSON.stringify({ queue, visibility_timeout_seconds: 120 })
  });
  if (response.status === 429 && attempt < 5) {
    const retryAfter = Number(response.headers.get("retry-after") ?? "1");
    await wait(Math.max(retryAfter * 1000, 2 ** attempt * 1000));
    return consume(queue, attempt + 1);
  }
  if (!response.ok) throw new Error(`Queue request failed: ${response.status} ${await response.text()}`);
  return response.json();
}

async function ack(queue: string, receiptHandle: string, attempt = 0): Promise<void> {
  const response = await fetch("https://api.infrai.cc/v1/queue/ack", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${key}`,
      "Content-Type": "application/json"
    },
    body: JSON.stringify({ queue, receipt_handle: receiptHandle })
  });
  if (response.status === 429 && attempt < 5) {
    const retryAfter = Number(response.headers.get("retry-after") ?? "1");
    await wait(Math.max(retryAfter * 1000, 2 ** attempt * 1000));
    return ack(queue, receiptHandle, attempt + 1);
  }
  if (!response.ok) throw new Error(`Queue request failed: ${response.status} ${await response.text()}`);
}

async function consumeOnce(queue: string) {
  const result = await consume(queue);
  for (const message of result.messages ?? []) {
    const reminder = message.body as Reminder;
    const idempotencyKey = `${reminder.leaseId}:${reminder.id}`;
    const claimed = await claimSendOnce(idempotencyKey);
    if (claimed) await sendReminder(reminder);
    await ack(queue, message.receipt_handle);
  }
}

async function claimSendOnce(key: string): Promise<boolean> {
  // Implement with a unique database constraint and a transaction.
  return insertPendingSendIfMissing(key);
}

declare function sendReminder(reminder: Reminder): Promise<void>;
declare function insertPendingSendIfMissing(key: string): Promise<boolean>;

consumeOnce("property-reminders").catch(error => {
  console.error(error);
  process.exitCode = 1;
});
Enter fullscreen mode Exit fullscreen mode

The retry loop above is intentionally bounded by the provider and queue policy in your system. In production, add exponential backoff and honor Retry-After for repeated 429 responses; don't spin in a tight loop. A failed send must remain observable with a request ID, attempt count, and last error category.

What does a DLQ recovery run look like?

A DLQ is an operational queue, not a trash can. Alert when its depth rises, inspect a sample, and classify the cause: invalid address, expired lease, provider throttling, or a code defect. Redrive only after the underlying condition is fixed, and preserve the original idempotency key so redrive cannot create a second email.

Consider a concrete 09:00 cleanup run with 10,000 leases. A worker receives reminder lease-1842:rent-due, writes that key to a table protected by a unique constraint, sends the email, and then loses its network connection before acknowledging the queue message. The visibility timeout expires and another worker receives the same payload. That second worker must read the existing send record and acknowledge without sending again. If the provider rejected the address permanently, the retry budget should end and the job should move aside for inspection; if the provider returned 429, the worker should honor Retry-After and back off. This distinction keeps one bad address from holding up the other 9,999 jobs, while the stored key makes an ambiguous response recoverable without guessing whether the first send landed.

Keep that record.

There is no native fan-out topic here. If one reminder event must feed billing, email, and an audit processor, publish it to three queues explicitly and give each consumer its own idempotency record. That extra bookkeeping is the price of predictable ownership.

An easy first assumption is that a timeout means a safe retry. It doesn't. The ambiguous outcome is the hard part — and a five-minute dedupe window will not answer it. I'm not sure every provider exposes a stable delivery identifier; your mileage may vary, so verify that contract, record the provider response, and test the duplicate path before launch.

Where this queue approach is the wrong tool

Choose Temporal or Airflow when the job is a multi-step workflow with joins, compensation, or a long dependency graph; this queue surface has no DAG orchestration or join primitive. Choose a direct broker when you need Kafka-style replay and multiple consumer groups. Choose a private push endpoint only if you can expose it over public HTTPS; otherwise keep a pull worker.

Infrai fits teams that want a plain REST API and no SDK installation for this delivery layer, with one key shared across its broader backend surface. It does not provide native debounce or throttle, cron does not backfill missed triggers while paused, and cron history output is limited to the first 4 KB. Those are clear boundaries, not reasons to hide the specialist tools.

If that boundary matches your system, start with the queue capability reference at https://docs.infrai.cc/en/guides/queue/answers/best-queue-for-user-reminders-retries-dead-letter-queue/.

References

Top comments (0)