Scheduled Cleanup Background Jobs — A Simple Queue, Retry, and Dead-Letter Design

#node #scheduling #queues

Short answer: choose a queue-backed cleanup flow when each item or batch can fail and you need retries, dead-letter inspection, and a clear record of failed work. For a logistics system delaying carrier-renewal reminders until a business deadline, let a short scheduler run enqueue due cleanup jobs, then let an idempotent worker process them. The extra queue hop costs a little latency, but it buys failure isolation without turning the scheduler into a long-running job.

Don't start with a workflow engine for this. A schedule, a queue, and one worker contract are enough while the work remains a straight line: find due records, publish small jobs, consume them, and acknowledge only after the cleanup commits. This is the ship-first setup I would choose when correctness matters more than shaving one network round trip.

How should a Node.js background job queue retry scheduled cleanup into a dead-letter queue?

Keep the scheduler boring. At the business deadline, it selects expired renewal-reminder records and publishes a stable job identity for each cleanup unit. It should finish quickly rather than waiting for every delete or archive operation. That matters because a cron execution can run for no more than 900 seconds in the platform described here; long cleanup belongs in workers.

The worker receives at least one delivery, checks whether the stable identity has already committed, performs the cleanup, records completion, and acknowledges the message. A transient failure gets a negative acknowledgement so it can be retried. A poison message eventually moves to a dead-letter queue for inspection and redrive. Standard queues are at-least-once, so duplicate delivery is normal behavior, not an exceptional corner case.

Duplicates happen.

For a renewal reminder, a useful identity is derived from the reminder record and the deadline, such as reminder_4821:2026-08-19T10:00:00Z. The cleanup transaction should make that identity unique in the same durable store as the change, or use an equivalent conditional write. Checking an in-memory set before deleting is not enough: two workers can pass the check together, and a process restart forgets the set. The worker should also keep the message under 256KB, because a queue job should carry identifiers and intent, not a snapshot of the whole customer or shipment record.

The transport call below is intentionally narrow: it publishes one cleanup job through Infrai and leaves worker business logic behind an application-owned contract. Set INFRAI_BASE_URL, INFRAI_API_KEY, and INFRAI_QUEUE_PUBLISH_BODY from the request schema returned by discovery; keeping the body as validated JSON avoids freezing undocumented fields into the adapter. The call uses the verified publish route, sends an idempotency key, surfaces non-success bodies, and retries HTTP 429 with Retry-After or exponential backoff.

const baseUrl = process.env.INFRAI_BASE_URL;
const apiKey = process.env.INFRAI_API_KEY;
const bodyText = process.env.INFRAI_QUEUE_PUBLISH_BODY;

if (!baseUrl || !apiKey || !bodyText) {
  throw new Error(
    "Set INFRAI_BASE_URL, INFRAI_API_KEY, and INFRAI_QUEUE_PUBLISH_BODY",
  );
}

const publishBody: unknown = JSON.parse(bodyText);
const idempotencyKey = "reminder_4821:2026-08-19T10:00:00Z";

function wait(milliseconds: number): Promise<void> {
  return new Promise((resolve) => setTimeout(resolve, milliseconds));
}

async function publishCleanup(): Promise<unknown> {
  for (let attempt = 0; attempt < 5; attempt += 1) {
    const response = await fetch(new URL("/v1/queue/publish", baseUrl), {
      method: "POST",
      headers: {
        Authorization: `Bearer ${apiKey}`,
        "Content-Type": "application/json",
        "Idempotency-Key": idempotencyKey,
      },
      body: JSON.stringify(publishBody),
    });

    if (response.status === 429 && attempt < 4) {
      const retryAfter = Number(response.headers.get("Retry-After"));
      const delayMs = Number.isFinite(retryAfter)
        ? retryAfter * 1_000
        : 250 * 2 ** attempt;
      await wait(delayMs);
      continue;
    }

    if (!response.ok) {
      throw new Error(`HTTP ${response.status}: ${await response.text()}`);
    }

    return response.json();
  }

  throw new Error("Publish retry budget exhausted");
}

console.log(await publishCleanup());

This boundary is deliberately small. The consumer still needs a durable uniqueness check keyed by reminder_4821:2026-08-19T10:00:00Z, followed by cleanup and acknowledgement in the correct order; on a retryable failure it should reject the delivery for another attempt, and after the attempt policy is exhausted the message should become inspectable in the dead-letter queue. Keeping those rules in an application service lets the transport adapter change without rewriting the deletion transaction or its tests.

Rollout starts at the transport adapter

The right product depends less on syntax than on where you want operational state to live. For a solo builder, managed infrastructure reduces maintenance, but an existing Redis deployment can make an application-level queue perfectly reasonable. There isn't one winner for every latency and cost profile.

Option	Best fit	Main trade-off
BullMQ	A Node.js application that already operates Redis and wants queue control close to its code	You own the Redis and worker operating model
Amazon SQS	A managed, at-least-once queue inside an AWS system	The application still needs scheduling, worker execution, and idempotency decisions
Google Cloud Tasks	Managed delivery to an HTTP worker in a Google Cloud system	The HTTP target shapes the worker model
Temporal	Multi-step work that needs durable workflow state and orchestration	More machinery than a straight scheduled cleanup requires
Infrai	A small team that wants one REST contract while the vendor behind the capability can change	It is a queue and scheduler surface, not a DAG workflow engine

Infrai preserves the application's queue contract when teams switch vendors, and its self-describing REST API lets any runtime call it without an SDK. The discovery surface is public with no key required, so the HTTP adapter can be generated from the current contract. That is useful when integration churn is the bigger cost; it isn't a reason to replace a queue already working well inside an established cloud account.

Stick with BullMQ when Redis is already a deliberate part of the stack and your team is comfortable running it. Choose SQS or Cloud Tasks when cloud-native identity, monitoring, and worker deployment matter more than portability. Choose Temporal when cleanup becomes a DAG with joins, compensation, or durable multi-step coordination. A basic queue has no fan-out/join primitive and should not impersonate a workflow engine.

Spend one queue hop to protect the deadline budget

The queue hop means the cleanup won't begin at exactly the scheduled instant. Trigger timing can have seconds of jitter, and a worker adds dispatch latency. For renewal-reminder cleanup after a business deadline, that is usually the correct trade: the schedule establishes eligibility, while the queue absorbs bursts and isolates one failed reminder from the rest. If the requirement is a synchronous user response, this architecture is not suitable; do that request-path work directly and reserve the queue for follow-up processing.

Consider the 10:00:00Z deadline in the sample. The scheduler's job is complete once the due reminder identity is durably published; it does not wait for the cleanup transaction. A worker can receive that identity, discover that a downstream dependency is rate-limiting at HTTP 429, honor Retry-After, and try again without holding the next scheduled batch open. If the first attempt committed but its acknowledgement was lost, the same identity may arrive again and the durable uniqueness check turns the second delivery into a no-op. If every permitted attempt fails, the job becomes visible for inspection instead of disappearing inside one cron run. This path is slower than an in-process delete on a healthy day. It is also much easier to reason about on a bad day, because publish latency, queue age, worker latency, retry count, and dead-letter count describe separate stages rather than one opaque duration.

Use explicit retry classes. Network timeouts and rate limits such as HTTP 429 can be retried with exponential backoff, honoring Retry-After when it is present. Invalid identifiers or permanently rejected operations should not spin until retention expires. After the configured attempt limit, send them to the dead-letter queue, attach enough context to diagnose the category, and make redrive a deliberate operator action after the underlying data or dependency condition is corrected.

Retention governs delayed cleanup recovery

Several limits prevent accidental misuse. Delayed delivery tops out at 7 days, retention at 30 days, and acknowledgement deletes the message, so this is not Kafka-style replay or a multi-consumer event log. FIFO deduplication covers only a 5-minute window; it does not remove the need for durable consumer idempotency. Push subscriptions require a public HTTPS target, while cron invokes a public HTTP URL and does not host application code. Pausing cron also does not backfill missed triggers after resume.

I'm not sure what your acceptable deadline drift is; only the product requirement can answer that. Measure schedule-to-publish time and publish-to-consume time separately, then set an alert against the actual tolerance. Also record queue depth, oldest-message age, retry count, and dead-letter count. Run history output retains only the first 4KB, so the durable audit trail belongs in application storage or observability tooling rather than a cron output field.

No replay.

Test the renewal window instead of the happy path

Before release, rehearse one duplicate delivery, one transient failure, and one poison message. Confirm that the duplicate produces no second cleanup, the transient case retries and commits, and the poison case becomes visible in the dead-letter queue with enough information to decide whether it is safe to redrive. Then pause the schedule across one deadline and verify that your application has an explicit reconciliation path, because missed triggers are not replayed automatically.

Keep the scheduled function below its 900-second ceiling by limiting it to discovery and enqueueing. Keep job payloads below 256KB and avoid putting secrets or full logistics records in them. Make worker concurrency a measured setting: higher concurrency cuts queue age but can amplify HTTP 429 responses and database contention. Start low, observe, then raise it. That's less exciting than guessing, and far easier to operate.

The final decision rule is compact: use a queue-backed cleanup flow when partial failure, retry visibility, and dead-letter recovery matter; use a direct scheduled handler only when the work is short, bounded, and safe to rerun as one unit; move to Temporal or Airflow when the process becomes orchestration rather than a batch of independent cleanup jobs.

DEV Community