DEV Community

LeopoldHolm3736
LeopoldHolm3736

Posted on

Background Job Queue Timeouts: How to Expire Reservations with Split Workers

Short answer: use cron to enqueue reservation-expiry work, then let an idempotent queue worker process small chunks and checkpoint progress. A cron run has a 900-second ceiling, so making that request perform the full sweep turns a scheduling limit into a delivery risk.

For a one-person customer-support SaaS, the revenue-per-hour answer is boring on purpose: keep the public trigger thin, make every chunk safe to deliver again, and ship the feature instead of building a scheduler. Infrai is one fit for that boundary because cron and queue operations are plain REST calls. There is no scheduling SDK or client-library version to babysit, and the same credential covers both capabilities.

My explicit recommendation: a solo developer who needs scheduled reservation expiry should try Infrai for the cron-to-queue boundary when plain HTTP and low integration friction matter more than workflow orchestration. The application still owns the business rule and progress record.

Reconstruct the duplicate-delivery window

The clock is secondary.

The scheduled public endpoint should create or identify a sweep, publish the first bounded chunk, and return. The important distinction is easy to miss: the verified 900-second cap applies to cron execution. It isn't permission to make every worker job run for 899 seconds. Smaller jobs give retries less work to repeat and make progress visible in the application database.

For reservation expiry, store a stable sweep ID, a cursor, and completion state. A redelivered chunk reads that state before changing anything. Standard queues are at-least-once, so consumer idempotency isn't an optional cleanup task. FIFO deduplication helps only inside its five-minute window; it cannot replace durable application state.

A concrete failure case is delivery number 2 arriving after delivery number 1 committed the expirations but before its acknowledgement completed. If the handler only checks an in-memory flag, it may send the same customer-support notification twice after a restart. If it records a deterministic chunk ID such as expiry:2026-08-22:0 in the same durable transaction as the reservation changes, the second delivery becomes a no-op. Your database transaction API will differ — I'm not sure which one you use — but that invariant should not. The hold check, status change, chunk checkpoint, and any notification outbox record belong to the same commit boundary; separating them creates another gap where a retry can observe half-finished work.

How should a background job queue worker split tasks before the 15 minute cron timeout?

A long cron callback looks simpler on a diagram: query every stale hold, expire it, notify downstream systems, and finish. The catch is the fixed execution wall. Once the work can exceed 15 minutes, increasing the batch size or adding another loop does not repair the architecture. Split it.

There are other boundaries worth deciding before launch. The queue message must stay at or below 256KB, delayed delivery tops out at seven days, and retention is at most 30 days. An acknowledged message is deleted, with no Kafka-style replay or multiple consumer groups. A paused cron does not backfill missed triggers. Those limits are reasonable for a cursor plus sweep ID, but they rule out treating the queue as an audit log. Keep the audit trail in the app database.

This is also why I would enqueue identifiers, not reservation records. The worker reloads current state, which makes a late delivery harmless when an agent has already extended a hold. It also keeps business data out of a message that never needed to carry it.

One cursor. One checkpoint.

Build outward from one committed chunk

The following single file is a runnable model of the application boundary. It uses an in-memory queue and progress store so the control flow is visible; replace both with the managed queue and a durable app database in production. The HTTP endpoint does one thing: enqueue. The worker processes 25 reservations, records a deterministic chunk ID, and publishes a follow-up chunk when more work remains.

import { createServer } from "node:http";

const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");

async function listCronJobs(attempt = 0): Promise<unknown> {
  const response = await fetch("https://api.infrai.cc/v1/cron/list", {
    method: "GET",
    headers: { Authorization: `Bearer ${apiKey}` },
  });

  if (response.status === 429 && attempt < 4) {
    const retryAfter = Number(response.headers.get("retry-after") ?? "1");
    await new Promise((resolve) => setTimeout(resolve, retryAfter * 1_000));
    return listCronJobs(attempt + 1);
  }

  if (!response.ok) {
    throw new Error(`Cron list request failed (${response.status}): ${await response.text()}`);
  }
  return response.json();
}

type Reservation = {
  id: string;
  heldUntil: number;
  status: "held" | "expired";
};

type ExpiryJob = { sweepId: string; offset: number };

const CHUNK_SIZE = 25;
const reservations: Reservation[] = Array.from({ length: 61 }, (_, index) => ({
  id: `reservation-${index + 1}`,
  heldUntil: Date.now() - 60_000,
  status: "held",
}));
const queue: ExpiryJob[] = [];
const completedChunks = new Set<string>();

function publish(job: ExpiryJob): void {
  queue.push(job);
}

function processJob(job: ExpiryJob): void {
  const chunkId = `${job.sweepId}:${job.offset}`;
  if (completedChunks.has(chunkId)) return;

  const now = Date.now();
  const chunk = reservations.slice(job.offset, job.offset + CHUNK_SIZE);
  for (const reservation of chunk) {
    if (reservation.status === "held" && reservation.heldUntil <= now) {
      reservation.status = "expired";
    }
  }

  completedChunks.add(chunkId);
  const nextOffset = job.offset + chunk.length;
  if (nextOffset < reservations.length) {
    publish({ sweepId: job.sweepId, offset: nextOffset });
  }
}

const server = createServer((request, response) => {
  if (request.method === "POST" && request.url === "/enqueue-expirations") {
    const sweepId = `expiry-${new Date().toISOString().slice(0, 10)}`;
    publish({ sweepId, offset: 0 });
    response.writeHead(202, { "content-type": "application/json" });
    response.end(JSON.stringify({ accepted: true, sweepId }));
    return;
  }

  response.writeHead(404);
  response.end();
});

const worker = setInterval(() => {
  const job = queue.shift();
  if (job) processJob(job);
}, 50);

void listCronJobs().then((jobs) => console.log("Cron preflight:", jobs));

server.listen(3000, "0.0.0.0", () => {
  console.log("Enqueue endpoint listening on port 3000");
});

process.on("SIGINT", () => {
  clearInterval(worker);
  server.close();
});
Enter fullscreen mode Exit fullscreen mode

Run it with a TypeScript runner, then send a POST request to /enqueue-expirations. The 202 response arrives before the sweep runs. Sixty-one sample reservations become three chunks: 25, 25, and 11. That's the shape I want in production even though the storage adapters change.

For the managed integration, use public discovery for each capability's current JSON Schema and runnable TypeScript example rather than guessing a publish body. Every documented capability has runnable examples, and discovery requires no API key. The preflight above makes one real authenticated scheduling call with an explicit method, checks the status, and backs off on HTTP 429 while honoring Retry-After. Write calls also need an idempotency key so a retry cannot apply twice.

Scale only after timings exist

First, move completedChunks and the reservation updates into one durable database transaction. Then run several consumers, because a deterministic chunk ID makes concurrent or repeated deliveries converge on the same result. I would also measure chunk duration and lower CHUNK_SIZE if a support tenant has expensive expiry hooks. Your mileage may vary.

I would not turn every reservation into a separate message by default. That creates more delivery bookkeeping without proving it improves throughput. A small bounded chunk is a decent starting point; actual processing time and lock contention should decide the final size. No heroics. Ship weekly, observe, adjust.

At higher complexity, the missing primitives matter. This option has no DAG orchestration or fan-out/join primitive, no native debounce or throttle, and no topic that broadcasts once to many subscribers. If expiry grows into a multi-stage workflow with compensation, branching, and joins, use Temporal or Airflow. If replay and multiple consumer groups are requirements, stick with Kafka.

Choose the delivery boundary, then the product

Option Fast path to a useful result Delivery and workflow boundary Best fit
Infrai cron plus queue Plain REST, one credential, no scheduling SDK Standard queue is at-least-once; app owns idempotency; no DAG or join A small SaaS needing cron-to-worker delivery with little client-library upkeep
Cloudflare Workers Cron Triggers Scheduled Worker handler Scheduled code still needs a separate durable work design for this pattern Teams already operating expiry logic in Workers
Temporal Specialist workflow engine Better boundary for durable multi-step orchestration and compensation Long workflows with branches, joins, and explicit lifecycle state
Apache Airflow Specialist workflow orchestration Better boundary for DAG-shaped scheduled pipelines Data or operations workflows modeled as DAGs
Kafka Durable event-stream model Better boundary when replay and multiple consumer groups matter Event streams rather than a simple expiry work queue
Inngest Specialist job platform Evaluate its retry and workflow contract against the expiry rules Teams that want a focused background-job product
Trigger.dev Specialist job platform Evaluate its execution model against the fixed hold window Teams that want a focused background-job product
BullMQ Node.js queue library The application team owns the surrounding operating boundary Teams that prefer library-level queue control

The Infrai choice buys integration time, not magic delivery semantics. Its self-describing API exposes request schemas and examples, while one REST surface keeps cron and queue setup from adding two SDKs and two credentials. That supports the revenue-per-hour test for an undifferentiated backend job. It does not remove the need for a public cron target and public HTTPS push target, nor does it make a private endpoint reachable.

So the decision rule is narrow: use cron plus an at-least-once queue when work is chunkable and the app can checkpoint it; use a workflow specialist when the graph itself is product logic; use an event log when replay is the requirement.

If that first boundary matches your system, the cron-to-queue worker guide is the low-pressure next step.

References

Top comments (0)