A crash-safe job queue for server-side Dart, in two packages

If a worker is killed mid-job, most Dart queue options lose the job. This one does not.

A request comes in, you accept it, and you push the slow part onto a background queue: send the email, charge the card. The request returns 200. Then the worker process gets kill -9, or the pod is evicted. The job was in flight, and now there is no record it was ever supposed to happen.

An in-process scheduler holds the job in a list in memory, so the job dies with the process. That is fine until the day it is not. The other option is a full platform. Those work, but they arrive with a dozen transitive dependencies and a mental model to match, which is a lot when what you wanted was "run this later, and do not lose it."

redis_task_queue sits in between. It is a Redis-backed queue for server-side Dart with at-least-once delivery. Two packages total: it, and the redis client it talks through. No other transitive dependencies.

What "not lost" requires

The property is that a job survives the worker dying mid-run. That is not free, and it is worth being precise about how it is bought.

A worker does not read a task off the queue. It claims one, moving the task from the pending list onto a per-worker in-flight list in a single Redis operation (LMOVE). The task is now on the worker's own list and nowhere else. The worker runs the handler. Only when the handler finishes, retries, or is dead-lettered does the task come off the in-flight list.

Walk the kill -9 case through. The worker claimed the task, moved it to its in-flight list, started running, and died. The task is still sitting on that in-flight list in Redis. Nothing removed it, because removal is the last step and the last step never ran. When the worker starts again under the same id, it sweeps its own in-flight list and requeues whatever is there.

That last clause is the catch, and the one people get wrong. Recovery is scoped to one worker's own in-flight list. A restarted worker only recovers work it can prove was its own, so it has to come back with the same identity. The worker id is an identity, not a nonce.

final worker = await Worker.connect(
  // A StatefulSet pod name. Stable across restarts, so recovery works.
  workerId: 'worker-2',
);

If you pass a random UUID on every boot, the orphaned task on worker-2's list has no one who will ever claim it, because the process that could is now worker-9f3a and only looks at its own list. Use the pod name from a StatefulSet. Do not use Uuid().v4().

Idempotency is your job, and there is a key for it

At-least-once means at least once. A worker can finish the work and die before it records that it finished, and the replacement will hand the task back and run it again. No queue avoids this. Acknowledging the work is a separate step from doing it, so something can always happen in between. Handlers have to be idempotent.

The tool for that is TaskContext.id. It is assigned once, at enqueue, and it is identical on every attempt and every recovery. Write against it, in the same transaction as the effect.

worker.handle('invoice:charge', (task, context) async {
  await db.execute(
    'INSERT INTO charges (task_id, invoice, amount) '
    'VALUES (@id, @inv, @amt) ON CONFLICT (task_id) DO NOTHING',
    {'id': context.id, 'inv': task.payload['invoice'], 'amt': 4200},
  );
  // Second run: same context.id, ON CONFLICT eats it, charged once.
});

Charge the card and record the task_id in one statement. The second run collides on the id and does nothing. That turns at-least-once into exactly-once effects, which is the honest version of what everyone actually wants.

The rest of it

Queues are weighted. {critical: 6, default: 3, low: 1} serves critical roughly six times as often as low, but low is never starved. The worker rotates a cursor over the weighted order rather than draining high priority first. Strict priority queues starve their tails. This does not.

Failed tasks retry with exponential backoff and jitter. When retries are exhausted the task lands on a dead-letter list you can read and act on, rather than vanishing:

for (final dead in await client.deadLetters(limit: 50)) {
  print('${dead.id}: ${dead.error}');
}
await client.replayDeadLetter(someId); // atomic; safe against double replay
await client.purgeDeadLetters();

stats() reports queue depth using SCAN, not KEYS, so calling it does not stall a busy Redis. stop() is awaitable. On SIGTERM, await it and the worker stops claiming new tasks and lets the one in progress finish before the loop exits, so a rolling deploy drains instead of abandoning work to recovery.

ProcessSignal.sigterm.watch().listen((_) async {
  await worker.stop();
  await worker.close();
});

When not to use it

Its durability is Redis's durability. Tasks survive a worker crash by construction. They survive a Redis crash only as well as your Redis is configured to, and default Redis persistence can lose the last fraction of a second. If losing one enqueued job is unacceptable no matter what fails, you want a queue that fsyncs to disk on the enqueue path, and this is the wrong tool.

If you already run Serverpod and want scheduling inside the same box with no extra infrastructure, its built-in future calls are a smaller operational footprint than standing up Redis. And if you need fan-out, DAGs, or step-level workflow orchestration, this does none of that on purpose. It runs one function per task, retries it, and does not lose it. That is the whole surface, and for a large number of "do this later" problems it is the right size.