Queues: at-least-once means never exactly-once

Rowan Feld — Sun, 01 Nov 2026 15:11:42 +0000

The most useful sentence I've read about queues: at-least-once delivery is easy, exactly-once processing is a property of your handler, not the broker.

Which means every consumer needs to survive a duplicate. The ways I do that, in order of preference:

Idempotent writes. UPDATE ... WHERE version = expected instead of blind writes.
A dedupe key with a unique index. Cheapest and most explicit.
A processed table with a TTL, for handlers with side effects.

The failure mode to design for isn't the duplicate, it's the poison message: one bad event that blocks the partition because the handler retries forever. Give every consumer a max attempt count and a dead-letter queue you actually read.

One more: make the retry delay grow. A flat 1-second retry turns a small outage into a stampede.

What's your dead-letter policy?

A job queue in Postgres, two months later

Rowan Feld — Thu, 29 Oct 2026 10:42:40 +0000

We moved a small job queue from Redis onto Postgres two months ago, mostly to get transactions and a single backup story. Here's what actually changed.

Throughput was never the problem. We run about 40 jobs a second at peak, and SELECT ... FOR UPDATE SKIP LOCKED handles that without breaking a sweat. The counter arguments about Postgres queues are usually about ten thousand a second, which we will never see.

What did need thought was stuck jobs. A worker that dies mid-job leaves a locked row, and SKIP LOCKED will skip it forever. We added a locked_until column and a reaper that clears expired locks every thirty seconds. That reaper is the actual queue implementation; the select is easy.

The other surprise was vacuum. A table that churns through rows fast needs its autovacuum settings tuned, or the table bloats and the queue gets slower for no obvious reason.

Two months in: fewer moving parts, one backup, and a reaper job I check on Mondays. I'd make the same trade again.

DEV Community: Rowan Feld

Queues: at-least-once means never exactly-once

A job queue in Postgres, two months later