DEV Community

Michael Sun
Michael Sun

Posted on • Originally published at novvista.com

Postgres SKIP LOCKED at 5K Messages/Second vs RabbitMQ: Our Production Benchmark Numbers

#webdev #programming #tools #devtools

When Postgres Beats RabbitMQ at Queueing (and When It Doesn't)

At 5,000 messages per second, our Postgres SKIP LOCKED setup matched RabbitMQ on throughput within 4%, but lost on P99 latency by a factor of 2.3x. For any workload below 50K msg/sec, the operational simplicity of "it's just Postgres" was worth every millisecond of that difference. These numbers aren't vendor claims or synthetic benchmarks—they're from our own testing, running identical workloads against both systems on identical hardware until they broke.

The Benchmark Breakdown

We ran a sustained 30-minute test at 5,000 msg/sec to compare Postgres with pgmq against a single-node RabbitMQ classic queue. The results were clear on trade-offs:

Throughput: Postgres with pgmq 1.5.0 sustained 4,920 msg/sec; RabbitMQ hit 5,140 msg/sec. Within margin of error, both kept up.

Latency: RabbitMQ won decisively here. P99 end-to-end latency was 4.9ms for RabbitMQ vs 11.4ms for Postgres. The gap was consistent across percentiles—P50 was 0.9ms vs 2.1ms. This isn't just a tail issue; RabbitMQ is consistently faster.

Resource Usage: RabbitMQ was more efficient, using 22% CPU across 16 cores compared to Postgres' 38%. Memory was interesting: Postgres held a flat 14GB in shared_buffers + ~3GB working set, while RabbitMQ grew from 800MB to 2.4GB over the test.

The Cliff: The real story is in failure modes. Postgres SKIP LOCKED on this hardware fell off a cliff around 38,000 msg/sec, where connection pool contention and WAL write amplification pushed P99 latency past 200ms. RabbitMQ classic queues kept scaling linearly past 80,000 msg/sec on the same hardware.

The Setup That Actually Works

You can't just slap SKIP LOCKED on a regular table and call it a queue. We tuned both systems aggressively.

Postgres Configuration

Defaults will get you embarrassed. The key was tuning autovacuum for churn, not growth:

shared_buffers = 16GB
work_mem = 64MB
max_connections = 400
synchronous_commit = on
wal_compression = on

# Autovacuum tuned for queue workload
autovacuum_vacuum_cost_delay = 2ms
autovacuum_vacuum_cost_limit = 2000
autovacuum_naptime = 10s
autovacuum_max_workers = 6
Enter fullscreen mode Exit fullscreen mode

pgmq Schema

pgmq generates the DDL, but the critical part is the visibility index:

CREATE TABLE pgmq.q_orders (
    msg_id BIGSERIAL PRIMARY KEY,
    read_ct INTEGER DEFAULT 0 NOT NULL,
    enqueued_at TIMESTAMPTZ DEFAULT NOW() NOT NULL,
    vt TIMESTAMPTZ NOT NULL,
    message JSONB
);

CREATE INDEX q_orders_vt_idx ON pgmq.q_orders (vt ASC);
Enter fullscreen mode Exit fullscreen mode

We tested both LOGGED (durable) and UNLOGGED variants. The UNLOGGED variant was 22% faster but loses crash recovery—only suitable for non-critical workloads.

The Verdict: When to Choose Which

If your workload is under 50K msg/sec, uses your existing Postgres database, and you don't have a dedicated platform team for RabbitMQ, Postgres wins on operational simplicity. The latency difference is often acceptable given the elimination of another moving part.

If you're above 50K msg/sec sustained or need heavy fan-out with topic exchanges, RabbitMQ classic queues scale better. Just avoid quorum queues—they peaked at 3,200 msg/sec in our tests due to Raft consensus overhead.

If you need both transactional semantics AND high throughput, you're facing a harder architectural problem than this solves—likely requiring a hybrid approach.

Read the full article at novvista.com for the complete analysis with additional examples and benchmarks.

Originally published at NovVista

Top comments (0)