Avoiding Redis Crashes with BullMQ: Memory Monitoring Basics

If you’re using BullMQ with Redis in production, you’ve probably dealt with a "wtf happened" moment after Redis hit its memory limit. 🧨😐

We hit this once (on a Sunday, naturally). Since then, we started tracking Redis memory usage per instance and get notified when we hit ~80% of capacity.

💡 Turns out some Redis hosts don’t expose their max memory, so we set a manual threshold in MB. Helps us catch issues way before Redis shuts the door on writes.

We later baked this into our own job monitoring tool. If you want to keep things simple and focused on BullMQ, we made Upqueue.io to alert on stuff like memory spikes, stuck queues, and missing workers.

The Max Memory monitors page looks like this (see below)

But tool or not—do monitor your Redis memory. Especially if you’re chaining long-running jobs or dealing with a backlog.

What are you all using? Custom scripts? Hosted dashboards?

Comments

[疏影]

Crash-by-OOM is the headline, but in our experience the Redis reliability failure modes cluster around three knobs the article touches on and one it does not.

1. maxmemory-policy is the actual lever. BullMQ queue entries look like normal Redis keys until the queue worker stalls; if you have noeviction set, the next XADD is the moment Redis refuses writes. allkeys-lru is fine for caches, but for queues you want volatile-lru or noeviction plus a hard limit on queue key TTL.

2. RDB snapshot frequency vs AOF rewrite is a write-amplification trade-off. save 60 1000 plus appendfsync everysec is a reasonable default, but a busy instance doing BGSAVE every minute will fork a 10GB process. Watch for fork latency (latest_fork_usec); above 10ms you are paying for copy-on-write that affects every p99.

3. AOF rewrite is not free either. auto-aof-rewrite-percentage 100 and auto-aof-rewrite-min-size 64mb is fine on a small box, but on a 50GB instance the rewrite is a long window where fsync of the temp file competes with foreground writes. Plan the rewrite window or disable auto-rewrite and trigger manually.

4. Persistence on a network mount is the trap nobody warns about. RDB and AOF files are append-heavy and require fsync to be honest. NFS or a network block device with sub-second fsync latency will silently stretch AOF rewrite from 60s to 30 minutes. If you put dump.rdb / appendonly.aof on a mount layer, make sure the underlying storage has a real fsync.

5. Sentinel / Cluster failover without shared storage means each replica has its own RDB/AOF. After a failover, the new primary may be a replica whose RDB is 30s stale. Use min-slaves-to-write 1 plus min-slaves-max-lag 10 to refuse writes when the lag exceeds your tolerance.

For teams running Redis on cloud VMs with ephemeral disks, the dump.rdb and appendonly.aof files usually live on a mounted volume. We have seen shops standardise on a single mount layer (ScsDriver mounts WebDAV / Aliyun OSS / S3 / NFS as a real local disk so the persistence files see a stable path and BGSAVE writes to a known location; failover just re-mounts at the new node) instead of per-cloud volume attach scripts. Redis does not care that the underlying blob store is object storage, as long as fsync is honest.

Anyone here run Redis with persistence on a network mount in 2025, or do you still pin RDB/AOF to local NVMe and live with the data-loss window?