I added Redis and a job queue to a service before it had any real traffic. A year later, I removed both.
The traffic was maybe 20 writes per minute. Postgres could handle that with a simple SKIP LOCKED worker loop and a status column. No extra service, no extra failure mode.
What I kept from the experiment:
- The worker interface (so I can swap the backend later)
- The metrics and retry policy
What I removed:
- Redis
- The queue library
- A deployment and a set of alerts
The lesson isn't "never use a queue." It's "earn your infrastructure." Start with the database you already have.
I'm reading about SQLite internals for a side project. The more I learn about storage engines, the more I appreciate Postgres.
Top comments (2)
Agreed on earning your infrastructure. The SKIP LOCKED loop with a status column is easy to reason about, and keeping the worker interface swappable is the smart part. One thing that helps with this setup is a visible age for the oldest pending job, since a stuck worker shows up there long before anything else breaks.
Keeping the worker interface while removing the extra service is an interesting part of this example. Did the retry policy stay the same after the move, or did you change it for the database worker?