DEV Community

Cover image for Redis Pub/Sub vs. Redis Streams: Choosing the Right Solution
LinceMathew
LinceMathew

Posted on

4 3 4 3 4

Redis Pub/Sub vs. Redis Streams: Choosing the Right Solution

While working on building a Go binary tool ("Runner") for our product LiveAPI (the super convenient API doc generation tool), we needed to establish communication between LiveAPI's backend and Runner. We initially tried the Redis pub/sub mechanism but later realized it was insufficient for our use case.

The Problems with Redis Pub/Sub

Redis Pub/Sub follows a traditional publish-subscribe model: Publishers send messages to channels, and subscribers receive messages from the channels they are subscribed to.

pub/sub

This presented several problems for LiveAPI:

Duplicate Operations and Wasted Resources: If a user ran the same Runner on multiple devices, the subscription behavior of pub/sub would cause the same operations to run on multiple devices, creating duplication and wasting resources.

Memory and Data Loss: Pub/sub doesn't retain or persist messages, so if the server crashed or a job failed due to a panic, there was no way to restart the operations.

Lack of Acknowledgement: The LiveAPI backend had no indication that a job request from the consumer had reached the subscriber.

Redis Streams as the Solution

Redis Streams provide a more robust and feature-rich messaging system. They are essentially append-only logs where producers add messages, and consumers read them.

Redis Streams addresses the issues we encountered with Pub/Sub:

Avoids Duplicate Operations: Even if the same Runner is running on multiple devices, only one instance can read and acknowledge a message, preventing the duplicate operations that occurred with pub/sub.

Message Persistence: Redis Streams persist messages. If the server crashes, operations can be restarted with the same message.

Guaranteed Delivery and Acknowledgement: Redis Streams provides message acknowledgement, assuring the LiveAPI backend that job requests reach the intended subscriber.

stream

Understand Your Usecase
Redis Streams was selected as a solution for LiveAPI due to its message persistence, guaranteed delivery, message acknowledgement, and ability to prevent duplicate operations.

While Pub/Sub is a simpler solution and better suited for real-time messaging, Streams offers more features and guarantees, making it ideal for use cases where data durability and reliability are critical.

Image of Timescale

🚀 pgai Vectorizer: SQLAlchemy and LiteLLM Make Vector Search Simple

We built pgai Vectorizer to simplify embedding management for AI applications—without needing a separate database or complex infrastructure. Since launch, developers have created over 3,000 vectorizers on Timescale Cloud, with many more self-hosted.

Read full post →

Top comments (0)

Billboard image

The Next Generation Developer Platform

Coherence is the first Platform-as-a-Service you can control. Unlike "black-box" platforms that are opinionated about the infra you can deploy, Coherence is powered by CNC, the open-source IaC framework, which offers limitless customization.

Learn more

👋 Kindness is contagious

Dive into an ocean of knowledge with this thought-provoking post, revered deeply within the supportive DEV Community. Developers of all levels are welcome to join and enhance our collective intelligence.

Saying a simple "thank you" can brighten someone's day. Share your gratitude in the comments below!

On DEV, sharing ideas eases our path and fortifies our community connections. Found this helpful? Sending a quick thanks to the author can be profoundly valued.

Okay