Pub/Sub Patterns: Independent Publishers and Subscribers
A practical guide to the publish/subscribe pattern itself — how publishers and subscribers decouple from each other, the mechanics of topic-based and content-based subscription, fan-out and delivery guarantees, and a working comparison across the pub/sub implementations covered elsewhere in this series: Redis, RabbitMQ, Kafka, Azure Service Bus, SignalR, and cloud-native pub/sub services.
Table of Contents
- Introduction
- The Defining Property: Publishers Don't Know Their Subscribers
- Topic-Based vs. Content-Based Subscription
- Fan-Out Mechanics
- Delivery Guarantees Across Pub/Sub Implementations
- Wildcard and Hierarchical Topics
- Pub/Sub Implementations Compared
- Cloud-Native Pub/Sub Services
- The Subscriber Lifecycle Problem
- Ordering Guarantees in Pub/Sub
- Combining Pub/Sub with Point-to-Point Messaging
- Choosing an Implementation
- Common Pitfalls
- Quick Reference Table
- Conclusion
Introduction
Publish/subscribe (pub/sub) is a messaging pattern where a publisher sends a message without addressing it to any specific recipient, and any number of subscribers, independently and without the publisher's knowledge, receive their own copy of that message if they've expressed interest in it. This guide focuses specifically on the pattern's mechanics — what genuinely defines pub/sub, the variations across topic-based and content-based subscription, and how the specific technologies covered elsewhere in this series (Redis, RabbitMQ, Kafka, Azure Service Bus, SignalR) each implement the same underlying idea differently, with meaningfully different guarantees.
// Publisher: has no idea how many subscribers exist, or who they are
await publisher.PublishAsync("order-events", orderPlacedEvent);
// Subscriber A and Subscriber B: both receive their own independent copy, entirely unaware of each other
await subscriber.SubscribeAsync("order-events", HandleOrderEvent);
If you've read this series' Event-Driven Architecture guide, this document is the more focused, mechanical companion to it — where that guide covers the broader architectural patterns (sagas, event sourcing, the outbox pattern), this one zooms in on pub/sub specifically: the subscription models, delivery guarantees, and concrete implementation trade-offs.
1. The Defining Property: Publishers Don't Know Their Subscribers
Contrast with point-to-point messaging
Point-to-point (a queue): one message → delivered to exactly ONE consumer among however many are listening
Pub/sub (a topic): one message → delivered to EVERY subscriber currently subscribed
This is the fundamental distinction from the point-to-point, competing-consumers model covered in this series' RabbitMQ, Kafka, and Azure Service Bus guides — in a queue, multiple consumers compete for messages (each message goes to exactly one of them, distributing load); in pub/sub, multiple subscribers each independently receive their own copy of every message (broadcasting the same information to everyone interested).
Why "the publisher doesn't know" is the actual point, not an implementation detail
// Adding a NEW subscriber requires ZERO changes to the publisher
await subscriber.SubscribeAsync("order-events", HandleOrderEventForFraud); // a fraud-detection service, added months later
The practical payoff of genuine pub/sub decoupling is that new subscribers can be added at any time, by anyone with access to subscribe to the topic, with no code change, no redeployment, and no coordination required from the publisher at all — this is the same core benefit covered in this series' Event-Driven Architecture guide's discussion of choreography, expressed here at the level of the specific pattern that makes it possible.
The trade-off this creates
Because the publisher genuinely doesn't know who's listening, it also has no built-in way to know whether a specific subscriber successfully processed a message, or even whether any subscriber exists at all for a given topic — pub/sub optimizes for decoupling and extensibility, not for the caller-gets-a-definitive-answer guarantee a direct call or a request/reply pattern provides.
2. Topic-Based vs. Content-Based Subscription
Topic-based: subscribe to a named channel
await subscriber.SubscribeAsync("order-events", handler);
await subscriber.SubscribeAsync("inventory-events", handler);
The simplest and most common model — subscribers declare interest in a specific, named topic (or channel), and receive every message published to that exact topic. Redis Pub/Sub (Section 6) and RabbitMQ's fanout exchange (covered in this series' RabbitMQ guide) are both, at their core, topic-based in this sense.
Content-based: subscribe based on the message's actual content
// A subscription filter evaluated against the message itself, not just its topic name
await adminClient.CreateRuleAsync("order-events", "high-value-orders",
new CreateRuleOptions("HighValueFilter", new SqlRuleFilter("Total > 500")));
Content-based subscription (as covered for Azure Service Bus's subscription rules in this series' companion guide) lets a subscriber express interest based on the actual properties or payload of a message, not just which topic it was published to — a subscriber might receive only orders over $500, regardless of what topic they were published under, giving finer-grained selectivity than a topic name alone can express.
RabbitMQ's topic exchange: a hybrid, pattern-based middle ground
await channel.QueueBindAsync("order-created-only", "orders", routingKey: "order.created");
await channel.QueueBindAsync("all-order-events", "orders", routingKey: "order.*");
As covered in this series' RabbitMQ guide, topic exchanges route based on a hierarchical routing key with wildcard matching — not a simple fixed topic name, and not full arbitrary content-based filtering either, but a structured middle ground that's expressive enough for most real subscription needs without the overhead of evaluating arbitrary filter expressions against every message.
Choosing between the models
Topic-based subscription is simpler to reason about and generally faster (no filter evaluation needed), and is the right default when subscribers naturally divide along clear topic boundaries. Content-based (or RabbitMQ's routing-key pattern) subscription earns its added complexity when subscribers need finer selectivity than a topic name alone provides — a fraud-detection service that only cares about high-value transactions shouldn't need to subscribe to every single transaction event and filter client-side if the broker itself can do that filtering more efficiently.
3. Fan-Out Mechanics
What actually happens when one message reaches many subscribers
Publisher publishes ONE message →
Broker/system makes N copies (one per currently-interested subscriber) →
Each subscriber receives and processes its own independent copy
Fan-out is the mechanical process of a broker taking a single published message and delivering an independent copy to every interested subscriber — critically, each copy is independent: one subscriber's processing (success, failure, retry) has no effect on any other subscriber's copy or processing outcome.
Fan-out at different layers
Redis Pub/Sub: fan-out happens in-memory, at the broker process itself, no persistence at all
RabbitMQ fanout exchange: fan-out happens by copying the message into every bound queue
Kafka consumer groups: fan-out happens because different CONSUMER GROUPS each read the same retained log independently
Service Bus topics: fan-out happens by copying the message into every subscription
Every pub/sub implementation covered in this series achieves the same conceptual fan-out, but the underlying mechanism — and therefore the guarantees that come with it — differ substantially, which is exactly why Section 4 (delivery guarantees) and Section 6 (implementation comparison) matter so much in practice; "it's pub/sub" alone doesn't tell you what happens if a subscriber is offline when a message is published.
Fan-out cost scales with subscriber count
A message published to a topic with 50 active subscribers genuinely results in up to 50 independent copies being stored and delivered (depending on the specific broker's implementation) — this is a real, scaling cost of the pattern, and it's part of why some systems (Kafka, in particular) implement fan-out more efficiently by having independent consumer groups read from the same underlying retained log rather than literally duplicating the message data per subscriber.
4. Delivery Guarantees Across Pub/Sub Implementations
The single most important practical question: what happens to an offline subscriber?
Scenario: Subscriber B is offline (crashed, deploying, network partition) when a message is published.
Does Subscriber B ever receive that message once it comes back online?
This single question splits pub/sub implementations into two fundamentally different categories, and getting this wrong (assuming a message will be there later when the implementation doesn't actually guarantee that) is one of the most consequential mistakes possible in a pub/sub design.
Category 1: "You had to be listening" — no retained history
// Redis Pub/Sub — if no subscriber is actively connected and subscribed at publish time, the message is simply gone
await subscriber.PublishAsync(RedisChannel.Literal("notifications"), message);
As covered in this series' Redis guide, Redis Pub/Sub delivers a message only to subscribers actively connected and subscribed at the exact moment of publication — there's no queue, no retention, no redelivery; a subscriber that wasn't listening simply never receives that message, permanently. This makes Redis Pub/Sub appropriate specifically for ephemeral, best-effort notifications (like fanning out a SignalR message across server instances, Section 6) and a poor fit for anything where a subscriber genuinely must not miss a message.
Category 2: durable, subscriber-independent delivery
// RabbitMQ, Kafka, Service Bus — the message is durably held and WILL be delivered
// once the subscriber reconnects, regardless of how long it was offline
RabbitMQ (via durable queues bound to a fanout/topic exchange), Kafka (via its retained log and per-consumer-group offset tracking), and Azure Service Bus (via durable subscriptions) all guarantee that a message published while a subscriber is offline will still be delivered once that subscriber reconnects — the message is held durably, independent of any specific subscriber's connection state, until it's been consumed (or expires per a retention/TTL policy).
Why this distinction matters more than any other single design decision
Choosing a "you had to be listening" pub/sub implementation for a use case that actually needs guaranteed eventual delivery is a design bug, not a configuration tweak to fix later — it's worth explicitly answering "is it acceptable for a subscriber to permanently miss a message if it happens to be down at the wrong moment" as the very first question when designing a pub/sub interaction, since the answer determines which entire category of implementation is even appropriate.
5. Wildcard and Hierarchical Topics
Structuring topic names hierarchically
orders.created
orders.shipped
orders.cancelled
inventory.restocked
inventory.low-stock
A common convention across many pub/sub systems is naming topics (or routing keys) hierarchically, using a consistent separator — this makes wildcard subscription (below) meaningful and predictable, and gives topic names a natural, browsable structure as the number of distinct event types grows.
Wildcard subscription
// RabbitMQ topic exchange
await channel.QueueBindAsync("all-order-events", "events", routingKey: "orders.*");
await channel.QueueBindAsync("everything", "events", routingKey: "#");
As covered in this series' RabbitMQ guide, * typically matches exactly one hierarchical segment and # matches zero or more — letting a subscriber express "everything under orders.*" without needing to enumerate every specific event type individually, and without the publisher needing to know in advance every possible subscription granularity a future subscriber might want.
MQTT: wildcard topics as a first-class IoT pattern
sensors/building-1/floor-3/temperature
sensors/+/floor-3/temperature ← + matches exactly one level, any building on floor 3
sensors/building-1/# ← # matches everything under building-1
MQTT (a lightweight pub/sub protocol widely used in IoT scenarios, distinct from but conceptually related to the brokers covered elsewhere in this series) uses this same hierarchical wildcard model natively as its core subscription mechanism — worth mentioning here because it's a particularly clean, minimal illustration of hierarchical topic design applied to a domain (sensor data from many devices) where the pattern fits especially naturally.
6. Pub/Sub Implementations Compared
Redis Pub/Sub
var subscriber = redis.GetSubscriber();
await subscriber.SubscribeAsync(RedisChannel.Literal("notifications"), (channel, message) => Handle(message));
await subscriber.PublishAsync(RedisChannel.Literal("notifications"), messageValue);
As covered in this series' Redis guide: in-memory, no persistence, "you had to be listening" delivery — extremely low latency, appropriate for ephemeral fan-out where occasional missed messages are acceptable (SignalR's cross-instance backplane being the canonical use case).
RabbitMQ fanout/topic exchange
await channel.ExchangeDeclareAsync("order-events", ExchangeType.Fanout, durable: true);
await channel.QueueDeclareAsync("email-service-queue", durable: true, exclusive: false, autoDelete: false);
await channel.QueueBindAsync("email-service-queue", "order-events", routingKey: "");
As covered in this series' RabbitMQ guide: durable, subscriber-independent delivery (as long as each subscriber has its own durable queue bound to the exchange), rich routing flexibility via topic/direct/headers exchange types, and manual acknowledgment for at-least-once delivery guarantees.
Kafka (consumer groups reading a topic)
var config = new ConsumerConfig { GroupId = "fraud-detection-service", BootstrapServers = "..." };
As covered in this series' Kafka guide: fan-out achieved not by duplicating messages, but by letting multiple independent consumer groups each read the same retained, replayable log at their own pace — the strongest option here for genuine replay (a new subscriber can read the entire topic's history from the beginning) and for very high message volumes.
Azure Service Bus topics and subscriptions
await using var sender = client.CreateSender("order-events");
await using var receiver = client.CreateReceiver("order-events", subscriptionName: "email-service");
As covered in this series' Azure Service Bus guide: durable, managed, with rich content-based filtering per subscription and built-in dead-lettering — a strong choice for teams wanting pub/sub semantics without operating broker infrastructure themselves.
SignalR groups
await Clients.Group("order-1001-watchers").SendAsync("OrderStatusChanged", newStatus);
As covered in this series' SignalR guide, SignalR groups are a pub/sub pattern specifically aimed at pushing real-time updates to connected clients (browsers, mobile apps) rather than server-to-server messaging — delivery is inherently "you had to be connected" (a disconnected client misses updates until it reconnects and, typically, re-fetches current state via a regular API call), making it conceptually closer to Redis Pub/Sub's guarantee category than to a durable broker's.
7. Cloud-Native Pub/Sub Services
Azure Event Grid
var client = new EventGridPublisherClient(topicEndpoint, new AzureKeyCredential(accessKey));
await client.SendEventAsync(new CloudEvent("orders/api", "OrderPlaced", orderPlacedEvent));
Azure Event Grid is a lightweight, fully managed pub/sub service purpose-built for reactive, event-driven automation — reacting to a blob being uploaded, a resource being created, or a custom application event — with push-based delivery (Event Grid calls a webhook/Function endpoint directly, rather than a subscriber pulling from a queue) and at-least-once delivery with automatic retry. It's deliberately lighter-weight than Azure Service Bus: no sessions, no transactions, no built-in dead-letter queue browsing UI (though dead-lettering to a storage account is supported) — appropriate for lightweight, high-volume reactive triggers rather than durable, feature-rich business messaging.
AWS SNS (Simple Notification Service)
await snsClient.PublishAsync(new PublishRequest { TopicArn = topicArn, Message = messageJson });
AWS SNS is a managed pub/sub service that fans out published messages to multiple subscriber types simultaneously — SQS queues, Lambda functions, HTTP endpoints, email, and SMS — commonly paired with SQS specifically (the "fan-out to SQS" pattern: SNS handles the pub/sub fan-out, and each subscribing SQS queue provides the durable, competing-consumers processing covered in this series' AWS Compute guide) to combine pub/sub's broadcast semantics with a queue's durability and retry guarantees.
Google Cloud Pub/Sub
Google Cloud's Pub/Sub service offers durable, at-least-once delivery with both push (webhook-style) and pull (subscriber-polls) subscription models, ordering keys for per-key ordering (conceptually similar to Kafka's partition-key ordering), and native integration with Google's broader data pipeline tooling — mentioned here for completeness, since it's a common choice for organizations on GCP, following the same durable-pub/sub category as Service Bus and SNS-plus-SQS rather than Redis Pub/Sub's ephemeral category.
8. The Subscriber Lifecycle Problem
What happens when a new subscriber joins after messages have already been published?
Kafka: a new consumer group can read from the BEGINNING of the retained log (subject to retention policy)
RabbitMQ/Service Bus: a new subscription only receives messages published AFTER it was created
Redis Pub/Sub: a new subscriber never sees anything published before it connected, ever
This is a specific, important variant of the delivery-guarantee question from Section 4: even among durable pub/sub implementations, there's a real difference between "can a brand-new subscriber see historical messages" (Kafka's replay capability, covered in depth in this series' Kafka guide) versus "a new subscription only sees what's published going forward" (RabbitMQ and Service Bus's typical model, where a subscription/queue must already exist to receive a message — it can't retroactively receive something published before it was created).
Designing for late-joining subscribers deliberately
// If a late-joining subscriber needs current state, not just future events,
// it often needs a separate mechanism to bootstrap: a snapshot/current-state query,
// not just subscribing to the event stream going forward
var currentState = await _orderService.GetCurrentOrdersAsync(); // bootstrap via a direct query
await subscriber.SubscribeAsync("order-events", HandleFutureEvents); // then stay current via events
For pub/sub implementations that don't support historical replay (the RabbitMQ/Service Bus/Redis category), a subscriber that joins later and needs to know about things that already happened typically needs a separate bootstrap mechanism — a direct query against a REST API or database for current state, followed by subscribing to the event stream to stay current from that point forward — rather than assuming the event stream alone can answer "what's the current state of everything."
9. Ordering Guarantees in Pub/Sub
Pub/sub and ordering are often in tension
Fan-out to multiple subscribers, each with independent processing speed and success/failure timing
→ different subscribers can end up "ahead" or "behind" relative to each other in processing order,
even if the underlying delivery mechanism preserves publish order
Even when a pub/sub implementation preserves the order messages were delivered to a given subscriber, independent subscribers processing at their own pace, with their own retry/failure behavior, naturally diverge in processing order relative to each other — this is rarely a problem (each subscriber's view of order is usually all that matters to it), but it's worth being explicit that "pub/sub" doesn't inherently promise any global ordering guarantee across subscribers.
Per-subscriber ordering
Kafka: ordering preserved WITHIN a partition, for a given consumer group's processing of it
RabbitMQ: ordering preserved within a single queue's delivery to a single consumer
Service Bus: ordering preserved within a session (per this series' companion guide)
What most implementations do guarantee is ordering within a single subscriber's stream of received messages, for messages sharing an appropriate key/partition/session — the same partition-key and session-based ordering mechanisms covered in this series' Kafka and Azure Service Bus guides apply directly here; pub/sub doesn't add a separate ordering concern beyond what those underlying mechanisms already provide.
10. Combining Pub/Sub with Point-to-Point Messaging
The common "fan-out then compete" pattern
Publisher → Topic → Subscription A (durable queue) → [Consumer 1, Consumer 2, Consumer 3] compete for THIS subscription's messages
→ Subscription B (durable queue) → [Consumer 4] processes THIS subscription's messages alone
This is arguably the single most common real-world pattern combining both models: a topic fans out each published message to multiple independent subscriptions (pub/sub, one copy per subscription), while multiple consumer instances attached to any one subscription compete for that subscription's messages (point-to-point, load-distributed processing) — exactly how Azure Service Bus topics/subscriptions, Kafka's consumer groups, and RabbitMQ's fanout-exchange-to-multiple-queues pattern all naturally support this combined model without requiring any special configuration beyond what's already covered in each technology's respective guide.
Why this combination is so widely used
It gives you both properties simultaneously: genuine decoupling between logically independent concerns (each subscription represents one independent "thing that needs to happen" in response to an event), and horizontal scalability within each of those concerns (scale out the number of consumers on any single subscription independently, based on that specific workload's actual throughput needs) — a design that naturally falls out of composing the patterns already covered in this series' messaging guides, rather than requiring a distinct third pattern to learn.
11. Choosing an Implementation
Need low-latency, ephemeral, best-effort fan-out (missed messages are acceptable)?
│
├── Yes, primarily for real-time client updates → SignalR (groups)
├── Yes, primarily for server-to-server, single-process/cluster scope → Redis Pub/Sub
│
└── No — messages must not be silently lost if a subscriber is briefly unavailable
│
├── Need very high volume, replay, or multiple independent consumer groups reading history? → Kafka
├── Need rich content-based filtering, managed service, enterprise features (sessions/transactions)? → Azure Service Bus
├── Need flexible routing (topic/direct/fanout/headers) with self-hosted or cross-cloud portability? → RabbitMQ
└── Need lightweight, push-based reactive automation triggers (not durable business messaging)? → Event Grid / SNS
The recurring theme across this entire series' messaging guides
As with the RabbitMQ-vs-Kafka-vs-Service Bus comparisons covered in their respective guides, there's no single universally correct pub/sub implementation — the right choice depends on the specific combination of delivery-guarantee needs (Section 4), replay requirements (Section 8), routing/filtering flexibility (Section 2), and operational preferences (self-hosted vs. managed) that a given use case actually has.
12. Common Pitfalls
| Pitfall | Why it hurts | Better approach |
|---|---|---|
| Using Redis Pub/Sub (or SignalR) for something that must not be silently missed | "You had to be listening" delivery means genuine, permanent message loss for an offline subscriber | Use a durable pub/sub implementation (RabbitMQ, Kafka, Service Bus) for anything requiring guaranteed delivery |
| Assuming a new subscriber automatically sees historical messages | Only Kafka-style retained logs support this; most durable brokers only deliver messages published after the subscription exists | Provide an explicit bootstrap/snapshot mechanism for late-joining subscribers that need historical state |
| Expecting global ordering across independent subscribers | Pub/sub inherently doesn't promise this; each subscriber processes independently at its own pace | Design for per-subscriber/per-partition/per-session ordering only, where actually needed |
| Confusing topic-based and content-based filtering costs | Content-based filtering (SQL-like rule evaluation) is more expensive per message than a simple topic-name match | Use topic-based subscription for the common case; reach for content-based filtering only where genuinely needed |
| Not distinguishing pub/sub fan-out from competing-consumer load distribution | Conflating the two leads to confusion about why "only one consumer got the message" in a genuinely pub/sub scenario, or vice versa | Be explicit about which property (broadcast vs. load-distribute) a given consumer group actually needs |
| Building custom polling-based "pub/sub" instead of using an existing implementation | Reinvents fan-out, delivery guarantees, and ordering poorly, from scratch | Use one of the well-understood implementations covered in this guide, matched to actual requirements |
Quick Reference Table
| Concept | Purpose |
|---|---|
| Topic-based subscription | Subscribe to a named channel/topic; the simplest, most common model |
| Content-based subscription | Filter delivery based on message properties, not just topic name |
| Fan-out | One published message, independently delivered to every interested subscriber |
| "You had to be listening" | Redis Pub/Sub, SignalR — no retention, offline subscribers permanently miss messages |
| Durable pub/sub | RabbitMQ, Kafka, Service Bus — messages held until delivered, independent of subscriber connection state |
| Replay | Kafka-specific: a new subscriber can read a topic's retained history from the beginning |
| Wildcard topic | Pattern-based subscription across a hierarchical topic namespace (orders.*, #) |
| Fan-out then compete | Combining pub/sub (multiple subscriptions) with point-to-point (multiple consumers per subscription) |
| Push vs. pull delivery | Event Grid/SNS push to a webhook; Kafka/Service Bus consumers pull at their own pace |
Conclusion
Pub/sub's core promise — a publisher that broadcasts a fact without needing to know or coordinate with whoever eventually cares about it — is genuinely valuable, but the pattern name alone tells you almost nothing about the guarantees a specific implementation actually provides. The single most consequential design question is whether an offline subscriber can permanently miss a message, and every implementation covered in this series (Redis, RabbitMQ, Kafka, Azure Service Bus, SignalR, Event Grid, SNS) sits on one side or the other of that line, with further meaningful differences in replay capability, filtering expressiveness, and ordering guarantees layered on top.
Choosing well means starting from the actual requirement — can this message be missed, does a late-joining subscriber need history, how selective does subscription need to be — and only then picking the specific technology whose guarantees genuinely match, rather than reaching for whichever pub/sub system happens to be already running and hoping its particular guarantees turn out to be sufficient.
Found this useful? Feel free to star the repo, open an issue with corrections, or share the "you had to be listening" surprise that taught you to check a broker's actual delivery guarantee before trusting it.
Top comments (0)