Azure Service Bus: Topics, Subscriptions, and Dead Letter Queues

#azure #messaging #cloud #dotnet

Azure Service Bus is an enterprise-grade cloud messaging service. It decouples applications so they do not need to be online at the same time to communicate. Instead of calling each other directly, services send messages to Service Bus and other services pick them up when ready. Service Bus was used extensively at Blue Yonder to connect Logic Apps, Function Apps, and external systems like Salesforce and ServiceNow - it was the backbone of every reliable integration built there.

Why Messaging Instead of Direct API Calls

A direct API call creates tight coupling: System A calls System B over HTTP, and if B is down, A fails immediately; if B is slow, A waits and blocks. Service Bus replaces that direct call with loose coupling instead - System A sends a message to Service Bus, and System B reads it whenever it's ready. This means B can be temporarily down and the message simply waits safely, A never blocks since it's fire-and-forget, multiple systems can read the same message independently, and every message leaves a full audit trail.

Service Bus core concepts — a queue for one consumer, a topic that fans out to multiple independent subscriptions, and a dead-letter queue for anything that fails repeatedly.

Queues vs Topics

A queue is one sender, one receiver - a message goes in one end, exactly one consumer reads it, and once read the message is gone. This fits scenarios where one system should process each message exactly once, like an Order Service sending work to a Fulfillment Service through a queue, where each order gets processed by exactly one worker.

A topic is one sender, many receivers - a message is published once, and every subscriber attached to that topic gets their own independent copy. This fits scenarios where multiple systems need to react to the same event, like an Order Service publishing to a topic where an Inventory subscriber, a Notification subscriber, and an Analytics subscriber each independently receive the same order event.

Subscriptions and Filters

Each topic subscription can have filters so it only receives the messages it actually cares about, which is extremely powerful for routing in enterprise integrations. On a topic named order-events, a subscription called high-priority-orders might filter on Priority equals High and receive only high priority orders. A subscription called international-orders might filter on Region not equal to US and receive only non-US orders. A subscription called all-orders with no filter at all receives every single order event published to that topic.

Dead Letter Queue

The Dead Letter Queue, or DLQ, is one of the most important concepts in Service Bus. When a message cannot be processed successfully after the maximum number of retries, Service Bus automatically moves it to the DLQ instead of losing it forever.

In the normal flow, a producer sends a message to a queue, the consumer processes it successfully, and the message is deleted after that successful processing. In the failed message flow, a producer sends a message, the consumer throws an exception, retry one fails, retry two fails, retry three fails and hits the maximum retry count, and the message is then moved to the Dead Letter Queue while the original queue becomes unblocked for everything behind it. The DLQ itself lives at a predictable path - for a queue named myqueue, the dead letter path is myqueue/$DeadLetterQueue.

This matters enormously in production for several concrete reasons: messages are never silently lost, failed messages do not block healthy messages behind them in the queue, the exact failure can be inspected directly, the underlying issue can be fixed and the message reprocessed, and there's a full audit trail available for compliance purposes.

Sending Messages in C

var client = new ServiceBusClient(connectionString);
var sender = client.CreateSender("order-topic");

var message = new ServiceBusMessage(
    JsonSerializer.Serialize(new {
        OrderId = "ORD-001",
        Priority = "High",
        Region = "EU"
    })
);

message.Subject = "OrderCreated";
message.ApplicationProperties["Priority"] = "High";
message.ApplicationProperties["Region"] = "EU";
message.MessageId = Guid.NewGuid().ToString();

await sender.SendMessageAsync(message);

Receiving Messages in C

var processor = client.CreateProcessor("myqueue");

processor.ProcessMessageAsync += async args =>
{
    var body = args.Message.Body.ToString();
    var order = JsonSerializer.Deserialize(body);

    await ProcessOrderAsync(order);

    // Tell Service Bus: processed successfully
    // Service Bus deletes it from the queue
    await args.CompleteMessageAsync(args.Message);
};

processor.ProcessErrorAsync += async args =>
{
    Console.WriteLine("Error: " + args.Exception);
    // Message automatically retried then dead-lettered
};

await processor.StartProcessingAsync();

The ProcessMessageAsync handler receives each message, processes it, and calls CompleteMessageAsync to tell Service Bus the message was handled successfully so it gets removed from the queue. The ProcessErrorAsync handler catches processing failures - Service Bus handles the retry logic automatically and moves the message to the dead letter queue once retries are exhausted, without any extra code required to make that happen.

Reading from the Dead Letter Queue

var receiver = client.CreateReceiver(
    "myqueue",
    new ServiceBusReceiverOptions {
        SubQueue = SubQueue.DeadLetter
    }
);

var messages = await receiver.ReceiveMessagesAsync(10);
foreach (var msg in messages)
{
    Console.WriteLine("DLQ Message: " + msg.Body);
    Console.WriteLine("Reason: " + msg.DeadLetterReason);
    Console.WriteLine("Error: " + msg.DeadLetterErrorDescription);

    // Fix the issue then complete or resubmit
    await receiver.CompleteMessageAsync(msg);
}

Setting SubQueue to SubQueue.DeadLetter on the receiver options points that receiver specifically at the dead letter queue rather than the main queue. Each dead-lettered message carries a DeadLetterReason and a DeadLetterErrorDescription, which together explain exactly why that message failed - genuinely useful information when deciding whether to fix and resubmit it or discard it entirely.

Message Sessions - Ordered Processing

Sessions guarantee that messages sharing the same SessionId are processed in order by the same consumer, which is critical for workflows where order genuinely matters. Setting message.SessionId to a customer ID means all messages for that same customer route to the same session, get processed in order by the same consumer instance, while other consumers handle other customers' sessions entirely in parallel. At Blue Yonder, this pattern was used with SessionId set to an IntegrationFlowId, so all messages belonging to one integration run were processed in sequence by the same Logic App instance rather than potentially arriving out of order.

Service Bus with Logic Apps

This is the most common pattern in enterprise Azure integration - a Logic App triggers on new Service Bus messages and orchestrates the workflow from there. In a typical pattern, a Logic App triggers when a message is received on a queue named incoming-orders, then parses the message body, checks a condition on the Priority field, calls the ServiceNow API based on that condition, and finally completes the message to remove it from the queue.

A second common pattern is fan-out with topics: a Logic App publishes a single message to a topic, and three separate subscriptions each trigger their own independent Logic App - one for inventory, one for notifications, one for analytics - all reacting to that same published message without any of them needing to know about the others.

Monitoring Service Bus

In the Azure Portal, under a Service Bus namespace and the specific queue or topic, the key metrics worth watching are Active Messages (waiting to be processed), Dead Letter Count (failed messages needing investigation), Incoming Messages (the rate messages are arriving), and Outgoing Messages (the rate messages are being processed). Sensible alert thresholds worth setting are a Dead Letter Count greater than zero, and Active Messages greater than some reasonable backlog threshold like 1000, depending on expected normal volume.

Service Bus vs Storage Queue

Service Bus is the right choice for enterprise messaging that needs ordering guarantees, topics and subscriptions, sessions, transactions, or dead lettering, and for anything connecting to Logic Apps or Function Apps. Storage Queue is the right choice for a simple task queue, very high volume at the lowest possible cost, where basic retry behavior is genuinely enough and none of Service Bus's richer features are actually needed.

Key Lessons From Production

Check the Dead Letter Queue daily in production - it's the single most reliable early warning signal that something is going wrong.
Set MessageId explicitly to prevent duplicate processing of the same logical message.
Use CorrelationId to trace a single message's journey across multiple connected systems.
Topics with subscriptions consistently beat maintaining multiple separate queues for the same kind of fan-out scenario.
Sessions are essential specifically when the order of processing matters - skipping them in an order-sensitive workflow is a common source of subtle bugs.
Store connection strings in Key Vault, never directly in code.
Use the Standard tier for development and the Premium tier for production workloads needing VNET isolation.

Summary

Azure Service Bus is the most reliable way to connect distributed systems on Azure. Queues handle point-to-point delivery, Topics handle publish-subscribe delivery to multiple independent consumers, Dead Letter Queues ensure a message is never silently lost, and Sessions provide ordered processing when sequence genuinely matters. Combined with Logic Apps and Function Apps, these pieces form the complete Azure integration stack that real enterprise systems are built on.

Originally published at TechStack Blog:
https://www.techstackblog.com/post.html?slug=azure-service-bus-deep-dive