Azure Service Bus Overview

What is Azure Service Bus?

Azure Service Bus is a messaging service on cloud used to connect any applications, devices, and also services running in the cloud to any other applications or services. As a result, it acts as a messaging backbone for applications available in the cloud or across any devices.

Service Bus Concepts

At the time of this webinar, there were three basic concepts in Azure Service Bus:

• Relay -> To establish a secure connection between applications which is bridged through the cloud
• Messaging -> Used to push/pull messages from the queue
• Event Hub -> High performing stream-based event system

Relays and also Event Hubs were moved out as separate services and Service Bus will only refer to Messaging that consists of Queues / Topics

Multi-Tenant Service Bus environment

In this environment, we can create a namespace which would be the container for our Service Bus artifacts in the cloud. The namespace can be available in different regions.

Each namespace comprises messaging entities like Queues, Relays or Topics. Most importantly, a namespace will be dedicated only to one type of Service Bus.

Message processing statistics (as of 2015)

The statistics generated by the Microsoft team working on Service Bus are as follows,

• Event Hub = 30 Billion Events per day
• Messaging = 7 Billion Messages per day

It can handle a colossal number of messages in a very executable manner.

Service Bus Relay

In Service Bus Relay, we will have an endpoint that will be hosted in the cloud, but the difference would be the fact that our on-premise service will make an outbound connection to register with the Service Bus relay endpoint.

At a conceptual level, we send a message to an endpoint in the cloud from where it will be forwarded to an on-premise service. As a result, it gives us the advantage to connect from one organization to another organization without having to set VPNs.

Some of the key features are,

 

Service Bus Relay can be used to solve problems in scenarios like,

1. Information passed between two data centers.
2. Building an integration architecture through SAP to forward messages into microservice architecture using Service Bus Relay.

Example Scenarios:

• Many of the Microsoft API apps connect to on-premise systems use relay under the hood
• Danish pension and also insurance supplier exposes IBM WebSphere integration platform to partners through relay
• Northumbria University use the relay to bridge cloud-hosted application to integrate with on-premise WCF service

Demo-Relay

This demo explains the process of hybrid connectivity with Service Bus Relay. We would initially work on a WCF service (on-premise service). The demo constructed on a .NET framework with console application is to call the back-end service.

In Service Bus Explorer, we will have a registered endpoint under the “relay” section. Now, a virtual directory (router) will be linked to the configuration file “web” under user defined location to establish a connection. The folder would also comprise of two more files, which are,

• Bin directory
• WcfRoutingService(SVC file)

In the SVC file, there would be service pointing to WCF routing service endpoint. In the configuration file, there would a WCF service which uses the HTTP relay binding along with other credentials to access the Service Bus Relay.

Finally, Service Bus Relay accepts a message and forwards it to the back-end service. When executed on the console application, every message will be processed within a time frame of 100 milliseconds.

Messaging

In Azure Service Bus, We can transfer the data between services or applications in message format like XML, JSON, or text.

In this messaging protocol, Microsoft would handle most of the scaling. We just have to create a namespace using platforms like Azure Portal, Service Bus Explorer or Serverless360.

Let us consider some example scenarios,

• Construction companies use sensors to monitor buildings in real time to ensure adherence to safety more standards.
• Using Service Bus, “Qoros” (Chinese car manufacturer) connects a car to the internet to stream messages.
• Metro Bank Use Service Bus Messaging to manage near real-time messaging between SharePoint and also Dynamics CRM.

Queue

A messaging protocol based on the first-in-first-out pattern. Here a sender would send a message to a queue from where the receiver would collect it in initial or later stage. Besides that, queues enable us to store the message until the receiver is available to receive and also process them. The user will receive the message upon request.

Topics

Topics are similar to queues. Most importantly, topics make multiple subscriptions available. A sender will send a message to a topic then a routing rule can determine which subscription may receive the slotted copy of the message.

The receiver can receive the message from the subscription. Multiple receivers could be allocated to a subscription.

Other messaging patterns include,

• RPC
• Scatter Gather
• Pub/Sub

Demo-Messaging

In this demo, we will send a message to a queue and also collect it back. Initially, a queue (testqueue) will be specified and also the following properties will be determined in Service Bus Explorer as mentioned below,

• Path
• Queue properties
• Lock duration

To send a message to a queue and also to collect it back, we define the Service Bus connection string along with reference to Service Bus NuGet package.

We will now create a Service Bus message and sent it. On receiving side, we have a competing consumer pattern which enables multiple concurrent consumers to process messages received on the messaging channel. Dynamic message delivery method is employed in a single-thread pattern i.e., first-in-first-out. But in multi-threading pattern, messages will be processed at a higher throughput for multiple receivers.

On receiving side, using “OnMessagingOption” we can specify the number of concurrent receivers required,

We will then access the body of the message through the event (OnMessage) and also write it to the console window.

The message can either be removed from the queue or abandoned. Service Bus handles serialization with the help of SDK’s.

Let us consider, when we send a message it would be processed in a loop format. So, we will take the .NET type specified as “TestMessage” and also moves it to “BrokeredMessage” to handle the serialization of the .NET type when passed to the queue.

The content of the message will be extracted. Then the format of the message which delivered, received should be verified.

Now if we run our demo on two separate console applications, one to send and also another to receive. The respective messages would be sent and also received from the queue.

Event Hubs:

Event hubs will manage massive scale event stream processing and are similar to Service Bus messaging. It is similar in the case that it supports sending the message over AMQP (Advanced Message Queuing Protocol).

The key difference between Event Hubs and also Service Bus messaging is that there is no concept of deleting a peek lock message like in Service Bus. Instead, Microsoft has used a reader across a stream which provides a high level of scaling.

Let us consider an IOT scenario where Event Hub comes in use,

The above picture comprises many subsystems, each with an individual offering. Event Hub can stream a high volume of data from different applications and also store it.

Most importantly, users can process the streamed data and apply it for their individual purpose.

Let us consider some example scenarios:

• Microsoft uses Event Hub under the hood as a part of application insights and also API management.
• A company called “Autolib” uses Event Hub as a part of Microsoft IOT stack to exchange information between devices and also vehicles as a part of a car-sharing scheme in Paris.

In Event Hub, multiple senders store their telemetry data in the message stream. A receiver cannot remove the message from the stream. Rather, they only get deleted when their retention period gets over. A consumer group can access message from any stream.

Application Telemetry

The above picture represents a typical message flow of a system. The idea is to feed telemetry data to Power BI through Event Hub. Message from the user would pass through various systems and also would reach Event Hub.

Here, Stream Analytics and Machine learning process collect the data from Event Hub and send it to Power BI for any business analytics service.  

Demo-Event Hubs

This demo illustrates the purpose of Event Hub and is similar to Azure Service Bus messaging.

On Sender Side,

In the console application, we have a connection string followed by an Event Hub name (key name). In the code portion, we have

• EventHubClient -> To create an instance of a client through the connection string
• Client.Send -> To send an object

On Receiver Side,

In the console application, we have an “EventHubName” and also storage account followed by a ServiceBus.EventProcessorHost which is used to create a listener for different partitions in the Event Hub instance. In the Azure storage account, this package creates a blob for each partition to manage the index of its location in the stream.

When a message is received, it will be fired to the event listed in the below piece of code,

Our position in the stream can also be acknowledged so if the whole process blew up, we can simply move to the last index we read and also continue our progress. Now, we can see how the operation gets executed. As a result, Event Hub can receive and store any kind of data and events. 

Fact

Service Bus team provided a cost estimation analysis for processing a billion events per day which lead to,

• 12 Throughput units
• 12,000 events per second
• 720,000 events per minute
• 43,200,000 events per hour
• Total Cost -> $37.67

Service Bus Costs

Relay:

• 10,000 listening hours=6.60 pounds
• 100 million messages=6.20 pounds

Messaging:

• 5 million operations per month=6.20 pounds
• 13-100 million operations = 50pounds per million
• 100-2500 million = 31pounds per million

Event Hubs:

• 100 million events = 1.72 pounds
• The rate of Processing = Throughput units = 14 pounds per month per unit

Service Bus + BizTalk

Although Service Bus and BizTalk provide connectivity in many aspects for a seamless experience, they could be more aligned in the following aspects;

SB- Adapter (latest version not available)

• Service Bus Relay
• Service Bus Messaging
• Event Hubs-it does not provide out-of-box support

Addition of features list includes,

• AMQP Adapter- Supports non-Microsoft Service Bus type products
• Event Hub Adapter

User scenarios of BizTalk+Relay

Scenario 1:

Let us consider a BizTalk server hosted in Azure as a Virtual Machine needs to connect to an on-premise web service. The simplest method to do that would be through SB adapter in BizTalk.

Here, BizTalk Server will send a message to Relay and the rest of the process will be carried out in WCF service.

Scenario 2:

If we have BizTalk server on-premise followed by a SaaS application in the cloud, a communication can be established between them through Service Bus Relay.

This can be achieved through Service Bus Relay or by assigning a REST API between SaaS component and the server. So, the Relay offers an easy way to expose a BizTalk endpoint from an organization.

Scenario 3:

Let us consider the scenario where two BizTalk Servers in separate companies needs to communicate. The best way to do that will be to send the message through Relay by exposing their endpoints. This flow does not require any separate infrastructure support.

User scenarios of BizTalk+Messaging

Scenario 1:

This scenario would focus more on transferring a massive number of messages from a web application in the cloud to on-premise Server. Here, BizTalk Server pulls the messages from the queue at a manageable level.

Scenario 2:

To publish an event in pub/sub format to various receivers, we will push the message from on-premise server to a queue where the application can access it. You cannot directly link the applications to the BizTalk Server located on-premise.

The key difference between Service Bus and BizTalk is that,

• BizTalk does not support poll pattern
• Service Bus does not support push pattern

Therefore, combining both the services would lead to dynamic productivity. The receiver can access the message available on Service Bus Topic at any time.

Scenario 3:

Let us consider a scenario where Service Bus Topic will share the resource between two BizTalk application. But if we want to delete the shared resource in the application and patch a new resource, the dependency management in BizTalk would make it harder.

When the BizTalk application has to be updated, we cannot undeploy it because the subscriptions would be eliminated. Therefore, this causes overcomplexity when trying to solve it. 

Demo

This demo explains on combining both BizTalk and Service Bus, through which messages can be sent and also received between BizTalk applications. Most importantly, you can execute the process in JSON format.

Let us assume there are three BizTalk applications (app1, app2, app3), each comprising of “Hello World” scheme followed by a unique namespace. We will have a sender in each application to receive a message from the file system, which will be routed to Service Bus through Send Ports. Furthermore, you can convert the messages to JSON format before sending it using JSON encoder.

Most importantly, when we receive the message in Service Bus, it would hit the topic comprising of two subscriptions. Each subscription will forward the content to the desired queue. The Topic can specify who can send and also receive the message in it, but if there are multiple receivers, there is a probability where one receiver may fetch the message of another receiver. Hence, this problem can be solved by using “Forward to” option in Topic and can also specify which receiver should receive the appropriate message.

Most importantly, the agenda is to send a message from any application to topics before it gets forward to the selected queues.

After the initialization process gets over, the virtual machine will receive a message. Using the JSON decode, we will change the format of the message in the receive pipeline of the virtual machine. Finally, the respective Biztalk applications receive the message.

User scenarios of BizTalk+Event Hub

Scenario 1:

Here Event Hubs forwards the messages to BizTalk server after processing from event processor host. If the scenario does not involve a huge number of messages to forward to BizTalk server, then the event processor host will act as an isolated BizTalk host without any adapter for a smoother transition.

Scenario 2:

In this scenario, you share the telemetry data and logging information with Service Bus Event Hub when a message flows through the pipeline of BizTalk server. Most importantly, powerful analysis machines like Power BI and also Cortana Analytics make use of the telemetry data from Service Bus.

Wrap up

This blog gives an overview of various Service Bus components available with demo scenarios for a better understanding of the audience. There are a lot of features available in Azure Service Bus and also more to come soon.