DEV Community: Simon Napper

Catching the Bus? How a Service Bus and Azure Functions Can Help Your Integration Reliability

Simon Napper — Thu, 07 Nov 2024 15:18:38 +0000

What’s the first thing that you think of when you hear the word “bus”?

It’s not often how reliable they are, but this article would like to change your perception on the word bus and how it can be a byword for greater reliability, scalability, and stopping integrations from being tightly coupled with each other.

So, what is a Service Bus?

Well it gets its name from the communication system inside a computer that transfer data to and from components (thanks Wikipedia) however it’s really more analogous to a secure mailroom, as it’s a safe store for “message” that pass to and from different computer systems.

Digital Void of Oblivion

Have you ever had a situation where you’ve had to integrate with an API provider that boosts 99% uptime, only to find that you seem to be in that unwanted 1% every time you use it? When that happens your call to the API and the data therein could be lost forever, cast into the digital void of oblivion, which always happens when your client is using the system.

Chasm of Doom

This is where the Service Bus comes in. It’s aim is to help you bridge that gap over the chasm of doom, by sending your data to the Service Bus as a “message” (yes this terminology does get confusing when you’re integrating with email systems) which is then held by the Service Bus, until it can successfully deliver it to the API that you’re integrating with.

Silicon Heaven

If that API is down, then the Service Bus will keep on trying for a certain number of attempts at a certain time interval, until it is successful. If it can never be delivered, then the data is moved out of the way into what’s called a “Dead Letter”, waiting to be looked at and either resent or deleted, but most importantly, it still exists with the data intact and hasn’t just vanished to silicon heaven.

Utopia

So now we are approaching a Utopia, a system that never loses its data, keeps trying automatically to resend that data and will store it for you if it can’t deliver it, perfection. Well maybe not perfection, but we are adding resilience into our architecture and ensuring that our communications between systems like Umbraco and third party API’s now have a higher chance of success or at the very least, we’re not losing our data to the vast, digital great beyond.

Service Bus Code Example

Here’s an example of how you can set up a Service Bus helper and hook that into firing when content is published in Umbraco. This use the Azure.Messaging.ServiceBus nuget package.

Create a Helper for Interacting with the Service Bus

public class ServiceBusHelper
{
    private readonly string _connectionString;
    private readonly string _topicName;

    public ServiceBusHelper(string connectionString, string topicName)
    {
        _connectionString = connectionString;
        _topicName = topicName;
    }

    public async Task SendMessageAsync(string message)
    {
        await using var client = new ServiceBusClient(_connectionString);
        ServiceBusSender sender = client.CreateSender(_topicName);

        ServiceBusMessage busMessage = new ServiceBusMessage(message);
        await sender.SendMessageAsync(busMessage);
    }
}

Next, inject this into your ContentPublishedNotification

public class ContentPublishHandler : INotificationHandler<ContentPublishedNotification>
{
    private readonly ServiceBusHelper _serviceBusHelper;
    private readonly ILogger<ContentPublishHandler> _logger;

    public ContentPublishHandler(ServiceBusHelper serviceBusHelper, ILogger<ContentPublishHandler> logger)
    {
        _serviceBusHelper = serviceBusHelper;
        _logger = logger;
    }

    public async void Handle(ContentPublishedNotification notification)
    {
        foreach (var content in notification.PublishedEntities)
        {
            await _serviceBusHelper.SendMessageAsync($"Content published: {content.Name}");
            _logger.LogInformation($"Sent message for content: {content.Name}");
        }
    }
}

Don’t forget to register it!

builder.Services.AddSingleton(new ServiceBusHelper("your-connection-string", "your-topic-name"));
 builder.AddNotificationHandler<ContentPublishedNotification, ContentPublishHandler>();

The Betrayal

So now we have our Umbraco solution sending messages to the Service Bus and the API is receiving data and you’re happy, I’m happy, the client is happy, job well done and time to put your feet up.

Well… not quite as I may have mislead you a smidgen.

As you might have noticed, the title of this has the words “Azure Functions” in it and as of yet, I’ve not mentioned a thing about them, however they are a very important part of this proposed set up, in fact this won’t work without them.

Groovy Baby

So what are and where do Azure Functions feature in all this.

Well according to the inter-web, Azure Functions is a serverless compute service provided by Microsoft Azure that allows you to run small pieces of code, or "functions," in the cloud without managing the underlying infrastructure. It’s designed to let developers focus on their code and application logic without worrying about server provisioning, scaling, or maintenance.

Well that sounds groovy, but what does it mean practically?

“Alright, Dave?”

Well for the purposes of this example, Azure Functions are bits of code that will be triggered (ok it's a bad Only Fools and Horses joke) by the arrival of a Service Bus message and will then act on that message accordingly and the data that the message contains (so could be form data, etc).

For this example of the Service Bus, were going to use a Topic in order to differentiate between the different messages that we’re sending.

When we send a Service Bus message, we specify the Topic that it belongs to, so that it ends up in the right place. We can then set the Function to poll for that Topic and to then process it when it arrives.

That action could be sending an email, sending data to an API, doing both and logging stuff, whatever you want but importantly, it will run that code and return a success or failure on the action you’re performing.

The Service Bus will then either mark message as successful and remove it or if there’s an exception, keep trying until it either gets a successful return or it hits the limit of the amount of tries it should do. So if you’re trying to send something to a busy or dare I say it, unreliable API, the combination of the Service Bus and Azure Functions will keep trying to send your data to the API until it goes through (or doesn’t).

Code Example for Azure Function

The following code takes in a message sent to the correct Topic in the Service Bus and then attempts to send an email with that message. This code has to be in an Azure Functions project in Visual Studio to work:

public class ServiceBusEmailFunction
{
    private readonly ILogger<ServiceBusEmailFunction> _logger;

    public ServiceBusEmailFunction(ILogger<ServiceBusEmailFunction> logger)
    {
        _logger = logger;
    }

    [Function("ServiceBusEmailFunction")]
    public async Task RunAsync(
        [ServiceBusTrigger("your-queue-or-topic-name", Connection = "ServiceBusConnection")] string message)
    {
        _logger.LogInformation($"Received Service Bus message: {message}");

        // Define email details
        string recipient = "recipient@example.com"; 
        string subject = "Service Bus Notification";
        string body = $"Message received from Service Bus: {message}";

        try
        {
            using (var smtpClient = new SmtpClient("smtp.yourserver.com", 587))
            {
                smtpClient.Credentials = new NetworkCredential("your-email@example.com", "your-password");
                smtpClient.EnableSsl = true; // Enable SSL if required

                var mailMessage = new MailMessage("your-email@example.com", recipient, subject, body)
                {
                    IsBodyHtml = false
                };

                await smtpClient.SendMailAsync(mailMessage);

                _logger.LogInformation("Email sent successfully.");
            }
        }
        catch (Exception ex)
        {
            _logger.LogError($"Failed to send email: {ex.Message}");
        }
    }
}

“Wibbly wobbly, timey wimey”

What else can Azure Functions do?

Well you can also have timers which will run at a set time or interval and execute your code. Perfect for daily reports and the such like or replacing Scheduled Tasks that you might still have knocking about on a server (I’m not judging, but it’s not 1999 anymore). You could have the timer Function create a Service Bus message which is then sent to a trigger Function, perhaps one that deals with sending emails or it could simply be standalone and do its thing.

Subscription or Queue?

There is the concept in Azure Service Bus of a Queue or a Subscription/Topic setup. We use the latter for more flexibility. However, for completeness, the difference between them is set out below. For the purposes of this article, we’ll be sticking with the Subscription/Topic setup.

1. Queue (Point-to-Point Messaging)

A queue is designed for one-to-one messaging, also known as point-to-point messaging.
Messages sent to a queue are delivered to a single receiver, which then processes each message.
Multiple receivers can connect to a queue, but only one will process each message, providing a load-balancing effect.
Queues are best suited for cases where there is a single consumer for each message, such as when an order-processing service handles incoming orders one at a time.

2. Topic/Subscription (Publish-Subscribe Messaging)

A topic is used in publish-subscribe scenarios, where a single message can be delivered to multiple subscribers.
Each topic can have multiple subscriptions, which are like "virtual queues" that receive copies of the messages sent to the topic.
When a message is sent to a topic, it can be received by multiple subscribers based on subscription rules and filters, allowing each subscription to define specific criteria for the messages it wants to receive.
Topics and subscriptions are ideal for one-to-many scenarios, such as when notifications need to be broadcasted to multiple services or applications (e.g., sending alerts to multiple monitoring services).

In Summary:

Use queues when you have a single consumer or want messages distributed evenly to multiple competing consumers.
Use topics/subscriptions when you want a message to be sent to multiple recipients, each of which may have unique filtering criteria for the messages they receive.

The Ol’ Switcheroo

A good advantage of using this architectural approach is that it simplifies your interactions in a DXP composable solution.

As long as your Service Bus message is reused, you can create a new Function to switch from say Mandrill to SendGrid and just disable one, enable the other and Bob’s your uncle.

“Your scientists were so preoccupied with whether they could, they didn’t stop to think if they should”

So there are a couple of considerations to keep in mind when using this approach.

Besides the financial cost, which will vary depending on the size of your integration, it's another thing to manage as part of your solution. This then increases the skill set you need in a developer to maintain and develop your solution.

Last thing of note is whilst nearly all the Service Bus functionality can be ran via the API, you can’t run all of the functionality for the Service Bus via the Azure Portal. For example, you can’t delete a dead letter message through there, you have to do that via the API, so you might need to code that for maintenance.

And Finally…

In conclusion, adding a combination of a Service Bus and Functions can greatly increase the resilience of your architecture when dealing with third party integrations. In the right solution and with monitoring set up, it’ll ensure that you have much more reliable and successful communication and is perfect for composable DXP architecture.

All aboard the Service Bus, ding ding!

Bake Your Own Content Cake: Using Umbraco Services to Whip Up Document Types, Data Types and Content

Simon Napper — Mon, 20 May 2024 17:21:12 +0000

In this article, I’ll show you how to create a document type with properties and then use it to create a content page in the CMS. Think of it like you’re making a cake…

Ingredients for Our Content Cake

Like a cake, creating content requires ingredients and a recipe. Here’s what you’ll need:

Several data types created from property editors
A property type group to organize properties in a tab
A document type to add these tasty data types to
A template to serve up this lovely dish
A startup routine to ensure our content is created in the CMS

Umbraco Services Overview

But first, a bit of background before we dive into the code.

In the beginning, there was Umbraco 3—a beacon of hope in the messy world of CMSs. Then along came Umbraco 4, bringing with it the Umbraco Library, and a way of creating content programmatically. After the chaos of Umbraco 5 (we don’t talk about 5 😐), Umbraco 6 and 7 arrived with improved programmatic content creation organised into services and thus order was rising from the chaos. Now, like a proud beast climbing the food chain, Umbraco has a full set of powerful services that can pretty much let you do whatever you want programmatically. That’s the wizardry we’ll use to magic up our content cake.

This article assumes you have some experience with Umbraco services and dependency injection. If not, you might want to have a look at this documentation from Umbraco which gives a good introduction on what dependency injection is and how you can use it with Umbraco.

Services Galore

There are many services in Umbraco. To save you from digging through the complete list (which you can find here), we’ll use the following to whip up our content cake:

ContentTypeService
DataTypeService
FileService
ContentService
PropertyEditorCollection (not a service, but we need it)

You say Potato, I say… err.. Potato

Now here’s an oddity for you if you’re not familiar with the history of Umbraco. For some reason with the advent of Umbraco 8, the decision was made to rename Document Types to Content Types as part of an initiative to simplify the terminology being used. However, nobody liked that, so from Umbraco 9 onwards, content types were switched back to being called Document Types and everyone rejoiced and danced in the streets.

But…

This wasn’t the case for the Content Type service which is still called the ContentTypeService and uses ContentTypes, rather than calling it the DocumentTypeService using DocumentTypes. So during this article you may see the interchanging use of Content Types and Document Types and they are referring to the same thing as they are, actually, the same thing.

Get Your Pinny On: Injecting Services

To get these services working, you need to set up dependency injection. We've not got the full injection code here, but these are the services that we're going to use and the variable names that you'll see in the examples below.

private readonly IContentTypeService _contentTypeService;
private readonly IDataTypeService _dataTypeService;
private readonly IShortStringHelper _shortStringHelper;
private readonly PropertyEditorCollection _propertyEditors;
private readonly IFileService _fileService;
private readonly IContentService _contentService;

Preparation: Sorting Out the Template

A Template is assigned to a document type so that when a content using that document type is viewed on a website, the template will contain the code needed to render that page.

So when we create our template for our document type, it can't be empty as that would just render a blank page, so we need to set the content of the template file with some Razor code so we can see stuff. There's many ways of doing this, but I'm using Razor code embedded in a resource file (TestPageHTML.TestPageHtmlContent), but you could reference a static file, have all your code as a very long string (no, maybe not do that) or whatever works best for you.

ITemplate testPageTemplate = _fileService.GetTemplate(alias);
if (testPageTemplate == null)
{
    testPageTemplate = new Template(_shortStringHelper, name, alias)
    {
        Content = TestPageHTML.TestPageHtmlContent
    };
    _fileService.SaveTemplate(testPageTemplate);
}

On Your Marks: Creating the ContentType and PropertyGroup

There is always the possibility that we've been here before, so perhaps we already have a content type (document type) in the CMS or one has been created coincidentally with the same name. Safety first then, let's check if it already exists and if not, we can safely use our alias to create our new content type.

var name = "Test Content Page";
var alias = name.ToSafeAlias(_shortStringHelper);

if (_contentTypeService.Get(alias) == null)
{
    var testContentPage = new ContentType(_shortStringHelper, -1)
    {
        Alias = alias,
        Name = "Test Content Page",
        Icon = "icon-map",
        AllowedTemplates = new [] { testPageTemplate },
        DefaultTemplateId = testPageTemplate.Id
    };
}

As we can, we might as well add our template that we created earlier and set it to be the default so that's all ready to go when we create our content later on.

We could save our content type now using the ContentTypeService, but let's add our properties first.

Get Set: PropertyTypes and DataTypes

As we know, in Umbraco we have a whole smorgasbord of built in Umbraco property types, such as the Rich Text Editor, dropdowns, textboxes, etc and other wonderful Community created options such as the excellent Contentment package from Lee Kelleher.

We’ll need to use property editors to create data types that we can then add to our property group. Property editors can only be added to a property group by using a PropertyTypeCollection which we’ll need to create before we can start.

var propertyTypes = new PropertyTypeCollection(false);

Next, we’ll create our data types that we want on our new content type. For this example, we’re going to create a textstring to for a title to be put in it and an RTE for the main content on the body of the page.

There’s a few stages to accomplish this. First of all, you need to get the DataType that you want to use by getting it from the PropertyEditorCollection. Whilst not strictly speaking a service, you can inject the PropertyEditorCollection with your other services and as you might of guessed from the name, this brings back a collection of the property editors that are available to you in Umbraco, that all use the IDataEditor interface. By getting the editor from this collection, you can then use it create a new DataType.


if (_propertyEditors.TryGet("Umbraco.TextBox", out IDataEditor? textboxPropertyEditor))
{
    var textBoxConfig = new TextboxConfiguration();
    var textBoxDataType = new DataType(textboxPropertyEditor, _configurationEditorJsonSerializer)
    { 
        Name = "Test - Title",
        Configuration = textBoxConfig
    };
    _dataTypeService.Save(textBoxDataType);

    var propertyTypeTitle = new PropertyType(_shortStringHelper, textBoxDataType, "title")
    {
        Name = "Title"
    };
    propertyTypes.Add(propertyTypeTitle);
}

In the above code we’re looking for an Umbraco.Textbox property editor that, it we have, we can use to create a DataType. To create the DataType, you need to have the correct configuration. This contains things like MaxChars, etc. that you can set now if you want to.

Once we have the populated DataType object, we can then save it using the DataTypeService.
Now we have our DataType in Umbaco, we can create our PropertType and add this to the PropertyTypeCollection we created earlier.

Congratulations, you now have your first PropertyType in the collection! You can now repeat as many times as you wish for each property and here’s an example of doing the same thing but with an RTE.

if (_propertyEditors.TryGet("Umbraco.TinyMCE", out IDataEditor? richTextPropertyEditor))
{
    var richTextConfig = new RichTextConfiguration();
    var richTextDataType = new DataType(richTextPropertyEditor, _configurationEditorJsonSerializer)
    { 
        Name = "Test - Text",
        Configuration = richTextConfig
    };
    _dataTypeService.Save(richTextDataType);

    var propertyTypeText = new PropertyType(_shortStringHelper, richTextDataType, "text")
    {
        Name = "Text"
    };
    propertyTypes.Add(propertyTypeText);
}

Unless you like your content types chaotic and enjoy giving editors a headache, it’s not a bad idea to organise your properties by tabs or sections. As we’re responsible developers and like to be nice, we’re going to create a property group for our data types that will eventually be added as a tab on the document type.

var propertyGroup = new PropertyGroup(false)
{
    Name = "Content",
    SortOrder = 1,
    Alias = "content"
};
propertyGroup.PropertyTypes = propertyTypes;

Now we have all our properties in our collection, we can add them to the ContentType and save it using the ContentTypeService.


testContentPage.PropertyGroups.Add(propertyGroup);

_contentTypeService.Save(testContentPage);

Bake! Adding the Content

We now have all we need to create the content in the CMS, a shiny new content type with properties and new data types, all ready to go.

To create the content page, we’ll need to:

Find where we want to add the content.
Set permissions so this new ContentType can be created.
Set up the content.
Save and publish the content.

So the first thing to do is get the place in the content structure that you want this new bit of content to be created. For simplicities sake, we’re going to assume that we have a site with a hompage at the root and we’re going to create it under there.

First of all, we need to get that content, find it’s ContentType and add permissions for our new one to be created under it.

var homePage = _contentService.GetRootContent().FirstOrDefault();
var homepageContentType = _contentTypeService.Get(homePage.ContentType.Key);
var testContentType = _contentTypeService.Get(testContentPage.Alias);

if (homepageContentType != null && testContentType != null)
{
    var allowedContentTypes = homepageContentType.AllowedContentTypes?.ToList();
    allowedContentTypes.Add(new ContentTypeSort
    {
        Alias = testContentType.Alias,
        SortOrder = allowedContentTypes.Count() + 1,
        Id = new Lazy<int>(() => testContentType.Id)
    });
    homepageContentType.AllowedContentTypes = allowedContentTypes;
    _contentTypeService.Save(homepageContentType);
}

Now that we’ve added our new ContentType as an allowed thing to be created under our Homepage ContentType, we can add the new content, set some values and then save and publish it.

var testPage = new Content("Test", homePage, testContentType);
testPage.SetValue("title", "Test Title");
testPage.SetValue("text", "Test Content");

_contentService.SaveAndPublish(testPage);

Cherry on Top: Create on Startup

If you want to enforce this setup when Umbraco starts, you can add a NotificationHandler and reference it in Program.cs (or Startup.cs).

public class UmbracoApplicationStartingNotificationHandler : INotificationHandler<UmbracoApplicationStartingNotification>
{
    private readonly ISetupService _setupService;

    public UmbracoApplicationStartingNotificationHandler(ISetupService setupService)
    {
        _setupService = setupService;
    }

    public void Handle(UmbracoApplicationStartingNotification notification)
    {
        _setupService.StartUp();
    }
}

For extra fun, you could have it add random stuff on April 1st before deleting it, but you didn’t get that idea from me…

I hope this has been helpful and at the very least gives you a start in writing what you need to create content programmatically in Umbraco. If you’ve got it working, go reward yourself with a nice coffee… and some cake!