DEV Community: Jeff Hollan

Dapr is like jQuery for microservices

Jeff Hollan — Mon, 21 Mar 2022 15:04:23 +0000

There's an emerging piece of open source tech I'm very excited about. If you haven't heard of Dapr, it's one of the newest additions to the Cloud Native Computing Foundation (CNCF) and promises to provide portable and reliable microservices. While on the surface that may sound great, a lot of you may be wondering "ok but what is it? how does it work?" While analogies and comparisons will never be perfect, I love to use them to help me contextualize. So while the comparison is imperfect, I personally think of Dapr like jQuery for microservices.

Ok hear me out. I realize that sentiment may evoke a range of emotions, but let me explain. This is heavily influenced by my experiences in web development. So let’s head back in time.

I remember over two decades ago building some of my first web pages. These were the days of GeoCities, and later Blogger and MySpace. I remember using text editors to write static HTML files, and spicing it up with the latest and greatest tech including MIDI files, guest books, and page visit counters.

But as we moved into the 2000s, the web shifted. You may hear folks now reference this as Web 2.0, but to me it was discovering a new paradigm of web experiences, led by Gmail, YouTube, and Facebook.

These pages revolutionized what a website could do. A web page could be dynamic, with controls or content that was able to update without clicking a hyperlink or navigating pages. As I was building web pages, I didn’t even know where to start to try to weave in this type of rich functionality. But in the late aughts that all changed when I got introduced to jQuery. To me, jQuery was magic. Someone like me, with only basic web development skills, could now create dynamic sites. Commands like $("button").click, $.ajax(), and $("property").hide unlocked these capabilities by pulling in the jQuery script files onto any page and using that beautiful $ alias and APIs. At the time, I would recommend jQuery to anyone and everyone building websites because it made the complex simple.

Times have changed some, and now in 2022, I don’t use jQuery when building sites. Those patterns have been baked into modern browsers and JavaScript and frameworks like Angular, React, and Vue. But jQuery, at least for me, was what democratized this new wave of dynamic websites.

While dynamic web pages are no longer a mystery, modern microservices and cloud native is still heavily evolving. Microservices can trace some roots back to SOA, which is anything but "new," but there are still a set of common challenges when building microservices, including:

How can I make my services resilient to failures?
How can I have better logging and tracing of my services?
How do I secure and encrypt my sevices?
How can I manage state efficiently and uniformly across my services?

There are a variety of ways to solve each of the above. I could write custom retry logic in my code to catch and retry. I could pull in SDKs for observability and logging solutions, or could integrate service meshes to manage inter-service communication. But in the same way dynamic webpages felt advanced and overwhelming for me without jQuery, solving each of these problems on its own can be a daunting task for any team building microservices. This is why for me, Dapr is doing for microservices what jQuery did for dynamic web pages: providing a simplified and developer friendly way to solve a variety of common challenges.

As mentioned at the beginning of the blog, Dapr is a set of APIs for building portable and reliable microservices. I can pull Dapr into my microservices architecture and have out of the box APIs to perform tasks like secure service invocation, publish & subscribe, and secrets management. Let’s take a simple example.

Imagine I have two services: service A and service B. I have deployed both services, and now want to call service B from service A within an orchestrator like Kubernetes. Without much effort I could call service B over HTTP (http://serviceB/hello). This works, but it’s missing a few useful capabilities. How can I make sure the network traffic between A and B is secured and encrypted? What if service B has a transient issue I want to automatically handle and retry? How can I make sure whenever service A calls service B I can trace that call and “follow the message” from origin to eventual destination? With these considerations, my simple call over HTTP now falls short.

The good news? I can simply swap out the call over HTTP with the Dapr API to “invoke” and get service A to call service B, but Dapr now layers on resiliency, encryption, and tracing.

await daprClient.invoker.invoke('serviceB' , 'hello', HttpMethod.GET);

Service invocation overview

That’s just one of many examples in how a simple API call to Dapr will layer on best practices and solutions to common microservices challenges.

Dapr Building Blocks

I’ve used Dapr a bit in the last year and have really come to value it in my toolchain. I personally find the service invocation APIs and observability APIs the most useful. I may use Dapr for state management, but other times I choose to leverage the state SDKs (like Redis or Cosmos DB) directly in my code. But that’s one part I like about how Dapr was built: you can mix and match just the APIs that make sense for your current project and team.

If you haven’t tried Dapr, it’s simple to get started using the Dapr quickstarts. There are options to try out locally or hosted in Kubernetes.

As our team has been building Azure Container Apps, a serverless container offering built on Kubernetes, we wanted to make it as easy as possible for users to leverage these Dapr APIs for containers and microservices. So we’ve baked Dapr into the service as a feature that can be easily enabled for any container. If you want to try creating a hosted Dapr app, check out this tutorial of creating a microservice with Dapr in container apps.

The same way I would recommend any web developer in the 2000s use jQuery, I strongly recommend Dapr for anyone building microservices in 2022 and beyond. Is it possible in 20 years these patterns and practices from Dapr will be built into these underlying microservice platforms the way jQuery APIs were for web? Who knows. But until I’ll be using Dapr in my microservice development.

Jeff Hollan is Director of Product for Azure PaaS and Serverless. @jeffhollan on Twitter.

Serverless circuit breakers with Durable Entities

Jeff Hollan — Mon, 21 Oct 2019 15:01:11 +0000

I remember a call with a customer from my first few months on the Azure Functions team. The customer’s function triggered from a queue message and ran a stored procedure in a SQL database. The solution had been working fantastic for months at a high scale until one fateful day the SQL database crashed. What resulted is a serverless version of kicking someone while they're down. The function executions started to fail, and as they failed, the message would be re-queued and retried (which is great! Retrying generally is a best practice, especially for transient failures). But more queue messages kept landing in the queue, in addition to a growing number of retries. Within minutes they had gone from hundreds of failed executions to thousands. What's worse, once the SQL database failure was resolved, Azure Functions was ready to hammer that poor SQL database back into a failed state with a mountain of queue messages and retries it had been holding up. Eventually, things got back into a steady-state, but the team was left with thousands of messages in a dead-letter queue and some battle scars. It's a powerful reminder that sometimes serverless scale can make a bad situation worse.

What the customer wanted was simple enough to understand.

"Can we make it so our function stops triggering if lots of failures start happening?"

It’s a simple problem that has been asked by many a developer. I want to show how you can use the new Durable Entities feature of Azure Functions to create a stateful circuit breaker to achieve exactly that.

Traditional circuit breakers and serverless scale

The pattern isn't a new one. Generally, it's known as the circuit breaker pattern. You can think of it as an electric circuit with a gate. Imagine electricity is flowing through a circuit to a destination. A gate is there bridging the connection. If you ever need to stop the flow of electricity, the gate "opens" creating a gap in the circuit, and stopping electricity. When electricity should resume, you "close" the gate connecting the circuit.

Many libraries exist that can implement this pattern in traditional apps. Polly for .NET is my favorite, but there are plenty to choose from. You can define a threshold number of exceptions within a range - if I get more than 30 exceptions in 1 minute - and it "opens" the gate to stop processing. While this works for many applications, serverless presents new challenges where this model doesn't fit.

Single server vs distributed apps

Assume I'm processing all of my queue messages on a single server. Here, libraries like Polly have worked great. Polly keeps an in-memory count of exceptions as they occur, and it can easily keep track of the threshold because every exception will only occur on that single instance.

Distributed applications like serverless functions are much different. Within seconds under load, an Azure Function can scale to dozens of instances. Imagine now that you have 50 active function instances triggering on events. You could have a circuit breaker library running in your app - but you'd have 50 instances of that circuit breaker library. Now imagine that you get 50 exceptions all at once, but each exception spreads evenly across your instances. Each library, like Polly, will think everything is healthy. "Hey I'm good, I only saw one exception, and my threshold is 30, so let's keep the circuit closed." But the global story is much different. Globally the threshold was crossed, but because each instance was processing in isolation, there was nothing that knew the state and health of the entire circuit. Not to mention, even if one instance "halted" processing, the other 49 may keep chugging along.

The scaling and abstraction of all of these instances make traditional circuit breaker libraries unpredictable. Sometimes 30 exceptions could open the circuit breaker if they all happen on a single instance, while other times it could take many many more exceptions to get a successfully opened gate.

State management and distributed circuits

The missing piece here for a distributed circuit breaker is some external state that can monitor the health and status of the entire circuit. There's no single answer on how that state could manifest. The state could be Azure Alerts and metrics that aggregate exceptions and take some action. The state could be an Azure Logic App like an example I created a few years back before we had Durable Entities. My current weapon of choice though is a Durable Entity.

Let's step away from this circuit breaker problem for a beat and introduce what Durable Entities are.

Durable Entities and stateful functions

Serverless functions are no longer the stateless ephemeral snippets of code you've likely heard of. With new capabilities like Azure Durable Functions you can now write Azure Functions that maintain state for an indefinite amount of time. This is extremely useful when you need to orchestrate or coordinate work as it moves through a system.

Recently we've released a new flavor of Durable Functions called Durable Entities. You can think of Durable Entities like a function that can have infinite instances, each with its unique state and ID. The basic example is something like a counter. Let's say you've now become the lead architect for Fitbit and need to build a solution that receives step counts for each of the thousands of users and keep them stored in state. Each Fitbit device will only ever send one signal: "Add Step." You could write a stateless function that interacts with a database, and does something like "lookup this user, get the current step count, add one, and save it back to the database." But it's a bit cumbersome to write, and things start to get messy when I think about how I can prevent two signals from executing at the same time on the same user, and what was two unique step signals only end up incrementing the database by one.

Durable Entities lets me describe that entity - an instance of a counter - while maintaining state and single-threaded operations on each instance. The code ends up looking like this:

public class Counter
{
    [JsonProperty("steps")]
    public int CurrentValue { get; set; }
    public void AddStep() => this.CurrentValue++;
    public void Reset() => this.CurrentValue = 0;

    [FunctionName(nameof(Counter))]
    public static Task Run([EntityTrigger] IDurableEntityContext ctx)
        => ctx.DispatchAsync<Counter>();
}

That's it - that's an entire Azure Function app. Once I publish this, I can use the Durable SDK or HTTP API to do things like "AddStep for Jeff" or "AddStep for Chris." I could have infinite instances of these counters, each one storing the state for that user. And these Durable Entities are just as serverless as Azure Functions. If I send 1,000 steps for Jeff, my function will scale and run and process those 1,000 events (guaranteeing that after all 1,000 are processed my count is "1000"). I pay only for those 1,000 steps. If I never call that function again, the state still lives (in Azure Storage by default), but I never pay for the function compute until I use it again.

Durable Entities make managing state extremely intuitive. We've seen tremendous interest in scenarios like IoT, where each IoT device could persist and expose state operations as a durable entity. My personal favorite use of entities though is to solve our circuit breaker dilemma.

Durable Entities and Circuit State

Let's pull these threads back together. If you recall from the beginning, we have a problem: we need to be able to manage the state and rate of exceptions external to each function. We also have a pretty slick new tool of Durable Entities. Now I want to show how you can combine these for a serverless circuit breaker.

First, let's break down the flow of how things will work. There are two main components. The function app that will be triggering and processing messages from something like an Azure Event Hub or queue, and a durable entity that will monitor and manage the state of the circuit.

Azure Function

Trigger and try to process the message
If there is an exception, send a signal to the durable entity to let it know it hit an exception

Azure Function Code

[FunctionName("MyFunction")]
public async Task Run([QueueTrigger("myqueue")] Message message, ILogger log)
{
    try
    {
       // try to process the message
    }
    catch (Exception ex)
    {
       // hit an exception
       // send a signal to the Durable Entity
       await _client.PostAsJsonAsync(entityUrl, new FailureRequest
            {
                FailureTime = DateTime.UtcNow
            });               

       // Throw the exception so the retries will kick in
       throw ex;
    }
}

Durable Entity

Keep track of how many exceptions have been reported across all scaled out instances of a function
If several exceptions within a certain period are reported, break the circuit
When breaking the circuit, use the Azure API to stop the Azure Function

Durable Function Code

[JsonObject(MemberSerialization.OptIn)]
public class Circuit
{   
    [JsonProperty]
    [JsonConverter(typeof(StringEnumConverter))]
    public CircuitState state = CircuitState.Closed;

    // Current rolling window of failures reported for this circuit
    [JsonProperty]
    public IDictionary<string, FailureRequest> FailureWindow = new Dictionary<string, FailureRequest>();

    public void CloseCircuit() => state = CircuitState.Closed;
    public void OpenCircuit() => state = CircuitState.Open;

    public async Task AddFailure(FailureRequest failure)
    {
        // Check to make sure the circuit isn't already opened
        if(state == CircuitState.Open)
        {
            _log.LogInformation($"Tried to add additional failure to {Entity.Current.EntityKey} that is already opened.  Close the circuit to resume processing");
            return;
        }

        // Add this failure to the stateful aggragate
        FailureWindow.Add(failure.RequestId, failure);

        // Calculate the time window we should evaluate exceptions for
        var thresholdCutoff = failure.FailureTime.Subtract(windowSize);

        // Filter the window only to exceptions within the cutoff timespan
        FailureWindow = FailureWindow.Where(p => p.Value.FailureTime >= thresholdCutoff).ToDictionary( p => p.Key, p => p.Value);

        if(FailureWindow.Count >= failureThreshold)
        {
            _log.LogCritical($"Break this circuit for entity {Entity.Current.EntityKey}!");

            // Kick off a call to disable the Azure Function App
            await _durableClient.StartNewAsync(nameof(OpenCircuitOrchestrator.OpenCircuit), failure.ResourceId);

            // Mark the circuit as open
            state = CircuitState.Open;
        }
        else 
        {
            _log.LogInformation($"The circuit {Entity.Current.EntityKey} currently has {FailureWindow.Count} exceptions in the window of {windowSize.ToString()}");
        }
    }

    [FunctionName(nameof(Circuit))]
    public static Task Run(
        [EntityTrigger] IDurableEntityContext ctx) => ctx.DispatchAsync<Circuit>(client);

    }

There are other ways you can chain these together - like adding some in-memory retries to the function or even having the function explicitly check the state of the circuit before processing the message, but this flow is my favorite and optimizes for high throughput and low cost. I can deploy the durable entity and function app, and once the durable entity detects the number of failures is too high it will automatically disable the Azure Function app so it stops processing. It is worth noting you can use the same durable entity to monitor and manage the circuit for many different function apps in your subscription at the same time.

The full code for my sample is on GitHub.

Sample circuit breaker scenario

To help make this pattern clear, let's walk through an example. Imagine I have the Azure Function above and the Durable Entity deployed in my subscription. We'll use the same scenario as we started this post with - the function triggers on the queue, interacts with a SQL server, and completes processing.

My function could be running for months without issue. All during this time, my durable entity is sitting idle (and free) in my subscription. Assume the function is triggering on queue messages across 100 active instances.

Suddenly, the SQL server starts having some issues. A few of the messages start throwing exceptions. An instance hits an exception and lets the durable entity know. The durable entity keeps track that 1 exception has happened in the last 30 seconds. Moments later, 20 other instances hit an exception. Each sends a signal to the durable entity, which now knows that 21 exceptions have happened in the span of a few seconds. Finally, the failures start to mount, and quickly the durable entity detects the threshold has crossed - it has the state for over 30 exceptions in a 30 second window.

The logic activates the breaking and opening of the circuit. It makes a call to the Azure APIs and disables the function app so it stops triggering. No messages are lost - the queue messages stay safely in the Service Bus queue. But rather than creating cascading failures, I've gracefully broken the circuit until the health of the systems can be confirmed, the circuit closed, and processing resumes.

With durable entities and Azure Functions we've very efficiently solved the problem we started with:

"Can we make it so our function stops triggering if lots of failures start happening?"

More than just functions

You can use Durable Entities to manage the state of any distributed app. Polly recently announced support for Durable Entity powered circuit breakers that can manage the state of any application. I'd encourage you to check them out, and give durable entities a spin!

Develop Azure Functions using .NET Core 3.1

Jeff Hollan — Thu, 17 Oct 2019 20:15:35 +0000

<< Updated on Dec 9 2019 with go-live announcement of Azure Functions on .NET Core 3.1 >>

We're extremely excited to announce you can now develop and publish Azure Functions built on top of the .NET Core 3.1 runtime.

Things to know

Azure Functions 3.0 is now go-live and ready for production. It is a highly backwards compatible release. One of the immediate benefits of this release is you can write Azure Functions targeting netcoreapp3.1. You should also be able to move existing ~2 apps to ~3 without issue. A list of considerations on changes between 2.x and 3.x can be found in our docs.

You can write any Azure Function language in 3.0 (Java, JavaScript, PowerShell, TypeScript, Python) - though note some of the tooling and integrations will continue to receive updates until mid-January. Instructions on changing your tools or apps to target 3.x for other languages can be found in our docs. For now let's walk through creating a .NET Core 3.1 app.

Getting started

Install the .NET Core 3.1 SDK

Install the 3.x core tools (optional for Visual Studio development)

Using npm:

npm install -g azure-functions-core-tools@3

Using brew:

brew tap azure/functions
brew install azure-functions-core-tools@3
# If you already have func installed
brew link --overwrite azure-functions-core-tools@3

Create an Azure Functions project

You have a few options here. You could create a new project using our Visual Studio, Visual Studio Code, or CLI tools. For other tools, once you install the 3.x core tools you should be set. If attempting to target .NET Core 3.1 just be sure to update your .csproj file template with netcoreapp3.1. Also be sure to target the `Microsoft.NET.Sdk.Functions package version of at least 3.0.1.

Creating with Visual Studio 2019

If you want to develop using .NET Core 3.1, you must use Visual Studio 2019 16.4 or newer.

Open Visual Studio and select to create a new Azure Functions project
- You should see an indicator at the bottom as it downloads the runtime and templates of "Making sure all templates are up to date." This could take a few minutes, but wait for it to finish.
Once the templates are up to date, go back one screen and forward so template options are refreshed.
Select the now available "Azure Functions v3" option from the template drop-down.
Create your project.
If you want to target .NET Core 3.1, edit the project properties and select .NET Core 3.1.

You should be set! You can now debug, test, and publish this function.

Publish your Function App

You can publish your function using the regular tools. Depending on the tool you will likely see a prompt to update your FUNCTIONS_EXTENSION_VERSION application setting from the default ~2 to ~3. This does need to occur before a 3.x app can run successfully in Azure.

Upgrading a v2 app to v3

If you have an existing app and want to test it out with the new release, the following settings would need to be modified:

Non-.NET Functions

Locally once you start using the 3.x CLI your apps will debug and run in the 3.0 environment automatically. For apps in the cloud, you simply need to update the app setting of FUNCTIONS_EXTENSION_VERSION to ~3 - but make sure to do some validation tests first before changing anything in production 😅.

.NET Functions

If using Visual Studio, make sure you've already followed all of the steps in the "Creating with Visual Studio 2019" section to ensure you have the latest templates and project options available before attempting to upgrade. You may need to go to the new app template screen to get the 3.x bits to pull in on your machine, so if you get an error like "no runtime available that matches the version," try creating a new 3.x app from scratch first to make sure VS has fetched all the latest bits.

To change a project between ~2 and ~3 locally, update the .csproj file of the project and change:

AzureFunctionsVersion from v2 to v3
Microsoft.NET.Sdk.Functions package version to at least 3.0.1
Optionally, you can change to target netcoreapp3.1 if you want to target the latest .NET Core version

You should not change the host.json version to 3.0. The host.json schema version is different than the function runtime version, and is still 2.0.

Path to GA

While you can begin using 3.x in production today, we are tracking a few work items around tooling and platform features before we announce full GA for tools and runtime in January 2020. You can see some of those work items called out in GitHub

Announcing the RabbitMQ extension for Azure Functions

Jeff Hollan — Thu, 12 Sep 2019 23:06:50 +0000

We're excited to announce today the public preview of the RabbitMQ Extension for the Azure Functions runtime. With this new extension you can now have functions trigger in response to queue messages, or connect to RabbitMQ for an output binding. Currently the preview trigger is supported when publishing functions to the Functions Premium plan with runtime driven scale enabled, or a dedicated App Service Plan. You can also be able to use RabbitMQ with Azure Functions containers in Kubernetes alongside KEDA.

How to get started with the RabbitMQ Trigger

To get started with using the RabbitMQ trigger, you need to include the extension in your function project.

.NET Functions

For .NET functions, you can pull in the RabbitMQ NuGet extension.

Install-Package Microsoft.Azure.WebJobs.Extensions.RabbitMQ -Version 0.2.2029-beta

JavaScript, TypeScript, Python, Java, and PowerShell Functions

For other functions, you need to install the extension into your project using the Azure Functions core tools

func extensions install Microsoft.Azure.WebJobs.Extensions.RabbitMQ -v 0.2.2029-beta

You can then create a function that can activate and run whenever a message is dropped in a defined RabbitMQ queue.

Example C# Trigger and Output Binding

public static void Run(
    [RabbitMQTrigger("queue", ConnectionStringSetting = "RabbitMqConnection")] string inputMessage,
    [RabbitMQ(
        ConnectionStringSetting = "RabbitMQConnection",
        QueueName = "downstream")] out string outputMessage,
    ILogger logger)
{
    outputMessage = inputMessage;
    logger.LogInformation($"RabittMQ output binding function sent message: {outputMessage}");
}

Example function.json

{
"bindings": [
    {
      "type": "rabbitMQTrigger",
      "connectionStringSetting": "RabbitMqConnection",
      "queueName": "queue",
      "deadLetterExchangeName": "dtx",
      "name": "myQueueItem"
    }
  ]
}

The connectionStringSetting resolves to an application setting (local.settings.json when developing locally) that should follow a format like: amqp://user:PASSWORD@127.0.0.1:5672.

For instructions on how to use Azure Kubernetes Service to host a RabbitMQ endpoint for testing, you can follow this guide.

Redelivery and Deadletter

Like the other queue-based triggers for Azure Functions, the RabbitMQ trigger supports retries and deadletters. If an execution results in an exception, the message can be retried. Once the message has exceeded its redelivery count threshold, we will route it to the dead letter exchange defined, where messages can end up in a poison queue. The extension will automatically append "-poison" to the original queue name when the DLX option is filled.

You can learn more about how to configure deadletter and poison queues here.

Open and Evolving

This is just the initial public preview release. Like all Azure Functions extensions, this extension is completely open source. If you have any issues, feedback, or requests - please file issues on the GitHub repository. We also welcome any contributions and additions from the community.

We're excited with this announcement and look forward to even more event sources being compatible with Azure Functions everywhere.

Ordered queue processing in Azure Functions with Sessions

Jeff Hollan — Thu, 30 May 2019 14:40:52 +0000

Let's chat about ordering. It's one of my favorite topics, and something I've blogged about extensively before. Previously ordered processing in Azure Functions was only possible with event streams like Azure Event Hubs, but today I want to show how you can preserve order for Service Bus queues and topics as well.

On the surface it seems pretty straight-forward: I want to be able to process messages from a queue in the exact order that I received them. For a simple service running on a machine, it's pretty easy to achieve. However, how do I preserve the ordering of queue messages when I want to process at scale? With something like Azure Functions I may be processing messages across dozens of active instances, how can I preserve ordering?

Let's use a simple example of a messaging system that deals with patients at a hospital. Imagine I have a few events for each patient:

Patient arrives
Patient assigned a room
Patient receives treatment
Patient is discharged

I want to make sure I never process a message out of order and potentially discharge a patient before I've processed their treatment!

Let's run some quick experiments to see what happens. For this I'm going to simulate 1000 patients each sending these 4 messages (in order) and processing them (ideally in order as well).

Default and out of order

Let's try this with a simple Azure Function that just triggers on a queue. I'm not going to do anything special, just trigger on the queue and push the operation it's processing to a list on Redis Cache.



public async Task Run(
    [ServiceBusTrigger("queue", Connection = "ServiceBusConnectionString")]Message message, 
    ILogger log)
{
    log.LogInformation($"C# ServiceBus queue trigger function processed message: {Encoding.UTF8.GetString(message.Body)}");
    await _client.PushData((string)message.UserProperties["patientId"], Encoding.UTF8.GetString(message.Body));
}

After sending 1000 patients worth of data (4 messages each) to this queue, what does the Redis Cache look like after processing? Well some of the patients look great. When I lookup Patient #4 I see:



>lrange Patient-$4 0 -1
1) "Message-0"
2) "Message-1"
3) "Message-2"
4) "Message-3"

Great! All 4 events were sent for Patient 4, and got processed in order. But if I look at patient 2:



>lrange Patient-$2 0 -1
1) "Message-1"
2) "Message-2"
3) "Message-0"
4) "Message-3"

In this case it didn't finish processing the "patient arrives" message until after 2 other messages had already been processed. So what happened here? Azure Service Bus does guarantee ordering, so why are my messages out of order?

Well by default, the queue trigger will do a few things. First, for every instance that spins up, it will process a set of messages concurrently. By default an instance concurrently processes 32 messages. That means it may be processing all 4 messages for a patient at the same time and they finish in different order than they were sent. Well that seems easy enough to fix, let's just limit the concurrency to 1.

Anti-pattern: limit scale out and concurrency

Here's maybe the most common solution to the above problem that I see. Let's limit the concurrency to only process 1 message at a time instead of 32. For that I modify my host.json file and set the maxConcurrentCalls to 1. Now each instance will only process 1 message at a time. I run the same test again.

First off, it's super slow. It takes me a long time to chew through the 4000 queue messages because each instance only processes 1 at a time. And worse yet? When I check the results afterwards, some of the patients are still out of order! What's going on here? Even though I limited the instance concurrency to 1, Azure Functions has scaled me out to multiple instances. So if I have 20 function app instances that have scaled, I have 20 messages being processed concurrently (1 per instance). That means I still get into a spot where messages from the same patient could be processed at the same time - just on different instances. I'm still not guaranteed ordered processing.

The fix here? Many people want to limit the scale out of Azure Functions. While it's technically possible, it would hurt my throughput even more. Now only one message globally could be processed at a time, meaning during high traffic I'm going to get a large backlog of patient events that my function may not be able to keep up with.

Sessions to the rescue

Wouldn't this be such a sad blog post if I ended it here? There is a better way! Previously I would have said your best bet here may be to use Event Hubs which, because of partitions and batches, you can guarantee ordering. The challenge here though is that sometimes a queue is the right message broker for the job given its transactional qualities like retries and deadlettering. And now you can use queues and get ordering with Service Bus sessions 🎉.

So what are sessions? Sessions enable you to set an identifier for a group of messages. In order to process messages from a session, you first have to "lock" the session. You can then start to process each message from the session individually (using the same lock / complete semantics of a regular queue). The benefit of sessions is it enables you to preserve order even when processing at high scale across multiple instances. Think of before where we had something like 20 Azure Function app instances all competing for the same queue. Rather than not scaling to 20, now all 20 instances each will "lock" its own available session and only process events from that session. Sessions also ensure that messages from a session are processed in order.

Sessions can be dynamically created at any time. An instance of Azure Functions spins up and first asks "are there any messages that have a session ID that hasn't been locked?" If so, it locks the session and starts processing in order. When a session no longer has any available messages, Azure Functions will release the lock and move on to the next available session. No message will be processed without first having to lock the session the message belongs to.

For our example above, I'm going to send the same 4000 messages (4 patient events for 1000 patients). In this case, I'm going to set the patient ID as the session ID. Each Azure Functions instance will acquire a lock on a session (patient), process any messages that are available, and then move on to another patient that has messages available.

Using sessions in Azure Functions

Sessions are currently available in the Microsoft.Azure.WebJobs.Extensions.ServiceBus extension using version >= 3.1.0, and at the time of writing this is in preview. So first I'll pull in the extension.



Install-Package Microsoft.Azure.WebJobs.Extensions.ServiceBus -Pre

And then make the tiniest code change to my function code to enable sessions (isSessionsEnabled = true):



public async Task Run(
    [ServiceBusTrigger("queue", Connection = "ServiceBusConnectionString", IsSessionsEnabled = true)]Message message, 
    ILogger log)
{
    log.LogInformation($"C# ServiceBus queue trigger function processed message: {Encoding.UTF8.GetString(message.Body)}");
    await _client.PushData(message.SessionId, Encoding.UTF8.GetString(message.Body));
}

I also need to make sure I'm using a session-enabled queue or topic.

And when I push the messages to the queue, I'll set the right sessionId for each patient message I send.

After publishing the function I push the 4000 messages. The queue gets drained pretty quickly, because I'm able to process multiple sessions concurrently across scaled-out instances. After running the test I check Redis Cache. As expected, I see all messages were processed, and for every single patient I see they were processed in order:



>lrange Patient-$10 0 -1
1) "Message-0"
2) "Message-1"
3) "Message-2"
4) "Message-3"

>lrange Patient-$872 0 -1
1) "Message-0"
2) "Message-1"
3) "Message-2"
4) "Message-3"

So with the new Azure Functions support for sessions, I can process messages from a Service Bus queue or topic in order without having to sacrifice on overall throughput. I can dynamically add messages to a new or existing session, and have confidence that messages in a session will be processed in the order they are received by service bus.

You can see the full sample I used for testing and loading messages in my GitHub repo. The master branch will be all in order, and the out-of-order branch is the default and out of order experiment.

Using Entity Framework with Azure Functions

Jeff Hollan — Tue, 28 May 2019 16:41:47 +0000

<< Update: The instructions below are still good but I did update my sample to use .NET Core 3.1 here if interested: https://github.com/jeffhollan/functions-chsarp-entitycore-3 >>

One of the exciting developments from Build this year was support for dependency injection in Azure Functions. This means you can register and use your own services as part of functions. While you've been able to use Entity Framework Core in the past, the pairing with dependency injection makes it a much more natural fit. Let's create a simple Azure Function that can interact with stateful data using Entity Framework Core. I'm going to create a very simple API that can get and set blog data in an Azure SQL Database.

Adding entity framework to a function project

First up, I created a brand new v2 (.NET Core 2) Azure Functions project. I selected the HTTP Trigger template to start with. In order to use dependency injection I need to verify two things:

The Microsoft.NET.Sdk.Functions is at least 1.0.28
Install the Microsoft.Azure.Functions.Extensions package to provide the API to use dependency injection

Update-Package Microsoft.NET.Sdk.Functions
Install-Package Microsoft.Azure.Functions.Extensions

While we're at it, let's install the Entity Framework libraries. ⚠️ These versions are very particular with the version of .NET you will be running. So choose your version of these libraries thoughtfully. At the time of writing this Azure is running .NET Core 2.2.3 so that's what I'll use.

Install-Package Microsoft.EntityFrameworkCore.SqlServer -Version 2.2.3

I also want to take advantage of the design-time migration, so I'll install the tools needed to drive that.

Install-Package Microsoft.EntityFrameworkCore.Design -Version 2.2.3
Install-Package Microsoft.EntityFrameworkCore.Tools -Version 2.2.3

Add the entity models

Next, I'll create a very simple DbContext for the data models I want to interact with. Let's just stick with the simple Blog and Posts entities.

using Microsoft.EntityFrameworkCore;
using System;
using System.Collections.Generic;
using System.Text;

namespace functions_csharp_entityframeworkcore
{
    public class BloggingContext : DbContext
    {
        public BloggingContext(DbContextOptions<BloggingContext> options)
            : base(options)
        { }

        public DbSet<Blog> Blogs { get; set; }
        public DbSet<Post> Posts { get; set; }
    }

    public class Blog
    {
        public int BlogId { get; set; }
        public string Url { get; set; }

        public ICollection<Post> Posts { get; set; }
    }

    public class Post
    {
        public int PostId { get; set; }
        public string Title { get; set; }
        public string Content { get; set; }

        public int BlogId { get; set; }
        public Blog Blog { get; set; }
    }
}

Write the function code to inject the entity context

I need to verify that I'm using constructors in my function classes so I can inject in the right dependencies (in this case, the entity framework context). This means getting rid of static and adding a constructor. I'll also modify the constructor to accept a BloggingContext created above (which itself will need a DbContextOptions<BloggingContext> injected).

using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Logging;

namespace functions_csharp_entityframeworkcore
{
    public class HttpTrigger
    {
        private readonly BloggingContext _context;
        public HttpTrigger(BloggingContext context)
        {
            _context = context;
        }

        [FunctionName("GetPosts")]
        public async Task<IActionResult> Get(
            [HttpTrigger(AuthorizationLevel.Function, "get", Route = "posts")] HttpRequest req,
            ILogger log)
        {
            log.LogInformation("C# HTTP trigger function processed a request.");

            // Code that returns posts - you can see this code by going
            // to the full sample linked at the bottom
            return new OkResult();
        }
    }
}

To register services like BloggingContext, I create a Startup.cs file in the project and implement the FunctionsStartup configuration. Notice I use an attribute to signal to the functions host where it can run my configuration.

using Microsoft.Azure.Functions.Extensions.DependencyInjection;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.DependencyInjection;
using System;

[assembly: FunctionsStartup(typeof(functions_csharp_entityframeworkcore.Startup))]

namespace functions_csharp_entityframeworkcore
{
    class Startup : FunctionsStartup
    {
        public override void Configure(IFunctionsHostBuilder builder)
        {
            string SqlConnection = Environment.GetEnvironmentVariable("SqlConnectionString");
            builder.Services.AddDbContext<BloggingContext>(
                options => options.UseSqlServer(SqlConnection));
        }
    }
}

Enable design-time DbContext creation

As mentioned, I want to use the design-time DbContext creation. Since I'm not using ASP.NET directly here, but implementing the Azure Functions Configure method, entity framework won't automatically discover the desired DbContext. So I'll make sure to implement an IDesignTimeDbContextFactory to drive the tooling.

public class BloggingContextFactory : IDesignTimeDbContextFactory<BloggingContext>
{
    public BloggingContext CreateDbContext(string[] args)
    {
        var optionsBuilder = new DbContextOptionsBuilder<BloggingContext>();
        optionsBuilder.UseSqlServer(Environment.GetEnvironmentVariable("SqlConnectionString"));

        return new BloggingContext(optionsBuilder.Options);
    }
}

Getting the project .dll in the right spot

There's one last thing I need to do. When you build an Azure Functions project, Microsoft.NET.Sdk.Functions does some organizing of the build artifacts to create a valid function project structure. One of the .dlls it moves to a sub-folder is the project .dll. Unfortunately, for the design-time tools like entity framework migration, it expects the .dll to be at the root of the build target. So to make sure both tools are happy, I'll add a quick post-build event to copy the .dll to the root as well. I added this to my .csproj file for the project.

<Target Name="PostBuild" AfterTargets="PostBuildEvent">
  <Exec Command="copy /Y &quot;$(TargetDir)bin\$(ProjectName).dll&quot; &quot;$(TargetDir)$(ProjectName).dll&quot;" />
</Target>

Adding an entity framework migration

I went into the Azure Portal and created a simple Azure SQL database. I copied the connection string into the local.settings.json file with a new value for SqlConnectionString. You can see in my previous code samples I used that as the environment variable that would have the connection string. This way I don't have to check it into source control 🥽.

To make sure the connection string is also available to the CLI tooling for migrations, I'll open my Package Manager Console and set the environment variable before adding a migration and updating the database.

$env:SqlConnectionString="Server=tcp:mySqlServerStuffxxx"
Add-Migration InitialCreate
Update-Database

When browsing to the SQL database in the Azure Portal, I can see the tables were successfully created!

Writing the function app and publishing

That's all the heavy lifting. Now I just write a few simple HTTP triggered functions that can return and add data to the BloggerContext I injected in. When running the app, I can then call the different GET and PUT methods to validate data is being persisted and returned correctly.

After publishing, I just make sure to set the appropriate application setting for SqlConnectionString with my production SQL connection string, and we're in business!

You can see the full project code on my GitHub account