DEV Community: Laurentiu Iarosevici

Building a serverless “shopping Cart” microservice using Azure functions and Durable entities with c# [Tutorial]

Laurentiu Iarosevici — Mon, 21 Sep 2020 09:15:58 +0000

We will be building a fully functional “microservice” that will be able to add, remove, and return a list of products. For this example we will model the way this microservice will be consumed will be done via REST. Get your coding hat on, we are about to start.

The setup

Before we begin you should have a new function project setup and the durable functions set-up. If you are not very sure about how to achieve this, here is a link to the official documentation.

Pro tip: in some IDEs (I use Rider) you have the option on the project setup to chose durable functions, so you don’t need to install the required NuGet packages by hand.

Durable entities and the actor pattern

So, since you found this article via searching or by clicking somewhere on social media, you should already know what durable entities are, or at least have an idea about them. But, just to be sure that we are all on the same page we will be doing a small introduction on durable entities, and why, when, and where we might be able to benefit from them.

“Durable entities” or “entity functions”, how they are called nowadays, are mostly inspired by the virtual actors pattern from the Orleans project, which in term are quite inspired by Akka/ Akka.net. Going into this topic is well outside of our scope, but you will find an introduction in them in the links provided, also I highly recommend looking a bit into them, because understanding the strengths and weaknesses of the actor pattern will help you when trying to determine if the durable entities might be a good match for you or should you use a more traditional SQL or NoSql storage.

Well, probably you haven’t clicked any of the links above … in case you did, well done!

Anyway, long story short, an actor is an isolated construct that holds it’s own state and has its behavior, they do not share memory and the communication is done asynchronously using some kind of message box/queues. They are kept in memory while in use and when disposed they are usually serialized to disk. They can run in grate numbers, and depending on the implementation they can be hierarchical. In general actor, systems are self-healing.

So, more or less the durable entities are some kind of actors. They are very good at keeping track of some small to mid-sized objects state and behavior, and also quite interestingly they hold the state in the table storage of the storage account that is used by the function app. This can be seen as an advantage through the lens of scalability and availability since both of these are handled automatically by Azure storage. Also, what is quite important to know is that the entities themselves are event-sourced.

Before we dive in the actual code there is one more thing I still need to address, why use entities for a shopping cart rather than using a classic SQL or NoSql DB. And this is quite a good question. As I usually say, almost all the things could be built in a lot of ways, each of them having particular benefits and downsides, so next, we will be diving in and see why it might be beneficial to use entities rather than SQL or CosmosDB.

We are operating on the assumption that azure functions are running on a consumption plan.

So one of the most appealing parts of the whole serverless movement and compute offer is the near “infinite” auto-scaling. So, by utilizing something that is also “serverless” for holding the state seems to be a good match. One would argue that CosmosDB is also serverless, but although the connection between Cosmos and functions could be done via binding it still is not as seamless as with the entities functions. Regarding SQL, well it is kind of evident why something that can scale as much and as fast as required should not be paired with something that isn’t able to scale as fast.

Also, one of the benefits of actors in general, and in our case durable entities, is that they are kept in memory for a time whiteout being offloaded, using them we avoid the latency of a round trip to the DB, since we are essentially using in-memory objects, that get serialized and serialized behind the scene.

The Code

Regarding the overall design, the microservice will expose a restful endpoint with the following API methods:

AddItemToCart: [POST] http://localhost:7071/api/cart/{id}/add

GetCartForSession: [GET] http://localhost:7071/api/cart/{id}

RemoveItemFromCart: [POST] http://localhost:7071/api/cart/{id}/remove

As you can see here, there is an {id} that should be a GUID generated by the client, this way in theory this type of microservice could be used with any client software, without the need of knowing anything about it. Regarding the GUID type, this could be changed to a regular string, but this way ensures against collisions.

The heart of the whole demo is the durable entity, which is defined as follows:

[JsonObject(MemberSerialization.OptIn)]
public class ShoppingCartEntity : IShoppingCart
{

    [JsonProperty("list")]
    private List<CartItem> list { get; set; } = new List<CartItem>();

    public void Add(CartItem item)
    {
        // Get existing
        var existingItem = this.list.FirstOrDefault(i => i.ProductId == item.ProductId);

        if (existingItem == null)
        {
            this.list.Add(item);
        }
        else
        {
            existingItem.Count += item.Count;
        }
    }

    public void Remove(CartItem item)
    {
        var existingItem = this.list.FirstOrDefault(i => i.ProductId == item.ProductId);

        if (existingItem == null)
        {
            return;
        }

        if (existingItem.Count > item.Count)
        {
            existingItem.Count -= item.Count;
        }
        else
        {
            this.list.Remove(existingItem);
        }
    }

    public Task<ReadOnlyCollection<CartItem>> GetCartItems()
    {
        return Task.FromResult(this.list.AsReadOnly());
    }

     [FunctionName(nameof(ShoppingCartEntity))]
     public static Task Run([EntityTrigger] IDurableEntityContext ctx)
            => ctx.DispatchAsync<ShoppingCartEntity>();
}

After a glance at the code, some interesting things could be noticed. It looks more or less like a regular domain entity with properties and behavior. Also, the property that keeps the actual state is private (I wouldn’t go as far as saying encapsulated) and only modified using the exposed public methods, and also it’s content is also returned via a public method. One thing that we need to keep in mind is that a method of an entity function can only return void, Task or Task, nothing else.

For the state container we also use the following class:

public abstract class CartItem
{
    public Guid ProductId { get; set; }
    public int Count { get; set; }
}

In this class*, we could have all the imaginable properties as long as they are serializable*, for sake of the example I tried to keep it as short as possible, but here you could have also the price when added, or the name and description, even a link to a product image or whatever.

Now that we have set up the entity itself, we will manipulate it using rest triggers, again for simplicity’s sake, the recommended way of interacting with entities is using orchestrator/durable functions, but the interactions are required in our example could be done directly from the triggers.

So for adding an item to the cart here is the trigger code:

[FunctionName("AddItemToCart")]
public static async Task<ActionResult> RunAsync(
         [HttpTrigger(AuthorizationLevel.Function,  "post", Route = "cart/{id}/add")] HttpRequestMessage req,
         Guid id,
         [DurableClient] IDurableEntityClient client )
     {
         if (id == Guid.Empty)
         {
             return (ActionResult) new BadRequestObjectResult("Id is required");
         }
             var entityId = new EntityId(nameof(ShoppingCartEntity), id.ToString());

             var data = await req.Content.ReadAsAsync<CartItem>();

             await client.SignalEntityAsync<IShoppingCart>(entityId, proxy => proxy.Add(data));

             return (ActionResult) new AcceptedResult(); // req.CreateResponse(HttpStatusCode.Accepted);

     }

So, here we are using a command approach, with a simple accepted response.

For removing an item from the cart, the code is remarkably similar:

[FunctionName("RemoveItemFromCart")]
public static async Task<IActionResult> RunAsync(
           [HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = "cart/{id}/remove")] HttpRequestMessage req,
           Guid id,
           [DurableClient] IDurableClient client,  ILogger log)
       {

           if (id == Guid.Empty)
           {
               return (ActionResult) new BadRequestObjectResult("Id is required");
           }
               var entityId = new EntityId(nameof(ShoppingCartEntity),id.ToString());

               var data = await req.Content.ReadAsAsync<CartItem>();

               await client.SignalEntityAsync<IShoppingCart>(entityId, proxy => proxy.Remove(data));

               return (ActionResult) new AcceptedResult();
       }

And for retrieving the elements from a cart:

[FunctionName("GetCartForSession")]
public static async Task<IActionResult> RunAsync(
    [HttpTrigger(AuthorizationLevel.Anonymous, "get", Route = "cart/{id}")] HttpRequestMessage req,
    Guid id,
    [DurableClient] IDurableClient client,
    ILogger log)
{
    if (id == Guid.Empty)
    {
        return (ActionResult) new BadRequestObjectResult("The ID is required");
    }

    var entityId = new EntityId(nameof(ShoppingCartEntity), id.ToString());

    var stateResponse = await client.ReadEntityStateAsync<ShoppingCartEntity>(entityId);

    if (!stateResponse.EntityExists)
    {
        return (ActionResult) new NotFoundObjectResult("No cart with this id");
    }

    var response = stateResponse.EntityState.GetCartItems();

    return (ActionResult) new OkObjectResult(response.Result);
}

And more or less, this is it. We now have a serverless microservice that could scale on demand.

Before we end this, I just want to remind you that this is a high level presentation of the concept, if you have any question, or would like to see more content about this feel free to add a comment on reach out to me on twitter.

As usual, you could get a closer look at the code on this github repository.

Dynamically select orchestrators in Azure Durable Functions

Laurentiu Iarosevici — Fri, 11 Sep 2020 07:22:31 +0000

This is one of the awesome things that I think can be done in functions, which also enables us to build a lot of very interesting mechanics, it’s not that complicated to pull off, but the results of using this technique might lead to very impressive outcomes.

Before we begin though, there are some disclaimers:

Disclaimer #1: The code example that I prepared is quite ridiculous, but I couldn’t find anything more simple to show this, since any other ideas I had pulled the focus to the business logic.

Disclaimer #2: It’s really ridiculous, please don’t do things like this in real life ( string building via durable functions workflows :D)

Disclaimer #3: In the code examples there aren’t many logging statements nor many exception handling, please for production code abuse logging and use exception handling, as with #1, the code example is designed to show this functionality in a theoretical concept and make it as clear as possible.

Good, now that we got this out of the way, let’s dig in.

Requirements:

As usual, we need a function app with the durable function nuggets installed ( if you want to follow along). If you need any guidance check this link.
A basic understanding of azure functions and durable functions

What are we building?

In this example we will have the following scenarios:

So, we will have the default HTTP trigger that asks for a name, and instead of returning the name, we will forward the name to our “Dynamic Orchestrator”. This orchestrator will decide if it is a short or long name, and based on this will call the LongName or the ShortName orchestrator. Each of these sub-orchestrators will have an activity that will actually build a string with a greeting and then return it. I know, brilliant example! Anyway, enough with the talk and drawing, let’s get into the code.

As I said earlier the trigger is basically the one that you get from the template with a few small tweaks:

public static class DynamicOrchestratorTrigger
{
    [FunctionName("DynamicOrchestratorTrigger")]
    public static async Task<IActionResult> RunAsync(
        [HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)]
        HttpRequest req, ILogger log,
        ExecutionContext executionContext,
        [DurableClient] IDurableOrchestrationClient starter)
    {
        log.LogInformation("C# HTTP trigger function processed a request.");

        string name = req.Query["name"];

        var requestBody = await new StreamReader(req.Body).ReadToEndAsync();
        dynamic data = JsonConvert.DeserializeObject(requestBody);
        name ??= data?.name;
        if (name == null)
        {
            return new BadRequestObjectResult("Please pass a name on the query string or in the request body");
        }

        var orchestratorInput = new OrchestratorInput
        {
            CorrelationId =  new Guid(),
            Name =  name.ToString()
        };

        var instanceId = await starter.StartNewAsync(nameof(DynamicOrchestrator), orchestratorInput);

        DurableOrchestrationStatus durableOrchestrationStatus = await starter.GetStatusAsync(instanceId);
        while (durableOrchestrationStatus.RuntimeStatus != OrchestrationRuntimeStatus.Completed)
        {
            await Task.Delay(200);
            durableOrchestrationStatus = await starter.GetStatusAsync(instanceId);
        }

        return (ActionResult) new OkObjectResult(durableOrchestrationStatus.Output);
    }
}

Please note, that line 30 – 35 are more or less a hack for this example, what this does is basically polling the orchestrator until it’s completed to and than returns the output, the only reason this is done is for “you” to easily test this out if you get the project from GitHub (link will be at the end of the post). I don’t advise using something like this in production.

So, next, we will look at the DynamicOrchestrator, here is where the magic happens:

public class DynamicOrchestrator
{

    [FunctionName(nameof(DynamicOrchestrator))]
    public async Task<string> Run([OrchestrationTrigger] IDurableOrchestrationContext context,
        ExecutionContext executionContext)
    {
        var input = context.GetInput<OrchestratorInput>();

        string greeting = "";
        switch (input.Name.Length)
        {
           case <= 4 :
               greeting = await context.CallSubOrchestratorAsync<string>(nameof(ShortNameOrchestrator), input);
               break;
           default:
               greeting = await context.CallSubOrchestratorAsync<string>(nameof(LongNameOrchestrator), input);
               break;
        }

        return greeting;
    }
}

So, at this point, I think you are either thinking “now this was underwhelming… “, or “this is amazing!”. As I said in the beginning, this might not look like much, and of course, this example doesn’t help either, but this simple construct enables you to do some really cool stuff. If this is not obvious let me refactor this code into this:

public class DynamicOrchestrator
{

    [FunctionName(nameof(DynamicOrchestrator))]
    public async Task<string> Run([OrchestrationTrigger] IDurableOrchestrationContext context,
        ExecutionContext executionContext)
    {
        var input = context.GetInput<OrchestratorInput>();
        return await context.CallSubOrchestratorAsync<string>(
            await context.CallActivityAsync<string>(
                nameof(DetermineOrchestratorToRunActivity), input.Name)
            ,input);
    }
}

Basically, what has happened here, we turned this orchestrator more or less in a HOF (higher-order function). By doing this we can even swap the “DetermineOrchestratorToRunActivity” if we wanted to.

Speaking of “DetermineOrchestratorToRunActivity”, in our case, this is just a simple activity that should perform the business logic that will result in our next step:

public class DetermineOrchestratorToRunActivity
{

    [FunctionName(nameof(DetermineOrchestratorToRunActivity))]
    public async Task<string> Run([ActivityTrigger] IDurableActivityContext  context)
    {
        var input= context.GetInput<string>();
        string nameOfOrchestrator;

        switch (input.Length)
        {
           case <= 4 :
               return nameof(ShortNameOrchestrator);
               break;
           default:
               return nameof(LongNameOrchestrator);
               break;
        }
    }
}

Maybe this doesn’t look like much, but instead of the length switch here, you could do latterly anything here, and I think this is quite interesting.

The rest of the orchestrators and activities are quite dumb and simple and don’t think it has any point in showing them here, but you can see a working example in the GitHub repository.

Inject configuration values in Azure Functions

Laurentiu Iarosevici — Mon, 31 Aug 2020 14:06:49 +0000

One of the most important updates to the Azure function framework was the addition of the DI support. One of the use-cases for having a DI is to read and use "feature flags" in the form of configuration values/environment variables.

So for the big question, how do we go from here (local.appsettings.json / env variables):

  {
    "IsEncrypted": false,
    "Values": {
        "AzureWebJobsStorage": "UseDevelopmentStorage=true",
        "FUNCTIONS_WORKER_RUNTIME": "dotnet",
        "FeatureFlag" : "True",
        "ConfigValue" : "some random value"
    }
}

To here (constructor injected in the function / trigger):

 public class FeatureHttpTrigger
    {
        private readonly FeatureConfig _config;

        public FeatureHttpTrigger(FeatureConfig config)
        {
            _config = config;
        }

        [FunctionName("FeatureHttpTrigger")]
        public async Task<IActionResult> RunAsync(
            [HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)]
            HttpRequest req, ILogger log)
        {
          ......
        }
}

Although I know that there is quite a difference between a feature flag and an environment variable, a lot of people use them like this. Anyway ... moving on...
So, one of the most common things you tend to do is to rely on the DI framework to provide you with all sorts of useful constructs ( that you painstakingly set up beforehand) in order to simplify the way you use them. If you are not familiar with Dependency Injection, I suggest you look into it since this is one of the most important patterns used nowadays in most of the frameworks around web development.

Prepartiations

In order to get this working, we need to make sure we are using v3 of azure functions (should work with v2 though).
Next, we need to make sure that we have the DI setup. Here is a good resource to help you do this.

The actual steps

Before we continue, I want to add that I am a big fan of feature folders, and when I build this example I subconsciously used them, they are not required, and by all means, some other constructs could be bypassed, but this is the way I'm usually structuring my function code.

Good, now this is all out of the way, let's get down to business.

The first thing you'll want to do is to add a class that will hold your data in the shape you prefer to use via the constructor injection.

    public class FeatureConfig
    {
        public string ConfigValue { get; set; }
        public bool FeatureFlag { get; set; }
    }

Next, we need to make sure that we have the values set up either in the appsettings or the environment variables or in the function apps configuration (ideally everywhere :) ).

Now, the next thing I usually do, which is not required but I think it keeps the code more structured si to create a static class that will hold all the DI configuration:

internal static class ServiceRegistrations
    {
        public static void ConfigureServices(IServiceCollection builderServices, IConfiguration configuration)
        {
            builderServices.AddFeature();
        }

        private static IServiceCollection AddFeature(this IServiceCollection serviceCollection)
        {
            // here we would group all feature related DI code
            serviceCollection.TryAddSingleton<FeatureConfig>(sp =>
            {
                var config = sp.GetService<IConfiguration>();

                return new FeatureConfig
                {
                    ConfigValue = config.GetValue<string>("ConfigValue"),
                    FeatureFlag = config.GetValue<bool>("FeatureFlag" )
                };
            });

            return serviceCollection;
        }
    }

And boom, that is about it ...

If you want a closer look, here is a link to github