DEV Community: Oscar van Tol

Using Bicep to link external Apis to Environments in Azure Static Web Apps

Oscar van Tol — Sun, 09 Oct 2022 13:50:41 +0000

I was preparing a talk called Azure Static Web Apps and the Api Multiverse in which I wanted to hook up external apis instead of the managed functions. The first thing I wanted to do is have a front-end that I could deploy that calls some kind of api on the backend. I did not want to create the Static Web App 'the magic way' but I wanted to start with IaC, so for this I created a bicep file.

param location string = resourceGroup().location

resource swamv_ui 'Microsoft.Web/staticSites@2022-03-01' = {
  name: 'swamv'
  location: location
  sku: {
    name: 'Standard'
    tier: 'Standard'
  }
  properties: {
    stagingEnvironmentPolicy: 'Enabled'
    allowConfigFileUpdates: true
    provider: 'Custom'
    enterpriseGradeCdnStatus: 'Disabled'
  }
}

Build and deploy

Next up I created a simple WASM Blazor app from the template and started a build/deployment pipeline. In this pipeline the code is built and the Azure CLI is used to create the resource using Bicep. Now for publishing the app to the Static Web App resource, a way to do a push deployment is using the SWA cli. This publish command needs a deployment token, you could save it to some secret variable in the pipeline but I decided to just query it because we have de Azure CLI here anyway.

trigger:
- main

pool:
  vmImage: ubuntu-latest

steps:
- script: |
    cd swamv-ui
    dotnet publish -o bin/publish
  displayName: 'build blazor app'

- task: AzureCLI@2
  displayName: 'Create SWA (IaC)'
  inputs:
    azureSubscription: 'REDACTED'
    scriptType: 'pscore'
    scriptLocation: 'inlineScript'
    inlineScript: |
      az deployment group create -g swamv-rg -f swamv.bicep

- task: AzureCLI@2
  displayName: 'Deploy Blazor app to SWA'
  inputs:
    azureSubscription: 'REDACTED'
    scriptType: 'pscore'
    scriptLocation: 'inlineScript'
    inlineScript: |
      npm install -g @azure/static-web-apps-cli
      $token = az staticwebapp secrets list --name swamv --query "properties.apiKey"
      swa deploy -d $token -a ./swamv-ui/bin/publish/wwwroot/ --env production

Linking an api with Bicep

I wanted to hook up multiple apis in the demo I'm doing, the first one is a simple Azure Function. In the Azure Portal you can just link a backend by clicking a button and choosing your backend from a dropdown. If you want to define the link in Bicep it is its own resource. You need three components, the Static WebApp which we already created, a resource (reference) to the backend you want to connect and you describe the link resource itself.


resource swamv_backend_azure_functions 'Microsoft.Web/sites@2021-03-01' existing = {
  name: 'funcapiapp'
  scope: resourceGroup('functions-api-app-rg')
}

resource swamv_ui_backend_functions 'Microsoft.Web/staticSites/linkedBackends@2022-03-01' = {
  parent: swamv_ui
  name: 'swamv_backend_functions'

  properties: {
    backendResourceId: swamv_backend_azure_functions.id
    region: 'westeurope'
  }
}

Linking an api to an other Environment

But in my talk I wanted to show multiple different backends, I found out if you redefine the link it seems that Azure will not allow you to deploy it because it is trying to add multiple backends.

I decided to use different environments (which basically are staging slots) for the other backends, I was a bit confused because the linked backend resource did not have any field in there where you can specify the Environment.

But wait! I did not specify any environment in the Bicep, I just published directly using the SWA cli and it auto created them. As it turns out the environments are sub resources and called 'Builds' because of 'history' and naming is hard?


resource swamv_backend_appservice 'Microsoft.Web/sites@2022-03-01' existing = {
  name: 'swamv-appservice-api'
  scope: resourceGroup('swamv-appservice-api')
}

resource swamv_ui_environment_appservice 'Microsoft.Web/staticSites/builds@2022-03-01' existing = {
  parent: swamv_ui
  name: 'appservice'
}

resource swamv_ui_linkedbackend_appservice 'Microsoft.Web/staticSites/builds/linkedBackends@2022-03-01' = {
  name: 'swamv_backend_appservice'
  parent: swamv_ui_environment_appservice
  properties: {
    backendResourceId: swamv_backend_appservice.id
    region: 'westeurope'
  }
}

In this last part you can see that the bicep is referencing an exsiting app service resource called 'swamv-appservice-api', in this case it is using an existing environment resource 'build' called appservice and it is creating a link between the two in the defined resource called 'swamv_ui_linkedbackend_appservice'.

Playwright, NUnit, Page Object Models and Dependencies

Oscar van Tol — Mon, 11 Jul 2022 20:42:30 +0000

In my job I work with multiple teams that next to 'regular' developers also include Test Automation experts. It is very common to see them using a tool like Selenium. A while ago one of the teams needed to reboot a part of their project and we decided to start fresh.

Fresh and clean

I remembered seeing a Playwright demo and we started to investigate if it was something for us. As a C# developer I was pleased with the clean Async implementation and the clear documentation pages. The TA folks were open to explore the tool after I created a demo including a working pipeline running headless tests quickly in a hosted agent in Azure DevOps.

Follow the guidance

In the official documentation you can see how to do most things. What the documentation also states is to use NUnit, as a developer I would normally use XUnit as my go to but let's not be stubborn for once as the docs state there are some issues.

[Parallelizable(ParallelScope.Self)]
public class Tests : PageTest
{
    [Test]
    public async Task ShouldAdd()
    {
        int result = await Page.EvaluateAsync<int>("() => 7 + 3");
        Assert.AreEqual(10, result);
    }
}

Demos are simple...

Once the team was implementing real scenarios we instantly needed a bit more thinking around like configuration, secrets and connections to other apis. Remember we wanted to keep it fresh and clean! As a developer being very comfortable with the ASP.NET Core setup the answer to some of those issues are obvious.

Configuration

The older projects were based on .NET Framework and were still using the System.Configuration.Configurationmanager. To quickly get configuration values within the test class we used ConfigurationBuilder.

var configuration = new ConfigurationBuilder()
    .AddJsonFile("appsettings.json") //make sure the file is copied to output
    .AddEnvironmentVariables()
    // Add any other configuration sources
    .Build();

Secrets

In any real test you are problably using different accounts to login per test case. We really did not want any secrets like passwords in configuration files and although you can use secret vars in pipelines it did not solve it for local development. Also here we implemented the same solution as in our 'normal' dev work. This means Azure Key Vault, not only that but in combination with Managed Identity. If you are logged in with the Azure Cli and have access to the Key Vault the following example magically works.

using Azure.Identity;
using Azure.Security.KeyVault.Secrets;
// ---
var keyVaultUri = new Uri(configuration["keyvaultUrl"]);
var tokenCredential = new DefaultAzureCredential();
var secretClient = new SecretClient(keyVaultUri, tokenCredential);

var secret = await secretClient.GetSecretAsync("SomeSecretName");

Page Object Models

For test automation it is pretty common to use the POM structure to separate and group interaction per page. This means that you create a class per page to hide implementation from your actual unit test and with that also create some reusability. This can have as effect that the test code will be littered with new statements passing in everything needed inside the POM.

Dependencies

Yes you called it, shouldn't we do some kind of dependency management here? If we would want to keep this clean we want to just get the models, configuration and things like the Secret Client when we need them.

We were going for test that look a bit like this:

[Test]
public async Task CheckPlatforms()
{
    await Page.GotoAsync(Configuration["BaseUrl"]);
    var homePage = GetService<HomePage>();

    await homePage.OpenPlatformPopup();
    var platFormPopUp = GetService<PlatformPopUp>();

    Assert.IsTrue(await platFormPopUp.HasSpotify());
}

There are many ways to set this up, the following example creates a base class called TestStartup that inherits PageTest so that the actual test class can inherit TestStartup instead of PageTest. It exposes IConfiguration and a GetService method, the Services (POMs and other dependencies) are added on a ServiceCollection inside the TestStartup class.

TestStartup.cs

public class TestStartUp : PageTest
{
    private IServiceProvider _serviceProvider;
    private IConfiguration _configuration;
    private TokenCredential _tokenCredential;

    public TestStartUp()
    {
        _configuration = BuildConfiguration();
        _serviceProvider = BuildServices();
        _tokenCredential = new DefaultAzureCredential();
    }

    private IConfiguration BuildConfiguration()
    {
        var builder = new ConfigurationBuilder()
           .AddJsonFile("appsettings.json")
           .AddJsonFile("appsettings.dev.json", optional: true);
        return builder.Build();
    }

    private IServiceProvider BuildServices()
    {
        var services = new ServiceCollection();
        services.AddSingleton(_configuration);
        services.AddSingleton<SecretClient>(s =>
           new(new Uri(_configuration["keyvaultUrl"]), _tokenCredential));
        services.AddTransient(page => Page);
        services.AddTransient<HomePage>();
        services.AddTransient<PlatformPopUp>();

        return services.BuildServiceProvider();
    }

    public IConfiguration Configuration => _configuration;
    public T GetService<T>() => _serviceProvider.GetService<T>() ?? throw new NullReferenceException("Do not forget to register the service.");
}

Resources

This repo on GitHub hosts the example code to make this work. In the future I'll add some other implementation examples that are useful for us.
https://playwright.dev/
Betatalks #52 - Automated testing with Playwright, NUnit and Azure Pipelines.

oscarvantol.nl

Azure Functions in .NET 5 multiple output bindings

Oscar van Tol — Tue, 13 Apr 2021 23:03:30 +0000

The new Azure Functions Isolated Process Model is a bit different than the one you might be used to. If you are not familiar with the new model you should first read this: 2021-03-27 Azure Functions in .NET 5 and beyond

In the new isolated model the output binding for a simple http function is similar to the old class library model, the return value of your method is the http response. If you need additional output bindings things did change, a lot! You can no longer specify binding attributes on the arguments (like IAsyncCollector) of your function. If you want to use multiple output bindings for your function in the new model, this is also done through the return value of the function method.

Return all the output bindings

But how does this work? The idea is pretty straight forward, you create a class with properties for all the output values and decorate them with output binding attributes. Here is an example borrowed from the official Samples.

    public static class MultiOutput
    {
        [Function("MultiOutput")]
        public static MyOutputType Run([HttpTrigger(AuthorizationLevel.Anonymous, "get")] HttpRequestData req,
            FunctionContext context)
        {
            var response = req.CreateResponse(HttpStatusCode.OK);
            response.WriteString("Success!");

            string myQueueOutput = "some output";

            return new MyOutputType()
            {
                Name = myQueueOutput,
                HttpResponse = response
            };
        }
    }

    public class MyOutputType
    {
        [QueueOutput("myQueue")]
        public string Name { get; set; }

        public HttpResponseData HttpResponse { get; set; }
    }

source

Noisy

As you can see the function MultiOutput returns a type MyOutputType and the property Name is decorated with a QueueOutputAttribute. Although this works and was clear to me, I could not help but have the feeling this setup is a bit noisy. Because of that I started playing around with some of the new features of c# to see if I could create a cleaner solution. First thought I had was to use tuples as a return value but this is not a viable solution at this moment, also the result definition with the attributes would make it weird.

Next thought was RECORDS, yes the new record type that helps us get to the holy grail called immutability. Creating a record instead of a good old class has a lot of cool advantages but it does not make this problem go away. To make the record definition smaller we can use positional records, this means that in one line of code you define an immutable class with properties, backing fields and a constructor that expects all the values. This together with a target-typed new expression will do the trick, right? My next problem appeared, how to put the attribute on the property? After some Bingle magic I ended up in the language spec which had the answer, you can prefix the attribute name with property: and it will generate the attribute on the property.

End result

My proof of concept looked like this:

    public class MultiOutputFunction
    {
        public record MultiOutputResult(HttpResponseData Response, [property: BlobOutput("multioutput/example.txt")] string BlobContent);

        [Function(nameof(MultiOutput))]
        public async Task<MultiOutputResult> MultiOutput([HttpTrigger(AuthorizationLevel.Anonymous, "get", Route = "MultiOutput")] HttpRequestData req)
        {
            var response = req.CreateResponse();
            response.WriteString("Hello world");

            return new(response, DateTime.UtcNow.ToString());
        }
    }

source

Please be aware this is just little old me playing around with some code, it could be that the Azure Functions Team is thinking about different directions and might not recommend this approach.

oscarvantol.nl

Azure Functions in .NET 5 and beyond

Oscar van Tol — Sat, 27 Mar 2021 19:20:41 +0000

In November of 2020 .NET 5 got released, the promise of "one-dotnet". The 'core' addition has been dropped and Full Framework evolution has stopped at the 4.8 version. This does not mean Full Framework support ends, this is tied to the support of windows therefore you can expect an eternity of patches. But when you are building something new now you should use .NET 5 or .NET core 3.1 because of its long term support.

Right from the release of .NET 5 we could start using it in for example Azure App Service but for Azure Functions this was not the case. As a heavy user of Azure Functions this topic had my interest and I followed the development a bit and in this post you can read what I got away from this.

Why wasn't it there already?

The reason that this is a bit more challenging is because the way the .NET model for Azure Function is implemented. The functions that you create will build a library that is loaded dynamically and hosted inside a running process. That means that that hosted process determines what version of .NET running. Azure Functions V1 was running .NET Full Framework and from V2 this was .NET core. The design of Azure Functions originates from Azure WebJobs and as you can imagine these libraries have some history by now. This makes it a lot of effort to get every next version of .NET to work, but also to get everything on the Azure side working side by side with a lot of different versions of .NET.

What's different?

By the end of 2020 the Azure Functions team reported that they would start building a new model for .NET, the "out-of-process" or "isolated" model. This model means that the application you build is no longer loaded inside the host but it is running by itself and communicating with the Functions runtime. This model is similar to the way all other (non .NET) languages run in Azure Functions.

What's new?

Early March 2021 this article appeared and we got a 1.0 version of the new "isolated process" model, the classic model is now referred to as the class library model.

The loose coupling means less risk on dependency conflicts that you might have ran into in the past. The complete separation would also mean that you could start using future preview releases. The startup of your application is similar to the model in ASP.NET using the hostbuilder where you can handle your configuration, dependency injection and almost identical middleware support.

What's the catch?

It will definitely take some time to upgrade your functions, the leap might be even bigger than going from v1 to v2. The new model makes a lot of sense, it feels clean and light weight but is also not as mature. Although the triggers that we know are mostly available, in the current version some features like rich types are not available yet. The main branch is moving pretty fast and apis are being opened up for all kinds extensibility, be sure to monitor that.

If we are talking about missing features, the elephant in the room is most definitely Durable Functions. The design of DF rely on the tight coupling that we no longer have. Other languages also have an implementation of DF and is on the roadmap to get this back but this will take some time. In November there is a LTS release (.NET 6) where the class library model still needs to be supported. After the .NET 6 release the team will focus on getting DF in by .NET 7.

What should I do?

The new model is the future of Azure Functions for .NET and I would recommend getting some experience with it, but be aware it is not ready yet for all workloads. Early adopters and teams that thrive on the edge of chaos will be fine with juggling different models at the same time.
But if you and your team tend to move a bit more carefully or if you rely on Durable Functions don't worry, you still have years of support ahead going from .NET core 3.1 to .NET 6.

Show me some code!

The new model is a rewrite so you won't be needing to take on the known dependencies for Azure Functions. If you look at the .csproj of a new Azure Function project you will see multiple small dependencies starting with Microsoft.Azure.Functions.Worker.

  <ItemGroup>
    <PackageReference Include="Microsoft.Azure.Functions.Worker" Version="1.0.0" />
    <PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.CosmosDB" Version="3.0.9" />
    <PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.EventGrid" Version="2.1.0" />
    <PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.EventHubs" Version="4.2.0" />
    <PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.Http" Version="3.0.12" />
    <PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.Kafka" Version="3.2.1" />
    <PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.RabbitMQ" Version="1.0.0-beta" />
    <PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.ServiceBus" Version="4.2.1" />
    <PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.SignalRService" Version="1.2.2" />
    <PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.Storage" Version="4.0.4" />
    <PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.Timer" Version="4.0.1" />
    <PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.Warmup" Version="4.0.1" />
    <PackageReference Include="Microsoft.Azure.Functions.Worker.Sdk" Version="1.0.1" OutputItemType="Analyzer" />
    <PackageReference Include="System.Net.NameResolution" Version="4.3.0" />
  </ItemGroup>

The startup

The startup with control over configuration and dependency injection was a late addition to the old model, next to that it was not mandatory. If you were using this setup it would look something like this:

[assembly: FunctionsStartup(typeof(StartUp))]
namespace MyFunctions
{
    public class StartUp : FunctionsStartup
    {
        public override void ConfigureAppConfiguration(IFunctionsConfigurationBuilder builder)
        {
            //Add Azure AppConfiguration
            //
        }

        public override void Configure(IFunctionsHostBuilder builder)
        {
           builder.Services
                AddSuffHere();          
        }
    }
}

In the new model you need a main entry point where you need to create a HostBuilder, because it is c#9 you can even do it in a top level statement.

using ExampleFunction;
using Microsoft.Azure.Functions.Worker.Configuration;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Hosting;

var host = new HostBuilder()
    .ConfigureAppConfiguration(c =>
    {
        c.AddCommandLine(args);
        //c.AddEnvironmentVariables();
    })
    .ConfigureFunctionsWorkerDefaults(app =>
    {
        //app.UseMiddleware<ExampleMiddleware>();
    })
    .ConfigureServices(s =>
    {
        //s.AddSuffHere();
    })
    .Build();

await host.RunAsync();

The function

The functions themselves seem pretty familiar but they definitly have some differences. First thing you probably won't notice is that the attribute on the function is [Function] and not [FunctionName].
Next thing is that you cannot just inject in the function ILogger anymore, it is available through an optional FunctionContext.

        [Function(nameof(HttpFunction1))]
        public async Task<string> HttpFunction1([HttpTrigger(AuthorizationLevel.Anonymous, "get", "post", Route = "test")] HttpRequestData req, FunctionContext functionContext)
        {
            var log = functionContext.GetLogger<HttpFunction>();            
            log.LogInformation("You called the trigger!");

            return "Hello world";
        }

The biggest change is the way output bindings are changed and not clear from the example above, if you want to use multiple outputs you need to define them in a class and use it a a return type. This is illustrated in the example below that I have borrowed from the official repository.

    public static class Function3
    {
        [Function("Function3")]
        public static MyOutputType Run([HttpTrigger(AuthorizationLevel.Anonymous, "get", "post", Route = null)] HttpRequestData req,
            FunctionContext context)
        {
            var response = req.CreateResponse(HttpStatusCode.OK);
            response.WriteString("Success!");

            return new MyOutputType()
            {
                Name = "some name",
                HttpResponse = response
            };
        }
    }

    public class MyOutputType
    {
        [QueueOutput("functionstesting2", Connection = "AzureWebJobsStorage")]
        public string Name { get; set; }

        public HttpResponseData HttpResponse { get; set; }
    }

Getting started yourself

If you want to get your hands dirty, go to this readme on GitHub. In the same repository you can find a lot of Samples that the team created.

Next to that you can also check out my examples.

Be sure to check the Known issues prevent frustration.

As things evolve I will post updates here.

Package updates:

Microsoft.Azure.Functions.Worker => 1.1.0
Microsoft.Azure.Functions.Worker.Sdk => 1.0.2

oscarvantol.nl

The state of the cloud: serverless

Oscar van Tol — Thu, 24 Sep 2020 21:03:03 +0000

It looks like the world is changing faster than ever before, disrupting start-ups come from nowhere and challenge big companies. Technology provides connectivity as never before and the possibilities seem endless. This year the world got challenged like never before and we are using technology to overcome issues in our daily life that would not have been possible 10 or even 5 years ago. In the past decade I have talked to a lot of development teams, CTOs and decision makers about the cloud, mainly Azure. From just grasping the concepts to helping architect specific solutions. If I look back, the term that was defining was 'As A Service'.

As a service

Most of the discussions were about scale and cost control, for start-ups it was about not having to make investments upfront. In the end it is about not owning a big part of the stack. Not owning something also meant having a different level of control. To choose your comfort zone we always had a good analogy, Pizza as a Service. Still today I(nfrastructure)aaS, P(latform)aaS and S(oftware)aaS exist side by side for a reason. With IaaS you still get the maximum control over your environments, with PaaS you get to focus on your own code and with SaaS you are just a consumer.

Evolution

Within those groups there has been a lot of evolution though. The most difficult to place for me are Containers, I would arguably place them as the pinnacle of IaaS because of the amount of control you have. Platform as a Service was always about not caring about anything but your own code, providing out of the box scale and cost control options. The ultimate version of this is SERVERLESS. So is this just PaaS rebranded? Well, there are definitely some services that have been around for some years but suddenly showed up in lists of Serverless offerings. Some of these services did get some new tiers that would classify better. We can discuss that Serverless is a horrible name but most offerings will scale back to zero hosts and cost nothing for those seconds. Most people would associate the term with compute and F(unctions)aaS but we should also count several storage and database solutions as Serverless.

Cold start

Every application needs some start-up time and the so called 'cold-start' is nothing new. After deploying or even after a certain idle time most applications need to spin up for a short amount of time. Also when you are scaling out the new hosts need time to become responsive. So why is this different? As mentioned before in Serverless you will go back to zero hosts if there are no requests. The idea is that you run a tiny service on a part of infrastructure that is assigned on the fly and not booked ahead. For asynchronous tasks this won't be a big issue but if it is a user doing an http call a 10 second wait is not the best experience. Serverless offerings are adding all kinds of control and tricks to overcome this issue like predictive scaling and giving the owner control to have a minimum amount of hosts running. Is this still Serverless then? The real answer is: Do you care? The real power of Serverless compute is in something else.

Vendor lock in

This power that I mentioned is also an interesting subject. The power is in abstraction, everything you should not care too much about is not in the way. But this is 1-to-1 connected to the heavily discussed 'drawback' of Serverless that is called vendor-lockin. This means that if you start using serverless you will be heavily tied into the specific cloud service and the SDK's. But these abstractions that they provide actually help you to focus on your code, you don't have to write listeners or pull in all kinds of packages to make something work. Your code starts exactly where you are adding value, right after the framework gave you an event with all the context you need. The logic that you write is still yours and of course moving to a different cloud will be some work but all the stuff that creates the 'vendor lock in' you did not write and you do not have to maintain!

Now what?

Serverless is here to stay, we will definitely see more evolution and maybe different naming in the future. It is not that you should start using Serverless for everything but it is a mature option to consider. Picking the right tool for the job is really important, this means that you should have multiple tools in your belt and you should know how they work. Thanks to the cloud we moved far away from having just a database and a webserver as tools to create.

Resources:

For #ServerlessSeptember we created and collected a lot of content. It can be found here.

oscarvantol.nl

Azure App Configuration and Azure Functions

Oscar van Tol — Wed, 02 Sep 2020 21:41:48 +0000

Azure App configuration is an amazing service that we have been using in our project to centralize our configuration. If you are unfamiliar with the service I suggest reading Azure App Configuration: an introduction by @rickvdbosch.

We were really happy to move our apis to App Configuration in combination with Azure Key Vault references, this meant that we centralized our configuration, removed duplication and pushed all the secrets to Azure Key Vault. Next to that we have the extra benefit of the other features that it's offering like feature flags.

Sounds simple, let's do it!

The logical next step after updating the configuration setup for all the apis was to do the same for our Azure Functions. We have a lot of them and they contain a lot of configuration, but an Azure Function is not the same as an Asp.NET core api. This is where the journey starts for today.

The first step was to fire up google and search around for Azure App Configuration in combination with Azure Functions, this led me to the official Microsoft documentation.
The documentation shows following example:

private static IConfiguration Configuration { set; get; }

static Function1()
{
    var builder = new ConfigurationBuilder();
    builder.AddAzureAppConfiguration(Environment.GetEnvironmentVariable("ConnectionString"));
    Configuration = builder.Build();
}

Meh...

Okay, this will work but it doesn't feel as slick as the setup that I had in the apis. I really wanted a setup the is similar to the one in the webapps. I had a lot of functions to upgrade and they all are already depending on either an existing IConfiguration implementation or the IOptions pattern.

FunctionsStartup

In Azure Functions you cannot really get into the HostBuilder setup like in a webapp, the closest thing available is implementing a 'FunctionsStartup'. Well in most of the functions I already had that available because we are setting up our dependency injection here. If you want more background on this topic check the documentation or my examples on GitHub.

The biggest issue is that the configuration is already setup by the time the code in the 'Configure' method is called. My plan was to create a new configuration and replace it in the service collection.

    var configBuilder = new ConfigurationBuilder();
    // Add env vars to the new config
    configBuilder.AddEnvironmentVariables();

    // Replace IConfiguration
    builder.Services.RemoveAll<IConfiguration>();
    builder.Services.AddSingleton<IConfiguration>(configBuilder.Build());

Update: After publishing the initial version of this post I got a tip from Anthony Chu to use IFunctionsConfigurationBuilder instead. This new recommended way that I somehow missed in my quest makes this so much more clean! To be able to use this you need at least version 1.1.0-preview1 of the Microsoft.Azure.Functions.Extensions package installed. Since I have absolutely no fear of preview packages that is not a problem.

A bit of tweaking and testing later I ended up with the following implementation. This Adds Azure App Configuration and Azure Key Vault to be able to leverage the Key Vault References. What you can also see in here that it only requires an endpoint to App Configuration because we are using Azure Identity's Default Credential to authentication. You can also use a connection string, but the objective was to remove the secrets from configuration.

using System;
using Azure.Identity;
using Microsoft.Azure.Functions.Extensions.DependencyInjection;
using Microsoft.Extensions.Azure;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.DependencyInjection.Extensions;

[assembly: FunctionsStartup(typeof(AppConfigurationExample.StartUp))]
namespace AppConfigurationExample
{
    public class StartUp : FunctionsStartup
    {
        public override void ConfigureAppConfiguration(IFunctionsConfigurationBuilder builder)
        {
            //get the original configuration
            var tmpConfig = builder.ConfigurationBuilder.Build();

            // create a new configurationbuilder and add appconfiguration
            builder.ConfigurationBuilder.AddAzureAppConfiguration((options) =>
            {
                var defaultAzureCredential = GetDefaultAzureCredential();

                options.Connect(new Uri(tmpConfig["AppConfigUrl"]), defaultAzureCredential)
                // also setup key vault for key vault references
                    .ConfigureKeyVault(kvOptions =>
                    {
                        kvOptions.SetCredential(defaultAzureCredential);
                    });

                // configure appconfiguation features you want;
                // options.UseFeatureFlags();
                // options.Select(KeyFilter.Any, LabelFilter.Null);
            });

        }

        public override void Configure(IFunctionsHostBuilder builder)
        {
            // Setup DI
            //builder.Services.AddScoped<>();
            //builder.Services.AddTransient<>();
            //builder.Services.AddSingleton<>();
        }

        private DefaultAzureCredential GetDefaultAzureCredential() => new DefaultAzureCredential(new DefaultAzureCredentialOptions
        {
            //be explicit about this to prevent frustration
            ExcludeManagedIdentityCredential = false,
            ExcludeAzureCliCredential = false,
            ExcludeSharedTokenCacheCredential = true,
            ExcludeVisualStudioCodeCredential = true,
            ExcludeInteractiveBrowserCredential = true,
            ExcludeEnvironmentCredential = true,
            ExcludeVisualStudioCredential = true
        });

    }
}

Consuming the configuration

This part was already implemented in my functions, so I didn't have to do anything here. One option is to inject IConfiguration in a non status class containing a function like here:

    public class Function1
    {
        readonly IConfiguration _configuration;

        public Function1(IConfiguration configuration)
        {
           _configuration = configuration;
        }

        [FunctionName("Function1")]
        public async Task<string> Run([HttpTrigger(AuthorizationLevel.Function, "get", Route = null)] HttpRequest req)
        {
            return _configuration["greetingsconfigkey"];
        }
    }

The better way would be to use the IOptions pattern as shown in the next example:

Inside the Configure method in StartUp

        builder.Services.AddOptions<ExampleSettingsConfig>()
         .Configure<IConfiguration>((configSection, configuration) =>
         {
                configuration.GetSection("ExampleTestSettings").Bind(configSection);
         });

The Function constructor

        public Function1(IOptions<ExampleSettingsConfig> exampleOptions)
        {
            _config = exampleOptions.Value;
        }

But there is a warning...
This implementation has some limitations described here. The configuration needed for the binding triggers cannot be moved to App Configuration because it is used outside your Azure Function code by the platform. If you would have a Service Bus trigger, the scale controller that will determine if it needs to spin up hosts for your functions needs access to the queue or subscription you are listening to. I am sure this has the attention of the team responsible, but I can imagine it is not an easy problem to solve.

What else did we get?

Next to the details options you have in the Azure App Configuration setup like setting up label filters you also get the Feature Flags feature. You can enable this by calling 'options.UseFeatureFlags()' in the setup.

Sharing is caring

Example code
I added a simple example implementation on GitHub for anyone to do whatever they want with it. You can find it here.

oscarvantol.nl

Infrastructure as code on Azure

Oscar van Tol — Wed, 29 Apr 2020 19:12:09 +0000

From the moment a developer writes a piece of code to support a new feature or fix a bug, a journey starts. This journey will hopefully start with some sort of commit and push into a source control system. The end goal of the journey is pretty clear, production (whatever that is in the context). What happens in between can be short and simple or long and complicated, the time the journey takes will mostly be determined by the quality assurance process and the level of automation of the CI/CD pipeline.

Build and Release

These days it is pretty common for teams to have their builds automated and triggered on a new commit. In this first CI step it is also pretty common to run some form of (unit)testing. The next step is the release process, in most cases this process will take a build and deploys it to an environment whether it is a qa environment, a ring, canary or straight to production.

But where is it releasing to? It used to be a metal box but these days it is more likely a virtual machine, container or some hip cloud service. Because the deployment targets used to be more static and stateful, their creation is often not automated.
If you cannot recreate your environment with its configuration with up to date documentation or automation you are suffering from technical debt that you should not underestimate!

Keeping documentation perfectly up to date

Based on empirical evidence, there will be a remote possibility that this will happen. And if it would, it is still extremely inefficient.

// TODO: write some documentation
// TODO: Remove this code after 2009-01-01

Infrastructure as code

Creating your resources automatically will pay off real quickly. Let's say you dream up some new piece of code that uses some storage service and web service in Azure, it is only a few clicks to create these in the Azure Portal. But you will probably do this in a safe development environment were you have all the access you can dream of. Your two new services and their configuration will also be deployed in one or more test environments and the production environment. Scripting and parameterizing the creation of these resources them will make you more efficient fast. An extra benefit is that your scripts can be reviewed and that issues with for example naming convention can be spotted early on. If you treat you scripts as code and store them in a version control system you have perfect historical record of how a resource was created.

Great let's do this! Tell me how!

Like always, there is more than one way to accomplish this. Azure exposes apis to manipulate resources, there are a lot of tools that can talk to these apis. For now let's focus on the options closest to Azure itself that can fairly easily be triggered in an Azure Pipeline.

The following three options contain an example that is taken from the official Azure Documentation pages. These examples show the creation an Azure Web App with a Service Plan.

ARM Templates

Maybe the most common way to do 'Infrastructure As Code' in Azure is to use ARM Templates. ARM stands for Azure Resource Manager and has been the standard set of apis for some years now, the services that are referred to as 'Classic' in Azure are controlled with the ASM (Azure Service Management) api.

ARM Templates are a JSON representation of one or more resources in Azure. In a single template you can define a complex set of resources and their dependencies within a resource group. If you are in the Azure portal you can go to any resource and export a ARM template, this sounds like the way to start doesn't it? Even though it sounds like a perfect and simple solution this will generate a complex script that is a lot better readable by a machine than by a human.
If you look at the example below it might be readable enough, but for me something like this is hard write from scratch without too much experience.

{
  "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "webAppName": {
      "type": "string",
      "metadata": {
        "description": "Base name of the resource such as web app name and app service plan"
      },
      "minLength": 2
    },
    "sku":{
      "type": "string",
      "defaultValue" : "S1",
      "metadata": {
        "description": "The SKU of App Service Plan, by default is Standard S1"
      }
    },
    "location": {
      "type": "string",
      "defaultValue": "[resourceGroup().location]",
      "metadata": {
        "description": "Location for all resources"
      }
    }
  },
  "variables": {
    "webAppPortalName": "[concat(parameters('webAppName'), '-webapp')]",
    "appServicePlanName": "[concat('AppServicePlan-', parameters('webAppName'))]"
  },
  "resources": [
    {
      "apiVersion": "2018-02-01",
      "type": "Microsoft.Web/serverfarms",
      "kind": "app",
      "name": "[variables('appServicePlanName')]",
      "location": "[parameters('location')]",
      "properties": {},
      "dependsOn": [],
      "sku": {
        "name": "[parameters('sku')]"
      }
    },
    {
      "apiVersion": "2018-11-01",
      "type": "Microsoft.Web/sites",
      "kind": "app",
      "name": "[variables('webAppPortalName')]",
      "location": "[parameters('location')]",
      "properties": {
        "serverFarmId": "[resourceId('Microsoft.Web/serverfarms', variables('appServicePlanName'))]"
      },
      "dependsOn": [
        "[resourceId('Microsoft.Web/serverfarms', variables('appServicePlanName'))]"
      ]
    }
  ]
}

Azure CLI scripts

Command line tools everywhere have gained popularity in recent years, so has the Azure CLI. The Azure CLI is basically a command line version of the Azure Portal with good feature discoverability due to the inline help functionality. Because you can script the use of a command line tool in PowerShell or bash you can create parameterized automation scripts that you can store in source control and run in CD pipelines. The flow of creating resources will be similar to the steps you would normally take to create something in the portal.

az group create -l westeurope -n MyResourceGroup
az appservice plan create -g MyResourceGroup -n MyPlan -l westeurope
az webapp create -g MyResourceGroup -p MyPlan -n MyUniqueAppName

Azure Management Libraries for .NET

Normally you would see this library inside the parts of your application that needs to control infrastructure in Azure. For example, if you have a multi tenant SAAS application that has SQL databases for every tenant you can use the libraries to create a new db on a new customer signup. Maybe you have some really specific scaling requirements that you can automate from within your application.
But if the developers are comfortable writing C# all day this might also be a fine way to script/code the creation of resources. With the Azure Management Libraries you can define services using a fluent api.

IAzure azure = Azure.Authenticate(credFile).WithDefaultSubscription();

var webApp = azure.WebApps.Define(appName)
    .WithRegion(Region.USWest)
    .WithNewResourceGroup(rgName)
    .WithNewFreeAppServicePlan()
    .Create();

Making a choice

The most important part is embracing the fact that it is important to commit the definition of the infrastructure your code needs to run on into a source control system. It still seems like the official recommendation is to use ARM templates. The strong part of the ARM templates is that you do not define a script but you define a state that you want your resources in, this means that you have reproducible behaviour no matter of the state you start with.
The Azure CLI commands are simple to write from scratch, but you are not defining a complete state like with ARM templates so you might need to script out a bit of flow. Most of the commands are idempotent and will not throw errors on recreation of a resource, but I found out the hard way that some of the commands still have some unexpected behaviour.
Using the Azure Management Libraries provides maximum control in complex setups and they certainly have their place but you are building yet another application instead of just a script.

Useful links:

Writing an Azure DevOps Pipeline extension

Oscar van Tol — Sat, 29 Feb 2020 19:56:03 +0000

blogged: 2020.02.20

How I ended up having a Saturday night date with the Azure DevOps marketplace.

'At some moment we should really fix this!' Is this something you and your team sometimes say? We had one of those, we are building really cool software at Virtual Vaults using Dotnet Core, Angular and all the Azure toys. We use Azure DevOps and have a lot of common NuGet packages that we share internally using the Artefacts Feed.
Everyone who updates a package sometimes will at one point forget to bump the version before pushing and we will have a discussion about how we should branch and version the packages. But in the end we will complete the task at hand and move on. Only until the next time.

Enough, let's do this

At one point I was sick of this. "let's settle this once and for all". After some quick conversations with strong opinioned team members we decided what would be a good strategy. We wanted to have the automated builds pushing out packages without bumping the version by hand. Also if the CI build would be triggered from any other branch than master, the version should be marked as pre-release. This sounded pretty straight forward but I could not make it work with any of the standard options I had in the pipeline. I found a lot of extensions in the marketplace that helped me a bit but the yaml pipeline file quickly became a large messy script. Being a developer I decided to write my own extension for a custom build task.

Building an extension

Because I created a simple build task before I knew it would be pretty straight forward. This was both true and untrue, the documentation is really good and I had my basic logic running very quickly using some typescript (which is awesome). But to get the thing properly packaged including what was needed from the node_modules folder had me googling for a while. Lots of people writing extensions have their code on GitHub and this helped me figure out what I was doing wrong.

Using the extension

The next step was uploading the extension to the marketplace and share it privately with my organisation. I was ready to start testing it in the context of a build. I struggled a bit with setting up the permissions correctly because I needed to use the DevOps api inside the task but it worked like designed and I had an example pipeline running.

Finally I added the custom task to the pipeline of the package I was actually working on during 'normal hours' and told everyone to have fun with it.

Done, or maybe not?

Noooooo! Not done, I needed to write some documentation on our wiki or at least some comments in the yaml file because otherwise this will definitely get lost as some point. And were to store the code for the extension. I thought to myself I am doing this anyway, let's do it right! So I pushed to sourcecode to GitHub and wrote some minimal documentation. Then I wanted to make the extension publicly available on the marketplace. This meant some more reading and adding information and metadata to my profile and the package. Making this public also made me add some extra documentation, a yaml code example showing how to use the extension and last but not least, an icon.

What was the benefit of adding these extra steps?

I forced myself to write some documentation and clean up.
This might be of help for someone else writing an extension.
I might help my future self in need of something similar.
Someone else might actually use my extension😱

oscarvantol.nl

Orchestrating your incoming traffic like the big boys

Oscar van Tol — Mon, 30 Sep 2019 20:19:02 +0000

blogged: 2019.09.30

Creating a large application and exposing it to the web has a lot of challenges. You will have to think about performance, security, reliability. You are probably not Netflix but you might have an application that is:

composed out of different (micro)services
highly available and fast from anywhere on the globe
needs specific routing rules for something (a-b testing?)

Azure Front Door Service

There are a lot of different solutions that might solve your routing puzzle, but in this blog we will dive into the capabilities of "Azure Front Door Service" (from here FDS because I'm lazy).

FDS is a global service takes care of routing and monitoring your HTTP traffic. FDS is a nice name because it is a service that provides an entry point for your web application. It actually provides you multiple entry points, they are located all over the world and from the nearest 'point of presence' the route to your app will be determined by FDS.

The best way of exploring new stuff for me is diving straight in. For most Azure services it is really easy and low-cost to try something out and deleting it afterwards. For this blog I did just that and created an FDS and while making some screenshots along the way.

Setting up FDS

If you create your FDS you will have to select a resource group and a region to create it, like any other resource. But the actual service that is hosting the endpoints is running on different pops around the world. The setup of the service consists out of three parts that you will have to configure.

Frontend hosts

Here you can define your entry points. A frontend host has a host name (something.azurefd.net). You can if you want session affinity here and if you want to add a WAF (Web application firewall) by adding a WAF policy that you have defined.

Backend pools

You will have to create one or more Backend pools. In these pools you will add 'Backends', these backends are your web applications that you want to expose. I know what you are thinking and yes, from your applications perspective these are frontends. A configuration of a backend pool contains a definition for health probes. To be able to keep the promise of all the cool routing and failover features the pool needs to know if a 'Backend' is up and running. Per backend you can also define 'weight' if you want to give some backends more traffic then others. This might be useful if you are doing A-B testing or if you have canary releases.

Routing rules

To tie it all together you need at least one routing rule. If the service gets a request on a frontend host it has to match it to a pattern, the simplest catch all (/*) is a mandatory rule to have. In these rules you simply bind an incoming url path to one of your backend pools. You can basically to two things, forward or redirect. With these rules you can also define what incoming protocols to accept (http or https) and with what protocol to route to the backend.

Features

You can add a Custom Domain to the service as you would expect. After you have created the front door you need to click the 'plus sign' again in frontend hosts.

A feature that in my opinion should be present in all the Azure services that have a custom domain option is certificate management. Next to uploading your own you can have one created and rotated by FDS.

FDS supports end-to-end IPv6 connectivity and by supporting HTTP/2 it can reduce a lot of traffic overhead hitting your backends.

With the caching option that you can setup per routing rule you control what is cached and you will be able to serve some of your content to the user directly from the pop without hitting any backend.

Possible Scenarios

There are multiple scenarios that might benefit from a service like FDS. Most of them match an app that has a serious scale and/or budget.

Improving performance by serving from cache at the endpoints
Improving performance by reducing direct hits on your hosts
Exposing multiple apis and frontends on different paths of a single domain
Routing traffic to the nearest hosts
Failover (to different other region)
A-B testing or canary releases
All of the above!

Is this for you?

This is a service that you can use for big and complex applications with a global presence, considering this the price isn't too bad. But this might be a bit overkill for your side project. Please check out the pricing page to prevent big surprises.

If you think you need this service be sure to check out the official docs for all the details.

oscarvantol.nl

You are only paying a small part of your dev team

Oscar van Tol — Fri, 30 Aug 2019 19:19:04 +0000

The software your team builds relies on a lot of code that is written by people outside your organisation. Are you paying them or thanking them even?

Building software changes all the time, for me it started with writing desktop apps that you would put on a floppy for someone to use. More than 20 years later I am working with a team at Virtual Vaults building SAAS by using PAAS and all the serverless goodies in Azure. Not only infrastructure
and the delivery to the end user changed, but also the way we are able to continuously create, test and ship features with multiple people working closely together.

A big part of being able to build complex systems fast is because we can build on top of layers of abstraction.
These days you just don’t build everything from scratch, you use services, packages. If you would look at your entire stack,
ever wonder how little code that is executed, is actually yours? And how much of the external code is a library that you are paying for?
I am betting that just like the software I am working on, most of the external software is pulled into your project as a package via NuGet or NPM.
The majority of these packages have their source code in a Github project and have an open source license.

So the foundations of the software we are making money with is made available for free. Build something for free, who would do that?

Well, there are a few different groups doing this. First of all, enthusiast that started something just to learn or make their life a bit more easy. Some of them grow their career because of the project and become trainers, speakers or specialized consultants. We also see big tech companies like Google and Microsoft pushing a lot of open source, in a lot of cases it will connect people to their platform making it a solid strategy. But what about all the other businesses that build software, are we also giving back instead of only consuming and should we?

Yes we should, and some do! But how can we do that, not everyone at the office will be amused if we would fill our day with hobby projects and we certainly cannot just publish all our company's source code to the world. There are however a few options to balance it out.

The first one is obvious, when finding an issue or missing a feature in something you are using and you are able to fix it,
do it and make the pull request, don’t build your internal version and keep it to yourself. In this case there is a fair chance that your fix or addition is properly reviewed (for free). In the case the fix is not pulled in (fast enough) you can fork the project and pull it in there.
Another option, and maybe the one that is the most comfortable one starting with is documentation. A lot of projects also accept changes in their docs, from simple things like correcting spelling or grammar to adding some examples. This is a valuable way of contributing to the community and you also get to benefit yourself if you need the documentation later. After all, we cannot always remember every little detail. If your team lacks experience contributing to open source it's easy to start here.
You can also create a blogpost where you write down how you used some tech in your scenario or describe the process you went through to make it work at all. This might help some other poor soul out there and it can also be useful for new team members or even for your future self.
One of the more challenging options is creating and maintaining projects that you use specifically in your software.
This will raise some questions, after all someone is paying the bill for this coding time, "is this not our intellectual property?".
If it is something you need to create anyway, it is generic enough that other people might be interested in to use and contribute back into, it could be really interesting. Keep in mind that in this scenario you really need awareness and full support from your company.
You can also pay back in the simple form of a donation. If your organisation heavily relies on a particular project and you really want it to stay healthy it might be wise to donate to the project. If this sounds interesting you should check out Github sponsors.

I wrote this because I believe that not everyone in (tech)companies is aware that their development teams and their products are powered by the vast amount of communities and heroes working on pet projects. If we increase contributing and sharing in our cultures I believe we can achieve even more.

* Note: There is a thing to be very aware of, don't let your project become a security issue. If you share or contribute to an open source project you are also letting the world know that you are using it. Make sure nothing gets into the libraries you use that you do not want in there! But you are already checking that with everything you are using, right?

oscarvantol.nl