DEV Community: Sam Vanhoutte

Using the Options pattern with .NET Aspire

Sam Vanhoutte — Mon, 14 Apr 2025 06:58:16 +0000

When using .NET Aspire Hosted App, most samples use Environment Variables when configuring your services. However, when you are migrating existing services, the chance is that you are using the Options pattern for .NET Core.

So, I decided to write this post in order to share how I achieved this.

Using Environment variables

The default approach seems to be the following.

The AppHost

In the AppHost, you are just adding the service (or project, as they call it in Aspire) and you can pass environment variables to the services.

var builder = DistributedApplication.CreateBuilder(args);

builder
    .AddProject<MySolution_Service1>("worker")
    .WithEnvironment("config1", "value1")
    .WithEnvironment("config2", "value2");

await builder.Build().RunAsync();

The Service

In the Service project, you can easily access the environment variables, by using this code.

Console.WriteLine(Environment.GetEnvironmentVariable("config1"));

Or you can use the Configuration builder for it.

builder.Configuration.GetValue<string>("config1");

Using one centralized approach for Options pattern

In order to have the default AppSettings shared from the Hosting project, and made available with the Micro services projects, we define an appsettings.base.json file in the Hosting project and have that automatically copied to all the services that need it. Every service can then still define an appsettings.json file to override or add extra configuration items per service.

The actual file reference

Define a file appsettings.base.json in the Hosting project.
Update the services project(s) files to indicate files should be copied from the hosting project to the service project itself.

<Target Name="CopySharedAppSettings" AfterTargets="Build">
    <!-- Define the source files -->
    <ItemGroup>
        <SharedFiles Include="..\MyApp.Host\appsettings.base.json" />
    </ItemGroup>

    <!-- Copy files to the output directory -->
    <Copy SourceFiles="@(SharedFiles)" DestinationFolder="$(ProjectDir)" />
</Target>
<ItemGroup>
    <Content Update="appsettings.base.json">
        <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
    </Content>
</ItemGroup>

Injecting the IOptions

In .NET Aspire, runtime configuration is typically done, through the Service Defaults. In this project, we will be adding the configuration method to inject all the relevant (common) Options into the services. Obviously, this can also be done by performing this in the services project itself.

As you can see, we are first loading the base file, and after that, the overrides from the service specific file. And, as this mainly focuses on the dev/local experience (with the appsettings file), we also inject the Environment variables, so that we can also leverage the deployment configuration when deployed onto a cluster.

private static TBuilder ConfigureOptions<TBuilder>(this TBuilder builder) where TBuilder : IHostApplicationBuilder
{
    var cfgBuilder = new ConfigurationBuilder()
        .SetBasePath(Directory.GetCurrentDirectory());

    cfgBuilder
        .AddJsonFile($"appsettings.base.json", true, true)
        .AddJsonFile($"appsettings.json", true, true)
        .AddEnvironmentVariables();

    var configuration = cfgBuilder.Build();

    builder.Services.Configure<SqlOptions>(configuration.GetSection("sql"));
    return builder;
}

And we are calling this method from the "main method" of the Service Defaults project:

public static TBuilder AddServiceDefaults<TBuilder>(this TBuilder builder) where TBuilder : IHostApplicationBuilder
    {
        builder.ConfigureOpenTelemetry();

        builder.ConfigureOptions();
        // ... skipped for brevity
        return builder;
    }

Putting Azure APIM in front of your Service Bus queue

Sam Vanhoutte — Thu, 27 Mar 2025 19:47:26 +0000

For a specific scenario, we wanted to expose a queuing endpoint towards a customer, in a secure way, but at the same time abstracting our internal usage of Azure Service Bus.

As we already had Azure API Management in place to expose our API, we decided to leverage this and see if we could avoid the typical scenario where we'd have to develop a strong typed custom API that then just takes the incoming request and maps it to a Service Bus message that should be be processed to the right endpoint.

And this post describes exactly how you can achieve this.

Azure API Management policy

The following abstract is the full policy that we configured. So, you could see this as the "tldr;" of this blog post. In the rest of the article, we'll dive deeper in the details and discuss what's behind everything.

<policies>
    <inbound>
        <base />
        <validate-content unspecified-content-type-action="detect" max-size="262144" size-exceeded-action="detect" errors-variable-name="schema-validation-error">
            <content type="application/json" validate-as="json" action="prevent" schema-id="OrderRequest" allow-additional-properties="false" />
        </validate-content>
        <trace source="Order create">Request validated</trace>
        <authentication-managed-identity resource="https://servicebus.azure.net" output-token-variable-name="sbBearerToken" ignore-error="false" />
        <set-header name="Authorization" exists-action="override">
            <value>@{return $"Bearer {(String)context.Variables["sbBearerToken"]}";}</value>
        </set-header>
        <set-backend-service base-url="https://{{serviceBusNamespaceFQDN}}" />
        <rewrite-uri template="/orders/messages?api-version=2015-01" />
        <set-header name="MessageType" exists-action="override">
            <value>Order</value>
        </set-header>
    </inbound>
    <backend>
        <retry condition="@(new List<int>() { 408, 429, 500, 502, 503, 504 }.Contains(context.Response.StatusCode))" count="2" interval="1" max-interval="10" delta="1">
            <forward-request buffer-request-body="true" />
        </retry>
    </backend>
    <outbound>
        <base />
    </outbound>
    <on-error>
        <base />
    </on-error>
</policies>

Set up the API operation contract

As we want to have a strong typed operation, enforcing the contract, we have to define both the actual operation (http resource) as well as the incoming payload schema first.

In the following screenshot, you can see we define the resource (POST /order). We also indicate that the operation expects an instance of the definition CreateOrderRequest in the request.

The schema itself is defined in the Definitions tab (at the bottom), where you can define the schema itself, or have the schema generated from a sample instance.

Enable security and access rights on Service Bus

In order to allow the API Management service to send messages to the Azure Service Bus endpoint, it is important to assign the correct permissions. We will do this by providing the Azure Service Bus Data Sender role to the System Identity of Azure API Management.

Please verify the System Assigned Identity is enabled on API Management, by navigating to the 'Managed Identities' tab.
Take note of the Object (principal) ID.
Navigate to your Service Bus namespace in the Azure portal
Assign the right permissions in Role Assignments, as shown in the next screenshot

The result should be looking like the following

In case you use a User Assigned managed identity, the client id of the role should be stored as it will be used in the APIM policy , to authenticate against the API of Servicebus (here: 60ec8160***). (If needed.

The XML policy explained

Validating the schema

As we're not having custom code, hosted in an API to perform validation, we have to fall back on the out of the box schema validation, provided by Azure API Management. This way, we will ensure only valid messages are delivered into our messaging backend. This comes with limitations as we are not returning a 404, in case the ArticleNumber does not exist, for example. So, those exceptions have to be handled asynchronously.

<validate-content unspecified-content-type-action="detect" max-size="262144" size-exceeded-action="detect" errors-variable-name="schema-validation-error">
    <content type="application/json" validate-as="json" action="prevent" schema-id="OrderRequest" allow-additional-properties="false" />
</validate-content>

When the request is not valid, an HTTP 400 (BadRequest) will be returned to the caller of the API. We also indicate

Add tracing

The next fragment, just adds a trace that will be integrated in the APIM trace and even a connected Application Insights instance.

<trace source="Order create">Request validated</trace>

Configure managed identity authentication

The authentication-managed-identity fragment, enables the authentication of the Azure APIM managed identity (system identity in our case) against the Azure Service Bus resource. A call to the Microsoft Entra ID happens and the result is being stored in the sbBearerToken variable, to be used in the next step.

<authentication-managed-identity resource="https://servicebus.azure.net" output-token-variable-name="sbBearerToken" ignore-error="false" />

Setting the Service Bus Authorization header

In this step, we are setting (and overriding, should it exist) the Authorization header on the outgoing HTTP call that will be made.

<set-header name="Authorization" exists-action="override">
    <value>@{return $"Bearer {(String)context.Variables["sbBearerToken "]}";}</value>
</set-header>

Configure the actual Service Bus entity url

Two steps are taken to define the actual entity url for the service call.

<set-backend-service base-url="https://{{serviceBusNamespaceFQDN}}" />
<rewrite-uri template="/orders/messages?api-version=2015-01" />

We are setting the base-url of the backend service to the https address of the service bus. (the {{serviceBusNamespaceFQDN}} value is retrieved from a namedvalue, as to configure this with the IaC pipelines.

After that, we are providing the name of the topic or queue to which we want to send the incoming message. (As per following document, the 2015-01 is still the latest version).

Adding custom message properties

If you want to add custom message properties, you can do so by adding HTTP headers with the correct name and value to the HTTP request.
This is what happens in the last snippet.

<set-header name="MessageType" exists-action="override">
    <value>Order</value>
</set-header>

Setting the retry policies

For certain HTTP status codes, we are indicating a retry policy to be used. This is certainly relevant for the 429 status code (throttling), but can also be enabled for the other ones.
This snippet is to be set on the backend node.

<retry condition="@(new List<int>() { 408, 429, 500, 502, 503, 504 }.Contains(context.Response.StatusCode))" count="2" interval="1" max-interval="10" delta="1">
    <forward-request buffer-request-body="true" />
</retry>

Testing

When testing the API call from the Azure API portal, we can clearly see the message appear on our subscription (linked to the Orders topic we were sending to). We can also see the MessageType custom property in the Service Bus explorer view.

Conclusions

This approach clearly helps in directly exposing a Service Bus entity, while still abstracting that as an implementation detail and maintaing some basic validation on the incoming request.

However, that validation is somewhat limited and lacks certain synchronous validation logic (even business logic) to be executed, which is what you typically would use a custom WebAPI project for (that then would forward the validated - or enriched - request to the Service Bus.)

Processing Time Series data in Azure: the options

Sam Vanhoutte — Mon, 24 Mar 2025 20:17:58 +0000

Time series data are data points that are collected or recorded at successive points in time. This type of data is becoming increasingly important in today's digital world as more systems, sensors, and applications generate continuous streams of information.
From IoT devices and financial markets to website analytics and industrial monitoring, time-stamped data enables organizations to detect trends, predict outcomes, and respond to changes in real time. As industries strive for smarter operations and faster decision-making, the ability to efficiently collect, store, and analyze time series data has become a critical capability for modern data solutions.

When working with several companies in different industries, I came across many different options and approaches to store time series data and that's why I decided to write this blog post series.

The different options in Microsoft Azure

There are several approaches and options to process and store timeseries data in Microsoft Azure, and this article will quickly list and touch upon them.

Some technologies will get a more detailed article later on.

What is Time Series data?

Time series data is a set of events. In messaging solutions, there are several types of messages that get exchanged between systems:

Messages: these are intents. A message can be a command, a query, a request. It implies something should happen and is expected.
Events: these are facts. An event just states something has happened and can be a measurement, a report or notification.

Time series falls under that second category. However, they are not discrete events (who are independent and actionable on their own).

Time series are order in time and are typically partitioned by a given context. (for example: MeterId , WebSiteVisitorId...). So one application will have multiple time series, typically. And these can be analysable on their own, or compared to each other.

The 4 components of Time Series data

There are four main components that make up time series data.

The trend: the long term direction of time series data
Cyclical effect: the larger fluctuations around the long term variation. Typically over longer periods than seasonality and without a fixed frequency.
Seasonality: the variations that occur and repeat over fixed time periods (every year, every Sunday, etc). One time series can have multiple frequencies of seasonality.
Residual effect (also referred to as noise). Unpredictable variations without a pattern that cannot be linked to the previous components.

Storing and processing Time Series in Azure

Several possibilities to store time series in Azure are listed below. Just know that this list is not exhaustive.

Azure Data Explorer

Azure Data Explorer (ADX, aka Kusto) is a fast and highly scalable data analytics service optimized for analyzing large volumes of structured, semi-structured, and unstructured data in near real-time. It is particularly well-suited for time series data, where high ingestion rates and quick querying over large datasets are critical. ADX supports time series-specific capabilities like automatic aggregation, downsampling, and time-based joins, making it ideal for monitoring trends, detecting anomalies, and visualizing data over time.
Its powerful Kusto Query Language (KQL) enables users to extract insights efficiently, even from billions of records. With native support for high-throughput ingestion pipelines and seamless integration with other Azure services, Azure Data Explorer provides a robust and scalable solution for managing and analyzing time series data.

Azure Fabric Realtime Analytics

Azure Fabric Real-Time Analytics is a service within Microsoft Fabric, which is an all-in-one analytics platform that combines data engineering, data warehousing, real-time analytics, and B.I. It is built on top of Kusto, the engine that powers ADX, so it inherits ADX’s strengths—fast ingestion, time series support, and the Kusto Query Language (KQL). However, it adds deeper integration with the broader Fabric ecosystem, like OneLake (a unified data lake), Power BI, and other collaborative and governance tools in Microsoft 365.
This also comes with a much different pricing model, which mostly means it's more expensive.

Azure Database for PostgreSQL with TimescaleDB

PostgreSQL is an open-source relational database known for its robustness, extensibility, and support for complex queries, making it a solid foundation for time series data management. When combined with TimescaleDB, a PostgreSQL extension specifically designed for time series workloads, its capabilities are significantly enhanced. TimescaleDB adds features like automatic partitioning (hypertables), time-based compression, and continuous aggregates, enabling PostgreSQL to efficiently store, query, and analyse massive volumes of time-stamped data.

Azure Database for PostgreSQL provides a fully managed database service, in which TimescaleDB can also be provided.

Azure managed instance for Appache Cassandra

Apache Cassandra is a highly scalable, distributed NoSQL database designed for handling large volumes of data across multiple nodes with high availability and no single point of failure. It is particularly well-suited for time series data due to its write-optimized architecture, horizontal scalability, and support for high-throughput operations. Cassandra's data model allows for efficient storage and retrieval of time-stamped records using techniques like wide rows and composite keys, making it a strong choice for time series applications where speed, scalability, and fault tolerance are critical.

Microsoft Azure allows to set up a managed instance of Appache Cassandra in your Azure subscription, so the actual management and setup of the cluster goes much faster, compared to setting up your own cluster in the environment.

Azure Timeseries Insights

This was a very easy to use service, but the service has been deprecated, in favour for Azure Fabric and Azure Data Explorer. But especially the Time Series Explorer was a very intuitive and interesting tool to explore and visualize time series data. You can see this in the screen shot below.

Consuming an Azure ML service from .NET

Sam Vanhoutte — Mon, 24 Mar 2025 10:40:03 +0000

For a project in the cycling community, I had to predict how long pro-riders would take to complete a certain race on the calendar. Different options, such as the parcours type, the distance and elevation obviously play a role in this.

So I used Azure Machine Learning to create an inference service that got deployed on Azure Compute Instance and that exposes an API for this.

In this post, I will explain how you can easily call the inference services of AzureML from dotnet, by passing strong typed objects to the inference method.

All of the code is available in the Open Sourced library: Aerozure.

Calling the scoring service from Postman

You can call the service that is hosted on the following url : https://..inference.ml.azure.com/score.
To authenticate, you have to pass in a Bearer token that you can retrieve from the Azure ML portal.

Calling the scoring endpoint can be done by posting a JSON that looks like the following. As you can see, it's a list of the column names, and their given values.

{
  "input_data": {
    "columns": [
      "month",
      "year",
      "distance",
      "elevation",
      "PointsScale",
      "UciScale",
      "ParcoursType",
      "ProfileScore",
      "RaceRanking",
      "StartlistQuality",
      "Classification",
      "Category"
    ],
    "index": [1],
    "data": [
        [2, 2025, 192, 1523, "2.PRO.Stage", "UCI.WR.Pro.Stage", 1, 22, null, 641, "2.Pro", "ME - Men Elite"]
    ]
  }
}

Streamline this call from csharp

The following code converts a strong typed dotnet class to the required input for the API.

Defining the request in csharp

The following class represents the actual data that is being sent to the API endpoint.

Inputdata.cs

using System.Text.Json.Serialization;
namespace Aerozure.Azureml;

public class InputData
{
    [JsonPropertyName("columns")]
    public string[] Columns { get; set; }

    [JsonPropertyName("index")]
    public int[] Index { get; set; }

    [JsonPropertyName("data")]
    public object[][] Data { get; set; }
}

The AzureML request

The AzuremlRequest will be used to serialize payload that is posted to the API and most importantly will convert the strong typed (PoCo) class to this request. For this, some basic Reflection is used to loop through all the properties of the object and take these properties and their values to the actual payload.

AzuremlRequest.cs

using System.Reflection;
using System.Text.Json.Serialization;

namespace Aerozure.Azureml;

public class AzuremlRequest
{
    [JsonPropertyName("input_data")] public InputData InputData { get; set; }

    public static AzuremlRequest Create(object payload)
    {
        var columns = new List<string>();
        var data = new List<object>();
        foreach (PropertyInfo property in payload.GetType().GetProperties())
        {
            columns.Add(property.Name);
            data.Add(property.GetValue(payload, null));
        }

        return new AzuremlRequest
        {
            InputData = new InputData
            {
                Columns = columns.ToArray(),
                Index = [1],
                Data = [data.ToArray()]
            }
        };
    }
}

The AzureML Inference client

The AzuremlClient class can be injected and expects the AzuremlOptions to be available on startup. Two properties are required there to be set (BearerToken and InferenceEndpoint).

AzuremlClient.cs

using System.Net.Http.Json;
using Aerozure.Configuration;
using Flurl.Http;
using Microsoft.Extensions.Options;

namespace Aerozure.Azureml;

public class AzuremlClient
{
    private readonly AzuremlOptions mlOptions;

    public AzuremlClient(IOptions<AzuremlOptions> mlOptions)
    {
        this.mlOptions = mlOptions.Value;
    }
    public async Task<T?> CallInferenceAsync<T>(object request)
    {
        var mlRequest = AzuremlRequest.Create(request);
        var response = await mlOptions.InferenceEndpoint
            .WithHeader("Authorization", $"Bearer {mlOptions.BearerToken}")
            .PostJsonAsync(mlRequest);
        return await response.ResponseMessage.Content.ReadFromJsonAsync<T>();
    }
}

Using the above code in your project

If you are using Aerozure, you can just enable the integration on startup in your Program.cs, by executing the following lines of code.

builder.Services
    .AddAerozure()
    .Configure<AzuremlOptions>(options => configuration.GetSection("azureml").Bind(options));

This will make the AzuremlClient available, configured for your endpoint with the right BearerToken.

Calling the inference endpoint, can be done by executing the following lines (here using the information on a given Race object).

var raceData = new RaceData
{
    month = race.Date!.Value.Month,
    year = race.Date!.Value.Year,
    distance = race.Distance,
    elevation = race.Elevation,
    race.PointsScale,
    race.UciScale,
    race.ParcoursType,
    race.ProfileScore,
    race.RaceRanking,
    race.StartlistQuality,
    race.Classification,
    race.Category
};
var result = await azuremlClient.CallInferenceAsync<double[]>(dataObject);
if (result != null)
{
    results.Add(race.Id, new RaceDurationPrediction
    {
        Distance = (int)race.Distance,
        Duration = result.First()
    });
}

Expose your Open API specs with Azure API management

Sam Vanhoutte — Mon, 25 Mar 2024 11:44:40 +0000

When deploying your API's to Azure API Management, you can have different logical API's that you want to have integrated by external applications or partners.

For this, the API Developer portal can be deployed, but this could be somewhat tedious. Another common approach is to expose the API's (only the ones you want to document publically!) specifications. The Open API endpoint (or the Swagger endpoint as it used to be called).

As documented in a previous post Deploy multiple APIs in Azure API management, hosted in the same App service, you can easily have multiple API's deployed to Azure API management.

This post explains how you can expose the Open API spec endpoint for a logical API in Azure API Management.

Enable the Service Reader Role

The Azure API management service instance should be able to read its own definitions, in order to build the API spec. For this, it is required we provide the Managed Identity of the Azure API management service with the API Management Service Reader Role.

This can be achieved with the following steps.

Enable managed identity

Navigate to Security > Managed Identities in the configuration of your API Management instance. Make sure the status is set to On.

Assign the required role

After that , click the Azure role assignments button.
Click Add role assignment (Preview) button
Select the resource group as scope, and search for the API Management Service Reader role`

Create the actual OpenAPI API definition

Create the operation

Create a new HTTP API (name: Swagger API) in Azure API management
Add an operation manually that will download the API specification, by using the path parameter api-id: /docs/{api-id}/openapi.

Retrieve the OpenAPI spec from the management API

The following snippet is needed (see explanation below)

<policies> <inbound> <base />  <send-request mode="new" response-variable-name="result" timeout="60" ignore-error="true"> <set-url>@("https://management.azure.com/subscriptions/#subscriptionid#/resourceGroups/#resource-group#/providers/Microsoft.ApiManagement/service/#resourceid#" + "/apis/" + context.Request.MatchedParameters.GetValueOrDefault("api-id","") + "?export=true&format=openapi&api-version=2021-01-01-preview")</set-url> <set-method>GET</set-method> <authentication-managed-identity resource="https://management.azure.com/" /> </send-request>  <return-response> <set-status code="200" reason="OK" /> <set-header name="Content-Type" exists-action="override"> <value>application/yaml</value> </set-header> <set-body>@((string)(((IResponse)context.Variables["result"]).Body.As<JObject>()["value"]))</set-body> </return-response> </inbound> <backend> <base /> </backend> <outbound> <base /> </outbound> <on-error> <base /> </on-error> </policies>

Warning! The pretty print of dev.to seems to be unable to cope with the complex instructions, so the structure seems to be messed up.

This is the policy you can set on the operation. With three replacements to make:

#subscriptionid#: the subscription id of the API management instance
#resource-group#: the resource group name of the API management instance
#resourceid#: the name of the API management instance.

Testing the solution

Just calling the API endpoint will now return the actual Open API spec as yaml. For this you need to provide the api-id value. This is the system name of the API in your instance. The field that is greyed out in the following screenshot:

Deploy multiple APIs in Azure API management, hosted in the same App service.

Sam Vanhoutte — Mon, 04 Mar 2024 19:00:40 +0000

We are building an API that contains several controllers with many operations. This API has to be consumed by different client applications.

Our own management portal
External integration companies
A mobile application

While we prefer to keep the code in the same ASP.NET Core Web API project (this is easy for many reasons, of which simplicity may be the biggest one), we don't want to expose every part of functionality to every client.

And this post is describing how we declare our API in such a way that we have multiple logical API specs that we then deploy in Azure API Management as different API's.

The WebAPI project

Enabling nswag in the build action

After creating the new ASP.NET Web API project, we can add the following NSwag packages to our project.

<PackageReference Include="NSwag.AspNetCore" Version="14.0.3" />
<PackageReference Include="NSwag.MSBuild" Version="14.0.3">

These packages will be used to decorate our operations and to generate the OpenAPI spec at build/deploy time.

We also add a build action to generate the open api spec in our deployment/api folder.

    <Target Name="NSwag" AfterTargets="PostBuildEvent" Condition=" '$(Docker)' != 'yes' ">
        <Exec WorkingDirectory="$(ProjectDir)" EnvironmentVariables="ASPNETCORE_ENVIRONMENT=Development;IS_NSWAG_BUILD=true;API_TITLE=Events_Backend_Api;INCLUDE_ALL_OPERATIONS=true" Command="$(NSwagExe_Net80) run nswag-client.json /variables:Configuration=$(Configuration),Output=backend-openapi.json" />
        <Exec WorkingDirectory="$(ProjectDir)" EnvironmentVariables="ASPNETCORE_ENVIRONMENT=Development;IS_NSWAG_BUILD=true;API_TITLE=Events_Public_Api;API_TYPE=Public" Command="$(NSwagExe_Net80) run nswag-spec.json /variables:Configuration=$(Configuration),Output=backend-public-openapi.json" />
    </Target>

When we build our project, the PostBuildEvent actions will be executed and the nswag will be used to use the nswag-spec.json configuration file to output the open-api as json to the configured output file. (/deployment/api in our case).

In the above configuration, we are generating two different api specs:

backend-openapi.json: we are passing INCLUDE_ALL_OPERATIONS=true as the argument to the nswag command.
backend-public-openapi.json: we are passing API_TYPE=Public as the argument to the nswag command.

In the next steps, I will show how these arguments are being used to filter out the correct operations in the open API spec generation.

Decorating the operations

In order to filter out the correct operations, I have created a custom Attribute that I can use on controllers and operations to indicate in which API output spec they should be included or not.

The custom attribute looks like this:

[AttributeUsage(AttributeTargets.All)]
public class ApiTypeAttribute : Attribute
{
    private readonly string? apiType;
    private readonly bool alwaysInclude;

    public ApiTypeAttribute(string? apiType = null, bool alwaysInclude = false)
    {
        this.apiType = apiType;
        this.alwaysInclude = alwaysInclude;
    }

    public string? ApiType => apiType;
    public bool AlwaysInclude => alwaysInclude;
}

This attribute can then be used and applied to operations and controllers, like in the following example:

using Events.WebAPI.Models;
using Events.WebAPI.Responses;
using Events.WebAPI.Runtime;
using Events.WebAPI.Services;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Options;
using NSwag.Annotations;

namespace Events.WebAPI.Controllers;

/// <summary>
/// API endpoint that exposes Events functionality
/// </summary>
[ApiController]
[Route("events")]
[ApiType(apiType: ApiTypes.Admin)]
public class EventController(EventsService eventsService): ControllerBase
{
    /// <summary>
    /// Get all events
    /// </summary>
    [HttpGet()]
    [ApiType(apiType: ApiTypes.Public)]
    [SwaggerResponse(StatusCodes.Status200OK, typeof(EventsResponse), Description = "The available events.")]
    public async Task<IActionResult> ListEvents()
    {
        var events = await eventsService.GetEventsAsync();
        return Ok(new EventsResponse(events.ToArray()));
    }

    /// <summary>
    /// Get all events
    /// </summary>
    [HttpPost()]
    [SwaggerResponse(StatusCodes.Status200OK, typeof(void), Description = "The event was created.")]
    public async Task<IActionResult> CreateEvent(Event @event)
    {
        await eventsService.CreateEventAsync(@event);
        return Ok();
    }
}

In the above controller, you can see that by default, the controller and its operations are considered to be part of the "Admin" API. But the ListEvents operation is decorated with the [ApiType(apiType: ApiTypes.Public)] attribute, indicating it's part of the public API as well.

In order to set this up, the Program.cs file contains the following logic, that is being called at the time of nswag-build.

builder.Services.AddOpenApiDocument(document => { GenerateOpenApiSpec(document, "v1"); });

void GenerateOpenApiSpec(AspNetCoreOpenApiDocumentGeneratorSettings nswagSettings, string documentName)
{
    string apiTitle = "Savanh Events API";

    bool includeAllOperations = false;
    string? apiType = null;

    if (!string.IsNullOrEmpty(builder.Configuration["API_TYPE"]))
    {
        apiType = builder.Configuration["API_TYPE"];
    }

    if (!string.IsNullOrEmpty(builder.Configuration["INCLUDE_ALL_OPERATIONS"]))
    {
        _ = bool.TryParse(builder.Configuration["INCLUDE_ALL_OPERATIONS"], out includeAllOperations);
    }

    if (!string.IsNullOrEmpty(builder.Configuration["API_TITLE"]))
    {
        apiTitle = builder.Configuration["API_TITLE"];
        apiTitle = apiTitle.Replace("_", " ");
    }

    Console.WriteLine($"Generating api doc : AllOperations: {includeAllOperations} // ApiType: {apiType}");
    nswagSettings.OperationProcessors.Add(new OpenApiSpecOperationProcessor(includeAllOperations, apiType));
    nswagSettings.DocumentName = documentName;
    nswagSettings.Title = apiTitle;
    nswagSettings.Version = Assembly.GetExecutingAssembly().GetName().Version?.ToString();
}

In the above code snippet, you can see that the input variables (API_TYPE, INCLUDE_ALL_OPERATIONS, API_TITLE) that are passed in the PostBuild action are being used to apply the behavior for the open API spec generation.

And there is an important line (nswagSettings.OperationProcessors.Add(new OpenApiSpecOperationProcessor(includeAllOperations, apiType));) that passed those configuration values to an OperationProcessor that will then filter out the operation or not. And that Processor is seen in the following snippet.

public class OpenApiSpecOperationProcessor : IOperationProcessor
{
    private readonly bool includeAllOperations;
    private readonly string? apiType;

    public OpenApiSpecOperationProcessor(bool includeAllOperations = false, string? apiType = null)
    {
        this.includeAllOperations = includeAllOperations;
        this.apiType = apiType;
    }

    // This method return a boolean, indicating to include the operation or not in the output
    public bool Process(OperationProcessorContext context)
    {
        if (includeAllOperations)
        {
            return true;
        }

        // Looking for the method or controller to have the ApiType attribute defined

        ApiTypeAttribute? attribute = null;
        if (context.MethodInfo.IsDefined(typeof(ApiTypeAttribute), true))
        {
            // First we check the method (deepest level), as that can override the controller
            attribute = (ApiTypeAttribute?)context.MethodInfo.GetCustomAttribute(typeof(ApiTypeAttribute));
        }
        else
        {
            // If no attribute on the method, we check the controller
            if (context.ControllerType.IsDefined(typeof(ApiTypeAttribute), true))
            {
                attribute = (ApiTypeAttribute?)context.ControllerType.GetCustomAttribute(typeof(ApiTypeAttribute));
            }
        }

        // Neither the method, nor the controller have the attribute
        // So we return false as it should not be included
        if (attribute == null)
        {
            return false;
        }

        // Since we found an attribute, we are now applying the logic where the method
        // will be included in the open api spec, when AlwaysInclude is on,
        // or when the ApiType matches the requested api type
        if (!string.IsNullOrEmpty(apiType))
        {
            var include = attribute.AlwaysInclude || apiType.Equals(attribute.ApiType, StringComparison.CurrentCultureIgnoreCase); 
            return include;
        }

        return attribute.AlwaysInclude;
    }
}

The result is now that, on every build of the WebAPI project, we will have the generated OpenAPI specs (swagger) stored in the corresponding output folder. (which is configured in nswag-spec.json)

Deploying the WebAPI to Azure API Management

This article leaves out the deployment logic to deploy the actual API as a Docker container and host it in an App Service. That would lead us too far and can be found in the github repo of this article.

We now just focus on importing the OpenAPI specs in Azure API Management as different API's.

This happens by leveraging the following bicep code.

Bicep modules

The following bicep modules can just be reused and then called from within your own project specific bicep file.

api-management-api.bicep

//Parameters
param environmentAcronym string
param locationAcronym string
param apiManagementName string
param serviceUrl string
param apiSpec string
param format string
param displayName string
param description string
param apiRevision string = '1'
param subscriptionRequired bool = true
param path string
param name string
param policy string
param policyFormat string = 'rawxml'
param namedValues array

// Apply naming conventions
var apiName = '${locationAcronym}-${environmentAcronym}-api-${name}'

// Get the existing APIM instance
resource apiManagement 'Microsoft.ApiManagement/service@2021-08-01' existing = {
  name: apiManagementName
}

// Define API resource
resource apiManagementApi 'Microsoft.ApiManagement/service/apis@2021-12-01-preview' = {
  parent: apiManagement
  name: apiName
  properties: {
    displayName: displayName
    description: description
    apiRevision: apiRevision
    subscriptionRequired: subscriptionRequired
    serviceUrl: serviceUrl
    path: path
    format: format
    value: apiSpec
    protocols: [
      'https'
    ]
    // Changing the subscription header name
    subscriptionKeyParameterNames: {
      header: 'x-subscription-key'
      query: 'x-subscription-key'
    }
    isCurrent: true
  }
  dependsOn: apiManagementNamedValues
}

// Specifying the policy on the API level
resource apiManagementapiPolicy 'Microsoft.ApiManagement/service/apis/policies@2021-12-01-preview' = {
  parent: apiManagementApi
  name: 'policy'
  properties: {
    value: policy
    format: policyFormat
  }
}

// Add named values
resource apiManagementNamedValues 'Microsoft.ApiManagement/service/namedValues@2021-08-01' = [for namedValue in namedValues: {
  parent: apiManagement
  name: '${namedValue.name}'
  properties: {
    displayName: namedValue.displayName
    value: namedValue.value
    secret: namedValue.isSecret
  }
}]

output apiName string = apiManagementApi.name
output basePath string = path

We can also deploy an API product for every seperate API. That module can be found below.

api-management-product.bicep

param name string
param apimName string
param displayName string
param description string
param subscriptionRequired bool = true
param apiNames array


resource product 'Microsoft.ApiManagement/service/products@2021-01-01-preview' = {
  name: '${apimName}/${name}'
  properties: {
    displayName: displayName
    description: description
    subscriptionRequired: subscriptionRequired
    state: 'published'
  }
  resource api 'apis@2021-01-01-preview' = [for apiName in apiNames: {
    name: apiName
  }]
}

And then we can call these modules from our main .bicep file, as can be seen in the following snippet:

// Deploy actual API
var apiManagementName = '${locationAcronym}-${environmentAcronym}-${companyName}-apim'

// deploy APIs to the API management instance
var backend_api_policy = '''
<policies>
    <inbound>
        <authentication-managed-identity resource="{0}" />
        <base />
    </inbound>
</policies>
'''

module backendApi 'modules/api-management-api.bicep' = {
  name: 'backendApi'
  scope: backendResourceGroup
  params: {
    locationAcronym: locationAcronym
    environmentAcronym: environmentAcronym
    apiManagementName: apiManagementName
    apiSpec: string(loadJsonContent('../api/backend-openapi.json'))
    format: 'openapi+json'
    path: 'backend/api/v1'
    // The following should typically be taken from output of the actual bicep module for App Service
    // But this sample is focusing on the API Management side of things
    serviceUrl: 'https://weu-dev-samvhintx-app-backend-api.azurewebsites.net/'
    displayName: 'SVH IntX API (${environmentAcronym})'
    description: 'The API that provides all required functionality on the ${environmentAcronym} environment.'
    name: '${environmentAcronym}-api-backend'
    policy: format(backend_api_policy, appServiceClientId)
    namedValues: [ ]
  }
}

module publicApi 'modules/api-management-api.bicep' = {
  name: 'publicApi'
  scope: backendResourceGroup
  params: {
    locationAcronym: locationAcronym
    environmentAcronym: environmentAcronym
    apiManagementName: apiManagementName
    apiSpec: string(loadJsonContent('../api/backend-public-openapi.json'))
    format: 'openapi+json'
    path: 'public/api/v1'
    // The following should typically be taken from output of the actual bicep module for App Service
    // But this sample is focusing on the API Management side of things
    serviceUrl: 'https://weu-dev-samvhintx-app-backend-api.azurewebsites.net/'
    displayName: 'SVH IntX Public API (${environmentAcronym})'
    description: 'The API that provides all required functionality on the ${environmentAcronym} environment.'
    name: '${environmentAcronym}-api-public'
    policy: format(backend_api_policy, appServiceClientId)
    namedValues: [ ]
  }
}

The result

After our successful deployment, this is the result in the API Management instance, where we have two different API products, each showing a different set of Operations.

This can be seen in the following screenshots.

*The "backend" API exposing two operations. *

The "public" API exposing just one operation.

As always, all code can be found online. Here.

Logic Apps deployment from Github with Powershell

Sam Vanhoutte — Wed, 28 Feb 2024 06:45:36 +0000

Context

I was working with a startup that is building software to predict time series in a specific industry. One of the big challenges for them is to integrate with systems of their customers.

While they offer an API that can be called by customers to ingest data into their tenant, some of those customers, require that startup to go and fetch the data from one of their systems. And yes, that smells like customer specific integration.

For that, we have set up several Logic Apps, that we isolate from the central part of the runtime. And this post describes how we have structured those Logic Apps, and how we automatically deploy them from our automated Github workflow.

Structure

As written above, the structure of the repo is in such a way that we have put all Logic Apps in a folder, called Integrations. And at deployment time, we automatically take those workflows from the folder, leveraging the names of the sub folders, to build the right resource name (ensuring our naming conventions). And those Logic Apps get deployed to a consumption Logic App.

Code

All code is available on https://github.com/SamVanhoutte/azure-logic-apps-deployment

Azure authentication

As with any other Github action, it is important to leverage the right credentials to authenticate against your Azure Subscription. For that, the following script has to be executed in the Azure CLI, and the resulting lines should be kept as a secret in your Github environment.

az ad sp create-for-rbac --name "github-deploy-dev" --role contributor --scopes /subscriptions/b73995e3-caad-4882-8644-f2175789c3ff --sdk-auth

The output looks like the following (which should be handled as a secret!).

{
  "clientId": "a8fa378b....",
  "clientSecret": "cOF8....",
  "subscriptionId": "b73995e3-caad-4882-8644-f2175789c3ff",
  "tenantId": "37e57300...."
}

Note: you can leave out the urls and just keep the first for relevant parts.

The output has been saved in my Github repo secret (with name AZURE_CREDENTIALS in the Development environment on my repo.

Github pipeline

To deploy our infrastructure, we default to Github workflows.

The result of our deployment pipeline is shown in the next screenshot:

The definition of the Github workflow can be easily found in the Github repo. But the most relevant sections have been copied here:

name: Deploy the Backend environment

on:
  workflow_call:
    inputs: 
    # Removed for brevity
jobs:
  Deploy-Azure-Backend-Resources:
    runs-on: ubuntu-latest
    if: always()
    environment: ${{ inputs.environment }}
    steps:
    # Log in to Azure
    - name: Azure Login
      uses: azure/login@v1
      with:
        creds: '${{ secrets.AZURE_CREDENTIALS }}'
        enable-AzPSSession: true
    # Removed for brevity
  Deploy-Integrations:
    runs-on: ubuntu-latest
    if: always()
    needs : [ Deploy-Azure-Backend-Resources ]
    environment: ${{ inputs.environment }}
    steps:
    - name: Azure Login
      uses: azure/login@v1
      with:
        creds: '${{ secrets.AZURE_CREDENTIALS }}'
        enable-AzPSSession: true
    - name: Download powershell files from pipeline artifact
      uses: actions/download-artifact@v3
      with:
        name: powershell
        path: ./artifacts/powershell
    - name: Download integration files
      uses: actions/download-artifact@v3
      with:
        name: integrations
        path: ./artifacts/integrations
    - name: Deploy integrations
      uses: azure/powershell@v1
      with:
        inlineScript: ./artifacts/powershell/deploy-logicapp-folder.ps1 -environmentAcronym ${{ inputs.environmentAcronym }} -location ${{ inputs.region }} -rootFolder './artifacts/integrations/'
        azPSVersion: "latest"

Some remarks about the above script:

enable-AzPSSession: true ensures our Powershell script will be running authenticated against our Azure Subscription.
We download the artifacts that we initially uploaded to the runtime storage. We do this for our powershell folder & our integrations folder that contains our Logic Apps definitions.
And in the last step, we are calling our Powershell script (that is explained in the next step) that will receive a reference to our integrations folder from our repo.

Powershell script

The following script takes the folder reference, walks the tree iteratively and deploys every Logic App definition to the corresponding resource group, applying the naming conventions, based on the folder structure.

param ($rootFolder, $location, $environmentAcronym)


Function Deploy-LogicAppDefinition($logicAppDefinitionFile, $location, $environmentAcronym, $locationAcronym)
{
    # This function takes a given workflow definition file 
    # and deploys it to the resource group 
    # (based on the folder, location & environment)
    # It will create or update the Logic App

    ### SET UP LOGIC APP NAME
    # Taking the file name and remove the folder
    $logicAppName = Split-Path $logicAppDefinitionFile -leaf

    # and remove the suffix to keep the logical name
    $logicAppName = $logicAppName.Replace('.definition.json', '')

    # and apply naming conventions
    $logicAppName = "$locationAcronym-$environmentAcronym-int-la-$logicAppName"

    ### SET UP RESOURCE GROUP NAME
    # Taking the directory name to which the json file belongs
    $logicAppType = Split-Path (Split-Path $logicAppDefinitionFile -Parent) -Leaf
    # and construct the resource group name
    $resourceGroupName = "$environmentAcronym-$locationAcronym-logapp-rg-int-$logicAppType"

    # Check if the resource group exists and create if not
    $existingRG = Get-AzResourceGroup -Name $resourceGroupName -ErrorAction SilentlyContinue
    if(-Not $existingRG)
    {
        Write-Host "Creating resource group $resourceGroupName"   
        New-AzResourceGroup -Name $resourceGroupName -Location 'West Europe' -Force
    }

    ### CRUD OF LOGIC APP DEFINITION
    # Check if logic app exists
    $existingLogicApp = Get-AzLogicApp -ResourceGroupName $resourceGroupName -Name $logicAppName -ErrorAction SilentlyContinue
    if($existingLogicApp)
    {
        Write-Host "Update logic app $logicAppName to resource group $resourceGroupName"   
        Set-AzLogicApp -ResourceGroupName $resourceGroupName -Name $logicAppName -DefinitionFilePath $logicAppDefinitionFile -Force
    }
    else 
    {
        Write-Host "Create logic app $logicAppName to resource group $resourceGroupName"
        New-AzLogicApp -ResourceGroupName $resourceGroupName -Name $logicAppName -Location $location -DefinitionFilePath $logicAppDefinitionFile
    }
}

# Apply location acronym, based on location
switch ($location)
{
    'westeurope' { $locationAcronym = 'weu'}
    'northeurope' { $locationAcronym = 'neu'}
    'westus' { $locationAcronym = 'wus'}
    default { $locationAcronym = 'weu'}
}

# Start of logic
# Search through all folders for files ending with definition.json
Get-ChildItem -Path $rootFolder -Recurse -Filter *.definition.json |
    ForEach-Object {
        Deploy-LogicAppDefinition $_.FullName $location $locationAcronym $environmentAcronym
    }

Conclusion

I believe it's crucial to apply naming conventions in every step of your deployment logic. So, here we have a way to structure our logic apps in our repo and have them deployed, just because the appear in our repo (we don't have to update the pipeline or script for every new workflow).

In another post, I will describe how a custom connector can be deployed and used in another Logic App.