DEV Community: Alexandre Nédélec

Deploying a Developer Conferences Nuxt Static Website on Azure with Pulumi

Alexandre Nédélec — Sun, 06 Apr 2025 20:45:04 +0000

This is a submission for the Pulumi Deploy and Document Challenge: Fast Static Website Deployment

What I Built

The other day, to learn and experiment with GitHub Copilot Agent mode, I built a small website in Nuxt that provides an interactive calendar view of developer conferences around the world. I was happy with the result and I wanted to deploy it on Azure. So I created a Pulumi program in TypeScript to provision an Azure Static Web App, configure a custom domain, and retrieve a deployment token I could use to deploy the website in my GitHub Actions pipeline.

For that, I've used:

the version 3 of the Azure Native provider that has just been released
the new customizable resource auto-naming in Pulumi to easily set up a naming convention for my resources
a Pulumi ESC environment configured to deploy to Azure using OpenID Connect
Pulumi Copilot Visual Studio Code extension to help me with the infrastructure code

Live Demo Link

https://devconferences.techwatching.dev/

Project Repo

TechWatching / developer-conferences

Developer Conferences

A modern Nuxt 3 application that allows users to explore developer conferences from around the world.

Features

📆 Interactive calendar view of developer conferences
🌍 Filter conferences by country
🔗 Direct links to conference websites

Acknowledgments

This project is based on the amazing open source project Developer Conferences Agenda by Aurélie Vache that lists in a GitHub repository all the developer conferences and CFPs in the world. This Nuxt application completely relies on the data provided by the Developer Conferences Agenda repository and is not meant to be a replacement for it, nor a replacement for its website developers.events that provides a nice way to explore the conferences around the world.

This project is a personal initiative to learn and experiment with GitHub Copilot Agent Mode to create a simple website using Nuxt. For this experiment, I decided to build a small website that lists developer conferences, using…

View on GitHub

My Journey

Context

The website I built is inspired by the developers.events website, which is the official website of the open source developers-conferences-agenda GitHub repository. This repository, which everyone can contribute to, provides data about developer conferences and CFPs worldwide. So I used this data to develop a Nuxt application that people can use to find the events taking place at a specific date in different countries.

In this article I want to focus on the work I did to provision the cloud Infrastructure I needed and deploy the application on it.

Choices

To host a website on Azure, there are several options available, such as a storage account, app service, container app, or static web app. However, for a static website, I believe the best choice is an Azure Static Web App, which is specifically designed for this type of scenario.

My website was developed in Nuxt using TypeScript so it made sense to choose TypeScript to develop my infrastructure code too. Using Pulumi in TypeScript with the Azure Native provider was a natural choice. Since a new version of the Azure Native provider has just been released, it was the opportunity to test it.

My code repository is hosted in GitHub so I wanted to use GitHub Actions to provision the infrastructure and deploy the application.

Prompting My Way to Pulumi

I’ve used a lot GitHub Copilot for the application code, no reason not to do the same with the infrastructure code, especially when there is a Pulumi Copilot extension for vscode that lets you use an IA assistant for Pulumi directly in GitHub Copilot Chat.

My first prompt was the following:

@pulumi I want to create a pulumi program in TypeScript the infra folder for the website I'm developing here.
The program should use the azure native provider to create an azure static web app.

It made me create run the command to create the pulumi program, add the Azure Native provider and provided me with an initial version of the Azure Static Web App infrastructure code, which I could modify to suit my needs.

pulumi new typescript
pnpm add @pulumi/azure-native

💡Thanks to the SDK size reduction, the Azure Native SDK was much faster to download.

import * as azure from '@pulumi/azure-native'

const resourceGroup = new azure.resources.ResourceGroup('devconfs')

const staticWebApp = new azure.web.StaticSite('devconfs-site', {
  resourceGroupName: resourceGroup.name,
  location: resourceGroup.location,
  sku: {
    name: 'Free',
    tier: 'Free'
  },
  buildProperties: {}
})

The only downside of Pulumi Copilot is that it’s not yet available in Edit or Agent modes. When you're accustomed to using Edit and Agent modes, relying solely on the Ask mode doesn't feel sufficient.

You can notice I removed the buildProperties and some other properties when defining the Azure Static Web App. It’s because, I did not want Azure to automatically create a build and deploy pipeline in my GitHub repository, but instead I wanted to fully control this process from a CI/CD pipeline I would create manually.

Adopting A Naming Convention Thanks To Pulumi Auto-Naming

When working with cloud infrastructure, it’s important to define a naming convention for your resources and enforce it in your infrastructure code. The Azure documentation is a good starting point to learn about the recommended patterns and the naming limitations for some resource types (restricted characters or maximum length for names).

In addition to the naming convention, it’s important to have unique names to avoid conflicts between environments and deployments (different instances, regions, etc). Even if your resource names include the name of the environment or region, it's a good idea to add some randomness to them. It’s also very important to allow zero-downtime deployment, when a change involves deleting a resource and recreating a new one (mandatory on some resources when changing some specific properties).

Pulumi has always automatically handled that by appending a few random characters at the end of the provided name for a resource. With that auto-naming feature, when a resource needs to be replaced, Pulumi ensures the new resource is created before deleting the old one. What they recently released is the ability to configure this auto-naming feature and specify a custom naming pattern to use globally or by resource type.

For my projet, I configured auto-naming to use the following naming pattern ${name}-${stack}-${hex(8)} by default and specify custom patterns for some resource types (to add a specific prefix relative to the type of resource) like stapp-${name}-${stack}-${hex(8)} for the Static Web App. This gave me the following project configuration file:

name: developer-conferences
description: The infrastructure for the Developer Conferences website
runtime:
  name: nodejs
  options:
    packagemanager: pnpm
config:
  pulumi:autonaming:
      value:
        pattern: ${name}-${stack}-${hex(8)}
        providers:
          azure-native:
            resources:
              "azure-native:resources:ResourceGroup": 
                pattern: rg-${name}-${stack}-${hex(8)}
              "azure-native:web:StaticSite":
                pattern: stapp-${name}-${stack}-${hex(8)}

You can see the generated names for the resources:

I used a simple naming convention, but there are many options depending on your needs. In the future, I might create a Pulumi ESC environment called naming-conventions with all the patterns by resource type that I use, especially for those with naming restrictions. This way, by importing this environment into my other ESC environments or directly into my IaC project configurations, all resources in my organization's projects will automatically follow the same naming conventions.

📺 You can learn more about customizing auto-naming in the documentation or by watching this YouTube video on the topic.

Simplifying and Securing Deployment to Azure with Pulumi ESC

Speaking of ESC, I set up a new ESC environment to allow my Pulumi project to authenticate easily and securely to Azure using OpenID Connect, both locally and from my GitHub Actions pipeline. I created this new environment developer-conferences/prod directly from the Pulumi vscode extension.

To authenticate to Azure, using a Service Principal with a client secret that can expire or be compromised is not the best practice. You should instead rely on Workoad Identity Federation that uses OIDC like I explain in this article. This requires to do some configuration in Azure Entra ID to authorize a GitHub Actions (or Azure DevOps) pipeline from a specific repository to retrieve an Azure access token and perform some actions on an Azure subscription. All that is a bit cumbersome (although it can be automated using Azure CLI or Pulumi) and will not work locally.

This is where Pulumi ESC comes in: once properly configured an ESC environment allows us to authenticate with cloud providers using OpenID Connect tokens. I already had an ESC environment azure-authentication/visual-studio-enterprise configured to authenticate to my Visual Studio Enterprise Azure subscription. All I had to do was to import this environment in my new developer-conferences/prod environment and use this environment in my Pulumi project.

I used the following command to use the ESC environment in my current stack but you can also directly manually edit the stack configuration file.

pulumi config env add developer-conferences/prod

environment:
  - developer-conferences/prod
config:
  azure-native:location: westeurope

My Pulumi project could then provision resources on my Azure subscription as long as it was authenticated with Pulumi Cloud. Locally, I just needed to be logged into the Pulumi CLI. In my GitHub Actions pipeline, I first had to set up the OpenID Connect integration for GitHub in my Pulumi organization.

Here is the resulting GitHub Actions pipeline part for the Pulumi part (no secret is needed for it work):

name: Deploy to Azure Static Web App

on:
  push:
    branches:
      - main
  workflow_dispatch:
permissions:
  id-token: write
  contents: read

jobs:
  build-and-deploy:
    runs-on: ubuntu-latest

    steps:
      - name: Install infra dependencies
        run: pnpm install --dir infra

      - uses: pulumi/auth-actions@v1
        with:
          organization: TechWatching
          requested-token-type: urn:pulumi:token-type:access_token:personal
          scope: user:TechWatching

      - uses: pulumi/actions@v6
        with:
          command: up
          stack-name: techwatching/developer-conferences/prod
          work-dir: infra

Deploying the Website Using a Secret Stack Output

To the best of my knowledge, a front-end application can only be deployed on an Azure Static Web App using a deployment token. I could get this token from my provisioned Azure Static Web App resource and store it in the GitHub Action secrets, but that wouldn't be ideal: if I delete my resource or create a new environment, the deployment won’t work.

Instead, I let my infrastructure code retrieve the generated deployment token and set it as a stack output. This way, I can easily access the token from my stack outputs, and if my infrastructure changes, the stack output will update accordingly.

const staticSecretOutputs = azure.web.listStaticSiteSecretsOutput({
  resourceGroupName: resourceGroup.name,
  name: staticWebApp.name
})

export const staticWebAppDeploymentToken = pulumi.secret(staticSecretOutputs.properties.apply(p => p.apiKey))

I used the pulumi.secret function so that the staticWebAppDeploymentToken is marked as secret and automatically encrypted because it contains sensitive data.

I asked Pulumi Copilot to adjust my GitHub Actions workflow accordingly:

The answer was not perfect, it used an old syntax ::set-output and did not ensure the token was written in the logs. So I had to adjust the suggestion a bit, but at least it pointed me in the right direction. Here's what it looks like using the secret stack output for the deployment of the website:

      - uses: pulumi/actions@v6
        with:
          command: up
          stack-name: techwatching/developer-conferences/prod
          work-dir: infra

      # Securely get the deployment token from Pulumi outputs
      - name: Get Static Web App Deployment Token
        id: get-token
        run: |
          # Use Pulumi CLI to get the token and mask it in logs
          TOKEN=$(cd infra && pulumi stack output staticWebAppDeploymentToken --show-secrets)
          echo "::add-mask::$TOKEN"
          echo "deployment_token=$TOKEN" >> $GITHUB_OUTPUT

      - name: Deploy to Azure Static Web App
        uses: Azure/static-web-apps-deploy@v1
        with:
          azure_static_web_apps_api_token: ${{ steps.get-token.outputs.deployment_token }}
          action: "upload"
          app_location: ".output/public" # The location of the Nuxt generated static files
          skip_app_build: true # We've already built the app
          skip_api_build: true

To deploy the website from you local environment using the the CLI, you could have used the following command:

pnpx @azure/static-web-apps-cli deploy .output/public --env production --deployment-token (pulumi stack output staticWebAppDeploymentToken --show-secrets --cwd infra)

Using a Custom Domain for the Website

Configuring a custom domain on the Azure Static Web App was straightforward. I simply used the StaticSiteCustomDomain resource and specified the custom domain I owned.

new azure.web.StaticSiteCustomDomain('customDomain', {
  resourceGroupName: resourceGroup.name,
  name: staticWebApp.name,
  domainName: 'devconferences.techwatching.dev'
})

The devconferences.techwatching.dev is a subdomain I manually created using Netlify DNS. I could have used the Pulumi Netlify provider for this, and it’s something I might do in the future. I just haven't taken the time to do it yet.

Final Thoughts

As always, working with Pulumi was a great experience. Deploying a static website on Azure using Pulumi is quite simple. I was able to leverage advanced features of Pulumi, such as customizable auto-naming and Pulumi ESC, to meet my needs for naming convention and secure deployments. Pulumi vscode extensions helped me along the way, and Pulumi Copilot is nice, but I'm looking forward to it being upgraded to work with the latest features of GitHub Copilot (Edit and Agent Mode, Model Context Protocol, etc.).

You can find the complete code here, don’t hesitate to give it a start or make a comment here. Keep learning.

Unlocking the Power of Azure Functions Flex Consumption Plan with Pulumi

Alexandre Nédélec — Thu, 05 Sep 2024 11:38:28 +0000

In this article, we will explore how to provision a Function App in the new Azure Functions hosting plan: the Flex Consumption plan. We will do that using Pulumi and TypeScript.

What is the Azure Functions Flex Consumption plan?

Azure Functions Flex Consumption plan is a new hosting plan for Azure Functions that combines benefits from the Consumption plan and interesting additional features like always-already instances or VNet integration.

You can see a comparison of the different hosting plans for Azure Functions here. And if you want to know more about the Flex Consumption plan, you can check out a nice video about it here.

To be transparent, I was a bit bored by the announcement of "yet another hosting plan for Azure Functions." However, I became enthusiastic when I discovered it solved most of the issues I could have with the other plans.

That's why I decided to see how to deploy a Function App in the Flex Consumption plan using Infrastructure as Code.

The Azure resources to provision

Nothing complicated here. As usual, to provision a Function App, we need to set up:

a Service Plan (with a Flex Consumption SKU)
a Storage account
the Function App itself

💬 To keep things simple and focused on the topic, I will not configure any monitoring on this Function App. This means we won't provision the Application Insights and Log Analytics Workspace resources.

In the diagram above, you can see that we will create a Managed Identity for the Function App, a Blob Container to contain the deployment package, and assign the storage blob data contributor role to the Function App managed identity.

Indeed, with Flex Consumption plan, you deploy your application package to a blob container, and your function app runs from this package. That's why you need this blob container. You could enable access for the function app to this container by adding an application setting containing the storage connection string in your function app configuration. However, I prefer using Azure RBAC as it is more secure.

Implement the Infrastructure Code

To create your Pulumi program, you can start from the azure-typescript template for instance:

pulumi new azure-typescript

This template will already contain the dependency on the Pulumi Azure Native provider that we will use to create our Azure infrastructure.

💡When you create your project, you will be asked to choose a default azure location for your stack. Choose a region where Flex Consumption plan is available. You can check that with the following azure cli command az functionapp list-flexconsumption-locations -o table

Let's start by creating a resource group:

import * as pulumi from '@pulumi/pulumi'
import * as resources from '@pulumi/azure-native/resources'

const stackName = pulumi.getStack()
const resourceGroup = new resources.ResourceGroup(`rg-flexconsumption-${stackName}`)

You can see that I'm including a prefix corresponding to the resource type and the stack's name in my resource names. I think it's a good convention to follow because it allows you to quickly identify which environment a resource belongs to. However, feel free to adopt any naming convention you like, as long as you have one. Check here if you haven't chosen one yet.

import * as storage from '@pulumi/azure-native/storage'

const storageAccount = new storage.StorageAccount(`stflexconsump${stackName}`, {
  resourceGroupName: resourceGroup.name,
  allowBlobPublicAccess: false,
  kind: storage.Kind.StorageV2,
  sku: {
    name: storage.SkuName.Standard_LRS,
  },
})

Nothing specific for the storage account, just make sure to disable blob public access by setting the allowBlobPublicAccess to false.

Now we can create the blob container that will contain the application package we will deploy to the function app. I named it deploymentpackage but you use any name you like.

const blobContainer = new storage.BlobContainer('deploymentPackageContainer', {
  resourceGroupName: resourceGroup.name,
  accountName: storageAccount.name,
  containerName: 'deploymentpackage',
})

At the time of writing, Flex Consumption is relatively new and only available on the latest versions of the Azure APIs. Since the Azure provider we are using is a "native provider" (generated from Azure APIs to always be up-to-date and cover 100% of the resources in Azure Resource Manager), this is not an issue. However, we need to specify we want the latest version of the API when doing the import from the package to create the Function App resource.

import {AppServicePlan, WebApp} from '@pulumi/azure-native/web/v20231201'

💬If you have worked with ARM templates, Bicep, or the Terraform AzAPI provider, you should be familiar with the concept of resource API versions. With the Pulumi Azure Native provider, there is also a default API version. This means you don't have to specify a version each time you create a resource. You only need to do this when you want a specific version (newer or older than the default one), as in this case.

The Application Service Plan is defined using the FlexConsumption SKU:

const servicePlan = new AppServicePlan(`plan-flexconsumption-${stackName}`, {
  resourceGroupName: resourceGroup.name,
  sku: {
    tier: 'FlexConsumption',
    name: 'FC1'
  },
  reserved: true
})

It's worth noting that currently the Flex Consumption plan is Linux-only so you have to set the reserved property to true.

As usual function apps on other hosting plans, the Function App on the Flex Consumption is a WebApp with the kind property set to functionapp.

const config = new Config()
const functionApp = new WebApp(`func-flexconsumption-${stackName}`, {
  resourceGroupName: resourceGroup.name,
  kind: 'functionapp,linux',
  serverFarmId: servicePlan.id,
  identity: {
    type: 'SystemAssigned'
  },
  siteConfig: {
    appSettings: [
      {
        name: 'AzureWebJobsStorage__accountName',
        value: storageAccount.name
      }
    ]
  },
  functionAppConfig: {
    deployment: {
      storage: {
        type: 'blobContainer',
        value: pulumi.interpolate`${storageAccount.primaryEndpoints.blob}${blobContainer.name}`,
        authentication: {
          type: 'SystemAssignedIdentity'
        }
      }
    },
    scaleAndConcurrency: {
      instanceMemoryMB: 2048,
      maximumInstanceCount: 100,
    },
    runtime: {
      name: config.require('functionAppRuntime'),
      version: config.require('functionAppVersion'),
    }
  }
});

What changes with the Flex Consumption plan is the introduction of a brand-new section functionAppConfig on the resource. There are 3 subsections:

deployment: to define the blob storage container where the application package will be published and the authentication method to use to access it (here the function app managed identity will be used)
scaleAndConcurrency: to define the memory size of the instances, the maximum number of instances and also the always-ready instances if you need some (I haven't set any here)
runtime: to define the language and version runtime (in my example, I set them with values coming from the stack configuration)

💡You can get additional information to provision a function app on the Flex Consumption plan in Microsoft's documentation. The properties on the Azure resources are the same in Bicep/ARM templates as in Pulumi Azure Native provider, so it's not complicated to follow the documentation.

The only missing part is to assign the Storage Blob Data Contributor role on the managed identity of the function app so that it can access the blob container.

What I like to do is to put the Azure Built-In roles needed in the infrastructure code in a specific azureBuiltInRoles.ts file like this:

export const azureBuiltInRoles = {
  contributor: '/providers/Microsoft.Authorization/roleDefinitions/b24988ac-6180-42a0-ab88-20f7382dd24c',
  storageBlobDataOwner: '/providers/Microsoft.Authorization/roleDefinitions/b7e6dc6d-f1e8-4753-8033-0f276bb0955b',
  storageBlobDataContributor: '/providers/Microsoft.Authorization/roleDefinitions/ba92f5b4-2d11-453d-a403-e96b0029c9fe'
};

And, then use the role I need in the RoleAssignment resource.

new RoleAssignment('storageBlobDataContributor', {
  roleDefinitionId: azureBuiltInRoles.storageBlobDataContributor,
  scope: storageAccount.id,
  principalId: functionApp.identity.apply(p => p!.principalId),
  principalType: 'ServicePrincipal'
})

We can also create some stack outputs to retrieve interesting information from the created resources like the function app name and the default function app key (just make sure to make it a secret so that's it properly encrypted in the state).

export const functionAppName = functionApp.name
export const defaultFunctionAppKey = pulumi.secret(listWebAppHostKeysOutput({ name: functionApp.name, resourceGroupName: resourceGroup.name })
  .functionKeys?.apply(x => x?.default))

We can now just run the pulumi up command to provision our infrastructure.

Deploy an Azure Function to the new Function App

To ensure the Function App works properly, let's create a new Azure Function and deploy it to our new resource. We can take the .NET 8 isolated process template for instance with a basic HttpTrigger function.

public class HelloFlexConsumptionPlan
{
    private readonly ILogger<HelloFlexConsumptionPlan> _logger;

    public HelloFlexConsumptionPlan(ILogger<HelloFlexConsumptionPlan> logger)
    {
        _logger = logger;
    }

    [Function("HelloFlexConsumptionPlan")]
    public IActionResult Run([HttpTrigger(AuthorizationLevel.Function, "get", "post")] HttpRequest req)
    {
        _logger.LogInformation("C# HTTP trigger function processed a request.");
        return new OkObjectResult("Hello Azure Functions Flex Consumption Plan!");
    }
}

Remember that our function app runtime is set with values coming from the stack configuration? So we will have to ensure the configuration is property set for our .NET 8 isolated Azure Function to work properly.

config:
  azure-native:location: northeurope
  flexconsumption:functionAppRuntime: dotnet-isolated
  flexconsumption:functionAppVersion: "8.0"

We can use the Azure Functions Core Tools (make sure you have an up-to-date version) to deploy the Azure Function App:

func azure functionapp publish "func-flexconsumption-dev*****"

💬 Replace func-flexconsumption-dev***** by the functionAppName stack output value.
You can add another output in your Pulumi program to have the endpoint to use to run the function you have just deployed:

export const functionEndpoint = pulumi.interpolate`https://${functionApp.defaultHostName}/api/HelloFlexConsumptionPlan?code=${defaultFunctionAppKey}`

And then just make a get request on the function endpoint:

Summary

The Azure Functions Flex Consumption plan offers a flexible and powerful hosting option that addresses many limitations of other plans.

Using Pulumi, you can easily provision a Function App on the Flex Consumption plan and start leveraging fast scaling and features like always-ready instances for your Azure Functions. You can find the complete source code used for this article in this GitHub repository.

If you are using other IaC solutions (Bicep or Terraform/OpenTofu with the Az API provider), you can check this sample repository

Call your Azure AD B2C protected API with authenticated HTTP requests from your JetBrains IDE

Alexandre Nédélec — Mon, 11 Mar 2024 00:00:00 +0000

I have written several blog posts about HTTP clients in the past. I am a big fan of using HTTP text files versioned in a git repository alongside API code and executed by an IDE tooling. However, there was one use case where a GUI tool like Postman or a swagger page was more convenient: retrieving OAuth 2.0 users' tokens. Thanks to the latest OAuth 2.0 feature in JetBrains' IDE built-in HTTP client, this is no longer an issue.

Context

I am developing a web application composed of a Vue.js frontend and an ASP.NET Core backend (just describing my use case, technologies don't matter). The end users of this application are authenticated using Azure AD B2C, which is a customer identity access management solution like Auth0 or other competitors.

I often need to manually call the endpoints of the API to verify the code is working properly and that an endpoint is returning the expected result. HTTP files are a convenient way of writing and executing the HTTP requests. Once committed in the Git repository, they can easily be shared with other developers of the team who may not have worked on some endpoints and want to have proper examples with the query parameters and payloads.

As the API is protected by Azure AD B2C, I need to retrieve a valid access token and pass it to my requests.

Previous solutions

Passing a valid access token to my HTTP requests is something I was previously doing by:

signing in my frontend
grabbing the token in the web browser dev tools
copying the token to my HTTP environment variables (preferably the private environment file to avoid committing a secret in your repository)

That works but:

it's cumbersome
you have to do it each time your access token expires

Another solution is to use a tool that generates app-specific local JWTs and configure your local dev environment to authenticate with these tokens instead of using the Azure AD B2C configuration. In .NET, you can use the dotnet user-jwts to do exactly that. It allows you to generate a JWT token with the scopes, roles, and claims you want. So it's a good solution to debug your API locally without having to bypass the authentication and authorization mechanisms.

However, it has some downsides:

the tokens are only valid in your local machine so it only works for your local environment
the Azure AD B2C authentication is replaced by this "local JWT authentication" so you are not testing your API in real conditions

With the new HTTP Client OAuth 2.0 feature

Starting version 2024.1, HTTP Client in the JetBrains IDEs (in my case Rider 2024.1) support automatically authenticating HTTP requests, provided that you properly configured it.

🗨 Support for OAuth 2.0 started in version 2023.3, however, Authorization Code Flow with PKCE (PKCE challenge is required in the OAuth 2.1 specification is only supported since 2024.1.

OAuth 2.0 authorization code flow with PKCE

The OAuth 2.0 flow involved in retrieving a valid access token to make requests to an Azure AD B2C protected API is the authorization code flow with PKCE. There are 2 steps in the OAuth 2.0 authorization code flow:

Get an authorization code
Exchange the authorization code for an access token

Step 1 involves the user entering their credentials in the login form (Azure AD B2C login form in this case). At first sight, it might appear not very suitable for using HTTP files but the JetBrains HTTP Client handled it by opening the login form in the IDE embedded browser.

For Azure AD B2C,

the authorize endpoint is https://{tenant}.b2clogin.com/{tenant}.onmicrosoft.com/{policy}/oauth2/v2.0/authorize
the token endpoint is https://{tenant}.b2clogin.com/{tenant}.onmicrosoft.com/{policy}/oauth2/v2.0/token

where:

tenant is the name of the Azure AD B2C tenant
clientId is the application ID of the application registered in Azure AD the B2C tenant
policy is the name of the policy created in the Azure AD B2C tenant

💡When using a custom domain in Azure AD B2C, the endpoints are similar but the {tenant}.b2clogin.com part is replaced by the custom domain.

If you want to better understand how this flow works, there is a nice diagram in Auth0 documentation.

Configuration in the JetBrains HTTP Client

To make the authorization code flow work in the HTTP Client, all I have to do is provide the configuration for the Azure AD B2C tenant in the HTTP environment file.

Here is an example of such configuration:

{
  "apiUrl": "https://localhost:5001/api",
  "Security": {
    "Auth": {
      "CIAM": {
        "Type": "OAuth2",
        "Grant Type": "Authorization Code",
        "PKCE": true,
        "Client ID": "3a53c90d-20c4-40e9-b440-4825b70374d7",
        "Scope": "openid offline_access profile https://mytenant.onmicrosoft.com/security/user.read",
        "Auth URL": "https://mytenant.b2clogin.com/mytenant.onmicrosoft.com/b2c_1_sign_in/oauth2/v2.0/authorize",
        "Token URL": "https://mytenant.b2clogin.com/mytenant.onmicrosoft.com/b2c_1_sign_in/oauth2/v2.0/token",
        "Redirect URL": "https://localhost:8080/oidc-callback",
        "Acquire Automatically": true
      }
    }
  }
}

💡 Instead of setting PKCE to true, you can set if to a JSON object with the code challenge method and code verifier to use in it.

💬 In this example, I have set a local Redirect URL as my front was running locally. But I could also have set the Redirect URL to another environment where my web application is running.

You can check the JetBrains documentation to have more information about the HTTP Client support for OAuth 2.0 authorization.

Authenticated HTTP Requests in the HTTP file

Once the configuration is set, retrieving an access token can be done with a simple click in the configuration file.

The authentication process is logged so we can check the requests made and identify any mistakes made in the configuration.

Hopefully, we don't have to manually retrieve an access token each time we need to execute an HTTP request in an HTTP file of our IDE. We can just use the {{$auth.token()}} variable in the Authorization header of our requests, like this:

GET {{apiUrl}}/products
Authorization: Bearer {{$auth.token("CIAM")}}

The IDE will handle the rest for us.

Wrapping up

The HTTP Client OAuth 2.0 feature in JetBrains IDEs has greatly simplified making authenticated HTTP requests to secure APIs. While this article focused on Azure AD B2C, the same principles apply to other Authorization Servers, with only the authorize and token endpoints differing.

I hope other IDEs will adopt this feature, using the same convention for the $auth.token() variable and its configuration. The only drawback is for developers not using JetBrains IDEs, who will need to adjust requests containing the $auth.token() variable to run them in their IDEs.

Having Fun With IT Event Calendars

Alexandre Nédélec — Mon, 04 Mar 2024 00:00:00 +0000

In this post, we will discuss how to write a small .NET program that retrieves events from an IT event calendar and submits them to another one using AngleSharp.

Some context

There are plenty of websites that list IT events in the world. One that is particularly popular is the developers conferences agenda Github repository that was created by Aurélie Vache, a well-known French DevRel. This repository is an excellent resource where numerous tech conferences and CFPs (Call for Papers) are listed. Adding a new conference/CFP is very easy for any developer because you just have add it in the readme that contains all the conferences and make a PR. Additionally, there is now a website available to easily view the list of conferences.

Another one I like is the Tech Community Calendar created by Lee Englestone, a Microsoft MVP. What I find interesting it that it does not just list conferences and call for papers but also other tech events like hackathon or meetups. Events are displayed on small cards with thumbnails of the events websites, and you can filter them by country or type of event. Yet, it is less popular than the developers conferences agenda I mentioned before, so there are fewer events listed. There is a form to suggest new events, and I have been submitted events from time to time. However, most events I submit are developer conferences and CFPs that people have already added in the developer conferences agenda.

So I thought, what if I automate the process of retrieving events from the developer conferences agenda and submitting them to the tech community calendar?

It's just a PoC!

At first, I spent too much time thinking about how to schedule and host the program I hadn't even started writing 😁. Of course, time-triggered Azure Functions came to my mind, I even considered Durable Functions to break down the process into steps (retrieve events, check for existing events, submitting each event...). Then I thought about Jobs in Azure Container Apps, or Dapr with Azure Container Apps and even Dapr Workflows. Eventually, I realized it did not matter much since it was just a proof of concept. I decided to postpone the choice for later (if ever it goes beyond the poc) and just start coding.

I often like writing .NET tools or small programs using the Worker Service template because it's straightforward and includes useful features like dependency injection and configuration. However, this time I decided to keep things simple: just a .NET console application and all the code in Program.cs file. With top level statement, it feels similar to writing a Bash or PowerShell script, making it quite convenient for experimenting. Of course, this approach isn't what I would use for a real project.

Retrieve Developer Conferences

In addition to the readme file, the developers conferences agenda exposes all the data publicly in JSON here.

Developer conferences can be easily represented with a record (I only kept the properties I needed):

public record DeveloperEvent(
    string Name,
    long[] Date,
    string Hyperlink,
    string Location,
    string City,
    string Country
);

We can use an HttpClient to retrieve the events. The namespace System.Net.Http.Json contains an interesting method to make the GET HTTP call and deserialize the data using System.Text.Json.

using var httpClient = new HttpClient()
{
    BaseAddress = new Uri("https://developers.events/")
};

var events = await httpClient.GetFromJsonAsync<DeveloperEvent[]>("all-events.json");

Convert Events To The Proper Format

The form to submit events in the Tech Community Calendar look likes that:

The Tech Community Calendar events can be represented with the following record :

public record TechCommunityCalendarEvent(
    string Name,
    string Url,
    DateTimeOffset StartDate,
    DateTimeOffset EndDate,
    EventType EventType,
    EventFormat EventFormat,
    string Country,
    string City
)
{
    public string? TwitterHandle { get; set; }
};

💬 Positional parameters in a record are init-only. As I want to set the Twitter URL after the event has been created, I use a read-write property for it.

We can write a method to convert a DeveloperEvent to a TechCommunityCalendarEvent:

TechCommunityCalendarEvent ConvertToTechEvent(DeveloperEvent developerEvent)
{
    var startingDate = DateTimeOffset.FromUnixTimeMilliseconds(developerEvent.Date.First());
    var endingDate = DateTimeOffset.FromUnixTimeMilliseconds(developerEvent.Date.Last());
    var eventNameContainsYear = int.TryParse(developerEvent.Name.Split(" ").LastOrDefault(), out var year) 
                                && year == startingDate.Year;
    return new TechCommunityCalendarEvent(
        eventNameContainsYear ? developerEvent.Name : $"{developerEvent.Name} {startingDate.Year}",
        developerEvent.Hyperlink,
        startingDate,
        endingDate,
        EventType.Conference,
        developerEvent.Country is "Online" ? EventFormat.Virtual : EventFormat.In_Person,
        developerEvent.Country,
        developerEvent.City
    );
}

It allows us to convert all retrieved events after filtering on their date to only keep upcoming events.

var upcomingEvents = events
    .Where(e => e.Date.FirstOrDefault() > DateTimeOffset.UtcNow.ToUnixTimeMilliseconds())
    .Select(ConvertToTechEvent)
    .ToList();

Retrieve An Event Twitter Profile Link

In the submission form, there's an optional field for entering the Twitter Profile Link of an event. That's not something the events from the developers conferences agenda have but that's interesting data that could be useful to supply. All events have an associated website and most of them contain a link to their Twitter Profile on it.

This is where a library like AngleSharp, which can parse HTML according to W3C specifications, becomes useful. Although I have not used this library before, creating a method to find the Twitter URL on an event's webpage is straightforward.

async Task<string?> RetrieveEventTwitterProfileLink(string eventUrl)
{
    var context = BrowsingContext.New(Configuration.Default.WithDefaultLoader());
    var queryDocument = await context.OpenAsync(eventUrl);

    var twitterSelector = "a[href*='twitter.com'], a[href*='https://x.com']";
    var twitterSocialLink = queryDocument.QuerySelector(twitterSelector)
        ?.GetAttribute("href");

    return Uri.TryCreate(twitterSocialLink, UriKind.Absolute, out var twitterProfileUri) ?
        // Normalize X/Twitter profile URL by removing query parameters and fragments
        $"{twitterProfileUri.Scheme}://{twitterProfileUri.Host}{twitterProfileUri.AbsolutePath}" : null;
}

💬 As the DOM API exposed follows the W3C specifications, it is very convenient. If you can retrieve something with document.querySelector in your browser console, you will be able to retrieve it using the same selector in your AngleSharp code.

Submit An Event

Submitting forms is also possible using AngleSharp. We first have to retrieve the form element in the HTML document using the query sector form[action="/addevent/"]. Then we can directly submit the event.

async Task SubmitEventToTechCommunityCalendar(TechCommunityCalendarEvent techCommunityCalendarEvent)
{
    var context = BrowsingContext.New(Configuration.Default.WithDefaultLoader());
    var queryDocument = await context.OpenAsync("https://techcommunitycalendar.com/addevent/");
    var form = queryDocument.QuerySelector<IHtmlFormElement>("""form[action="/addevent/"]""");
    if (form is not null)
    {
        var response = await form.SubmitAsync(techCommunityCalendarEvent);
    }
}

💬 I intentionally named the properties in the TechCommunityCalendarEvent record with the same names as the fields in the form. This way, I can directly submit the event without any transformation. Otherwise, I would have to convert the event to an anonymous object with the correct names.

The Full Program

Here is the content of the complete Program.cs file.

using System.Net.Http.Json;
using AngleSharp;
using AngleSharp.Dom;
using AngleSharp.Html.Dom;

using var httpClient = new HttpClient()
{
    BaseAddress = new Uri("https://developers.events/")
};

var events = await httpClient.GetFromJsonAsync<DeveloperEvent[]>("all-events.json");
var upcomingEvents = events
    .Where(e => e.Date.FirstOrDefault() > DateTimeOffset.UtcNow.ToUnixTimeMilliseconds())
    .Select(ConvertToTechEvent)
    .ToList();

foreach (var upcomingEvent in upcomingEvents)
{
    upcomingEvent.TwitterHandle = await RetrieveEventTwitterProfileLink(upcomingEvent.Url);
    await SubmitEventToTechCommunityCalendar(upcomingEvent);
}

async Task<string?> RetrieveEventTwitterProfileLink(string eventUrl)
{
    var context = BrowsingContext.New(Configuration.Default.WithDefaultLoader());
    var queryDocument = await context.OpenAsync(eventUrl);

    var twitterSelector = "a[href*='twitter.com'], a[href*='https://x.com']";
    var twitterSocialLink = queryDocument.QuerySelector(twitterSelector)
        ?.GetAttribute("href");

    return Uri.TryCreate(twitterSocialLink, UriKind.Absolute, out var twitterProfileUri) ?
        // Normalize X/Twitter profile URL by removing query parameters and fragments
        $"{twitterProfileUri.Scheme}://{twitterProfileUri.Host}{twitterProfileUri.AbsolutePath}" : null;
}

async Task SubmitEventToTechCommunityCalendar(TechCommunityCalendarEvent techCommunityCalendarEvent)
{
    var context = BrowsingContext.New(Configuration.Default.WithDefaultLoader());
    var queryDocument = await context.OpenAsync("https://techcommunitycalendar.com/addevent/");
    var form = queryDocument.QuerySelector<IHtmlFormElement>("""form[action="/addevent/"]""");
    if (form is not null)
    {
        var response = await form.SubmitAsync(techCommunityCalendarEvent);
    }
}

TechCommunityCalendarEvent ConvertToTechEvent(DeveloperEvent developerEvent)
{
    var startingDate = DateTimeOffset.FromUnixTimeMilliseconds(developerEvent.Date.First());
    var endingDate = DateTimeOffset.FromUnixTimeMilliseconds(developerEvent.Date.Last());
    var eventNameContainsYear = int.TryParse(developerEvent.Name.Split(" ").LastOrDefault(), out var year) 
                                && year == startingDate.Year;
    return new TechCommunityCalendarEvent(
        eventNameContainsYear ? developerEvent.Name : $"{developerEvent.Name} {startingDate.Year}",
        developerEvent.Hyperlink,
        startingDate,
        endingDate,
        EventType.Conference,
        developerEvent.Country is "Online" ? EventFormat.Virtual : EventFormat.In_Person,
        developerEvent.Country,
        developerEvent.City
    );
}

public record DeveloperEvent(
    string Name,
    long[] Date,
    string Hyperlink,
    string Location,
    string City,
    string Country
);

public record TechCommunityCalendarEvent(
    string Name,
    string Url,
    DateTimeOffset StartDate,
    DateTimeOffset EndDate,
    EventType EventType,
    EventFormat EventFormat,
    string Country,
    string City
)
{
    public string? TwitterHandle { get; set; }
};

public enum EventFormat
{
    Unknown = 1,
    Virtual = 2,
    In_Person = 3,
    Hybrid = 4
}

public enum EventType
{
    Any = 0,
    Unknown = 1,
    Conference = 2,
    Meetup = 3,
    Hackathon = 4,
    Call_For_Papers = 5,
    Website = 6,
}

Keep it mind that it's a quick experiment to automate the submission of developer conferences to the Tech Community Calendar, not production-ready code.

Final Thoughts

I think this PoC is a good starting point to create a scheduled process that automatically submit the events from the developers conferences agenda to the tech community calendar.

💡Of course it would be great to do the opposite as well (automatically import events from the tech community calendar to the developers conferences agenda) but it seems complicated as events in the tech community calendar are stored in a database I don't have access to and it would involved parsing and writing in the README file of the developers conferences agenda repository.

Some ideas for improvement:

store somewhere the events already submitted to only process new events on each run
parallelize the processing of events as retrieving the twitter URL of submitting an event can take some time
reorganize the code

It was the first time I used AngleSharp, and I was happy with the result. It's a nice library that I would use again for similar needs.

A big thank you to the contributors of these IT event calendars. As someone who tries to attend tech events and speak at developer conferences, I find them incredibly useful. A special shoutout to Aurélie Vache and her developer conferences agenda for making this data openly available (JSON files with CFPs and conferences publicly accessible).

Using dependency injection with Azure .NET SDK

Alexandre Nédélec — Mon, 19 Feb 2024 00:00:00 +0000

I love how the Azure SDKs have evolved over the years. In the past, there was no consistency between the various Azure SDKs. However, that's not longer the case (at least for most Azure libraries), as they now adhere to the same principles and follow a set of well-defined guidelines.

💡You can learn more about these guidelines and how the Azure .NET SDKs work in this video from 2021 which is I think still relevant today.

Having consistency between libraries, it's easier to handle things like authentication and dependency injection consistently when you are using multiple Azure SDKs in your project.

One aspect often overlooked by people using Azure SDKs is the use of Microsoft.Extensions.Azure. This package facilitates registering and configuring the service clients for interacting with Azure APIs.

Let's see why using this package could be beneficial for your project.

Avoid making mistakes when registering service clients

It's mentioned in the documentation to use this package for dependency injection with the Azure SDK for .NET. Still, many people don't read the documentation and manually register the Azure service clients.

It's not a problem in itself if you know what you are doing. Otherwise,

you might choose the wrong lifetime for the Azure service clients, they must be singleton
you may forget to register a dependency that is needed for your use of the SDK

using Azure.Identity;
using DIWithAzureSDK;
using Microsoft.Extensions.Azure;

var builder = Host.CreateApplicationBuilder(args);
builder.Services.AddHostedService<Worker>();
builder.Services.AddAzureClients(clientBuilder =>
{
    clientBuilder.AddBlobServiceClient(new Uri("https://stdiwithazuresdk.blob.core.windows.net/"));
    clientBuilder.UseCredential(new DefaultAzureCredential());
});

var host = builder.Build();
host.Run();

In this sample, the AddBlobServiceClient handles the registration of all dependencies for us so that the BlobServiceClient can then be injected directly where needed.

public class Worker : BackgroundService
{
    private readonly ILogger<Worker> _logger;
    private readonly BlobServiceClient _blobServiceClient;

    public Worker(ILogger<Worker> logger, BlobServiceClient blobServiceClient)
    {
        _logger = logger;
        _blobServiceClient = blobServiceClient;
    }

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        await foreach (var blobContainer in _blobServiceClient.GetBlobContainersAsync(cancellationToken: stoppingToken))
        {
            _logger.LogInformation(blobContainer.Name);
        }
    }
}

💬 You may find it convenient to configure the dependency injection for all Azure service clients in a central place with the AddAzureClients method. When applications become larger with different csproj, I often prefer to separate service registration by business domain/module so having everything in a central place does not always suit my needs. That's not a problem, as the internal methods of the library make use of the TryAddd methods for registering services, I can call AddAzureClients in multiple places with only the services I want to register.

Easily manage the authentication to Azure services

All the SDKs use the Azure.Identity package to authenticate to Azure. There are different authentication methods available and you can easily specify which one to use with each client. Additionally, you can define a default authentication method for all clients, as demonstrated in the previous example.

builder.Services.AddAzureClients(clientBuilder =>
{
    clientBuilder.AddServiceBusClient("https://sb-diwithazuresdk.servicebus.windows.net/")
        .WithCredential(new ManagedIdentityCredential());

    clientBuilder.AddTableServiceClient(new Uri("https://stdiwithazuresdk.table.core.windows.net"))
        .WithCredential(new EnvironmentCredential());

    clientBuilder.AddBlobServiceClient(new Uri("https://stdiwithazuresdk.blob.core.windows.net/"));

    clientBuilder.UseCredential(new DefaultAzureCredential());
});

In the example above, we configured:

the service bus client to use the managed identity of the application to obtain a valid token for the service bus
the table client to use environment variables to obtain a valid token for the storage table
the blob client without any credentials so that it will use the one that we configured by default (with the UseCredential method)

Effortlessly configure the Azure clients' options

All Azure clients have options that can be effortlessly configured when registering them in the AddAzureClients method.

builder.Services.AddAzureClients(clientBuilder =>
{
    clientBuilder.AddBlobServiceClient(new Uri("https://stdiwithazuresdk.blob.core.windows.net/"))
        .WithCredential(new DefaultAzureCredential())
        .ConfigureOptions(options =>
        {
            options.TrimBlobNameSlashes = true;
            options.Retry.MaxRetries = 10;
            options.Diagnostics.IsLoggingEnabled = false;
        });
});

Some options are specific to the client (like the TrimBlobNameSlashes here for Blob client). Others can be configured globally and overridden on a client if necessary.

builder.Services.AddAzureClients(clientBuilder =>
{
   clientBuilder.AddBlobServiceClient(new Uri("https://stdiwithazuresdk.blob.core.windows.net/"))
        .WithCredential(new DefaultAzureCredential())
        .ConfigureOptions(options =>
        {
            options.TrimBlobNameSlashes = true;
            options.Retry.MaxRetries = 10;
            options.Diagnostics.IsLoggingEnabled = false;
        });

    clientBuilder.ConfigureDefaults(options =>
    {
        options.Retry.MaxRetries = 5;
        options.Retry.Mode = RetryMode.Exponential;
        options.Diagnostics.IsDistributedTracingEnabled = true;
    });
});

That's the purpose of the ConfigureDefaults method.

💡Please note that all this configuration (as well as the Uris of each client) can be loaded from the configuration like this clientBuilder.AddTableServiceClient(builder.Configuration.GetSection("Inventory:Tables"));

Use named clients for different Azure resources

Usually, you only need one client of each SDK in your application. Let's say you have multiple Azure Storage tables that are used in your application, you will only need to have one TableServiceClient. However, if you are interacting with tables in two different storage accounts, you will need multiple table clients.

To do that you can register your clients with a specific name:

builder.Services.AddAzureClients(clientBuilder =>
{
    clientBuilder.AddTableServiceClient(builder.Configuration.GetSection("Shop:Inventory"))
        .WithName("Shop");
    clientBuilder.AddTableServiceClient(builder.Configuration.GetSection("Warehouse:Inventory"))
        .WithName("Warehouse");
}

This way, you will be able to retrieve the specific client you need in your code:

public class WarehouseDeliveryService
{
    private readonly TableServiceClient _tableServiceClient;

    public WarehouseDeliveryService(IAzureClientFactory<TableServiceClient> azureClientFactory)
    {
        _tableServiceClient = azureClientFactory.CreateClient("Warehouse");
    }
}

Register a custom client factory

If you have specific needs, the AddClient method can help you register your Azure client while letting you control how you instantiate the client.

For instance, the Azure Cosmos Db .NET SDK is not built on the same foundation (Azure.Core) as the other SDKs. So at the time of writing, there is no AddCosmosServiceClient you can use in the AddAzureClients (there is an issue about that though). However, you can use the AddClient I've just mentioned.

builder.Services.AddOptions<CosmosDbConfiguration>().BindConfiguration("Warehouse:CosmosDb");
builder.Services.AddAzureClients(clientBuilder =>
{
    clientBuilder.AddClient<CosmosClient, CosmosClientOptions>((_, _, serviceProvider) =>
    {
        var cosmosConfiguration = serviceProvider.GetRequiredService<IOptions<CosmosDbConfiguration>>().Value;
        return new CosmosClientBuilder(cosmosConfiguration.Endpoint, cosmosConfiguration.AuthKey)
            .WithSerializerOptions(new () { PropertyNamingPolicy = CosmosPropertyNamingPolicy.CamelCase })
            .Build();
    }).WithName("Warehouse");
}

You can note that using the AddClient method allows us to take profit from the named clients' feature.

Wrapping up

As you have seen, the use of the Microsoft.Extensions.Azure package simplifies the registration and configuration of Azure clients. While providing you with a consistent way of handling the dependency injection for Azure SDKs, it also allows you to easily customize the authentication and other options available.

I hope you learned something. Don't hesitate to share your tips or what you like about the Azure SDKs in the comments.

Week 4, 2024 - Tips I learned this week

Alexandre Nédélec — Mon, 29 Jan 2024 00:00:00 +0000

Easily debug a non-HTTP-triggered Azure Function

The other day, I wanted to locally debug a Queue-triggered function without manually adding a queue message to my local storage.

My Azure Function looked like that:

public record Order(string Product,int Count);

public class ProcessOrder
{
    private readonly ILogger<ProcessOrder> _logger;

    public ProcessOrder(ILogger<ProcessOrder> logger)
    {
        _logger = logger;
    }

    [Function(nameof(ProcessOrder))]
    public void Run([QueueTrigger("orders")] Order sentOrder)
    {
        _logger.LogInformation($"Order contains {sentOrder.Count} {sentOrder.Product}");
    }
}

To trigger it, I could simply add a message in the order queue of my storage emulator like this:

You may notice that I don't even have to go to the Azure Storage Explorer to add the message, I can do it directly in the IDE. However, call me lazy but I wanted to execute the function just by making an HTTP call, like we do for HTTP-triggered functions.

This way, I could write the HTTP request in an HTTP file, commit it, and push it to my repository to share it with my colleagues, so they don't have to guess what message they should put in the queue to trigger the function.

Fortunately, the documentation explains how to do this.

Thus, for my use case, the resulting request is as follows:

POST http://localhost:7071/admin/functions/ProcessOrder HTTP/1.1
Content-Type: application/json

{
  "input": "{\n \"product\": \"laptop\",\n \"count\": 3\n}"
}

The content of your queue message goes in the value of the key "input" and must be escaped.

🚧 If like me, you skim through the documentation, you might miss the "escape" requirement and your request will fail so be sure to properly escape your content.

The Azure DevOps tip you did not know about: Azure Pipelines tasks name conflicts

I recently discovered that when you install extensions from the Azure DevOps marketplace, several Azure Pipelines tasks can have the same name. And if you use that name in your pipelines, Azure Pipelines won't know which task you are referring to and will prevent your pipeline from running.

This can easily occur if you install multiple extensions for Terraform in your Azure DevOps organization. For instance, the extensions Azure Pipelines Terraform Tasks from Jason Johnson and Terraform from Microsoft Dev Labs both have a task named the same way: TerraformInstaller.

To avoid these conflicts, you must use the full name of the tasks in your pipelines. You can find their full names in the GitHub repository of the extensions. Another way is to use these tasks in a test Release and click on the "View YAML" button to see the full name of the task you added.

Using metrics to understand your usage of Azure resources

I don't often use all my monthly free credits of my Azure subscription, but this month my spending limit was quickly reached and my subscription was disabled!

The cost analysis tab of my subscription showed me that an Azure Maps Account resource was responsible for consuming most of my credits but didn't provide more details.

So, I went to the Metrics tab of my resource and discovered that I could split the Usage metric by API name to determine exactly which Azure Maps API was heavily used by my applications. Combined with the pricing page, I can deduce which API requests I'm making too frequently and, therefore how to optimize costs.

Depending on the type of resource, you will use different metrics and split on different properties. Regardless, metrics can help you comprehend your resource usage and its associated cost.

And that's it for this week, happy learning!

Another year of sharing and learning - Dev Retro 2023

Alexandre Nédélec — Tue, 02 Jan 2024 00:00:00 +0000

Last year, I wrote my first annual retrospective. It was an interesting exercise that I intend to do every year. So for 2023, here is my year in review.

Plans for 2023 versus reality

My plans for 2023 that I shared in my 2022 retro were to:

keep learning about Vue.js and Nuxt.js
explore Azure Container Apps and Dapr
keep writing articles on my blog about topics I am interested in
keep sharing links and tips on social networks
improve my use of PKM tools like Obsidian
give at least 1 talk at a developer conference

I must admit that I didn't fully achieve my goals:

I continued learning Vue.js and Nuxt.js but not as extensively as I would have hoped
I didn't dive deeply into Azure Container Apps and Dapr although I did experiment with them a bit
I wrote articles on my blog but fewer than in the previous years
I shared links and tips on social networks but not consistently
I took my notes using Obsidian, but I haven't utilized it as a true PKM tool
I did give several talks at various developer conferences (more on that later)

It's not a big deal that I didn't accomplish everything. My primary goal for 2023 was for it to be a year of learning and sharing, just like in 2022. And I succeeded in doing that. 2023 was another year of learning and sharing, and it also provided numerous speaking opportunities. This is one of the reasons why I didn't have the time to do everything I planned.

Public speaking

In 2022, I gave my first talk at a developer conference (online). In 2023, I had the opportunity to speaker at five French developer conferences:

Global Azure France in Paris - May 2023
Cloud Est in Lyon - June 2023
Breizh Camp in Rennes - June 2023
BDX I/O in Bordeaux - November 2023
.NET Conf 2023 with MTG (online) - December 2023

The first three talks focused on Infrastructure as Code in general, and more specifically on Pulumi. I particularly enjoyed speaking at Breizh Camp, as many people attended my talk 🥰 and the organization was excellent.

The fourth talk showcased slidev, a tool for developers by Anthony Fu that allows creating slides in markdown (and using web technologies). It was a 15-minute talk titled "Oops, I Forgot to Make My Slides," during which a friend helped me create my slides about Vue 3 on stage. It was an incredibly fun talk to prepare and deliver. I cannot thank my co-speaker Xavier Noya enough for agreeing to do this talk with me. Additionally, I was delighted to be a speaker at this fantastic conference that takes place in my hometown.

The last talk was online (for a French event related to the .NET Conf 2023). It concentrated on the new features of C# 12 and .NET 8, and demonstrated how to implement Infrastructure as Code (IaC) using .NET.

🎥 The talks I gave are all in French, but if you are interested some of them have been recorded

I am incredibly proud of the numerous speaking opportunities I've had. While it may not seem impressive to experienced speakers, it means a lot to me. I am truly grateful for the chance to speak at these events and to have attended some fantastic talks as well.

Developer conferences' Call for Papers are highly selective, and they always receive many excellent proposals. Thus, I have no idea if I will be able to speak at multiple conferences in 2024, but I will certainly do my best.

Blogging

In 2023, I "only" wrote 11 articles on my blog, which is fewer than the 15 articles I wrote in 2022 and significantly less than the 19 articles in 2021.

For some of my articles, I created a GitHub repository with the code samples used in the article. That's something I intend to do more.

My blog's traffic decreased a little (not enough articles this year I guess):

27K users vs 28K
27K pages seen vs 37K

I kept cross-posting all my articles on Hashnode and dev.to, and published two of them on DZone.

Together with a friend, we initiated a blog post series about pnpm on a new team blog called "Bordeaux Coders". It was enjoyable, but our motivation waned after the summer. We need to regain our motivation, start writing again, and perhaps find others interested in collaborating on this blog.

I have also co-authored an article on my company's blog about something I have been doing for 5 years now: overseeing student projects at my former engineering school.

School Relationships and Teaching

As I mentioned, this year I once again supervised a group of students on a small software development project over a few months. This experience provided the opportunity to:

Explore new tools and technologies
Share knowledge with students and learn from them as well
Grow (by wearing different hats and utilizing educational management skills)
Promote my company's expertise and attract future talent

For the first time, in 2023, I taught a DevOps course at the same engineering school. It was an optional module on DevOps practices for 2nd-year students.

Building relationships with schools takes time and isn't always easy, but I enjoy doing it (and I'm not alone, as I have colleagues who help me). Unfortunately, I am uncertain whether my company will continue supporting me in this area next year, so I don't know what I will be able to do in 2024.

What's next?

Together with two friends, we have started a tech community called "MTG:Bordeaux," which will host meetups in Bordeaux to discuss Microsoft technologies (among others) several times a year. It is affiliated with MTG:France, which already encompasses numerous local communities in various French cities. The inaugural meetup is scheduled for February 1, 2024, and I hope it will be the first of many.

Instead of setting vague plans for 2024 that I might not fully achieve, I prefer creating a list of small, tangible goals for the year. I know I won't be able to accomplish all of them, but it will provide me with achievable objectives to work on throughout the year:

Organize 3 meetups for MTG:Bordeaux
Obtain the official Vue.js certification
Create a small speaker website in Nuxt, listing my previous talks
Build a small application using Dapr and running in Azure Container Apps
Write a blog post about Obsidian
Write 2 articles for the Vue CI/CD series on Bordeaux Coders
Present at least 2 different talks at developer conferences
Reach 1K followers on LinkedIn
Add missing sections to the Pulumi Azure Workshop
Develop a 1-day Pulumi training course
Create a YouTube video about a developer tool or technology

To conclude

Despite not fully achieving my goals, 2023 was an interesting year, especially regarding public speaking. Looking ahead to 2024, I have outlined a series of concrete goals that emphasize continuous learning and community involvement.

As I close the 2023 chapter, I want to thank 3 people:

Christian Bonnaud - you played a role in many aspects I mentioned in this article (school relationships, blogging, tech community, ...), and it's always a pleasure to collaborate with you.
Xavier Noya - it was nice to give a talk alongside you this year at BDX I/O
My life partner - I wouldn't be able to write these articles, prepare these talks, or accomplish everything I do without your support and understanding

Enjoy 2024, and keep learning.

Playing with the .NET 8 Web API template

Alexandre Nédélec — Tue, 19 Dec 2023 00:00:00 +0000

In this article, we will explore the latest C# 12 and .NET 8 features by applying them to the basic dotnet Web API template.

Getting started with the ASP.NET Core Web API template

First, let's install the latest .NET 8 SDK:

winget install --id Microsoft.DotNet.SDK.8

We can list the available templates:

Let's go for the basic ASP.NET Core Web API template but with the controllers:

dotnet new webapi --use-controllers -n WeatherApi

💡[Minimal APIs] are great too but having controllers is more suited to what I want to show in this article.

We can run the API and test the GET /weatherforecast endpoint using the generated request file:

@WeatherApi_HostAddress = http://localhost:5103

GET {{WeatherApi_HostAddress}}/weatherforecast/
Accept: application/json

This is included in the dotnet webapi template and is supported by Visual Studio, Rider, and vscode (using the REST Client extension)

💡Read my article about choosing an API Client and why I prefer versioned HTTP files rather than GUI tools like Postman.

If we put a breakpoint in the controller we can see one small ASP.NET 8 improvement concerning the debugging experience: better debug summaries are displayed for types like HttpContext.

Enhancing the Weather Forecast API

Currently, the template randomly generates weather forecasts in the controller. It would be nice to retrieve real weather data from a weather API.

To do that we can:

introduce an IWeatherService interface that contains a method to retrieve weather forecasts
extract the current logic that generates the random weather forecasts in a RandomWeatherService.cs that implements this interface
creates a new implementation OpenWeatherService of this interface that retrieves the weather data from the Open Weather Map API

The WeatherForecastController becomes:

[ApiController]
[Route("[controller]")]
public class WeatherForecastController : ControllerBase
{
    private readonly IWeatherService _weatherService;

    public WeatherForecastController(IWeatherService weatherService)
    {
        _weatherService = weatherService;
    }

    [HttpGet(Name = "GetWeatherForecast")]
    [ProducesResponseType(typeof(WeatherForecast), StatusCodes.Status200OK)]
    public Task<WeatherForecast[]> Get()
    {
        return _weatherService.GetWeatherForecasts();
    }
}

We can get rid of the typeof because there are now generic attributes for some common ASP.NET Core attributes like ProducesResponseType.

    [HttpGet(Name = "GetWeatherForecast")]
    [ProducesResponseType<WeatherForecast>(StatusCodes.Status200OK)]
    public Task<WeatherForecast[]> Get()
    {
        return _weatherService.GetWeatherForecasts();
    }

There are now 2 implementations of the IWeatherService interface:

RandomWeatherService that contains the code that previously was in the controller
OpenWeatherService that makes a call to the Open Weather Map API to retrieve the weather forecasts and then maps the obtained data to a list of WeatherForecast

public class OpenWeatherService : IWeatherService
{
    private readonly IOpenWeatherMapApi _openWeatherMapApi;
    private static readonly (double Latitude, double Longitude) BordeauxCoordinates = (44.837789, -0.57918);

    public OpenWeatherService(IOpenWeatherMapApi openWeatherMapApi)
    {
        _openWeatherMapApi = openWeatherMapApi;
    }

    public async Task<WeatherForecast[]> GetWeatherForecasts()
    {
        var weatherApiResponse = await _openWeatherMapApi.GetWeatherForecast(BordeauxCoordinates.Latitude, BordeauxCoordinates.Longitude);

        var computeWeatherSummary = (double temperature) =>
            temperature switch
            {
                < 0 => "Freezing",
                >= 0 and < 5 => "Bracing",
                >= 5 and < 12 => "Chilly",
                >= 12 and < 18 => "Cool",
                >= 18 and < 24 => "Mild",
                >= 24 and < 30 => "Warm",
                >= 30 and < 35 => "Balmy",
                >= 35 and < 40 => "Hot",
                >= 40 and < 45 => "Sweltering",
                >= 45 => "Scorching",
                _ => "Warm"
            };
        return weatherApiResponse.List
            .Select(x =>
                new WeatherForecast
                {
                    Date = DateOnly.FromDateTime(DateTimeOffset.FromUnixTimeSeconds(x.Dt).Date),
                    TemperatureC = Convert.ToInt32(x.Main.Temp),
                    Summary = computeWeatherSummary(x.Main.Temp)
                })
            .ToArray();
    }
}

The weather forecasts of a specific geolocation are retrieved. Indeed coordinates (corresponding to Bordeaux in France) are passed to the Open Weather Map API call. In C# 12, we can alias any type so we can introduce an alias "Coordinates" for the coordinates tuple:

using Coordinates = (double Latitude, double Longitude);

public class OpenWeatherService : IWeatherService
{
    private readonly IOpenWeatherMapApi _openWeatherMapApi;
    private static readonly Coordinates BordeauxCoordinates = (44.837789, -0.57918

Once this call is done, results are mapped to the expected model WeatherForecast. A lambda expression is used to get the "weather summary" from a temperature. If we want to have a default summary, that's something we can do thanks to the support of default lambda parameters in C#12.

var computeWeatherSummary = (double temperature, string defaultSummary = "Warm") =>
    temperature switch
    {
        < 0 => "Freezing",
        >= 0 and < 5 => "Bracing",
        >= 5 and < 12 => "Chilly",
        >= 12 and < 18 => "Cool",
        >= 18 and < 24 => "Mild",
        >= 24 and < 30 => "Warm",
        >= 30 and < 35 => "Balmy",
        >= 35 and < 40 => "Hot",
        >= 40 and < 45 => "Sweltering",
        >= 45 => "Scorching",
        _ => defaultSummary
    };

RandomWeatherService does not have this logic because Summaries are randomly selected from an array containing possible summaries.

private static readonly string[] Summaries = new [] { "Freezing", "Bracing", "Chilly", "Cool", "Mild", "Warm", "Balmy", "Hot", "Sweltering", "Scorching" };

With collection expressions, this array can be defined directly with square brackets.

private static readonly string[] Summaries = ["Freezing", "Bracing", "Chilly", "Cool", "Mild", "Warm", "Balmy", "Hot", "Sweltering", "Scorching"];

It would work with other types of collections as well. If we needed to have another list containing only cold summaries and avoid duplication between the two lists, we could also define the two lists and use the spread operator.

 private static readonly IList<string> ColdAdjectives = ["Freezing", "Bracing", "Chilly", "Cool"];
 private static readonly string[] Summaries = [..ColdAdjectives, "Mild", "Warm", "Balmy", "Hot", "Sweltering", "Scorching"];

The last C# 12 thing we could do in this example is to take advantage of the new class (and structs) primary constructors that were previously limited to records.

The WeatherForecast class could become the following:

namespace WeatherApi;

public class WeatherForecast(DateOnly date, int temperatureC, string? summary)
{
    public int TemperatureC { get; } = temperatureC;

    public DateOnly Date { get; } = date;

    public string? Summary { get; } = summary;

    public int TemperatureF => 32 + (int)(TemperatureC / 0.5556);
}

💬 I'm not sure this is completely relevant here, a record would probably be better but you get the idea.

You can use primary constructors in any class, it will work as well with dependency injection. However, be aware that the services you used to assign to a private read-only field of your class won't be read-only anymore like weatherService in this example:

public class WeatherForecastController(IWeatherService weatherService, ILogger<WeatherForecastController> logger) : ControllerBase
{
    [HttpGet(Name = "GetWeatherForecast")]
    [ProducesResponseType(typeof(WeatherForecast), StatusCodes.Status200OK)]
    public Task<WeatherForecast[]> Get()
    {
        return weatherService.GetWeatherForecasts();
    }
}

Having 2 different implementations of the IWeatherService is great, but what if you need one of them in some part of your code? The one you will have injected in your class is the last one registered in the DI container, but that may not be the one you want. You could get all of them by injecting IEnumerable<IWeatherService> and selecting the one you need. You could also create a sort of factory to retrieve the correct instance. Yet in .NET 8, you don't need to worry about all that because you have the keyed DI Services.

Specifying a key (that can be anything, not necessarily a string) is done when registering the services in the DI container.

builder.Services.AddKeyedTransient<IWeatherService, RandomWeatherService>("random");
builder.Services.AddKeyedTransient<IWeatherService, OpenWeatherService>("api");

With this key, retrieving a specific implementation becomes easy.

    public WeatherForecastController([FromKeyedServices("random")] IWeatherService weatherService, ILogger<WeatherForecastController> logger)
    {
        _logger = logger;
        _weatherService = weatherService;
    }

I did not discuss the code that requests the Open Weather Map API. It's quite simple thanks to the uses of Refit.

using Refit;

namespace WeatherApi.Services.OpenWeatherMap;

public interface IOpenWeatherMapApi
{
    [Get("/forecast?lat={latitude}&lon={longitude}&units=metric")]
    Task<WeatherMapResponse> GetWeatherForecast(double latitude, double longitude);
}

public record WeatherMapResponse(IList<WeatherMapForecast> List);

public record WeatherMapForecast(int Dt, WeatherMapMain Main);

public record WeatherMapMain(double Temp);

I created an HTTP Message Handler to take care of adding the Open Weather Map API key to the requests. This API key and the URL to the API come from the configuration and are mapped to a configuration object WeatherMapConfiguration.

In .NET 8, we can use data validation attributes for data like configuration options. There is also a source code generator that can implement the validation logic:

namespace WeatherApi.Services.OpenWeatherMap;

public class WeatherMapConfiguration
{
    [Required]
    public required string ApiKey { get; init; }

    [Required]
    [Url]
    public required string Uri { get; init; }

}

[OptionsValidator]
public partial class WeatherMapConfigurationValidator : IValidateOptions<WeatherMapConfiguration>
{
}

This way we can make sure that the configuration contains the API Key and the URI that has the Url format. The configuration in the Program.cs looks like that:

builder.Services.Configure<WeatherMapConfiguration>(builder.Configuration.GetSection("WeatherMap"));
builder.Services.AddSingleton<IValidateOptions<WeatherMapConfiguration>, WeatherMapConfigurationValidator>();

builder.Services.AddTransient<ApiKeyHandler>();
builder.Services.AddRefitClient<IOpenWeatherMapApi>()
    .ConfigureHttpClient((provider, client) =>
    {
        var configuration = provider.GetRequiredService<IOptions<WeatherMapConfiguration>>().Value;
        client.BaseAddress = new Uri(configuration.Uri);
    })
    .AddHttpMessageHandler<ApiKeyHandler>();

A few closing words

Here is the recap of what we talked about:

Feature	Area
Support for generic attributes	.NET 8
Primary constructors	C# 12
Collection expressions	C# 12
Optional parameters in lambda expressions	C# 12
Alias any type	C# 12
Debug customization attributes on ASP.NET Core types	ASP.NET Core 8
Options validation	.NET 8
Keyed DI Services	.NET 8

There are many more interesting features in C# 12, .NET 8, or ASP.NET Core 8. Yet, the ones I introduced in this article are the ones I will probably use the most.

You can find the complete code sample here. The repository also contains a folder infra to set up the Azure infrastructure to host this API. 2 IaC solutions that use .NET are shown: one using Azure SDK and one using Pulumi.

This article was published as part of the C# Advent 2023 which is a nice initiative. Make sure to check the other blog articles on the advent calendar.

Effortlessly Configure GitHub Repositories for Azure Deployment via OIDC

Alexandre Nédélec — Mon, 23 Oct 2023 00:00:00 +0000

What if we could script the creation and configuration of a GitHub Repository so that it is ready to provision or deploy Azure resources from a GitHub Actions pipeline? We will do that in this article using the Azure CLI and GitHub CLI.

The Objective

The goal is to go from nothing to running a GitHub Actions workflow that authenticates to Azure using Open ID Connect (so without secret credentials) in a newly created GitHub repository.

The workflow we plan to run is as follows:

name: Run Azure Login with OIDC
on:
  workflow_dispatch:

permissions:
  id-token: write
  contents: read
jobs:
  build-and-deploy:
    runs-on: ubuntu-latest
    steps:
      - name: 'Az CLI login'
        uses: azure/login@v1
        with:
          client-id: ${{ secrets.AZURE_CLIENT_ID }}
          tenant-id: ${{ secrets.AZURE_TENANT_ID }}
          subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}

      - name: 'Run az commands'
        run: |
          az account show
          az group list

This workflow is an example coming from the GitHub documentation showing how to configure GitHub Actions workflow to access Azure resources protected by Microsoft Entra.

To run this workflow we will need to automate the configuration of these resources:

💬 Looks familiar? That's the same diagram from my article about creating an Azure-Ready GitHub Repository using Pulumi. The purpose was the same but using Pulumi instead of CLI tools. If you prefer a declarative Infrastructure as Code approach using programming languages over CLI tools, you should definitively read it 😉

The Script

A word about the tools used

I will be using PowerShell which is cross-platform. However, if you prefer using a different shell, you will simply need to adjust some syntax (such as the environment variable declarations) to ensure compatibility.

To create and configure the Microsoft Entra ID resources, we will need the Azure CLI.

To create and configure the GitHub repository, we will need the GitHub CLI.

Create the repository on GitHub

Let's assume we are already in a new directory with the YAML workflow file .github\workflows\main.yml in it.

First, we can initialize the git repository.

git init
git add .
git commit -m "Intialize repository with the GitHub Actions workflow file"

Second, we can create the GitHub repository and push the git repository we just initialized in it.

$repositoryName = "MyAzureReadyRepository"
gh repo create $repositoryName --private --source=. --push

💡You can use the --public flag instead of the --private one if you want your GitHub repository to be public.

The repository's full name (containing the organization name) can be retrieved like this:

$repositoryFullName=$(gh repo view --json nameWithOwner -q ".nameWithOwner")

💡Passing the --json flag converts the output format to JSON which, combined with the --q flag can be handy for filtering or formatting a command output. More on that in the documentation

Create the Microsoft Entra ID resources

Later, we will need the subscription and the tenant identifiers. Let's retrieve them now and take this opportunity to check that we are logged in on the correct tenant with the correct subscription selected.

$subscriptionId=$(az account show --query "id" -o tsv)
$tenantId=$(az account show --query "tenantId" -o tsv)

💬 Similar to the GitHub CLI, the Azure CLI has a --query flag to filter a command output. There are also different output formats. The tsv (tab-separated values) one is useful for capturing a value in an environment variable. If you are not very familiar with the Azure CLI, you can check my article on the topic here

To create the app registration and its associated service principal, we can execute the following commands:

$appId=$(az ad app create --display-name "GitHub Action OIDC for ${repositoryFullName}" --query "appId" -o tsv)
$servicePrincipalId=$(az ad sp create --id $appId --query "id" -o tsv)

We can now assign the contributor role to the service principal on the subscription.

az role assignment create --role contributor --subscription $subscriptionId --assignee-object-id $servicePrincipalId --assignee-principal-type ServicePrincipal --scope /subscriptions/$subscriptionId

Creating federated credentials is a bit more complex as one of the arguments needs to be an in-line JSON string.

$parametersJson = @{
    name = "FederatedIdentityForWorkshop"
    issuer = "https://token.actions.githubusercontent.com"
    subject = "repo:${repositoryFullName}:ref:refs/heads/main"
    description = "Deployments for ${repositoryFullName}"
    audiences = @(
        "api://AzureADTokenExchange"
    )
}

💡The subject property here specifies that the GitHub Actions workflow from the created repository is only authorized to authenticate to Azure when it runs on the main branch. Of course, there are other possible configurations, such as those involving pull requests or environments. Consult the documentation to learn more about these options.

To make this JSON string an inline string with escaped quotes that works for the Azure CLI, we have to transform the string using a command I found in this blog article.

$parameters = $($parametersJson | ConvertTo-Json -Depth 100 -Compress).Replace("`"", "\`"")

And finally, we can create the federated credentials.

az ad app federated-credential create --id $appId --parameters $parameters

Configure the GitHub Actions and run the workflow

For the OIDC authentication to function properly, we need to set 3 GitHub Actions Secrets (could also be GitHub Actions variables as there are not really secrets):

The identifier of the Azure tenant
The identifier of the Azure subscription
The application identifier of the app registration

gh secret set AZURE_TENANT_ID --body $tenantId
gh secret set AZURE_SUBSCRIPTION_ID --body $subscriptionId
gh secret set AZURE_CLIENT_ID --body $appId

We can directly run the workflow from the GitHub CLI, and watch the run until it is completed.

gh workflow run main.yml
$runId=$(gh run list --workflow=main.yml --json databaseId -q ".[0].databaseId")
gh run watch $runId

Full script

# Initialize git repository with current code
# You should have added the main.yml workflow file in the `.github\workflows` directory 
git init
git add .
git commit -m "Intialize repository with the GitHub Actions workflow file"

# Create a new remote private GitHub repository
$repositoryName = "MyAzureReadyRepository"
gh repo create $repositoryName --private --source=. --push

# Retrieve the repository full name (org/repo)
$repositoryFullName=$(gh repo view --json nameWithOwner -q ".nameWithOwner") 

# Retrieve the current subscription and current tenant identifiers 
$subscriptionId=$(az account show --query "id" -o tsv)
$tenantId=$(az account show --query "tenantId" -o tsv)

# Create an App Registration and its associated service principal
$appId=$(az ad app create --display-name "GitHub Action OIDC for ${repositoryFullName}" --query "appId" -o tsv)
$servicePrincipalId=$(az ad sp create --id $appId --query "id" -o tsv)

# Assign the contributor role to the service principal on the subscription
az role assignment create --role contributor --subscription $subscriptionId --assignee-object-id $servicePrincipalId --assignee-principal-type ServicePrincipal --scope /subscriptions/$subscriptionId

# Prepare parameters for federated credentials
$parametersJson = @{
    name = "FederatedIdentityForWorkshop"
    issuer = "https://token.actions.githubusercontent.com"
    subject = "repo:${repositoryFullName}:ref:refs/heads/main"
    description = "Deployments for ${repositoryFullName}"
    audiences = @(
        "api://AzureADTokenExchange"
    )
}

# Change parameters to single line string with escaped quotes to make it work with Azure CLI
# https://medium.com/medialesson/use-dynamic-json-strings-with-azure-cli-commands-in-powershell-b191eccc8e9b
$parameters = $($parametersJson | ConvertTo-Json -Depth 100 -Compress).Replace("`"", "\`"")

# Create federated credentials
az ad app federated-credential create --id $appId --parameters $parameters

# Create GitHub secrets needed for the GitHub Actions
gh secret set AZURE_TENANT_ID --body $tenantId
gh secret set AZURE_SUBSCRIPTION_ID --body $subscriptionId
gh secret set AZURE_CLIENT_ID --body $appId

# Run workflow
gh workflow run main.yml
$runId=$(gh run list --workflow=main.yml --json databaseId -q ".[0].databaseId")
gh run watch $runId

# Open the repostory in the browser
gh repo view -w

Final Thoughts

I am very glad to have scripted the creation and configuration of a GitHub repository ready to deploy to Azure. Even if I had already done the same using Pulumi, having a small script can sometimes be more convenient than having a full IaC program. In my case, I needed to automate that for a workshop, so it was easier to give participants a script to execute.

However, I must admit that developing this script proved to be much more challenging than provisioning the same resources using Pulumi. I didn't expect it to take so much time: browsing the CLI documentation, finding the correct syntax, and understanding the cause of failures. In contrast, using the GitHub and Azure Pulumi providers in my TypeScript code turned out to be a much more enjoyable experience.

Nevertheless, I was pleased to be introduced to the GitHub CLI, which I hadn't explored extensively until now. While I found it very useful, a few things bothered me. Not all commands can be used with the --json and -q parameters, which is not very convenient for scripting. Commands that create things (repo, workflow runs) don't return the identifier of the thing they create. I wish GitHub CLI would be more similar to Azure CLI in these matters. I have no doubt these will be improved over time.

As for Azure CLI, I am still a big fan, although a bit disappointed to have struggled with the inline JSON string.

Keep learning, keep sharing.

Deploying to Azure from Azure DevOps without secrets

Alexandre Nédélec — Thu, 21 Sep 2023 00:00:00 +0000

If you are deploying your application to Azure from Azure Pipelines, you might want to leverage the ability to do so without using secrets thanks to Workload identity federation. In this article, I will demonstrate how to automate the configuration of your Azure DevOps project, with everything pre-configured to securely deploy applications to Azure.

Why should you use Workload Identity Federation for your deployment pipelines?

I already wrote about the problem of secret credentials, but let me remind you 2 reasons why I think you should always avoid using secrets in your deployment pipelines:

It's more secure if you don't need a secret to authenticate to Azure
It's more practical if you don't need to handle secret rotation when secrets expire

This is true whatever the CI/CD platform you are using.

Workload identity federation leverages OpenID Connect to solve these problems and avoid using secrets in your pipelines to authenticate to Azure. I previously published an article about using Azure OpenID Connect with Pulumi in GitHub Actions, but that also works with Azure Pipelines.

ℹ Microsoft has announced the public preview of Workload identity federation for Azure Pipelines on the 11th September 2023.

How can you use Workload Identity Federation to deploy to Azure from Azure Pipelines?

Azure Pipelines tasks use service connections to authenticate with external services. Specifically, for Azure, it is necessary to create an Azure Resource Manager service connection.

You can create an Azure Resource Manager service connection that uses workload identity federation by configuring it in your Azure DevOps organization portal (check the documentation here).

Or ... you can automate that using Infrastructure as Code 😉.

Yet, who wants to manually configure things from a wizard when everything can be automated in versioned code? So let's go the IaC way.

💬 All kidding aside, I genuinely believe that there are many advantages to provisioning your Azure DevOps projects and their associated resources (Repos, Service Connections, policies, pipelines, ...) using Infrastructure as Code. It takes time to properly configure Azure DevOps projects, and if they are often organized similarly, it's more efficient to automate their configuration rather than doing it manually.

I will use Pulumi and its Azure DevOps provider to provision the necessary resources. The infrastructure as code will be written in C# but you could easily convert the C# code to any language that Pulumi supports (like TypeScript, I am a big fan of using TypeScript to write infrastructure code 🔥).

Here is the complete solution to implement:

Automate the configuration of Workload identity federation in Azure DevOps

Create the Pulumi .NET project

Let's start by scaffolding a new Pulumi project using .NET:

pulumi new csharp -n AzureDevOpsWorkloadIdentity -s dev -d "A program to set up an Azure-Ready Azure DevOps repository"

This command creates a new pulumi project and stack from the csharp template:

The name of the project "AzureDevOpsWorkloadIdentity" is specified using the -n option
The description of the project "A program to set up an Azure-Ready Azure DevOps repository" is specified using the -d option
The stack of the project "dev" is specified using the -s option

This project will need 3 different providers:

the Azure Native provider
the Azure Active Directory provider (provider for Microsoft Entra ID)
the Azure DevOps provider

So we can add the following Nuget packages to our project:

Create the Azure DevOps project

First, we must select the Azure DevOps organization where we wish to create a project and set its URL in our Pulumi configuration.

pulumi config set azuredevops:orgServiceUrl XXXXXXXXXXXXXX --secret

Second, we need to supply the necessary Azure DevOps credentials. For that, we can create a personal access token and add it to our Pulumi configuration.

pulumi config set azuredevops:personalAccessToken YYYYYYYYYYYYYY --secret

🔐 I followed the documentation but to be honest, I don't think it's necessary to include the --secret option for the organization URL as it's not really a sensitive value that needs to be encrypted. However, it's mandatory to include it for the access token so that we can safely commit the configuration files without creating security risks.

Third, we can create the DevOps project:

var project = new Project("AzureReadyADOProject", new()
{
    Description = "New project with everything correctly configured to provision Azure resources or deploy applications to Azure",
    Features = new()
    {
        ["boards"] = "disabled",
        ["repositories"] = "enabled",
        ["pipelines"] = "enabled",
        ["testplans"] = "disabled",
        ["artifacts"] = "disabled"
    },
});

I intentionally disabled Azure Boards, Azure Test Plans, and Azure Artifacts as we only need Azure Repos and Azure Pipelines for this demo but you can enable what you need for your projects.

By default, when we create an Azure DevOps project, a Git repository is created for us with the same name as the project. This repository can be retrieved using the following code:

var repository = GetGitRepository.Invoke(new()
{
    ProjectId = project.Id,
    Name = project.Name
});

We can also choose to create a new Git repository like this:

var repository = new Git("AzureReadyADORepository", new()
{
    ProjectId = project.Id,
    Initialization = new GitInitializationArgs()
    {
        InitType = "Clean",
        SourceType = "Git",
        SourceUrl = "https://repo.com",
        ServiceConnectionId = ""
    },
    DefaultBranch = "refs/heads/main"
});

ℹ We should not have to set the SourceType, SourceUrl and ServiceConnectionId properties as we are initializing a clean Git repository, not importing one, but it's a workaround because of this issue on the provider.

Configure the ARM Service Connection in Azure DevOps

In the Azure DevOps provider, the Azure Resource Manager service connection is called a ServiceEndpointAzureRM. We can create such a resource like this:

var serviceConnection = new ServiceEndpointAzureRM("AzureServiceConnection", new()
{
    ProjectId = project.Id,
    ServiceEndpointName = "azure-with-oidc",
    ServiceEndpointAuthenticationScheme = "WorkloadIdentityFederation",
    AzurermSpnTenantid = tenantId,
    AzurermSubscriptionId = subscriptionId,
    AzurermSubscriptionName = subscriptionName,
    Credentials = new ServiceEndpointAzureRMCredentialsArgs()
    {
        Serviceprincipalid = servicePrincipal.ApplicationId,
    }
});

Do not worry about the service principal, we will see in the next section how to create it. The tenant and the subscription identifiers can be retrieved from the current configuration of the Azure Native provider (using the GetClientConfig.Invoke function):

var azureConfig = GetClientConfig.Invoke();
var tenantId = azureConfig.Apply(c => c.tenantId);
var subscriptionId = azureConfig.Apply(c => c.SubscriptionId);

For the subscription name, it's more complicated as we don't have it, and no easy way to retrieve it. To be frank, I think having to provide the subscription name while we already provide the subscription identifier is completely useless but that's how the Azure DevOps provider works.

The Azure Classic provider offers a function to get a subscription by its identifier but it's not available in the Azure Native provider. I don't want to add the Azure Classic provider to my project solely for this purpose. However, it's not a big deal as it allows us to experience one of the advantages of using Pulumi: when something is not available you can just implement it or use any library that can help you, such as the Azure SDK in this case.

var subscriptionName = subscriptionId.Apply(s =>
{
    var armClient = new ArmClient(new DefaultAzureCredential());
    var subscription = armClient.GetSubscriptionResource(new ResourceIdentifier($"/subscriptions/{s}")).Get();
    return subscription.Value.Data.DisplayName;
});

Set up the necessary Microsoft Entra ID resources

We need to set up the following resources in Microsoft Entra ID:

an Application that represents the Azure DevOps service connection identity
a Service Principal (related to the application above) that has the contributor role on the Azure subscription
credentials for the CI/CD pipeline to authenticate to Azure on behalf of this Microsoft Entra ID application

Let's take care of the first 2 points:

var azureConfig = GetClientConfig.Invoke();
var aadApplication = new Application("ADOAzureReadyApp", new()
{
    DisplayName = "ADO Azure Ready App"
});
var servicePrincipal = new ServicePrincipal("AzureReadyServicePrincipal", new()
{
    ApplicationId = aadApplication.ApplicationId,
});

var subscriptionId = azureConfig.Apply(c => c.SubscriptionId);
new RoleAssignment("contributor", new()
{
    PrincipalId= servicePrincipal.Id,
    PrincipalType= PrincipalType.ServicePrincipal,
    RoleDefinitionId = AzureBuiltInRoles.Contributor,
    Scope = Output.Format($"/subscriptions/{subscriptionId}")
});

ℹ️ It's worth mentioning that using an Application and its associated Service Principal is not the only way to proceed, we could have created instead a User Assigned Identity

Now that everything is created, we can create the Federated identity credentials:

new ApplicationFederatedIdentityCredential("ADOAzureReadyAppFederatedIdentityCredential", new() 
{
    ApplicationObjectId = aadApplication.ObjectId,
    DisplayName = "AzureReadyDeploys",
    Description = "Deployments for azure-ready-repository",
    Audiences = new(){"api://AzureADTokenExchange" },
    Issuer = serviceConnection.WorkloadIdentityFederationIssuer,
    Subject = Output.Format($"sc://{organisationName}/{project.Name}/{serviceConnection.ServiceEndpointName}")
});

You can observe that the federation subject adheres to a particular format (sc://<org>/<project>/<service connection name>), which identifies the service connection authorized for authentication with Azure.

Create the deployment pipeline

We have completed the configuration of an ARM Service Connection that employs Workload Identity Federation for authentication with Azure. While we could stop at this point, it would be nice to automate the creation of a pipeline that utilizes this service connection and seize the opportunity to ensure everything works properly.

For this purpose, I have written a very simple YAML pipeline that runs the AzureCLI task to show information about the Azure subscription associated with the previously created service connection.

trigger:
  - main

pool:
  vmImage: ubuntu-latest

steps:
  - task: AzureCLI@2
    inputs:
      azureSubscription: 'azure-with-oidc'
      scriptType: 'pscore'
      scriptLocation: 'inlineScript'
      inlineScript: 'az account show --query id -o tsv'

We can add this file in the Git repository:

var pipelineFile = new GitRepositoryFile("AzurePipeline", new()
{
    File = "azure-pipelines.yaml",
    RepositoryId = repository.Apply(r => r.Id),
    CommitMessage = "Add preconfigured pipeline file",
    Content = File.ReadAllText("azure-pipelines.yml"),
    Branch = "refs/heads/main"
});

Now, we have to create the pipeline itself:

var pipeline = new BuildDefinition("deployToAzure", new()
{
    ProjectId = project.Id,
    Repository = new BuildDefinitionRepositoryArgs()
    {
        RepoId = repository.Apply(r => r.Id),
        BranchName = "refs/heads/main",
        YmlPath = pipelineFile.File,
        RepoType = "TfsGit"
    }
});

To complete the automation process, we can authorize the pipeline to utilize the service connection, eliminating the need for manual intervention through the portal:

new PipelineAuthorization("azureOidcPipelineAuthorization", new()
{
    ProjectId = project.Id,
    Type = "endpoint",
    PipelineId = pipeline.Id.Apply(int.Parse),
    ResourceId = serviceConnection.Id
});

The last thing we can do is create a stack output to expose the URL of the created pipeline:

return new Dictionary<string, object?>
{
    ["pipelineUrl"] = Output.Format($"{organizationUrl}{project.Name}/_build?definitionId={pipeline.Id}")
};

Now we can execute the pulumi up command to provision all these resources and then open the pipeline page in our browser to test the pipeline.

💡On Windows, you can use the start $(pulumi stack output pipelineUrl) command to directly open the browser on the pipeline page. If you are using Nushell the command will be pulumi stack output pipelineUrl | start $in

Everything is working as expected.

To conclude

In this article, we demonstrated how to automate the configuration of an Azure DevOps project using Workload Identity Federation for secure deployments to Azure. We covered the provisioning of the Microsoft Entra ID and Azure DevOps resources necessary to make this work. It's very similar to what can be done for GitHub but with the specificities of Azure DevOps.

It was an opportunity for me to work with the Azure DevOps provider. Even if it does the job, I must admit I was somewhat disappointed with the developer experience which I found to be not very intuitive, with poorly named resources and an overreliance on strings as parameters. I assume that the Azure DevOps APIs are primarily responsible for this, as they are what the provider calls upon.

One thing I find interesting with Azure DevOps is that YAML pipelines do not need to be updated to take advantage of workload identity federation as long as the Azure Pipelines tasks you are using support it and your ARM service connection has been converted to workload identity federation.

Anyway, regardless of the CI/CD platform you are using, I believe that employing Workload Identity Federation to deploy code to Azure from pipelines is the right approach.

You can find the complete source code used for this article in this GitHub repository.

Create an Azure-Ready GitHub Repository using Pulumi

Alexandre Nédélec — Fri, 21 Jul 2023 00:00:00 +0000

Creating an application and deploying it to Azure is not complicated. You write some code on your machine, do some clicks in the Azure portal, or run some Azure CLI commands from your terminal and that's it: your application is up and running in Azure.

Yet, that's not real life, at least not what you will do when working on a professional project. Your code needs to be versioned and pushed to a location where your colleagues can work on it. The provisioning of Azure resources and deployment to Azure should be carried out using a properly configured CI/CD pipeline with the necessary authorization.

That's a lot of work that would need to be done each time you start a new project. So let's automate that using Pulumi to simplify the process and create an "Azure-Ready GitHub repository".

What's an Azure-Ready GitHub repository?

"Azure-Ready GitHub repository" is not an official term or concept, it's just something I've come up with to describe a Github repository that has everything correctly configured to provision Azure resources or deploy applications to Azure from a GitHub Actions CI/CD pipeline.

The GitHub part

On the GitHub side, to have an Azure-Ready GitHub repository, we need:

the GitHub repository itself (already initialized with a main branch)
the necessary GitHub Actions variables/secrets to authenticate to the correct Azure subscription
a YAML file located in the .github/workflows/ folder that contains the CI/CD pipeline that provisions resources in Azure

The Azure part

On the Azure side, to have an Azure-Ready GitHub repository, we need:

the existing Azure subscription to which resources are deployed
an identity in the Azure Active Directory of the desired tenant so that the GitHub CI/CD pipeline can authenticate to Azure and interact with the subscription

ℹ️ Azure Active Directory has recently been renamed Microsoft Entra ID (as of the time of writing). However, I will continue to use the term Azure Active Directory throughout the rest of the article. Please note that both terms refer to the same service.

The problem with secret credentials

People tend to use secret credentials to authenticate their pipeline to Azure and that's not the best thing to do.

From a security standpoint, depending on secrets always poses a security risk. Even if in that case the secret would be safely stored in a GitHub secret and never exposed publicly, it's still better to avoid secrets when we can.

🔐 That's precisely why when hosting applications in Azure, we use Managed Identities and IAM roles instead of relying on secrets. Yet, here we can't use Managed Identities for GitHub Actions pipelines.

From a practical standpoint, depending on secrets can quickly become problematic as they expire and thus require rotation. Of course, you can set up alerting or automate secret rotation but that's something you would prefer to avoid managing.

💬 I recently encountered a situation in Azure DevOps where a deployment failed due to the expiration of an Azure AD Application secret associated with the Service Connection used in the pipeline, and we were not alerted about it. That's the kind of scenario that can easily happen with secrets and that you want to avoid.

So what can we do about that?

👉 We can stop using secret credentials and use Workload identity federation instead. I suggest you have a look at this GitHub documentation page as well to better understand how it works but basically, you can remember the following:

this mechanism relies on Open ID Connect and trust between Azure and GitHub
the GitHub pipeline does not need an Azure AD application secret anymore to authenticate to Azure
it's not an Azure thing only, it's an open standard that also works with other cloud providers and other platforms than Github

To establish the trust relationship between the Azure AD application and the GitHub repository, a Federated Identity Credential must be created in the Azure Active Directory. You can find how to do that manually from the portal in the documentation but we are going to directly automate that 😉.

The complete solution to implement

Why use Pulumi in that context?

You might wonder why I chose to automate this process using Pulumi instead of writing a Bash or PowerShell script that would execute commands from the GitHub CLI and the Azure CLI.

💡By the way, you should check GitHub CLI if you have not done it yet, it's very handy. And if you have read my article about Azure CLI, you know it's a very convenient tool as well.

I think Pulumi is a better choice here because:

a script is imperative by nature, but declarative infrastructure seems more suitable to avoid dealing with idempotency
Pulumi can interact with both GitHub and Azure using its providers
the code will be easier to write and maintain
the code could be integrated into any application (including a future self-service infrastructure portal) using Pulumi Automation API

In this article, the Pulumi code will be in TypeScript but it would work in any language supported by Pulumi.

Automate the creation of the Azure-Ready GitHub Repository

Create the Pulumi project

Let's start by scaffolding a new Pulumi project using TypeScript:

pulumi new typescript -n AzureOIDC -s dev -d "A program to set up an Azure-Ready GitHub repository"

This command creates a new pulumi project and stack from the TypeScript template:

The name of the project "AzureOIDC" is specified using the -n option
The description of the project "A program to set up an Azure-Ready GitHub repository" is specified using the -d option
The stack of the project "dev" is specified using the -s option

ℹ By default, the pulumi new command installs the dependencies when creating the project. You can prevent this by specifying the -g option, which is useful when you want to use another package manager than the default one (pnpm instead of npm for instance).

This project will need 3 different providers:

So we can add the following packages to our package.json file:

@pulumi/azure-native
@pulumi/azuread
@pulumi/github

Create the repository on GitHub

To use the GitHub provider, we have to provide GitHub credentials. For that, we can create a personal access token (I prefer to create a fine-grained personal access token although a classic personal access token would also work). Next, we simply set the GitHub token in our Pulumi configuration, and the GitHub provider will automatically use it:

pulumi config set github:token XXXXXXXXXXXXXX --secret

🔐 Don't forget to include the --secret option when setting sensitive configurations, as this ensures that Pulumi encrypts the information. By doing so, we can safely commit the configuration files without creating security risks.

Now, it's time to create our GitHub repository!

import * as github from "@pulumi/github";

const repository = new github.Repository("azure-ready-repository", {
  name: "azure-ready-repository",
  visibility: "public",
  autoInit: true
});

export const repositoryCloneUrl = repository.httpCloneUrl;

Pulumi has an auto-naming capability that is very convenient to prevent name collisions or to ensure zero-downtime resource updates. Yet, in this context, I prefer to avoid a random suffix in my GitHub repository name, that's why I am specifying the name property to override the auto-naming behavior.

The last line creates a stack output named repositoryCloneUrl so that we can easily get the URL to clone our newly created repository.

ℹ I wanted the repository to be initialized, that's why I set the autoInit property to true but you should set it to false if you have an existing local git repository that you want to push on this GitHub repository.

Create the identity in Azure Active Directory for the GitHub Actions workflow

Creating an Azure AD application and its service principal is not very complicated:

import * as azuread from "@pulumi/azuread";

const aadApplication = new azuread.Application("AzureReadyApp", { displayName: "Azure Ready App" });
const servicePrincipal = new azuread.ServicePrincipal("AzureReadServicePrincipal", {
  applicationId: aadApplication.applicationId,
});

The OIDC trust thing is a bit more complex. Fortunately, Microsoft's documentation has a detailed page Configuring an app to trust an external identity provider that explains everything and shows how to add a federated identity for GitHub Actions using the Azure Portal, Azure CLI, or Azure PowerShell.

Let's do the same thing using TypeScript and Pulumi Azure AD provider:

new azuread.ApplicationFederatedIdentityCredential("AzureReadyAppFederatedIdentityCredential", {
  applicationObjectId: aadApplication.objectId,
  displayName: "AzureReadyDeploys",
  description: "Deployments for azure-ready-repository",
  audiences: ["api://AzureADTokenExchange"],
  issuer: "https://token.actions.githubusercontent.com",
  subject: pulumi.interpolate`repo:${repository.fullName}:ref:refs/heads/main`,
});

The subject property is what identifies the repository where the GitHub Actions workflow will be authorized to exchange its GitHub token for an Azure access token. It's worth noting that it will only work if the GitHub Actions workflow is run on the git reference (branch or tag) or the environment you specify in subject. You can also specify that only workflows triggered by a pull request should be authorized. Here, I have used the main branch but I could create multiple Federated Identity Credentials with different subjects if needed.

With this configuration, the GitHub Actions workflow we create next will be able to obtain a valid Azure access token.

If you are interested in gaining a better understanding of how all this works, you can refer to this diagram from Microsoft's documentation (with GitHub serving as the external identity provider in our case).

Authorize the Service Principal to provision resources on the subscription

We have created everything we need to get a valid Azure access token, but we still have not authorized the application to provision resources on our subscription.

We can do that by giving the Contributor role to our service principal.

import * as authorization from "@pulumi/azure-native/authorization";
import { azureBuiltInRoles } from "./builtInRoles";

new authorization.RoleAssignment("contributor", {
  principalId: servicePrincipal.id,
  principalType: authorization.PrincipalType.ServicePrincipal,
  roleDefinitionId: azureBuiltInRoles.contributor,
  scope: pulumi.interpolate`/subscriptions/${subscriptionId}`,
});

I intentionally did not declare the variable subscriptionId in the code above. It's because it's up to you to choose how you will provide it. You may want to set it in the configuration and retrieve it from it :

const config = new pulumi.Config();
const subscriptionId = config.get("subscriptionId");

Or your might want to retrieve it from the current configuration of the Azure native provider :

const azureConfig = pulumi.output(authorization.getClientConfig());
const subscriptionId = azureConfig.subscriptionId;

Concerning, the contributor role definition identifier, I could have dynamically retrieved it using Azure APIs (like here). But honestly, as these identifiers don't change it's much easier to hardcode it in a dedicated builtInRoles.ts file.

export const azureBuiltInRoles = {
  contributor : "/providers/Microsoft.Authorization/roleDefinitions/b24988ac-6180-42a0-ab88-20f7382dd24c"
};

💡Please note that you don't have to work on the subscription scope. If you prefer to assign the contributor role (or any other role) to an existing resource group rather than the entire subscription, you can certainly do that as well.

Add the configuration for the GitHub Actions workflow

The next step is to correctly set the configuration for the GitHub Actions of our Azure-Ready GitHub repository.

The workflow requires three pieces of information for the OIDC authentication to function properly:

The identifier of the Azure tenant
The identifier of the Azure subscription
The application identifier (also known as client ID) of the previously created Azure AD application

These identifiers are not secrets, they are just identifiers so we could directly set them as GitHub Actions variables like this:

new github.ActionsVariable("tenantId", {
  repository: repository.name,
  variableName: "ARM_TENANT_ID",
  value: azureConfig.tenantId,
});

However, I like to keep my tenant id and my subscription id private so we will store them in GitHub secrets but that's not mandatory at all.

const azureConfig = pulumi.output(authorization.getClientConfig());

new github.ActionsSecret("tenantId", {
  repository: repository.name,
  secretName: "ARM_TENANT_ID",
  plaintextValue: azureConfig.tenantId,
});

new github.ActionsSecret("subscriptionId", {
  repository: repository.name,
  secretName: "ARM_SUBSCRIPTION_ID",
  plaintextValue: azureConfig.subscriptionId,
});

new github.ActionsSecret("clientId", {
  repository: repository.name,
  secretName: "ARM_CLIENT_ID",
  plaintextValue: aadApplication.applicationId,
});

ℹ Please note that could also use environments for deployment and their associated secrets and variables.

Create the GitHub Actions workflow

Everything seems to be properly configured to provision Azure resources from a GitHub Actions workflow in this new repository, except for the workflow itself. The goal here is to have a properly configured pipeline in the repository to get started provisioning Azure infrastructure.

Here is such a pipeline:

name: infra

on:
  workflow_dispatch:

permissions:
      id-token: write
      contents: read
jobs:
  provision-infra:
    runs-on: ubuntu-latest
    steps:
      - name: 'Az CLI login'
        uses: azure/login@v1
        with:
          client-id: ${{ secrets.AZURE_CLIENT_ID }}
          tenant-id: ${{ secrets.AZURE_TENANT_ID }}
          subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}

      - name: 'Run az commands'
        run: |
          az account show
          az group list

This workflow first authenticates to Azure using OIDC with the azure/login action and then performs some Azure CLI commands to interact with Azure resources. That's fine and probably enough to get you started but you surely want to provision your infrastructure using a more declarative solution than an Azure CLI script. So let's see a more interesting pipeline still authenticating via Azure OIDC but using Pulumi to provision the Azure resources.

name: infra

on:
  workflow_dispatch:

permissions:
  id-token: write # required for OIDC auth
  contents: read # required to perform a checkout

jobs:
  provision-infra:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Install pnpm
        uses: pnpm/action-setup@v2
        with:
          version: latest

      - name: Set node version to 18
        uses: actions/setup-node@v3
        with:
          node-version: 18
          cache: 'pnpm'

      - name: Install dependencies
        run: pnpm install

      - name: Provision infrastructure
        uses: pulumi/actions@v4.4.0
        id: pulumi
        with:
          command: up
          stack-name: dev
        env:
          ARM_USE_OIDC: true
          PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}
          ARM_CLIENT_ID: ${{ secrets.ARM_CLIENT_ID }}
          ARM_TENANT_ID: ${{ secrets.ARM_TENANT_ID }}
          ARM_SUBSCRIPTION_ID: ${{ secrets.ARM_SUBSCRIPTION_ID }}

A permission section is required with 2 settings (more details here):

id-token: write ➡️ needed to request the OIDC token
contents: read ➡️ needed to perform checkout action

ℹ When you start to specify specific permissions, you have to specify all the permissions you need for the job because the default permissions won't apply anymore.

The 3 steps following the checkout step are actions to specify the Node.js version to use, install and correctly configure pnpm. We assume here the infrastructure will be provisioned using TypeScript (and Pulumi of course) but there would have been similar steps with other runtimes/languages (a setup-dotnet and a dotnet retore action for .NET for instance).

The last action is the Pulumi action to provision the infrastructure by running the pulumi up on the dev stack. We can see that this action uses environment variables whose values are based on the GitHub Actions secrets we defined earlier. To tell Pulumi to use OIDC, we just have to set the ARM_USE_OIDC environment variable to true.

        env:
          ARM_USE_OIDC: true
          PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}
          ARM_CLIENT_ID: ${{ secrets.ARM_CLIENT_ID }}
          ARM_TENANT_ID: ${{ secrets.ARM_TENANT_ID }}
          ARM_SUBSCRIPTION_ID: ${{ secrets.ARM_SUBSCRIPTION_ID }}

A GitHub Actions secret we did not talk about is PULUMI_ACCESS_TOKEN that is a Pulumi access token to use Pulumi Cloud as our backend to store the infrastructure state and encrypt secrets. This token should be:

Created from Pulumi Cloud (following the documentation here)
Stored in the stack configuration using the following command pulumi config set pulumiTokenForRepository ******* --secret
Stored in a GitHub Actions secret using this code

The last thing to do is to add this workflow file to the GitHub repository:

import { readFileSync } from "fs";

const pipelineContent = readFileSync("main.yml", "utf-8");
new github.RepositoryFile("pipelineRepositoryFile", {
  repository: repository.name,
  branch: "main",
  file: ".github/workflows/main.yml",
  content: pipelineContent,
  commitMessage: "Add preconfigured pipeline file",
  commitAuthor: "Alexandre Nédélec",
  commitEmail: "15186176+TechWatching@users.noreply.github.com",
  overwriteOnCreate: true,
});

This code:

reads the main.yml file that contains the workflow we saw previously
creates a file with this content in the repository in the .github/workflows/ folder for the GitHub Actions workflows
makes a commit when creating the file (or modifying it)

💬 To read the YAML file, I use the readFileSync method from the File System API fs. That's one of the things I love about Pulumi: you use the things you already know and that already exist in your ecosystem. No need to look for a module or wait for someone to write one, there is probably something standard or a popular community library you can use.

Test the Azure-Ready GitHub Repository

Now that the infrastructure code to provision the Azure-Ready GitHub repository is written, let's run it with the pulumi up command and see if it works!

All the resources are correctly created and our new GitHub repository is ready to be used.

Let's clone it.

git clone https://github.com/TechWatching/azure-ready-repository; cd azure-ready-repository

We want to verify that the GitHub project is properly configured and can provision Azure resources from its GitHub Actions workflow.

Let's add some infrastructure code that provisions a few Azure resources to check that:

pulumi new azure-typescript -n "AzureReadyGitHuRepository" -y --force

The --force option allows us to create the code within a non-empty directory.

I used the azure-typescript template that creates a storage account and outputs retrieve its primary access key.

🔒 In the SDK, the outputs of the function that lists the storage access keys are not currently marked as secrets. There is currently an open issue to change that but in the meantime, I have just modified the code to label the stack output as secret ensuring its encryption.

Let's run a pnpm install to install the dependencies and generate the pnpm-lock.yaml file. Then, we can push the code to GitHub and run the pipeline to see how it goes.

That's it, we succeeded to provision a storage account from our new GitHub repository whose creation and configuration were entirely automated using Pulumi.

To conclude

Additional information

There are different platforms you can use to host your Git repositories: GitHub, GitLab, and Azure DevOps to name a few. We use GitHub in this article but you can easily apply the same logic with other platforms (Pulumi has providers for GitLab and Azure DevOps as well).

Even though the Azure-Ready GitHub repository is provisioned using Pulumi, there's nothing stopping you from using another Infrastructure as Code solution that supports Azure OIDC (such as Azure CLI, which was mentioned in the article, Azure Bicep, or even Terraform) in the GitHub Actions workflow of the created repository. You don't even have to provision infrastructure; you can use this workflow to simply deploy an application to an existing Azure resource.

Potential Enhancements

There are many aspects that could be improved in the infrastructure code provisioning the Azure-Ready GitHub repository, but I believe the current solution serves as a good starting point. Nevertheless, here are some ideas for potential enhancements:

make additional items, such as the commit author, configurable
authorize an environment and not only a branch to retrieve an Azure token
use environment variables/secrets instead of variable/secrets at the repository scope

I think it would be interesting as well to put that code behind an API or a Web application using Pulumi Automation API to have a self-service solution to create Azure-Ready GitHub repository on the fly.

Here are some articles on the same topic I wanted to mention:

Stop using static cloud credentials in GitHub Actions by Lee Briggs

➡️ This post provides examples for configuring OIDC authentication with GitHub Actions for AWS, Azure, and GCP. The code for Azure is quite similar to the code I showed here. Yet, it doesn't go so far as to initialize a pipeline ready to deploy resources with Pulumi. Anyway, it's awesome to have the code for all 3 major providers.
Configuring GitHub Actions to Azure authentication with OIDC by Xavier Geerinck

➡️ This post also shows how to configure OIDC authentication with GitHub Actions and Azure but using an Azure CLI script. Although the GitHub repository creation and configuration are done manually, automating the Azure part with a few lines of script is nice.
Getting Rid of Passwords for Deployment with Pulumi OIDC Support by Sam Cogan

➡️ If you don't care about automating everything and simply want to configure OIDC authentication through the Azure portal, that's the post you will want to read. There is also an example of a pipeline to provision Azure infrastructure using a .NET Pulumi program.

Complete code solution

In this article, I aimed to provide a step-by-step explanation of how to automate the creation of a GitHub repository with a properly configured workflow to interact with Azure using OpenID Connect. Consequently, the article turned out to be quite lengthy. I apologize for that, but I didn't want to present the code without adequate explanation.

Anyway, now that we've covered everything, here is the complete code, which is just 75 lines long:

import * as pulumi from "@pulumi/pulumi";
import * as github from "@pulumi/github";
import * as azuread from "@pulumi/azuread";
import * as authorization from "@pulumi/azure-native/authorization";
import { azureBuiltInRoles } from "./builtInRoles";
import { readFileSync } from "fs";

const config = new pulumi.Config();

const repository = new github.Repository("azure-ready-repository", {
  name: "azure-ready-repository",
  visibility: "public",
  autoInit: true
});

export const repositoryCloneUrl = repository.httpCloneUrl;

const aadApplication = new azuread.Application("AzureReadyApp", { displayName: "Azure Ready App" });
const servicePrincipal = new azuread.ServicePrincipal("AzureReadyServicePrincipal", {
  applicationId: aadApplication.applicationId,
});
new azuread.ApplicationFederatedIdentityCredential("AzureReadyAppFederatedIdentityCredential", {
  applicationObjectId: aadApplication.objectId,
  displayName: "AzureReadyDeploys",
  description: "Deployments for azure-ready-repository",
  audiences: ["api://AzureADTokenExchange"],
  issuer: "https://token.actions.githubusercontent.com",
  subject: pulumi.interpolate`repo:${repository.fullName}:ref:refs/heads/main`,
});

const azureConfig = pulumi.output(authorization.getClientConfig());
const subscriptionId = azureConfig.subscriptionId;

new authorization.RoleAssignment("contributor", {
  principalId: servicePrincipal.id,
  principalType: authorization.PrincipalType.ServicePrincipal,
  roleDefinitionId: azureBuiltInRoles.contributor,
  scope: pulumi.interpolate`/subscriptions/${subscriptionId}`,
});

new github.ActionsSecret("tenantId", {
  repository: repository.name,
  secretName: "ARM_TENANT_ID",
  plaintextValue: azureConfig.tenantId,
});

new github.ActionsSecret("subscriptionId", {
  repository: repository.name,
  secretName: "ARM_SUBSCRIPTION_ID",
  plaintextValue: azureConfig.subscriptionId,
});

new github.ActionsSecret("clientId", {
  repository: repository.name,
  secretName: "ARM_CLIENT_ID",
  plaintextValue: aadApplication.applicationId,
});

new github.ActionsSecret("pulumiAccessToken", {
  repository: repository.name,
  secretName: "PULUMI_ACCESS_TOKEN",
  plaintextValue: config.requireSecret("pulumiTokenForRepository"),
});

const pipelineContent = readFileSync("main.yml", "utf-8");
new github.RepositoryFile("pipelineRepositoryFile", {
  repository: repository.name,
  branch: "main",
  file: ".github/workflows/main.yml",
  content: pipelineContent,
  commitMessage: "Add preconfigured pipeline file",
  commitAuthor: "Alexandre Nédélec",
  commitEmail: "15186176+TechWatching@users.noreply.github.com",
  overwriteOnCreate: true,
});

You can find the complete source code used for this article in this GitHub repository.

I hope you enjoyed this article. Please feel free to share your thoughts in the comments, ask questions, or make suggestions. Keep learning.

Who is using pnpm?

Alexandre Nédélec — Thu, 06 Jul 2023 15:25:40 +0000

You may have come across pnpm through discussions with fellow developers, reading blog posts, watching videos, or attending developer conferences. You have probably heard its praises: it's fast, disk-space efficient, and great for monorepos.

However, you might wonder: who is actually using pnpm?

A growing popularity

At the time of writing, pnpm has over 24k stars on GitHub, and this number is rapidly increasing. The pnpm Twitter account maintains a thread that tracks the number of stars. Each time the GitHub repository gains 1k stars, a new tweet is posted. For quite some time now, it has been growing by 1K every two months.

// Detect dark theme var iframe = document.getElementById('tweet-1666004997840986116-648'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1666004997840986116&theme=dark" }

Another indicator of its growing popularity is its number of downloads. If you go to npm stats you can see how this number evolved compared to npm and yarn.

I believe this diagram speaks for itself 🚀.

Which companies are using pnpm?

There is a page on pnpm's documentation about well-known companies using pnpm.

You can also see some other companies on the StackShare website (but it seems not many companies took the time to fill in the fact that they were using pnpm in their stack).

Which popular open-source projects are using pnpm?

If you see a pnpm-lock.yaml or a pnpm-workspace.yaml file in a GitHub repository, then that project is definitively using pnpm to manage its dependencies. You can use this technique to find GitHub projects using pnpm by querying them with GitHub code search.

I thought it would be interesting to explore which package managers are utilized in the development of popular JavaScript framework projects. And guess what? Many JavaScript frameworks are developed using pnpm 💖.

💡To check that these projects were using pnpm, I not only verify the presence of pnpm specific files but also checked their continuous integration pipelines (contained in the .github folder) to see what they were using to manage their dependencies.

Here is a non-exhaustive list of popular JavaScript web frameworks that use pnpm as their package manager:

Vue
Nuxt
Next.js
SvelteKit
SolidStart
Astro
Qwik

That's quite an impressive list: most modern JavaScript web frameworks seem to have chosen pnpm. That's also the case for popular frontend tooling projects like Vite or Turbo.

💡The fact that pnpm is utilized by maintainers for internal development of these frameworks does not imply that these frameworks can only be used with pnpm. Typically, JavaScript frameworks are "package manager" agnostic, allowing you to use your preferred package manager when developing a project with one of these frameworks.

Should you use pnpm because others do?

Short answer: no.

Choosing a technology solely based on its popularity is not advisable. While popularity is a factor to consider, it should not be the only determining aspect. Thus, you should not use pnpm because well-known companies or popular open-source projects use it.

However (and here's the long answer 😉), you should consider exploring pnpm, as there must be a reason why all these intelligent individuals have chosen it over npm or yarn. Investigate the issues pnpm resolves for them; perhaps you face similar challenges in your projects. See what problems pnpm solves for them, maybe you have the same problems in your projects. You might not even be aware of certain problems (such as lengthy CI due to time-consuming package installations, excessive space occupied by node modules, or issues with hoisted node modules), but pnpm could potentially make some things easier. Nevertheless, if you are satisfied with your current package manager, there is no need to switch just to imitate the popular frameworks projects.

I believe people are familiar with npm since it is the default package manager for Node.js projects. They might also know about yarn because it was initially developed by Facebook (who created React) and addressed some issues with npm. However, people recognize and utilize pnpm due to its performance and ability to resolve the problems they might encounter with npm package management. That's also why I use pnpm; it does the job, and it does it quickly.

Now you know that you're not alone in using pnpm; from renowned companies to popular open-source projects, many people are utilizing it.