DEV Community: Massimiliano Donini

Publish Dacpac to Azure SQL with Entra-only authentication using GitHub Actions

Massimiliano Donini — Fri, 23 Aug 2024 13:01:19 +0000

In Azure, most services nowadays supports Microsoft Entra based authentication, for example via a system managed identity or a user assigned one.
This authentication method is preferred to the old connection string based one because it gets rids of secrets and with that the need of secret rotation.

Some services then push this concept even further and allow you to completely disable a secret based connection, Azure SQL is one of them. When Entra-only authentication is enabled it's pretty easy to configure a service running in Azure to connect to the service in question but what about the CI/CD pipelines?

This article explains how to connect to Azure SQL when Microsoft Entra-only authentication is enabled.

Prerequisites

Create a service principal used by GitHub actions to connect to the Azure subscription. This can be created with or without using a shared secret. Using the shared secret option is easier but also less secure and can be achieved with the following:

az ad sp create-for-rbac -n YourServicePrincipalNameHere --role Owner --scopes /subscriptions/YourSubcriptionIdHere

If you want to use the secretless approach, you can refer to my other post here that explains step-by-step how to set up and configure a Microsoft Entra application and configure a federated identity credential on the application here

TIP The article above uses an App registration with a service principal, the same can also be achieved via a user-assigned managed identity. You can find more details on the Microsoft documentation.

Enable Entra-only authentication

Enable Entra-only authentication has to be done at the Azure SQL Server level and can be achieved in several ways:

Terraform azurerm_mssql_server

resource "azurerm_mssql_server" "sql_server" {
  name                = local.server_name
  resource_group_name = azurerm_resource_group.rg.name
  location            = azurerm_resource_group.rg.location
  version             = "12.0"
  minimum_tls_version = "1.2"

  azuread_administrator {
    login_username              = var.admin_username
    object_id                   = var.admin_object_id
    azuread_authentication_only = true                  # Add this to enable Entra-only authentication
  }
}

AZ CLI

az sql server ad-only-auth enable --resource-group mygroup --name myServer

Azure Portal

Create a SQL user

Manual script

Now we need to create the user in the SQL Database that represent the service principal and grant the necessary permissions, we achieve this by running the following script:

CREATE USER [ServicePrincipalName] FROM EXTERNAL PROVIDER;
GO
ALTER ROLE db_owner ADD MEMBER [ServicePrincipalName];

-- Depending on your requirements you can also use a less privileged role, e.g. db_ddladmin

Note When creating a user mapped to an Azure service principal (e.g., when using FROM EXTERNAL PROVIDER), you must connect to the database using Microsoft Entra authentication. If you try to run this script using a regular username and password connection, you will get an error message like the following:

Failed to execute query. Error: Principal 'ServicePrincipalName' could not be created. Only connections established with Active Directory accounts can create other Active Directory users.

Terraform mssql_user

The official Azure Terraform provider, azurerm, doesn't support creating Azure SQL users. However there's a community provider that does support Azure SQL user creation.

If you apply your infrastructure changes manually, configuring this might be worthwhile. However, if you use CI/CD pipelines to apply changes, it creates a chicken-and-egg problem since mssql_user still requires an Entra connection to the Azure SQL database.

GitHub action workflow

Now that everything is set up, we can examine the action workflow. My workflow is a simplified example that generates the DACPAC, connects to Azure, and publishes it.

In a more realistic setup, the workflow might be divided into multiple jobs. The first job would build the DACPAC and create an artifact, while the subsequent jobs would deploy the same artifact across different environments, with steps like approval, baking time, and so on.

Building the Dacpac

For this demo, I've used the community built MSBuild.Sdk.SqlProj, but the same can be achieved with a regular database project created in Visual Studio. This SDK uses regular csproj to generate a dacpac hence it's very convenient to use because all it takes is just a simple command:

dotnet build path-to-the-project.csproj --configuration Release

Get the connection string

Since the connection string does not contain secrets anymore, there’s no need to put it in a GitHub Action secret. So for the sake of simplicity in this example a connection string is created in a workflow step, but you can create it in any way you like.

The connection string format should be the following:

Server=tcp:{SqlServerName}.database.windows.net,1433; Initial Catalog={DatabaseName}; Authentication=Active Directory Default; Encrypt=True; TrustServerCertificate=False; Connection Timeout=30;"

Note It would be nice if az cli had support for creating the correct connection string but, as of now, this is not supported.

Make sure you replace SqlServerName and DatabaseName with the appropriate values.

Publishing the dacpac

Publish the dacpac can be done with the azure/sql-action, this is an action that wraps sqlcmd and provided by Microsoft. The action requires a connection string, the dacpac file and the action to perform.

Here below, you can see the interesting part of the workflow:

jobs:
  deploy:
    runs-on: ubuntu-latest
    timeout-minutes: 5
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Setup dotnet SDK
        uses: actions/setup-dotnet@v4

      - name: Create Dacpac
        run: dotnet build Database/Database.csproj --configuration Release -o ./out

      - name: Login to Azure
        uses: azure/login@v1
        with:
          creds: ${{ secrets.AZURE_CREDENTIALS }}

      - name: Construct connection string
        run: |
          connection_string="Server=tcp:${{ vars.SQL_SERVER }}.database.windows.net,1433; Initial Catalog=${{ vars.SQL_DATABASE }}; Authentication=Active Directory Default; Encrypt=True; TrustServerCertificate=False; Connection Timeout=30;"
          echo "::add-mask::$connection_string"
          echo "connection_string=$connection_string" >> $GITHUB_ENV

      - name: Deploy Database
        uses: azure/sql-action@v2.3
        with:
          connection-string: ${{ env.connection_string }}
          action: 'publish'
          path: ./out/Database.dacpac

      - name: Logout of Azure
        run: az logout

And now what's left is just running the action itself!

All the code for this blog post can be found here:

ilmax / azure-sql-entra-only-id

Sample code that demonstrates how to deploy a dacpac to Azure SQL with Entra-only authentication enableddeploy from GitHub Action linux runners

This repository contains the accompaignig code for the blog post published here

It shows how to deploy Azure SQL database using dacpac and dotnet via GitHub actions.

Key Features

Generate dacpac on linux runners
Deploy to a database with Entra-only authentication enabled

View on GitHub

Conclusion

Enabling Entra-only authentication can help us improve our Azure SQL security baseline. Additionally, the process of updating GitHub Action workflows to incorporate Entra-only authentication is straightforward and manageable. This means we can seamlessly integrate this security measure into our CI/CD pipelines, taking full advantage of the benefits it provides.

Till the next time!

GitHub Actions Azure Vnet Integration

Massimiliano Donini — Sun, 18 Aug 2024 08:02:20 +0000

In today's post, we will look at an interesting challenge, having GitHub actions interact with Azure PaaS services for which we have disabled public access.

Problem Statement

If you are working on improving your cloud security posture on Azure, one of the first things that you should look into when deploying PaaS services (like for example Azure Storage, Azure Cosmos DB, or Azure SQL Server) is to disable the public access.

Most PaaS services nowadays allow you to disable public access, meaning that you can't connect to those services over the Internet anymore. For the applications deployed in Azure, you can take advantage of Virtual Networks, Private Endpoints and Private DNS Zones to enable private connectivity without changing any single line of code.

This is all well and good, but what about those services that aren't deployed in Azure, like for example the CI/CD runners?

Chances are that you interact with your infrastructure in CI/CD pipelines (like for example running terraform apply) and, as soon as you close the firewall, some operations will start to fail due to connectivity problems.

At this point, we have several ways of fixing the issue, here's a quick non-exhaustive list off the top of my head:

Open and close the PaaS Service public access when the pipeline starts and revert the operation at the end
Allow access to the PaaS Services from within your office network and self-host your runners within your office network
Self-host your runners in Azure Virtual Machines with network connectivity to those services

All the options in the above list will fix the issue but they all have some disadvantages, so can we do better?
Not long ago, we got a new and better alternative which is to take advantage of GitHub private networking for hosted runners.

GitHub private networking

This relatively new feature (Public beta started in November 2023 and went GA in April 2024), allows GitHub-hosted runners to use a network interface card (NIC) created in a subnet under your control. This way we don't have to manage our own hosted runners, GitHub will keep managing the runners for us, while we will still be able to connect to PaaS service using private connectivity.

To set this up we need to configure a few things:

Get your GitHub the Enterprise ID or Organization ID (more on this below)
Create the Azure VNET that will host the NICs used by the GitHub runners
Create a GitHub network configuration in Azure
Create a Hosted Computed Network configuration in GitHub
Create Runner Group(s) and Runner(s) on GitHub
Change your GitHub action runs-on to reference the runner

Note: This functionality is only supported by GitHub Enterprise and GitHub Team plans

Azure private connectivity

Let's now take a look into a few concepts that are necessary for understanding how this can be configured.

Private Endpoints

Private Endpoints can be thought of as read-only Network Interface Cards (NICs) for your PaaS services. Those NICs are created in the subnet you specify and are assigned a private IP from the VNET address space. The connection between the private endpoint and the PaaS service uses a secure private link.

When using Private Endpoints, the traffic never leaves the Microsoft backbone network as opposed to going through the public internet, making it not only more secure but also a faster way to access your PaaS services.

Private DNS Zones

When a PaaS service enables the Private Endpoints, at the DNS level the service FQDN turns into a CNAME to the private link zone for the related service.
Let's see the changes in resolving an Azure Storage Account hostname with the dig command (available in Linux and MacOS, when using Windows you can use dig on WSL or nslookup on cmd/PowerShell)

No Private Endpoints



dig example.blob.core.windows.net

;; ANSWER SECTION:
example.blob.core.windows.net. 60 IN CNAME blob.ams08prdstr13c.store.core.windows.net.
blob.ams08prdstr13c.store.core.windows.net. 86400 IN A 1.2.3.4

With Private Endpoints



dig examplepe.blob.core.windows.net

;; ANSWER SECTION:
examplepe.blob.core.windows.net. 60 IN CNAME examplepe.privatelink.blob.core.windows.net.
examplepe.privatelink.blob.core.windows.net. 60 IN CNAME blob.am5prdstr12a.store.core.windows.net.
blob.am5prdstr12a.store.core.windows.net. 60 IN A 1.2.3.5

As you can see above, after we turn on Private Endpoints for a particular service, we get another DNS indirection. This zone coincides with the name of the Private DNS Zone in which we have to create our records to enable private connectivity.

Note: You can read more about how this works in the Microsoft documentation.

Different services have different DNS Zones and those are mentioned in the documentation here.

An A record is then created in the respective Private DNS Zone that resolves to the IP of the NIC that represents your PaaS service.

Private Endpoint can be linked to one or more Private DNS Zones to make sure that whenever the IP of the NIC connected to the Private Endpoint changes, the DNS record(s) are automatically updated by the platform for us.

VNET Peering

If you need to connect to PaaS services via Private Endpoint from a different VNET than the one where the Private Endpoint NICs live, you can take advantage of network peering and you should be able to communicate without any problems. VNET Peering is very flexible and can peer networks that are in the same regions as well as different regions, subscriptions or even tenants.

To be able to resolve the hostname to the private IP of the NIC created by the Private Endpoints, we need to make sure that the private DNS Zone is linked to all the VNETs that have to connect to the PaaS service.

Note: Bear in mind that network peering is not transitive, so if you need to traverse several networks, you need to configure a network virtual appliance (NVA) that knows how to route traffic

All three components briefly described above are used to configure private access to PaaS services and GitHub-hosted runners can take advantage of this infrastructure. Let's see how below.

Configuration

To get this working we need to configure the networking in Azure, create the hosted network configuration in GitHub, create runner groups and runners in GitHub and, as the last step, we can change the workflow's runs-on to specify the new runner name, let see how to do this in detail.

Azure configuration

In Azure, you have to decide in which VNET the GitHub-hosted runner NICs will be created. You can use the same network where the private endpoint for your PaaS services lives or create another VNET to keep things separate, this decision is up to you and depends on your networking configuration/requirements. What's worth noting is that just a subset of Azure regions are supported, so you may be forced to create a VNET in a region different from the VNET that contains your Private Endpoints.

As of today (July 2024) the only supported regions are:

EastUs
EastUs2
WestUs2
WestUs3
CentralUs
NorthCentralUs
SouthCentralUs
AustraliaEast
JapanEast
FranceCentral
GermanyWestCentral
NorthEurope
NorwayEast
SwedenCentral
SwitzerlandNorth
UkSouth
SoutheastAsia

If the VNET that contains the Private Endpoints is not in any of those regions, you have to create a new VNET and use Regional VNET Peerings or create another set of Private Endpoints in the designed VNET. If you use a HUB/Spoke network topology, you may want to create a dedicated spoke that will host the GitHub NICs.

When you have multiple spokes that need to communicate, you can either peer them together, configure traffic routing through an NVA or connect them through a VPN gateway. Please refer to the documentation on how to achieve that.

In my case, I went with the easy option to use network peering between the two spoke VNETs.

After the networking part has been taken care of, we need to:

Register a new resource provider
Create a new resource of type GitHub.Network/networkSettings
Copy the tag.GithubId output

Register the resource provider can be done in several ways, via Terraform (see below) or via az cli running the following command:



az provider register --namespace GitHub.Network

In GitHub, depending on whether you have an Enterprise Cloud or Team Plan, you can configure the Hosted Compute Networking on the Enterprise level or at the organization level. If you have GitHub Enterprise Cloud, you can still select whether to configure it at the Enterprise or Organization level.

The GitHub network settings need to know about your Enterprise/Organization so, before creating the network settings resource in Azure, we need to get a hold of the Enterprise ID/Organization ID from GitHub. As far as I know, this is not displayed anywhere in the UI so we need to execute a GraphQL API call as shown below:

GitHub Enterprise ID

We get the organization ID via the following GraphQL call, before the call we also need to generate a personal access token with the required grants.



curl -H "Authorization: Bearer BEARER_TOKEN" -X POST \
  -d '{ "query": "query($slug: String!) { enterprise (slug: $slug) { slug databaseId } }" ,
        "variables": {
          "slug": "ENTERPRISE_SLUG"
        }
      }' \
https://api.github.com/graphql

Note: The documentation for configuring the private networking for GitHub-hosted runners in your Enterprise can be found here

GitHub Organization ID



curl -H "Authorization: Bearer BEARER_TOKEN" -X POST \
  -d '{ "query": "query($login: String!) { organization (login: $login) { login databaseId } }" ,
        "variables": {
          "login": "ORGANIZATION_LOGIN"
        }
      }' \
https://api.github.com/graphql

Note: The documentation for configuring private networking for GitHub-hosted runners in your Organization can be found here

Create the GitHub network setting

Here's the Terraform code to create the GitHub network settings resource in Azure:



terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = ">3.0.0"
    }

    azapi = {
      source  = "Azure/azapi"
      version = "~> 1.14.0"
    }
  }
}

# Register the GitHub.Network resource provider
resource "azurerm_resource_provider_registration" "github_resource_provider" {
  name = "GitHub.Network"
}

resource "azurerm_resource_group" "resource_group" {
  location = "North Europe"
  name     = "My-Rg"
}

resource "azurerm_virtual_network" "vnet" {
  name                = "My-vnet"
  location            = azurerm_resource_group.resource_group.location
  resource_group_name = azurerm_resource_group.resource_group.name
  address_space       = "10.0.0.0/8"
}

resource "azurerm_subnet" "runner_subnet" {
  name                 = "My-runner-subnet"
  resource_group_name  = azurerm_resource_group.resource_group.name
  virtual_network_name = azurerm_virtual_network.vnet.name

  address_prefixes = "10.0.1.0/24"

  delegation {
    name = "delegation"

    service_delegation {
      name    = "GitHub.Network/networkSettings"
      actions = ["Microsoft.Network/virtualNetworks/subnets/join/action"]
    }
  }
}

# Create the GitHub Network settings
resource "azapi_resource" "github_network_settings" {
  type                      = "GitHub.Network/networkSettings@2024-04-02"
  name                      = "github_network_settings_resource"            # The name of the networksettings
  location                  = "West Europe"                                 # The region in which the networksetting resource will be created
  parent_id                 = azurerm_resource_group.resource_group.id      # Parent Id that should point to the ID of the resource group
  schema_validation_enabled = false
  body = jsonencode({
    properties = {
      businessId = var.github_business_id                                   # GitHub EnterpriseID or Organization ID based on Enterprise vs Organization level configuration
      subnetId   = azurerm_subnet.runner_subnet.id                          # ID of the subnet where the NICs will be injected
    }
  })
  response_export_values = ["tags.GitHubId"]                                # Export the tags.GitHubId

  lifecycle {
    ignore_changes = [tags]
  }
}

output "github_network_settings_id" {
    description = "ID of the GitHub.Network/networkSettings resource"
    value = jsondecode(azapi_resource.github_network_settings.output).gitHubId.value
}

You can find the whole source code at the end of the article in the references section

GitHub configuration

I decided to create the Hosted Compute Networking configurations at the Organization levels because it is where it makes the most sense for my use case, but creating it at the Enterprise level is pretty much the same thing, so you can easily adapt this tutorial to it.

Go to your Organization Settings
In Hosted Compute Networking, create a new Network Configuration and pick Azure Private Networking
Add a name to the configuration and then click on the Add Azure Virtual Network button
Paste the ID outputted by Terraform while creating the GitHub Network settings resource
Save the configuration

After the network configuration is created, we have to create a Runner Group that uses the network configuration just created.

Go to Organization settings > Actions > Runner Groups
Give the Runner Group a name
In the network configuration dropdown, select the network configuration created in the previous step
Save the Runner group

After the Runner Group has been created, it's now time to create a runner (or more) within the Runner Group

Click on the Runner Group just created
Click on the New runner > New GitHub-hosted runner
Specify name, OS, Image (OS Version) and Specs (Size)
Make sure the Runner Group is the previously created Runner Group
Save the runner

Now that we have configured everything, we can change the runs-on label on a workflow with the name of one of the runners created above and it will use the new runner with VNET connectivity with our PaaS service.



name: Sample workflow

on:

  pull_request:

jobs:

  build:

    name: Build

    runs-on: azure-vnet-runner # Use the name of the runner just created

References

Conclusion

Thanks to GitHub private networking for hosted runners, we can ensure our CI/CD pipeline works seamlessly even when we deny public access to the PaaS services we use, allowing us to enhance the security posture of our Azure Subscription.

In this repository, I have a simple Terraform configuration where I deploy an Azure Storage Account, disable Public Access, and create the Private Endpoints for blob and tables within a VNET. In another VNET I have configured the GitHub network settings, the output of such configuration is the token that you can input in GitHub when creating the Hosted Compute Networking configuration.

I hope you enjoyed this article, if you have some questions, don't hesitate to reach out.
Till the next time!

Ace the AZ-104 Microsoft Azure Administrator Certification Exam: Tips and Strategies

Massimiliano Donini — Wed, 01 Feb 2023 21:09:43 +0000

I recently passed the AZ-104 certification exam, making it the third Microsoft certification under my belt so far. I started with AZ-900, then AZ-204 and recently AZ-104.
I'm planning to get the AZ-305 and eventually the AZ-400 soon™.

To successfully prepare for any exam you need to come up with an effective study plan. In this article you can find the tips I use to get prepared to undergo the AZ-104 certification exam.

I'm using Azure daily so I am familiar with many of the certifications objectives, but not all of them. Having a developer background, I never needed to configure Azure Active Directory for MFA, conditional access policies or Site-to-Site VPN connections for example.

Below you will find how I prepared as well as some tips I used, but keep in mind that you should tailor the study plan to fit your own way of learning. Different people learn in different ways and to be successful you need to figure out what works best for you.

My study agenda

I'm a full-time employee and I work 40 hours per week, so finding the time to study was not always easy, luckily for me though, here in The Netherlands the winter is quite depressing: It's dark when I leave home for the office and it's already dark when I come back. This somehow helped me stay at home during the evening so I could dedicate time to study and prepare for the certification.

The time I put into studying was around 1.5/2 hours every day and around 4 or 5 hours during the weekends. It took me more or less 1.5 months to prepare for the certification.

My weekly goal was to put in around 15 hours. I wasn't studying every day to not get overwhelmed, so I took some free days to relax and blow some steam off.

Resources

Here's a list of materials that I used to get prepared, in no particular order:

AZ-104 Study guide

The Study guide is a great resource to check how familiar you are with a specific subject. From the study guide, I flagged the areas I was least familiar with and I started focusing on these first.

John Savill's AZ-104 Study cram

This study cram was super useful to understand concepts I wasn't familiar with because John can explain concepts in a very easy-to-understand way. I used this one to kick-start my knowledge of Azure Active Directory configuration, VPN connections and to better understand how networking works in Azure.

Microsoft Learn

Contrary to many, I didn't complete the AZ-104 learning path. I found the study path too high level and not very useful, what I did instead was to use the study path to dive deeper using the resources usually listed at the end of every chapter.

In my opinion the learning path does not prepare you enough for the exam. Test questions go into a greater detail than what the learning path covers so, I'd suggest to just skim over the learning path and rather focus more on the resources you find at the end of each learning path module.

Azure free trial subscription

Activating a trial subscription allowed me to have hands-on experience with all the areas I wasn't very familiar with e.g. Azure Active Directory. An Azure trial subscription is free, you have 170.00 € of budget that will last for 30 days. Even though you have to give your credit card details you won't be billed. If you ran out of credits, all the services will be decommissioned unless you upgrade to a Pay-As-You-Go subscription as described in the Azure free account FAQ.

Find a practice test

There're plenty of practice tests out there, some better than others so you have to put in some research to find a good one, and then go for it. Price vary from around 20.00 € up to around 100.00 € but I think it's worth the money.

Make sure you're not using dumps because it can invalidate all your certifications and prohibit you to take new ones ever again.

Microsoft sponsors the MeasureUp practice tests, so that will set you up for success.

Tips

Here's a collection of tips that helped me get the certification.

Plan the exam date as the first step

In order to not postpone taking the exam forever, I decided to plan the exam before starting to study. I did a bit of research to understand how much it takes to get ready for the exam, then gave me some more time and planned it. I ended up giving myself two months of time to prepare, this helped me to put the time in.

Planning it in advance still allows you to change the exam date up to 24 hours before the exam, so it's not a strict deadline, but I found it helpful to have a due date to look at.

Alternate your study routine

Mixing up your study routine can be beneficial as it helps to keep things interesting and can prevent boredom. Additionally, different study methods can appeal to different learning styles and can help to reinforce the material in different ways. For example, reading documentation may help to build a strong understanding of the theoretical concepts, while practice exams can help to build test-taking skills and identify areas where you need to improve. YouTube videos can be helpful in providing additional explanations and examples of the concepts that you are studying. So, switching up your routine can help to make the most of your study time and increase your chances of success.

Carefully review wrong practice test answers

While taking the practice test, carefully review your answers, especially the wrong ones. This may uncover some gaps in your knowledge and will give you an indication of where you need to spend some time on.

This point was especially important for me because I personally learn a lot more from a wrong answer than a correct one. Every time I give the wrong answer in the practice test, I dig deeper in the documentation and it really helps me understand the subject better.

Whenever I got the same question wrong multiple times, I wrote down the question and the correct answer in a small document that I used to go over and over almost daily.

Retake the same practice test twice

I found it very helpful to re-take the same practice test twice, possibly in the same day. Retaking the test in the same day can help to reinforce the material that you have studied and identify areas where you may still have difficulty. This can be helpful in focusing your study efforts and pinpointing areas where you need to spend more time studying.

Get hands-on experience

Hands on experience is very important. Having only theoretical knowledge derived from the documentation may not be enough for this exam so I advise you to spend some time on practicing what you're learning with the free Azure account.
This can be invaluable to get familiar with the az cli or PowerShell cmdlet. You can do so using the Azure Cloud Shell so you don't even have to install anything on your PC.

Enjoy the journey

The added value of a certification in my opinion is not the certification badge, but rather all the knowledge you accrue preparing for the actual exam.

The exam

The exam I went through had 49 questions and I had 1 h and 40 minutes to complete it. The exam started with a case study comprised of 6 questions.
I found the level of difficulty of the exam reasonable, all the questions were clear and easy to understand. The time for me was enough, but if you find yourself running out of time you can skip the question you're not sure of using the mark for review functionality, so you can go through them at the end, avoiding the risk of running out of time.

Please note that you can't review questions from the case study at the end of the exam, you can review them at only at the end of the case study.

Summary

In this article I wanted to share my approach to prepare for a certification exam together with some tricks I used to get ready in a reasonable amount of time and how to figure out what are the knowledge gaps I had.

To sum up, I wanna point out that this is what worked well for me. It's not guaranteed that this approach works equally well for you, but you may use some of the approaches I described to get yourself ready for the certification.

I hope you find this helpful, if you have any questions/suggestions don't hesitate to leave a comment below. If you're preparing for AZ-104 or any other certification, I wish you best of luck 🍀.

Till the next time.

Goodbye secrets 👋, Hello token exchange: Connect Your GitHub Actions to Azure securely

Massimiliano Donini — Wed, 25 Jan 2023 14:47:54 +0000

OpenID Connect (OIDC) integration between Azure Active Directory and GitHub allows your GitHub Actions workflows to securely access resources in Azure, without needing to store the Azure credentials in the GitHub action secrets.

This functionality has been available for quite a while, it was first announced on October 2021 and up until now, it has been on my "things to look into" list.

Recently I've been working on a project to migrate Azure DevOps to GitHub so I decided that time has come to look into this functionality.

How a typical connection is configured

Usually, to connect to Azure as an application (i.e. when running a GitHub Action) you need to:

Create a Service Principal in Azure Active Directory
Create a Service Principal Credential
Grant to the Service Principal permissions on the subscription(s)
Copy the secret created in step 3 on your GitHub secrets
Authenticate the workflow using the secret created above

Please note that the az ad sp create-for-rbac can simplify the process a bit since it can do steps from 1 to 3 in a single go, more info can be found in the documentation

Why should you use secret-less connections

Having secret-less connections is far better than using some form of a shared secret.
Since you don't have any secrets, you can't leak any moreover shared secrets usually comes with a fixed validity. Ideally, you should rotate them often to limit the risk derived from a secret leak.

Additionally, in the case of automated infrastructure deployments, the Service Principal will have high privileges on your subscription(s) and/or possibly Azure Active Directory since it has to create resources, potentially assign RBAC role assignments (that require the Owner role) making the event of a secret leak even more dangerous.

With secret-less connections, on the other hand, even if the GitHub Actions secrets are leaked, an attacker can't gain direct access to the Azure subscription.

Please note that even if you store the shared secret in GitHub secrets, it's still fairly easy to get access to the original value, see this StackOverflow question for example.

I'm sold, now how can I set it up?

To configure OpenID Connect Integrations between Azure Active Directory and GitHub you need to do a couple of things:

Create an App Registration in Azure Active Directory
Create a Service Principal for the App Registration created above
Add the Federated Credential in the App Registration
Copy the configuration values CLIENT_ID, SUBSCRIPTION_ID and TENANT_ID in GitHub
Configure your workflow permissions
Grant to the Service Principal the desired permissions on your subscriptions(s)
Use the action azure/login@v1 to login to Azure using token exchange

For a step-by-step guide, refer to the GitHub documentation

How does it work?

It's not the goal of this post to dig into the nitty-gritty details, so my explanation will be quite brief.
Under the hood, this uses Azure Workload identities to exchange a token issued by GitHub with a token issued by Azure Active Directory.

For the token exchange to be successful, you need to establish a trust relationship between the GitHub token issue and the Azure Active Directory. This trust relationship boils down to configuring the Federated Credential (created in step 3) in Azure Active Directory filling in the content of two of the claims issued by GitHub to the workflow, more specifically you need to fill in:

The issuer (iss claim of the access token issued by GitHub)
The subject (sub claim of the access token issued by GitHub)
The audience (fixed value of api://AzureADTokenExchange)

Here below you can find how GitHub explains it in their announcement post.

If you want to dig deeper, here's the documentation.

Manual configuration

If you go to Azure Active Directory, after you created an App Registration, when you try to add Federated Credentials, the Azure Active Directory Portal helps you with filling in the required details for setting up GitHub Federated Credentials.

The screen looks like the following:

If you have to configure multiple repositories, the manual approach falls short so let's have a look at how we can configure it with Terraform.

Terraform configuration

Since Terraform has a provider-based approach, we can configure a GitHub repository (or many) and, at the same time, create the required setup in Azure Active Directory.

Let's see how is done:

// Look-up GitHub Actions token issuer discover document
data "http" "github_actions_oidc_discovery_document" {
  url = "https://token.actions.githubusercontent.com/.well-known/openid-configuration"

  request_headers = {
    Accept = "application/json"
  }

  lifecycle {
    postcondition {
      condition     = contains([200], self.status_code)
      error_message = "Status code invalid"
    }
  }
}

locals {
  github_issuer = jsondecode(data.http.github_actions_oidc_discovery_document.response_body)["issuer"]
}

// Create an Azure Active Directory Application representing GitHub Actions
resource "azuread_application" "github_app_registration" {
  display_name = "GitHub-App-Registration"
}

// Create a Service Principal for the GitHub Actions App Registration
resource "azuread_service_principal" "github_service_principal" {
  application_id = azuread_application.github_app_registration.application_id
  use_existing   = true
}

// Create the Federated Credential for the App Registration
resource "azuread_application_federated_identity_credential" "github_federated_credentials" {
  application_object_id = azuread_application.github_app_registration.object_id
  audiences             = ["api://AzureADTokenExchange"]
  display_name          = "GitHub-FederatedCredential"
  issuer                = local.github_issuer
  // You need to decide how to configure this one accordingly to your use case
  subject               = "repo:${var.organization}/${var.repository_name}:environment:${var.environemnt}"
}

// Look-up current subscription and tenant id
data "azurerm_client_config" "current" {}

// Create a GitHub repository
resource "github_repository" "repository" {
  name        = var.repository_name
  description = var.repository_description
}

// Optional
resource "github_repository_environment" "environment" {
  environment  = var.environemnt
  repository   = github_repository.repository.name
}

// Can be just regular repository secret if you don't use environments
resource "github_actions_environment_secret" "client_id" {
  environment     = github_repository_environment.environment.environment
  repository      = github_repository.repository.name
  secret_name     = "CLIENT_ID"
  plaintext_value = azuread_application.github_app_registration.application_id
}

// Can be just regular repository secret if you don't use environments
resource "github_actions_environment_secret" "subscription_id" {
  environment     = github_repository_environment.environment.environment
  repository      = github_repository.repository.name
  secret_name     = "SUBSCRIPTION_ID"
  plaintext_value = data.azurerm_client_config.current.subscription_id
}

// Can be just regular repository secret if you don't use environments
resource "github_actions_environment_secret" "tenant_id" {
  environment     = github_repository_environment.environment.environment
  repository      = github_repository.repository.name
  secret_name     = "TENANT_ID"
  plaintext_value = data.azurerm_client_config.current.tenant_id
}

//providet.tf
terraform {
  required_providers {
    github = {
      source  = "integrations/github"
      version = "~> 5.16.0"
    }

    azuread = {
      source  = "hashicorp/azuread"
      version = "~> 2.32.0"
    }
  }

  backend "azurerm" {
    ...
  }
}

# Configure the GitHub Provider
provider "github" {
  owner = var.organization
}

Please note that this Terraform configuration requires multiple providers and, to run the apply successfully, you need to be authenticated into both.

Quirks

As you can see, the configuration is quite straightforward, but there's a catch.
Since you have to configure the subject (sub) claim in the Federated Credential with the same value of the sub claim that GitHub is issuing to your workflow and, by default, the repository name will be part of the claim value, this means you will have to create one App Registration, Federated Credential and Service Principal for each repository.

This may be fine if you have a limited number of repositories, but in my case, I had around 60 repositories that needs to be deployed.
On top of that, I (like probably most of you) have several environments e.g. Development, Testing, Acceptance, and Production and since I don't want the same Service Principal to have access to different subscriptions for security reasons, the number of App Registrations, Federated Credentials and Service Principal gets multiplied by a 4 factor (one for each environment) reaching a whopping 240.

As you can imagine this was not ideal so I decided to look at possible alternatives. What I wanted to achieve was creating 4 App Registrations, one for every environment, and use the same one across all the repositories.

Please note that In order to achieve this you need to use GitHub deployment environments. Environments, environment secrets, and environment protection rules are available in public repositories for all products. For access to environments, environment secrets, and deployment branches in private or internal repositories, you must use GitHub Pro, GitHub Team, or GitHub Enterprise. For access to other environment protection rules in private or internal repositories, you must use GitHub Enterprise, see documentation

To achieve what I described above, I needed a way to change the content of the subject claim issued by GitHub, luckily for us, this functionality is supported by GitHub using this api, even better, this functionality is also supported by the GitHub Terraform provider.

As far as I know, there's no UI support to change the content of the access token issued by GitHub yet.

Configure the subject claim with Terraform

Here below you can see how to configure the subject claim for our use case:

resource "github_actions_repository_oidc_subject_claim_customization_template" "sub_claim_template" {
  repository = github_repository.repository.name

  use_default = false
  include_claim_keys = [
    "repository_owner",
    "context",
  ]
}

// Create the Federated Credential for the App Registration
resource "azuread_application_federated_identity_credential" "github_federated_credentials" {
  application_object_id = azuread_application.github_app_registration.object_id
  audiences             = ["api://AzureADTokenExchange"]
  display_name          = "GitHub-FederatedCredential"
  issuer                = local.github_issuer
  // You need to decide how to configure this one accordingly to your use case
  subject               = "repository_owner:${var.organization}:environment:{var.environment}"
}

Please note that what you incude in the include_claim_keys depends on your specific scenario, this configuration allows me to add the organization and the environment in the subject claim so I can reuse the same Service Principal across all repositories for a given environment.
Also bear in mind that however you configure the GitHub sub claim, MUST match what's configured in the App Registration Federated Credential.

There're more customizations possible and you can learn about these in the GitHub documentation.

The last change you need to implement, besides granting RBAC privileges to the Service Principal on your subscriptions(s), is to configure the workflow to use token exchange to authenticate into Azure, there are two parts to it:

Configure the required permissions in the workflow
Configure the action azure/login@v1 to use the token exchange.

Here's an example of a workflow that will work with the configuration above:

name: Azure Login with OIDC
on: [push]

permissions:
  id-token: write
  contents: read

jobs: 
  build-and-deploy:
    runs-on: ubuntu-latest
    steps:
      - name: 'Az CLI login'
        uses: azure/login@v1
        with:
          client-id: ${{ secrets.AZURE_CLIENT_ID }}
          tenant-id: ${{ secrets.AZURE_TENANT_ID }}
          subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}

      - name: 'Run az commands'
        run: |
          az account show
          az group list

Summary

As you can see, the configuration is quite straightforward and it allows you to get rid of secrets, improve your security posture and, as a bonus, forget about the expiring credentials issue.

I hope you found this useful.

Till the next time

Terraform Tips & Tricks: Managing Large-Scale Azure Resource Imports

Massimiliano Donini — Thu, 19 Jan 2023 17:28:52 +0000

This post describes my journey to import several hundred Azure resources in Terraform. Before digging into the what and the how let me give you a brief description of our environment's infrastructure.

In my current company, we manage many Azure resources for each environment and we have a few of them (DEV, TEST, etc.). Every environment looks pretty much the same and it mostly differs by product SKUs, database sizes, etc.

My fellow team members and I are building a microservices solution, we have around 50 independent services deployed in Azure that require some specific resources (imagine a SQL database or a Storage container), and on top of that, we have all the service agnostic infrastructure.

All these resources aren't created/updated in the same way, some use ARM templates, some use PowerShell scripts, some use Azure cli scripts and so on.
This is mostly because we need to work around some tooling limitations e.g. you can't create resources in Azure Active Directory using ARM templates.

All the aforementioned scripts are placed in a shared repository and every project references what it needs in its deployment pipeline.

This approach works, but it has several problems:

We use different technologies to deploy infrastructure and that makes it harder for newcomers to get up to speed quickly
Deployments take longer than necessary
It's impossible to have a preview of the changes that are going to be applied at deployment time
Re-deploying infrastructure may not fix all the configuration drift especially the PowerShell/az cli scripts

I'm sure I missed several other points but these are the most painful ones.

Hence we decided to move to Terraform since it can address all the points above and, according to GitHub Octoverse 2022, HCL was the fastest growing language in 2021-2022.

The import challenge

If you start on a greenfield project everything is quite easy, but as you may know, if a resource has been created outside Terraform, it needs to be imported to be managed with Terraform in the future.
Importing resources is not difficult, on the provider documentation site, at the bottom of the page of every resource, you can find the command to execute to import a given resource.

My problem was that I had hundreds of them, around 160 global resources, multiplied by all the various environments + between 5-10 resource service dependent multiplied by the number of services ~50 multiplied by the number of environments.

As you can imagine this adds up very quickly, especially because importing resources is a tedious task.
To import a resource in Terraform you need a couple of things:

Create the resource in your configuration files
The Terraform resource id
The Azure resource id

For example:

                |--------------Terraform resource id-------------------| |-----------------------------------------Azure resource id---------------------------------------------------------------|
terraform import module.servicebus.azurerm_servicebus_namespace.example  /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/mygroup1/providers/Microsoft.ServiceBus/namespaces/sbns1

To get the Terraform resource id, you may first need to come up with the final module structure because if you use modules, the module name would be part of the Terraform resource id (as can be seen above).

Figuring out the Azure resource id can also be challenging for some nested resources, e.g. a cosmos role assignment requires some az cli gymnastics.

This seemed like a herculean effort so I started looking around hoping to find a tool that could help with a bulk import.

Azure Terrafy - Aztfy

Aztfy is a tool developed by Microsoft that allows you to bulk import resources, it has some configuration so you can specify what to import, the names to import and so on.
After spending some time with the tool, I quickly realized it may be a no-go. The problem I had with this tool is twofold:

It doesn't generate reproducible Terraform configurations
It doesn't generate Terraform idiomatic code

Let me expand on those:

Not generating reproducible configurations means that after an import, when you run Terraform plan, you may still have changes that you need to fix manually, or worse, Terraform may fail due to validation problems.
This limitation is documented in the project README and I could've lived with it.

Not generating idiomatic code is a bit more problematic, since it requires manually changing most parts of the imported code to make use of variables or to reference a parent resource.
This means that all the code that aztfy will output, needs to be adjusted/modified. Moreover, if you decide to reorganize the code and move resources in a different module (hence changing the Terraform resource id) after it has been imported, then you have to either start modifying the Terraform state manually, or you may need to import it all over again.

Given the above downsides, I decided to use it only marginally and instead start writing my configuration from scratch.

My approach

Before starting I came up with a set of principles to use as guidelines when writing HCL modules, which are:

One module for every different resource type used
For every module that needs access to resources defined in other modules, read these resources with a data source
All the module's inputs are defined in a file called variables.tf
All the permissions-related stuff (RBAC, AAD group membership) will go in a file called permission.tf
All the networking configuration (Firewall rules, VNet, Private endpoints and so on) will go in a file called networking.tf
All the module's outputs will go in a file called output.tf
These lower-level modules will be invoked by a higher-level module where most of the naming logic will be
Lower-level modules can only be called by higher-level modules
Several higher-level modules will be created:
- Environment specific with all the global resources
- Several service-specific ones, one for each type of service
Client code can only reference higher-level modules
Higher level modules should have the least number of secret possible
Lower-level modules shouldn't reference other lower-level modules

Please note that these are principles I came up with and that make sense in my specific scenario, your mileage may vary

Directory structure

The principles stated above helped me come up with a directory structure that looks like the following:

├── environments
│   ├── acc
│   ├── dev
│   ├── prod
│   └── tst
├── modules
│   ├── private                --> Lower-level modules
│   │   ├── global
│   │   │   ├── global_azure_service_1          e.g. Cosmos Db 
│   │   │   ├── global_azure_service_2          e.g. vNET
│   │   │   ├── global_azure_service_3          e.g. App Service Plan
│   │   └── service
│   │       ├── service_specific_resource_1     e.g. App service
│   │       ├── service_specific_resource_2     e.g. Cosmos container
│   │       ├── service_specific_resource_3     e.g. Sql Database
│   └── public                 --> Higher-level modules
│       ├── environment     
│       └── webapp

Import Script

After coming up with this list of principles, I started creating the Terraform module for each resource type, importing it in a local state, running Terraform plan to ensure there are no changes and repeating till I created all the modules.

To make the plan/import phase quick, I was applying the changes on a single lower-level module basis.

Since I ended up importing the resources over and over and over again, I decided to write a small PowerShell script to help me speed up the process.

This script tries to address the two main paint points:

Do not re-import a resource that's already imported
Simplify reuse across environments via PowerShell string interpolation, whenever Azure resource Ids are predictable.

Please note that the last point depends on your resources naming conventions.

The script looks like this:

Terraform init // Comment this out after the first execution

# Get all the items from Terraform state and put it inside an array
$stateItems = $(Terraform state list)

function ImportIfNotExists {
    param (
        [String]$resourceName,
        [String]$resourceId
    )

    if ($resourceId -eq $null -or $resourceId -eq "") {
        Write-Warning "Resource id for $resourceName is null"
        return
    }

    if ($stateItems -notcontains $resourceName.Replace("\", "")) {
        Write-Host "Importing $resourceName with id $resourceId"
        Terraform import "$resourceName" "$resourceId"
        if ($LASTEXITCODE -ne 0) {
            Write-Warning "Error importing $resourceName with id $resourceId"
        } else {
            Write-Host "$resourceName imported"
        }
    } else {
        Write-Host "$resourceName already exists"
    }
}

$env = "DEV"
$subscriptionId = "your-subscription-id-here"
$spokeResourceGroupName = "myrg-spoke-$env".ToLower()
$hubResourceGroupName = "myrg-hub-$env".ToLower()

$ErrorActionPreference  = "Stop"

## Resource group import
ImportIfNotExists 'module.environment.azurerm_resource_group.spoke_rg' "/subscriptions/$subscriptionId/resourceGroups/$spokeResourceGroupName"
ImportIfNotExists 'module.environment.azurerm_resource_group.hub_rg' "/subscriptions/$subscriptionId/resourceGroups/$hubResourceGroupName"

This script allows me to quickly import resources and iterate faster since it allows me to re-run the same over and over without worrying about re-importing a resource that's already part of the state.

As stated above, you can make the script reusable for multiple environments with few modifications. Some Azure resource ids are a bit more complex to figure out hence I usually do a manual lookup and find-and-replace.
Some of those resources include:

Cosmos Sql Role Definition (The Role id uses a guid so it's different for every role definition)
Cosmos Sql Role Assignment (same as above)
RBAC role assignment
Automation account job schedules
AAD Groups
AAD Groups membership

Please note that your mileage may vary depending on the resources you use

If you want to, you can enhance the script to also look up these resources using the azure cli and a bit of JMESPath, for example, I'm doing this to look up AAD groups since in my case they follow a naming convention:

ImportIfNotExists 'sample.azuread_group.your_group_name' $(az ad group show --group "{your-group-name-prefix}-$env" --query id --output tsv)

Here below you can see an example where I'm employing JMESPath to further filter the result of az cli to look up the role assignment for a given role and group:

ImportIfNotExists 'sample.azurerm_role_assignment.your_group_assingments' $(az role assignment list --scope {your-scope} --query "[?principalName=='{you-principal-name}' && roleDefinitionName=='{your-role-name}'].id" -o tsv)

where:

{your-scope} is the resource you assigned the RBAC role assignment to (e.g. the resource group or a specific resource)
{you-principal-name} is the user name, group name or managed identity name of the principal that will be granted the role
{your-role-name} is the name of the RBAC roles you assigned (e.g. Contributor)

This is quite powerful and allows you to make the script parametric enough to allow you to reuse it for all environments.

It's also worth considering though that the import operation will be executed just once, so it may be quick to just do a find replace at times.

Quirks

If you have declared resources that use for_each in HCL, the name of the resource may contain (based on what you're foreach-ing) a string, e.g.
imagine you're creating several service bus topic using a for_each in the following way:

resource "azurerm_servicebus_namespace" "example" {
  name                = "tfex-servicebus-namespace"
  location            = azurerm_resource_group.example.location
  resource_group_name = azurerm_resource_group.example.name
  sku                 = "Standard"
}

resource "azurerm_servicebus_topic" "topics" {
  name         = each.value
  namespace_id = azurerm_servicebus_namespace.example.id

  for_each var.topics
}

Then the Terraform identifier will be something like the following: module.servicebus.azurerm_servicebus_topic.topics["{topic-name}"].

To make Terraform and PowerShell play nicely together in the import script, you have to write the above this way:

ImportIfNotExists 'module.servicebus.azurerm_servicebus_topic.topics[\"{topic-name}\"]' "{servicebus-resource-id}"

To avoid the Terraform error: import requires you to specify two arguments.

Useful resources

To import a resource you need to find its unique identifier in Azure and this is not always easily doable from the portal so I took advantage of the following tools to make my life simpler

The first one will be familiar to everyone, it's the azure command line tool and it's a must-have, the second one is a bit less known in my opinion but still an excellent resource to look into the definition of the various resources.

This work took quite a bit of time but in the end, I was able to import all the resources in all the environments and come up with idiomatic HCL code.

I hope you find this helpful!

Till the next time.

EF Core 7 is here - Welcome typed entity id 🍾

Massimiliano Donini — Thu, 17 Nov 2022 18:59:46 +0000

Source code

EF 7 has been released at dotnetconf and it brings a heap of new and exciting features. To read about all the new goodnes in this release you can go through the What's new in EF Core 7 docs page.

One of the feature I'm more excited about that hasn't been properly advertised (hence this post), in my opinion, is support for what they call Value generation for DDD guarded types.

This neat new feature allow us to create custom types that wrap identifiers and supports value generation on the database side.

You could already do this in the past if you were providing a value yourself, but it was not supported to generate the value on the database.

The feature is also negatively advertised with a warning on the EF Core docs page saying that it adds complexity to the code so let's find out what's this about and what it allow us to do.

What's primitive obsessions?

If you're a seasoned DDD practitioner you're probably familiar with this concept and you can skip this section altogether, if that's not the case keep reading.

Primitive obsession is a code smell and it's been defined as follows on hackernoon:

Primitive Obsession is a code smell in which primitive data types are used excessively to represent your data models.

What this means is that we fail to properly model some domain concepts and we instead use more permissive primitive data types.

Sometimes using a more permissive type is a tradeoff forced by the limitations of the tools we use in our applications.

With EF Core we were limited in how to model the entity primary key with the value to be generated by the database.
If we wanted to use a sequence in the db to generate a monotonically increasing number for our entity id, we had to use an int as the primary key property type.

As an example let's use the following model used in most of the EF Core samples:



public class Blog
{
    public int Id { get; private set; }
    public string Name { get; set; }
    public List<Post> Posts { get; } = new();
}

public class Post
{
    public int Id { get; private set; }
    public string Title { get; set; }
    public string Content { get; set; }
    public DateTime PublishedOn { get; set; }
}

As you can see, both Blog and Post entities have an int primary key.

This allows one subtle mistake not to be caught by the compiler: We can erroneously use the Blog.Id value in places where we should use the Post.Id or viceversa because both types are int and satisfies the type system requirements event though they're conceptually two completely different things.
Using the same type to represent different things besides opting out of compiler help, also hinders readability.

DDD typed id to the rescue

Now with EF Core 7 we can easily avoid this problem defining two different types to represent the primary key of each entity and thanks to the C# feature record struct we can even get away with it with similar performance characteristics.

Let's see it in action in the new model:



public class Blog
{
    private Blog(BlogId id, string name)
    {
        Id = id;
        Name = name;
    }

    public BlogId Id { get; private set; }

    public string Name { get; set; }

    public List<Post> Posts { get; private set; } = new();

    public static Blog Create(string name)
    {
        if (name == null)
        {
            throw new ArgumentNullException(nameof(name));
        }

        return new Blog(default, name);
    }
}

public record struct BlogId(int Value);

public class Post
{
    private Post(PostId id, string title, string content, DateTimeOffset publishedOn)
    {
        Id = id;
        Title = title;
        Content = content;
        PublishedOn = publishedOn;
    }

    public PostId Id { get; private set; }
    public string Title { get; set; }
    public string Content { get; set; }
    public DateTimeOffset PublishedOn { get; set; }

    public static Post Create(string title, string content)
    {
        if (title == null)
        {
            throw new ArgumentNullException(nameof(title));
        }

        if (content == null)
        {
            throw new ArgumentNullException(nameof(content));
        }

        return new Post(default, title, content, DateTimeOffset.UtcNow);
    }
}

public record struct PostId(int Value);

In order for EF Core to understand how to map the two new types PostId and BlogId to the dB, we need to use value converters like the following:



public class BlogIdIdConverter : ValueConverter<BlogId, int>
{
    public BlogIdIdConverter()
        : base(v => v.Value, v => new(v))
    { }
}
public class PostIdIdConverter : ValueConverter<PostId, int>
{
    public PostIdIdConverter()
        : base(v => v.Value, v => new(v))
    { }
}

// register value converters, we can take advantage of the new model building conventions feature and register the value converters only once for our whole context 

protected override void ConfigureConventions(ModelConfigurationBuilder configurationBuilder)
{
    configurationBuilder.Properties<BlogId>().HaveConversion<BlogIdIdConverter>();
    configurationBuilder.Properties<PostId>().HaveConversion<PostIdIdConverter>();
}

// Last step is to configure the value generation for these entity keys in the OnModelCreating method

modelBuilder.Entity<Blog>().Property(blog => blog.Id).ValueGeneratedOnAdd();
modelBuilder.Entity<Post>().Property(post => post.Id).ValueGeneratedOnAdd();

Implementing the entity ids this way allow use to fix the aforementioned problem since now the two types are different so we're unable to pass a Blog.Id where we expect a Post.Id or vice versa.
This may seems like a small feature, but if you search the web, there're tons of articles that describe why this is useful (i.e. more expressive code, compile support, easier refactoring, etc).

Put it all together

Let's see how this work:



var blog = Blog.Create("My First Blog!");
context.Add(blog);
context.SaveChanges();

The code above produces the following Sql:



SET IMPLICIT_TRANSACTIONS OFF;
SET NOCOUNT ON;
INSERT INTO [Blogs] ([Name])
OUTPUT INSERTED.[Id]
VALUES (@p0);

Note that since this is only inserting one value and the database already guarantees atomicity for a single insert, the statement is not wrapped into a transaction, one of the nice performance benefits that we will get for free just updating to EF Core 7.

Now, add few posts:



blog.Posts.Add(Post.Create("First post", "EF Core is awesome"));
blog.Posts.Add(Post.Create("Second post", "Typed Ids are amazing"));
context.SaveChanges();

Produces the following Sql:



SET IMPLICIT_TRANSACTIONS OFF;
SET NOCOUNT ON;
MERGE [Post] USING (
VALUES (@p0, @p1, @p2, @p3, 0),
(@p4, @p5, @p6, @p7, 1)) AS i ([BlogId], [Content], [PublishedOn], [Title], _Position) ON 1=0
WHEN NOT MATCHED THEN
INSERT ([BlogId], [Content], [PublishedOn], [Title])
VALUES (i.[BlogId], i.[Content], i.[PublishedOn], i.[Title])
OUTPUT INSERTED.[Id], i._Position;

And now the reading part, reading a blog from the db:



BlogId id = blog.Id;
var blogFromDb = context.Blogs.SingleOrDefault(blog => blog.Id == id);

Produces the expected sql:



SELECT TOP(2) [b].[Id], [b].[Name]
FROM [Blogs] AS [b]
WHERE [b].[Id] = @__id_0

As you can see everything works smoothly as you'd expect and with little more code, you also have some additional type safety that comes in handy especially at refactoring time, and if you, by mistake, use a Post.Id where a Blog.Id is expected, you get a nice compiler error.

Caveats

I implemented the BlogId and PostId using records to keep the code succinct, in real life you may want to add a bit more to it, like for example overriding ToString to only print value, and maybe add some validation to make sure you can't create a negative value and so on, using a struct also has some similar performance characteristics of using an int.

Please also note that EF Core 7 has few issues that will be resolved in the coming months so you may want to wait for some of these issues to be resolved before pushing it to prod.
// Detect dark theme var iframe = document.getElementById('tweet-1593244935939305474-661'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1593244935939305474&theme=dark" }

I hope you enjoyed this article, till the next time!

Azure WebJobs, Service Bus and Managed Identity: Lesson learned

Massimiliano Donini — Tue, 09 Aug 2022 13:20:02 +0000

Today I was converting some Azure webjobs to connect to Azure Service Bus using managed service identity (MSI).

The application is a simple C# Azure WebJob built using the Azure WebJob SDK that subscribe to a topic and process incoming message writing to a database.

These are the nuget packages used:

Microsoft.Azure.WebJobs v 3.0.33

Microsoft.Azure.WebJobs.Extensions.ServiceBus v 5.6.0 Please note that since Azure Functions are built on top of the WebJobs SDK, you may encounter the same issue there, I haven't verified though.

In order to grant the required permission, I created a security group and added the managed identity of the app service to the group, then I proceeded to grant this service group the Azure Service Bus Data Owner

According to the description, this role should have full access to the whole Service Bus namespace, so imagine my surprise when I tried to run the application and got an error that looks like the following:

Unauthorized access. 'Listen' claim(s) are required to perform this operation. 
Resource: 'sb://{namespace-name}.servicebus.windows.net/{topic-name}/subscriptions/{service-name}'.
TrackingId:4e956a067b044b1089b5e327c0d08fd0_G9, SystemTracker:gateway7, Timestamp:2022-08-05T15:32:58

After several trial and error, this is what I found:

If you assign the permission Azure Service Bus Data Owner to a group and the managed identity of the application is part of the group, it won't work, no matter which permission you grant ❌
If you assign the managed service identity the permission Azure Service Bus Data Owner it won't work ❌
If you assign the permission Azure Service Bus Data Receiver to a group and the managed identity of the application is part of the group, it will work ✅
If you assign the managed service identity the permission Azure Service Bus Data Receiver, it will work ✅

Please note it may take up to 5 minutes for the permission changes to be applied, so you may still experience failures after applying them.

The only way to get it working was to assign the managed identity the Azure Service Bus Data Receiver role to either the service identity or the security group.

This seems either a bug in the Azure SDK or in the Service Bus itself, I'm not the only one that ran into this issue and here you can find additional information.

Till the next time.

Dynamically scale down AppService outside business hours to save 💰💰

Massimiliano Donini — Mon, 25 Jul 2022 05:51:35 +0000

The other day I was on a quest to lower a bit our Azure spending.

Im my current company we have several environment that we use for different purposes, Development, Test, Acceptance and so on.

All these environments have slightly different tiers for various services and I was wondering how to lower App Service Plan tier outside business hours.

App Services have some built-in, albeit limited, capabilities to scale but this only involves scaling out.

Scaling out is the process of adding additional instances of our application to adapt to an increasing load.
Scaling up is the process of running the application on a more performant hardware.

Since there's no built-in support to scale up & down in App Services, I had to come up with a custom solution.

After a bit of research, I ended up creating an Azure automation account, two runbooks that execute on a schedule the scale down and scale up of our App Service Plan.

It turned out to be extremely simple to implement yet effective.

Disclaimer: This is just one of the possible way to scale services up & down outside business hours, you can achieve the same with a scheduled github action or Azure DevOps pipeline than runs your IaC code with different Sku parameter values for example.

Here's what I've created:

Azure automation account
Azure Runbooks
Azure automation schedule
Azure automation account variables

Azure automation account - Docs

This is the go to resource to automate processes in Azure, where you define the runbooks, the schedule and the variables.

Azure Runbooks - Docs

This is where I defined what needs to happen when the schedule triggers the runbook and starts a job.
There are several different types of runbooks, here I chose the powershell one.

Azure automation schedule - Docs

This is where I defined when to execute our runbooks, I went with a weekly schedule to scale down resource in the evening and scale them back up early in the morning.

Azure automation account variables - Docs

This is where I defined few variables used by the runbook.
This step is optional since you can potentially hardcode everything in the runbook itself, but if you want to use the same runbook across different environment, you can define variables and read them in the runbook.
I defined few variables, one for the resource group name, one for the app service plan name and the desired scale down sku and the one the needs to be used during business hours.

In order for the runbook to successfully change the App Service Plan, we also need to grant the identity - either managed identity or user assigned one - of the automation account enough grant on the App Service Plan. I went with managed identity.

Putting it all together

After creating the variables, the schedules and the runbooks I linked the schedule to the runbook.
I created two schedules called scale-down and scale-up, two runbooks named the same way and linked the schedule to the runbook. You link a runbook to a schedule in the overview page of the runbook itself.

Last missing part is the code of the runbook itself, so here's the shortest possible version of it (of course you can make it smarter based on your needs) used to scale down, the scale up version is exactly the same but read a different variable for the sku.

Till the next one!

Zero downtime deployment with Azure Container Apps and Github Actions - Part 1

Massimiliano Donini — Fri, 24 Jun 2022 16:21:26 +0000

Introduction

As you may know, Azure Container Apps went out of preview during Microsoft Build in late May this year.
Azure Container Apps is a very interesting service that runs on top of Kubernetes adding some additional powerful capabilities in a simple and covinient way.
Some of these capabilities are:

Built in support for Keda autoscalers
Built in support for Dapr components
Ability to scale to zero

There's a lot more to it, you can dig deeper on the official Microsoft documentation here

TL;DR; All the code described in this article is available on Github here
I'm still working on improvements so main branch may be updated by the time you read this.

Requirements

I would like to migrate several Azure App Services to Azure Container Apps, I have two different type of services, http api that write to a Service Bus queue/topic and web jobs that consume and process the messages, a pretty common setup these days.

Azure Container Apps allows me to easily scale the web jobs based on the amount of messages present in the queue/topic and also makes it easy to scale to zero outside business hours.

In order to migrate from Azure App Service to Azure Container Apps I want to implement a zero downtime deployment for the http api services.

Problem

Azure Container Apps has built in support for health probes, there are 3 types of health probes:

Liveness
Readiness
Startup

Given the built-in health probes support, I went under the assumption that, while using single revision mode, we could just deploy another revision and the control plane could take care of warming up and then swapping traffic to the new revision without downtime.
To my surprise I figured out that's not the case and if you're reading this blog post, you might have noticed that too.

I also double checked it on Discord with the Azure Container Apps team if I was missing something on my end, but they confirmed my findings.

After doing a bit of research, I figured it out that I can implement my own workflow to implement zero downtime deployment. This is far from ideal but still better than a deployment process that causes downtime.
Hopefully Azure Container Apps will implement built support for zero downtime deployment, but in the meantime the following approach is an acceptable workaround.

Solution

In order to implement zero downtime deployment in (pseudo) single revisions mode (meaning using multiple revision mode with a single active revision at a time, serving all the traffic), we need to do the following steps:

Redirect all the traffic to the latest revision by name (more on that later)
Deploy a new revision
Warm up the new revision
Redirect the traffic to the newly deployed revision

In this blog post I am using multiple revision mode because it happened to me that moving from single revision to multiple revision mode caused downtime - I'm currently investigating it and will update this post as soon as I found.
The idea is to have the container app configured in multiple revision mode but only have one active revision at a time.

Regarding step 1, Redirect all the traffic to the latest revision by name, has to do with the latest revision alias.
I found that if the traffic is set to the latest revision, when deploying a new revision, the traffic get redirected to the new revision even while it's still in the provisioning state, leading to possible timeouts and failures on the caller side.

The workflow above is a bit long but not particularly difficult, and we can easily implement it with the help of the Azure Cli containerapp extension.

Out of all the steps above, point 3 (Warm up the new revision) is the most tricky since different services may have different health probe configurations. I really didn't want to duplicate health probe configurations in the infrastructure and in the deployment pipeline but rather reuse what has been configured in the Container App Container instead.

Dynamically discovering and calling the health probe in bash is doable but I am a bit more proficient writing that code in a high level language so I decided to write a small C# application to do that.

In order to manage Azure resources, we can use the Azure Management SDK, this is a set of packages that allows you to manage Azure resources in your language of choice.
You can find all the supported resources here.

Luckily, Azure Container Apps already have an SDK available, although in beta at the moment of writing.

Since I should be able to invoke this application from a Github Action, I decided to implementing a web application that exposes an api to make my life easy.

Essentially all the steps described above but steps 4 are executed in a github action, while step 4 is executed by a web application invoked by cURL in the Github Actions.

The next point to solve is where should be this web application deployed, we have few options:

Have it always available
Make it available on demand

If you deploy an internal Azure Container App Environment and the application is not exposed to the internet, point two may be a bit more complicated since you need to make sure the github action runner can reach the Azure Container App you want to warm up.

I initially went with the first approach, having the application always available, but since I didn't want to also add authentication to the mix (to make sure only verified clients can call the warmup endpoint), I later decided against it.

In order to make this application available on demand, I decided to use Github Actions service container.
This post is getting already a bit too long so I won't go into detail of what service container is, let's just say that it allows you to run a container and made it available on the runner. If you wanna dig deeper, you can check the documentation here

After sorting the Github Actions service container, last problem that I had to tackle was how to authenticate the application against Azure.
Thanks to Azure.Identity package and the DefaultAzureCredential class, we can just set some environment variables and authenticate with a previously defined service principal.

Azure management SDK authentication

In order to get the required credentials to authenticate, we need to create a service principal with the Reader role on the resource group that contains the Azure Container Apps.
We can quickly create this with the az cli and the following command:

az ad sp create-for-rbac -n "HealthProbeSp" --role Contributor --scopes /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupId}

Remember to replace the subscription and resource group names

After setting all this up, I was able to implement zero downtime deployment.

Here's an extract of the Github Action:

jobs:
  build:
    name: build ${{ matrix.services.appName }}
    runs-on: ubuntu-latest
    permissions:
      contents: read
      packages: read
    services:
      health-invoker:
        image: ghcr.io/${{ github.repository }}/health-invoker:main
        ports:
          - 5000:80
        env:
          AZURE_TENANT_ID: ${{ secrets.AZURE_TENANT_ID }}
          AZURE_CLIENT_ID:  ${{ secrets.AZURE_CLIENT_ID }}
          AZURE_CLIENT_SECRET: ${{ secrets.AZURE_CLIENT_SECRET }}
          Azure__SubscriptionId: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
          Azure__ResourceGroupName: ${{ secrets.RESOURCE_GROUP_NAME }}

# Clone repo, Build and push omitted for brevity 

      - name: Deploy azure container app without downtime
        if: github.event_name != 'pull_request' && matrix.services.zeroDowntime == true
        run: |
          echo "Installing containerapp extension"
          az extension add --name containerapp --upgrade &> /dev/null
          echo "Get latest active revision name"
          latest_revision=$(az containerapp show -n ${{ matrix.services.appName }} -g ${{ secrets.RESOURCE_GROUP_NAME }} --query properties.latestRevisionName -o tsv)
          echo "Redirect traffic to active revision $latest_revision"
          az containerapp ingress traffic set -n ${{ matrix.services.appName }} -g ${{ secrets.RESOURCE_GROUP_NAME }} --revision-weight $latest_revision=100 &> /dev/null
          echo "Create new revision"
          az containerapp update -n ${{ matrix.services.appName }} -g ${{ secrets.RESOURCE_GROUP_NAME }} -i ${{ steps.image-tag.outputs.tag }} &> /dev/null
          new_revision=$(az containerapp show -n ${{ matrix.services.appName }} -g ${{ secrets.RESOURCE_GROUP_NAME }} --query properties.latestRevisionName -o tsv)
          echo "Warmup new revision at ${{ env.WARMUP_APP }}/warmup/${{ matrix.services.appName }}"
          health_response_status=$(curl -m 180 --write-out "%{http_code}\n" -s ${{ env.WARMUP_APP }}/warmup/${{ matrix.services.appName }} --output backend.txt)
          if [ $health_response_status = "200" ]; then
            echo "Redirect traffic to new revision $new_revision"
            az containerapp ingress traffic set -n ${{ matrix.services.appName }} -g ${{ secrets.RESOURCE_GROUP_NAME }} --revision-weight $new_revision=100 $latest_revision=0 &> /dev/null
            echo "Deactivate revision $latest_revision"
            az containerapp revision deactivate -n ${{ matrix.services.appName }} -g ${{ secrets.RESOURCE_GROUP_NAME }} --revision $latest_revision &> /dev/null
          else
            echo "Warmup failed with status code $health_response_status"
            cat ./backend.txt
            echo "Redirect traffic to active revision $latest_revision"
            az containerapp ingress traffic set -n ${{ matrix.services.appName }} -g ${{ secrets.RESOURCE_GROUP_NAME }} --revision-weight $latest_revision=100 &> /dev/null
            if [ ! -z "$new_revision" ]; then
              echo "Deactivate revision $new_revision"
              az containerapp revision deactivate -n ${{ matrix.services.appName }} -g ${{ secrets.RESOURCE_GROUP_NAME }} --revision $new_revision &> /dev/null
            fi
            exit 1
          fi

Here's the output of wrk while deploying a new revision:

wrk -t12 -c400 -d30s https://xxxxxxxx.azurecontainerapps.io/api/echo/ping
Running 30s test @ https://xxxxxxxx.azurecontainerapps.io/api/echo/ping
  12 threads and 400 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   268.53ms  123.73ms   1.02s    68.90%
    Req/Sec   137.15    103.17     1.15k    66.59%
  41604 requests in 30.10s, 8.89MB read
Requests/sec:   1382.32
Transfer/sec:    302.38KB

After deployment has been completed:

wrk -t12 -c400 -d30s https://xxxxxxxx.azurecontainerapps.io/api/echo/ping
Running 30s test @ https://xxxxxxxx.azurecontainerapps.io/api/echo/ping
  12 threads and 400 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   251.00ms  135.47ms   1.24s    77.22%
    Req/Sec   148.38    104.52   434.00     63.10%
  44970 requests in 30.09s, 9.61MB read
Requests/sec:   1494.36
Transfer/sec:    326.89KB

As you can see there's almost no difference and, most importantly, wrk doesn't indicate any non 2XX or 3XX response meaning that we were able to serve all requests while deploying a new revision.

The numbers are quite low because I'm using a very small configuration for testing purposes (0.25 Cores and 0.5 Gi of memory)

I hope you find this helpful and if you have suggestions , don’t hesitate to comment or reach me out via twitter at twitter.com/maxx_don.

Stay tuned for part 2 that will be out very soon!

Create Azure Container Apps with terraform

Massimiliano Donini — Thu, 26 May 2022 08:33:58 +0000

Microsoft announced at Microsoft Build that Azure Container Apps are now generally available (GA).

If you're not familiar with Azure Container Apps (ACA) I suggest you to go and check out the documentation here.

Fully managed serverless container service for building and deploying modern apps at scale

This is how Microsoft itself markets the product.

I think it's a very interesting platform and it offers some of the benefits of Kubernetes, abstracting a lot of concepts and complexity.

In order to start playing around with it, I usually create a repo with some terraform code so I can spin up a set of resources, play with them and then destroy all of them when I'm done.

As you may know not all Azure resources are available in terraform azure provider(s) from day 1, but for a month or so we have the awesome AzApi provider for terraform.

Follow this github issue that tracks adding support for it in the azurerm official provider.

The AzAPI provider enables you to manage any Azure resource type using any API version.

Thanks to this provider it is very easy to create an Azure Container App with terraform.

Here's how you configure the provider:

Here you can see how you can use azapi_resource to create an Azure Container App

Tips

In order to discover the properties, I first make the changes it in the Azure portal, then I'm running the following az-cli command:

az containerapp list

and then you need to transform the json to the terraform equivalent.

You can also install Terraform AzApi Provider Visual Studio Code Extension VS Code extension that should provide completion support.

Kudos to piizei for coming up with the suggestion here

If you want to see a more advanced example, you can give a look at my repo here.

Hope you find this helpful, if you have any question/suggestion, don't hesitate to comment!

Using Managed Identity with Azure WebJobs and Service Bus

Massimiliano Donini — Tue, 24 May 2022 14:09:21 +0000

Managed Service Identity (or MSI for short) allows Azure resources to connect to Azure services that supports AD authentication (see the full list here) without using secrets.
This is extremely useful because handling secrets the proper way it's far from easy.

How MSI works is beyond the scope of the article and you can find more information here, but in a nutshell:

You create a service principal object and an application in Azure AD to represents your service (this is done automatically when turning on managed identity)
You grant some permissions to access the downstream service it needs to communicate to
You configure the authentication to the downstream service to be done via MSI

In my case, I have a WebJob that processes messages via a ServiceBus queue, so I granted the service principal a permission to read from the queue using the built in Azure Service Bus Data Receiver role.

The WebJob SDK supports connecting to the ServiceBus using MSI as described here so I went ahead and configured the WebJob in the following way:

"ServiceBusConnection__fullyQualifiedNamespace" : "<service_bus_namespace>.servicebus.windows.net"
"QueueName" : "test-queue"

The queue and the permission were defined in terraform as follows:

// Define the queue
resource "azurerm_servicebus_queue" "msi-test-queue" {
  name                = "test-queue"
  namespace_id        = azurerm_servicebus_namespace.msi-test-sb.id
  enable_partitioning = true
}

// Grant the WebJob Azure Service Bus Data Receiver permissions
resource "azurerm_role_assignment" "consumer_service_bus_read" {
  scope                = azurerm_servicebus_queue.msi-test-queue.id
  role_definition_name = "Azure Service Bus Data Receiver"
  principal_id         = azapi_resource.consumer_container_app.identity.0.principal_id
  depends_on           = [azapi_resource.consumer_container_app]
}

The WebJob definition in C# is something like this:

[FunctionName("Processor")]
public async Task ProcessEvent(
    [ServiceBusTrigger("%QueueName%", Connection = "ServiceBusConnection", IsSessionsEnabled = false)]
    ServiceBusReceivedMessage message, ServiceBusMessageActions messageActions)
{
    ....
}

Where the string ServiceBusConnection points to the name of the configuration value that contains the connection string to the Service Bus and the string %QueueName% point to the configuration value that contains the queue name.

If you don't use the percent sign, the string will be the name of the queue and it will be hardcoded, adding the %% allows you to configure dynamically via a configuration lookup.

This unfortunately didn't work, the WebJob was throwing exception at startup complaining about permissions, the message was:

Unauthorized access. 'Listen' claim(s) are required to perform this operation

This was unexpected since the permission was there and I double checked it in the Azure portal.

The only way I was able to get this working was to grant Azure Service Bus Data Receiver permission on the whole service bus namespace.

resource "azurerm_role_assignment" "consumer_service_bus_read" {
  scope                = azurerm_servicebus_namespace.msi-test-sb.id
  role_definition_name = "Azure Service Bus Data Receiver"
  principal_id         = azapi_resource.consumer_container_app.identity.0.principal_id
  depends_on           = [azapi_resource.consumer_container_app]
}

One caveat of this approach is that it grants more permissions than strictly required, but at least it got me unblocked.

If you're interested into the source code, you can find it here.

I hope you find this useful and if you have any questions/suggestions feel free to comment here below!

Implement Azure AD Workload Identity on AKS with terraform

Massimiliano Donini — Thu, 24 Feb 2022 07:43:47 +0000

Azure makes it very easy to create managed identities for a variety of services (e.g. Azure Functions, App Services, Logic Apps...), but when we want to implement it for Azure Kubernetes Service, things gets just a bit more complicated.

First of all we have few options to choose from:

AAD Pod Identity (deprecated)
Azure AD Workload Identity What we discuss in this post, azwi for brevity.

Both solutions aims to associate a pod with an identity in Azure Active Directory so we can grant this identity permissions to access another resource (i.e. a storage account or an Azure Sql Database).

As described on the documentation, azwi is the suggested approach from now on since Azure AD Pod Identity has been (somehow) deprecated as you can read on the github repo and on the blog post here.

The documentation describes Azure AD Workload Identity as follows:

Azure AD Workload Identity for Kubernetes integrates with the capabilities native to Kubernetes to federate with external identity providers. This approach is simpler to use and deploy, and overcomes several limitations in Azure AD Pod Identity

Assuming you already have an AKS cluster up & running (I won't cover the creation of it here), in order to configure Azure AD Workload Identity we need to:

Configure the AKS cluster to enable OIDC issuer
Deploy the Azure AD Workload Identity helm chart to the cluster
Create a Federated Azure AD Application + a Service Principal
Create a kubernetes service account manifest with some azwi specific metadata
Configure our pods to run with the service account

1. Configure the AKS cluster to enable OIDC issuer

Unfortunately since OIDC issuer feature is still in preview at the time of writing (February 2022), there's no built-in support in terraform, but this is a one time only operation, you can read more about it here.

So we need to enable it from the azure cli with the following command:

Enable az cli preview feature

# Install the aks-preview extension
az extension add --name aks-preview

# Update the extension to make sure you have the latest version installed
az extension update --name aks-preview

Enable OIDC issuer on an existing cluster

az aks update -n aks -g myResourceGroup --enable-oidc-issuer

After we enable the OIDC issuer feature we need to get the OIDC issuer url that will be used in the next step to federate the Azure AD Application, this can be done with the following command:

az aks show --resource-group <resource_group> --name <cluster_name> --query "oidcIssuerProfile.issuerUrl" -otsv

2. Deploy the Azure AD Workload Identity helm chart to the cluster

We can deploy a helm chart in several ways, in this case I decided to deploy the chart using terraform.
You can achieve that with the following terraform code:

resource "kubernetes_namespace" "azure-workload-identity-system" {
  metadata {
    annotations = {
      name = "azure-workload-identity-system"
    }
    name   = "azure-workload-identity-system"
    labels = var.tags
  }
}

resource "helm_release" "azure-workload-identity-system" {
  name       = "workload-identity-webhook"
  namespace  = "azure-workload-identity-system"
  chart      = "workload-identity-webhook"
  repository = "https://azure.github.io/azure-workload-identity/charts"
  wait       = false
  depends_on = [kubernetes_namespace.azure-workload-identity-system]

  set {
    name  = "azureTenantID"
    value = var.azureTenantID
  }
}

Here we create a new kubernetes namespace and we deploy the helm release. I choose to deploy with terraform the helm charts that I depend on (i.e. my application dependencies, for example Azure AD Workload Identity and kong that I use as my ingress).
We need to set the azureTenantID value when we deploy the helm chart with the current azure tenant id.
I read the current tenant id in the root module with the data "azurerm_subscription" "current" {} data source and pass in as a variable in the child module.

3. Create a Federated Azure AD Application + a Service Principal

Here we need to create an Azure AD Application + a Service Principal and federate the application with the OIDC Issuer so that Azure AD can exchange a token issued to the pod with a token that can be used to access other Azure resources.

We can achieve it with a bit of terraform:

locals {
  namespace_name = "app-ns"
  ## This should match the name of the service account created by helm chart
  service_account_name = "app-${local.namespace_name}"
}

## Azure AD application that represents the app
resource "azuread_application" "app" {
  display_name = "sp-app-${var.env}"
}

resource "azuread_service_principal" "app" {
  application_id = azuread_application.app.application_id
}

resource "azuread_service_principal_password" "app" {
  service_principal_id = azuread_service_principal.app.id
}

## Azure AD federated identity used to federate kubernetes with Azure AD
resource "azuread_application_federated_identity_credential" "app" {
  application_object_id = azuread_application.app.object_id
  display_name          = "fed-identity-app-${var.env}"
  description           = "The federated identity used to federate K8s with Azure AD with the app service running in k8s ${var.env}"
  audiences             = ["api://AzureADTokenExchange"]
  issuer                = var.oidc_k8s_issuer_url
  subject               = "system:serviceaccount:${local.namespace_name}:${local.service_account_name}"
}

output "app_client_id" {
  value = azuread_application.app.application_id
}

Here we need to specify a couple of things:

The OIDC Issuer url that we got from step 1, (I'm using a variable here to hold it's value)
The subject that should follow a specific format: system:serviceaccount:{k8s_namespace}:{k8s_service_account_name}

The namespace should match the namespace you will use to install your app in kubernetes and the Service Account name should match what you define in the kubernetes manifest.

4. Create a kubernetes service account manifest with some specific metadata

Here we will create the service account manifest and add the required metadata to allow the azwi do it's magic.

Here's the code to create a service account and the corresponding value file:

# serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: {{ include "app.serviceAccountName" . }}
  labels:
    {{- include "app.labels" . | nindent 4 }}
    {{- with .Values.serviceAccount.labels }}
    {{- toYaml . | nindent 4 }}
    {{- end }}
  {{- with .Values.serviceAccount.annotations }}
  annotations:
    {{- toYaml . | nindent 4 }}
  {{- end }}

# value.yaml
serviceAccount:
  # Labels to add to the service account
  labels:
    azure.workload.identity/use: "true" # Represents the service account is to be used for workload identity, see https://azure.github.io/azure-workload-identity/docs/topics/service-account-labels-and-annotations.html
  # Annotations to add to the service account
  annotations:
    azure.workload.identity/client-id: "{Client Id of the azure ad application}"
    azure.workload.identity/tenant-id: "{Tenant Id of you Azure subscription}"
    azure.workload.identity/service-account-token-expiration: "86400" # Token is valid for 1 day

Please note that you can get the client id from the output of the step 3 and that the name of the service account should match what you set in the subject of the azuread_application_federated_identity_credential.

5. Configure our pods to run with the service account

We need to make sure our pods run with the service account created in step 4.
In order to do that we just need to specify the serviceAccountName with the name of the Service Account in our deployment.yaml file as shown below:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ include "app.fullname" . }}
  labels:
    {{- include "app.labels" . | nindent 4 }}
spec:
  {{- if not .Values.autoscaling.enabled }}
  replicas: {{ .Values.replicaCount }}
  {{- end }}
  selector:
    matchLabels:
      {{- include "app.selectorLabels" . | nindent 6 }}
  template:
    metadata:
      {{- with .Values.podAnnotations }}
      annotations:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      labels:
        {{- include "app.selectorLabels" . | nindent 8 }}
    spec:
      {{- with .Values.imagePullSecrets }}
      imagePullSecrets:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      serviceAccountName: {{ include "app.serviceAccountName" . }}
....

This is all it takes, after we're done here, we can grant our Service Principal some rights to, for example, allow it to access a storage account in the following way:

## Lookup our storage account
data "azurerm_storage_account" "storage" {
  name                = var.storage_account_name
  resource_group_name = var.storage_account_rg
}

## Role assignment to the application
resource "azurerm_role_assignment" "app_storage_contributor" {
  scope                = data.azurerm_storage_account.storage.id
  role_definition_name = "Storage Blob Data Contributor"
  principal_id         = azuread_service_principal.app.id
}

This is my required_providers configuration:

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 2.84"
    }
    azuread = {
      source  = "hashicorp/azuread"
      version = "~> 2.14.0"
    }

    helm = {
      version = "2.4.1"
    }
  }
}

That's all for now, I hope you find this interesting, if you have any questions/suggestions, don't hesitate to comment!