DEV Community: Richard Roché

Single Sign-On, Azure Static Web Apps and Azure Active Directory

Richard Roché — Tue, 01 Mar 2022 10:00:00 +0000

We often build and deploy web applications specifically for users internal to our organisation. Azure Static Web Apps is proving to be an excellent replacement for Azure App Service in these scenarios.

At a high-level the service provides you with a great set of features (outlined in the Azure release notes)

Globally distributed content for production apps

Tailored CI/CD workflows from code to cloud

Auto-provisioned preview environments

Custom domain configuration and free SSL certificates

Built-in access to a variety of authentication providers

Route-based authorization

Custom routing

Integration with serverless APIs powered by Azure Functions

A custom Visual Studio Code developer extension

A feature-rich CLI for local development

The desired experience?

The experience I wanted to achieve was that if one of our internal users browsed to any of our internal apps, they would be able to use SSO across them provided they were a member of the AAD group needed to access the app (or just a member of our tenant for organisation-wide apps) - no login button, just a seamless logged-in user experience.

It turns out this was super easy to get right! Follow below!

Authentication options

Azure Static Web Apps makes authentication easy to enable across the three pre-configured identity providers

Azure Active Directory (AAD)
Github or
Twitter

These options allow users to login using a login button linking to the desired provider.

Initially I tried to use the pre-configured AAD provider, and when trying to log in using my company account I was presented with this approval dialogue

Meaning if I was able to get an admin to grant the permission, all users from our tenant would be able to log in to all Azure Static Web Apps, regardless of who had deployed them making this a non-starter for me.

Fortunately we already deploy our static web apps using the Standard plan for $9/per app/month, giving us the ability to use custom authentication and use SSO with our organisations AAD tenant, internal app registrations and restrict access to groups / users in our tenants directory.

Getting it done!

To achieve the desired experience there are a number of components required

the static web app (the website)
an Azure App Registration for your app in your tenant
an Azure resource group for your project (I'm working on the assumption you already have a subscription, if not read here)
- an Azure Static Web App for the web app
- an Azure Key Vault to safely store the secrets for your app

I will be showing you how to create, configure and deploy these using GitHub Actions, Bicep (for creating the Azure resources) and some once off scripts.

Note: All code is available on GitHub over here

Note: It is a good idea to wire up your Key Vault to a Log Analytics workspace or similar to track audit events when in production.

Initial setup and deployment

First we are going to scaffold our web app, define our infrastructure and deploy to Azure without any auth in place. Once done, we will move onto the configuration of the app with auth.

Scaffold the web app

For the purposes of this post, I needed a simple static web app and used the SvelteKit skeleton project with the static adapter.

npm init svelte@next webapp
cd webapp
npm install --save-dev @sveltejs/adapter-static@next

You need to update the default svelte.config.js to use the static adapter, see an example here.

Define the Azure Infrastructure

Resource Group

First, create an Azure resource group. For this tutorial I'm creating it manually from my terminal and Azure CLI

az group create -g rg-swa-sso -l northeurope

Resources

We will be deploying a Static Web App as well as a Key Vault using Bicep. I've split out the Bicep files to make them easier to work with as follows (filenames link to the code on GitHub)

main.bicep - the main template that is deployed which makes use of the other templates
get-kv-secrets-refs.bicep - a helper to build up the Key Vault secret references
key-vault.bicep - defines the Key Vault resources and the role assignments needed
static-sites.bicep - defines the Static Web App resources needed

In the main.bicep there are the following highlights to make note of

Configuring the static web app
- to have app settings that are Key Vault references (AAD_CLIENT_ID, AAD_CLIENT_SECRET)
- to use the Standard sku

module swa 'static-sites.bicep' = {
  name: 'deploy-swa-${appName}'
  params: {
    appSettings: {
      AAD_CLIENT_ID: refs.outputs.aadClientIdRef
      AAD_CLIENT_SECRET: refs.outputs.aadClientSecretRef
    }
    ...
    sku: {
      name: 'Standard'
      tier: 'Standard'
    }
    ...
  }
}

And configuring the key vault to allow the static web app permissions to read from it

// https://docs.microsoft.com/en-us/azure/key-vault/general/rbac-guide?tabs=azure-cli#azure-built-in-roles-for-key-vault-data-plane-operations
var keyVaultSecretsUserRole = '4633458b-17de-408a-b874-0445c86b69e6'

module kv 'key-vault.bicep' = {
  name: 'deploy-kv-${appName}'
  params: {
    ...
    roleAssignments: [
      {
        roleDefinitionId: keyVaultSecretsUserRole
        principalType: 'ServicePrincipal'
        principalId: swa.outputs.siteSystemAssignedIdentityId
      }
    ]
    ...
  }
}

Deploy using GitHub

Deployment credentials

For a GitHub Action to be able to deploy to your resource group you need to have some form of deployment credentials. You can read about the options available here. I'll be using the Service Principal approach with Azure CLI.

az ad sp create-for-rbac --name "sp-swa-sso" \
    --role Owner \
    --scopes /subscriptions/{subscription-id}/resourceGroups/{resource-group-name}
    --sdk-auth

You will get a JSON output from this that you can then save as a GitHub secret with the name AZURE_CREDENTIALS.

Note:

Replace {subscription-id} with the ID of your subscription
Replace {resource-group-name} with the name of your resource group
I've made the service principal have Owner access to the resource group (so that I can assign roles).

Workflow token

Azure Static Web Apps needs access to your workflow when deploying. For this we'll set up a WORKFLOW_TOKEN secret using a GitHub personal access token with the workflow scope. Follow the instructions here to create the token.

Github Action Workflow

With the above secrets in place, we can now create a workflow (mines in .github/workflows/deploy.yaml) which

specifies some environment variables for names, tags and locations
checks out the repo
logs into Azure using ${{ secrets.AZURE_CREDENTIALS }}
deploys to the resource group using the azure/CLI@v1 action
gets the API from the deployed static web app (so that we can deploy code to it)
deploys the webapp using the Azure/static-web-apps-deploy@v1 action

See the full workflow below.

name: Deploy Infra and App

on:
  push:
    branches:
      - main
  pull_request:
    types: [opened, synchronize, reopened, closed]
    branches:
      - main

env:
  RESOURCE_GROUP: 'rg-swa-sso'
  RESOURCE_TAGS: '{"owner":"rick.roche", "app":"azure-swa-sso", "repo":"https://github.com/rick-roche/azure-static-web-apps-sso" }'
  APP_NAME: 'swa-sso'
  LOCATION: 'westeurope'

jobs:
  deploy-infra:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Repository
        uses: actions/checkout@v2

      - uses: actions/setup-node@v2
        with:
          node-version: '16'

      - name: Azure Login
        uses: azure/login@v1
        with:
          creds: ${{ secrets.AZURE_CREDENTIALS }}

      - name: Deploy Infra
        id: deploy_infra
        if: github.event_name != 'pull_request'
        uses: azure/CLI@v1
        with:
          inlineScript: |
            az deployment group create \
              --resource-group ${{ env.RESOURCE_GROUP }} \
              --template-file ./infra/main.bicep \
              --parameters \
                  appName='${{ env.APP_NAME }}' \
                  location='${{ env.LOCATION }}' \
                  repositoryUrl='https://github.com/rick-roche/azure-static-web-apps-sso' \
                  repositoryToken='${{ secrets.WORKFLOW_TOKEN }}' \
                  tags='${{ env.RESOURCE_TAGS }}'

      - name: Get Static Web App API Key
        id: static_web_app_apikey
        if: github.event_name != 'pull_request'
        uses: azure/CLI@v1
        with:
          inlineScript: |
            APIKEY=$(az staticwebapp secrets list --name 'stapp-${{ env.APP_NAME }}' | jq -r '.properties.apiKey')
            echo "::set-output name=APIKEY::$APIKEY"

      - name: Deploy WebApp to Static Web App
        id: static_web_app_deploy
        if: github.event_name != 'pull_request'
        uses: Azure/static-web-apps-deploy@v1
        with:
          azure_static_web_apps_api_token: ${{ steps.static_web_app_apikey.outputs.APIKEY }}
          repo_token: ${{ secrets.GITHUB_TOKEN }} # Used for GitHub integrations (i.e. PR comments)
          action: 'upload'
          # Build configuration for Azure Static Web Apps: https://aka.ms/swaworkflowconfig
          app_location: 'webapp'
          api_location: ''
          output_location: 'build' # relative to app_location

Finalising auth

At this stage you should have a GitHub workflow successfully deploying a Static Web App and a Key Vault to you resource group. You should also be able to browse to your web app using the generated URL (navigate into the Static Web App from the portal, and you will see your URL on the overview tab. e.g. https://gray-wave-03fb32a03.1.azurestaticapps.net).

Note: Your app won't ask you to authenticate just yet!

To enable auth our next steps will be to

create an AAD app registration
add it's client ID and secret as secrets in your key vault
configure the static web app to auto log you in if you aren't already or your token has expired

AAD App Registration

An AAD app registration allows us to bind our application to a desired set of authentication flows and restrictions. Think of it as giving your application an identity inside AAD. Mark Foppen wrote a lovely article that may help demystify this a bit.

In this tutorial I'll just be using Azure CLI to create one:

az ad app create --display-name aadapp-swa-sso \
    --available-to-other-tenants false \
    --identifier-uris api://stapp-swa-sso \
    --reply-urls 'https://gray-wave-03fb32a03.1.azurestaticapps.net/.auth/login/aad/callback' \
    --native-app false

Note:
Ensure you set the --reply-urls to be the generated URL of your app with the /.auth/login/aad/callback suffix. This allows AAD to call your app once the auth flow has completed.

Once created, we need to create an application secret for the application. For this tutorial you can do this via the portal following the instructions here.

Copy the value of the secret as well as the Application (client) ID of the AAD app (found under the Overview section of the app).

Setup Key Vault Secrets

Once again, I've used the portal for this tutorial. You can follow the guide here setting two secrets in your vault

aadClientId - this should be set to the Application (client) ID of your app registration
aadClientSecret - this should be set to the value of the application secret created above

Configuring the Static Web App

Configuration for Azure Static Web Apps is defined in the staticwebapp.config.json file, which controls, among other things, authentication and authorisation.

I've put mine in the root of the webapp folder and set it to do the following to enable the auto-login SSO magic

enable custom auth with Azure Active Directory Version 2 using the app settings references from earlier (AAD_CLIENT_ID, AAD_CLIENT_SECRET)

  "auth": {
    "identityProviders": {
      "azureActiveDirectory": {
        "registration": {
          "openIdIssuer": "https://login.microsoftonline.com/4af290c8-08df-440d-93db-cbac02bd9b19/v2.0",
          "clientIdSettingName": "AAD_CLIENT_ID",
          "clientSecretSettingName": "AAD_CLIENT_SECRET"
        }
      }
    }
  },

a navigationFallback route to index.html

  "navigationFallback": {
    "rewrite": "index.html"
  },

specific rules for the apps routes

creates a /login route redirecting to AAD allowing anonymous access

{
  "route": "/login",
  "rewrite": "/.auth/login/aad",
  "allowedRoles": ["anonymous", "authenticated"]
},

blocks all providers except AAD

{
  "route": "/.auth/login/github",
  "statusCode": 404
},
{
  "route": "/.auth/login/twitter",
  "statusCode": 404
},

creates a /logout route redirecting to AAD allowing anonymous access

{
  "route": "/logout",
  "redirect": "/.auth/logout",
  "allowedRoles": ["anonymous", "authenticated"]
},

enforces auth for all other routes (/*)

{
  "route": "/*",
  "allowedRoles": ["authenticated"]
}

sets up a response override that if an unauthenticated user hits a page, they should be redirected to the /login route
```
"responseOverrides": {
  "401": {
    "redirect": "/login",
    "statusCode": 302
  }
}
```

The combination of all of the above means that anyone accessing the site that isn't logged in will be routed to the /login route which will ensure that the AAD auth flow is completed!

Finishing up!

Commit all your outstanding changes, push to GitHub and watch your action deploy... once done, access your web app in the browser, and it should redirect you to login

Initially you will need to provide admin consent for your AD app registration to read the logged-in user details

If all has gone well you should see the default page after login completes

And that's it! Hope you enjoyed this tutorial and that it helps you setup SSO for your Static Web Apps!

Note:
A reminder that all code is available on GitHub over here

Featured image background by Hello I'm Nik on Unsplash

Bicep Modules: Refactor, Compose, Reuse

Richard Roché — Sun, 11 Jul 2021 11:00:00 +0000

In my previous post I touched on the things I learnt while migrating ARM templates to Bicep. Bicep also introduces the concept of modules to enable template reuse. I took some time to refactor a composite application that had already been converted from using ARM to Bicep templates, to use Bicep modules. This post will cover the things that I learnt by working through that process.

Why modules?

From the Bicep documentation:

Bicep enables you to break down a complex solution into modules. A Bicep module is a set of one or more resources to be deployed together. Modules abstract away complex details of the raw resource declaration, which can increase readability. You can reuse these modules, and share them with other people. Bicep modules are transpiled into a single ARM template with nested templates for deployment.

One of the traps we fell into with ARM templates was duplicating templates to make composing and deploying the resources we need easier. Any opportunity to make the deployment of infrastructure more readable, more reusable, and more composable are excellent reasons for me to give it a go. My goal when doing this refactor was to

get rid of any duplication by creating fine-grained modules, designed to be reused
ensure that all main templates are super easy to use by composing modules together in a way that makes sense for the application
enable reusability of the fine-grained modules in other projects going forward.

In short: let's make the infra readable, composable and reusable.

Notable learnings

Reusable Modules

I went with the approach of trying to make a reusable module for each Azure resource type and putting the modules into a folder called modules and making sub folders for template groupings. For example,

modules/
    appInsights.bicep
    appServicePlan.bicep
    functionApp.bicep
    logAnalytics.bicep
    storageAccount/
        storageAccount.bicep
        tables.bicep

Personally I prefer to have one module to create a storage account and another to add tables to that storage account etc. This level of granularity felt like a good place to start.

Referencing existing resources inside a module

By making fine-grained modules, there were a number of use cases where I would need to reference an existing resource. For example, creating tables in a storage account requires an existing storage account. Referencing an existing resource is really easy -- you only need to know its name and can reference it as follows,

// Lookup an existing resource
resource storageAccount 'Microsoft.Storage/storageAccounts@2021-02-01' existing = {
  name: storageAccountName
  // Optional if the existing resource is in a different resource group or subscription
  scope: resourceGroup(subscriptionId, resourceGroupName)
}

// You can now use the resource as if you had created it. e.g.
outputs storageAccountResourceId = storageAccount.id

Loops!

I really like the loops feature. This allows you to iterate over an array setting multiple properties or creating multiple resources etc. This came in super handy for a storage account tables module that can create multiple tables in one go. E.g.

param storageAccountName string
param tables array = [
  {
    container: 'default'
    name: 'replace'
  }
]

resource storageAccount 'Microsoft.Storage/storageAccounts@2021-02-01' existing = {
  name: storageAccountName
}

resource storageAccountTables 'Microsoft.Storage/storageAccounts/tableServices/tables@2021-02-01' = [for table in tables: {
  name: '${storageAccount.name}/${table.container}/${table.name}'
  dependsOn: [
    storageAccount
  ]
}]

output storageAccountTableNames array = [for (table, i) in tables: {
  name: storageAccountTables[i].name
}]

Note the use of the different styles of the loops in the storageAccountTables resource and the outputs.

Functions and Expressions

There are loads of functions and expressions that you can use in your Bicep files and I won't go into all of them. There were a few that I used regularly, and it's hopefully useful that I call them out.

Union

union(arg1, arg2, arg3, ...)
Returns a single array or object with all elements from the parameters. Duplicate values or keys are only included once.

I used union everywhere I wanted to have fixed (opinionated) defaults in the module, but allow additional parameters to be merged in. For example, with function apps I defined base settings I want all function apps to have and still allow for additional app settings to be passed in and merged with the base.

@description('Additional app settings for your function app')
param additionalAppSettings object = {}

var appSettingsBase = {
  FUNCTIONS_EXTENSION_VERSION: '~3'
  FUNCTIONS_WORKER_RUNTIME: 'dotnet'
  WEBSITE_RUN_FROM_PACKAGE: '1'
  AzureWebJobsStorage: 'DefaultEndpointsProtocol=https;AccountName=${storageAccount.name};EndpointSuffix=${environment().suffixes.storage};AccountKey=${listKeys(storageAccount.id, storageAccount.apiVersion).keys[0].value}'
}

var appSettings = union(appSettingsBase, additionalAppSettings)

We can also get rid of all the hardcoded schema API versions when looking up keys against a resource by using <resource>.apiVersion as well as hardcoded suffixes by using the environment function, in this case using the storage suffix: environment().suffixes.storage.

Another super useful call out would be to the listKeys function. It allows you to get connection strings or keys from your resources and is super handy (see the AzureWebJobsStorage example above). John Reilly went into detail on this over here, please have a read!

Ternary

I found that with Bicep I use the ternary operator a lot in my main templates when composing the modules. For example, only enabling a secondary region when the environment is nonprod or prod but not in dev can easily be described as,

var secondaryRegionEnabled = (contains(env, 'prod')) ? true : false

Much cleaner and readable without all the JSON around it.

Composing modules together

Every Bicep file can be consumed as a module which is an awesome feature. I chose to break my files up as follows:

infra/
  myApp/
    main.bicep - loads up the app.bicep and monitoring.bicep modules
    app.bicep - uses the fine grained modules - app service plans, functions, azure storage etc
    monitoring.bicep - uses the fine grained modules - app insights, log analytics etc
shared-infra/
  modules/
    <all the fine-grained modules>

In my main.bicep I reference the two local Bicep files as modules

module monitoring 'monitoring.bicep' = {
  name: '${appName}-monitoring'
  params: {
    tags: tags
    location: location
    appInsightsName: appInsightsName
    logAnalyticsWsName: logAnalyticsWsName
  }
}

module app 'app.bicep' = {
  name: '${appName}-app'
  params: {
    tags: tags
    location: location
    appName: appName
    storageAccountName: storageAccountName
    appInsightsName: monitoring.outputs.appInsightsName
    logAnalyticsWsName: monitoring.outputs.logAnalyticsWsName
    monitoringResourceGroup: monitoring.outputs.ResourceGroupName
  }
}

Note that the app module relies on the outputs of the monitoring module -- this helps Bicep figure out the dependencies so that you don't need to define dependsOn any more.

Then in monitoring.bicep I can reference fine-grained reusable modules that I'd like to share with other projects in much the same way

module log '../../shared-infra/modules/logAnalytics.bicep' = {
  name: '${appName}-log'
  params: {
    tags: tags
    location: location
    logAnalyticsWsName: logAnalyticsWsName
  }
}

module appi '../../shared-infra/modules/appInsights.bicep' = {
  name: '${appName}-appi'
  params: {
    tags: tags
    location: location
    appInsightsName: appInsightsName
    logAnalyticsWsName: log.outputs.logAnalyticsWsName
  }
}

Another thing to note is the name of your module is what is shown in the Deployments tab of the Azure Portal, so make these make sense to you for easy debugging if deployments are breaking.

Sharing modules

At this stage I've got different two styles of modules -- app specific modules breaking up my main.bicep, making it easier to read and maintain and fine-grained templates in a modules folder that I can use across all the apps in my infra folder. Readable -- check! Composable -- check! Reusable -- only inside this repo, so half a check!

The current version of Bicep (v0.4.63), does not have a native mechanism to externally share modules across projects. The good news is that this is being looked at and will hopefully be in the v0.5 release. The issues to watch are:

In the interim, I am using Git submodules to solve this, although this will not work with all CI/CD tooling when using private repos. To enable this, I moved the modules in shared-infra into its own repo and added it back to my project.

git submodule add <path-to-repo> shared-infra/
git submodule update --init

Testing locally

To quickly validate the individual modules and main templates on my local machine, I wrote a simple bash script to either

build the Bicep file, outputting to the terminal rather than writing to file or
validate the Bicep template against a resource group

#!/bin/bash
RG=replace-with-your-rg
SUB=replace-with-your-sub

op=$1 # lint, validate, create
main_shared=$2
if [ "$main_shared" == 'main' ]; then path='./infra' && filename='main.bicep'; fi
if [ "$main_shared" == 'shared' ]; then path='./shared-infra' && filename='*.bicep'; fi

lint () {
    az bicep build --file "$f" --stdout
}

validate () {
    az deployment group validate --resource-group "$RG" --subscription "$SUB" -f "$1" -p env=dev
}

for f in $(find "$path" -name "$filename") ; do
    echo "$f"
    $op "$f"
done

The script allows one to test either main or shared templates, linting them or validating them:

./infra.sh lint main
./infra.sh lint shared
./infra.sh validate main

Finishing up

After the refactor, my project is in a far cleaner state and the infrastructure is much easier to follow. A second plus is that we now have a separate repo of our shared modules that can become a shared asset across our various teams (using Git submodules for now and the Bicep registry in the future).

Featured image background by Michael Dziedzic on Unsplash

Migrating Azure ARM templates to Bicep

Richard Roché — Fri, 18 Jun 2021 07:00:00 +0000

You may have heard of Bicep, and you may be wondering how much effort it is going to take to move all your ARM templates to this new way of deploying Azure resources.

I gave migrating from ARM to Bicep a go. This post will cover going from JSON ARM templates to shiny new Bicep templates that have no errors and don't contain any warnings or linting issues!

Why should you migrate?

If you have ever deployed infra to Azure you have most likely used ARM templates before. They do the job, the docs aren't bad and there are built in tasks for ADO which make deploying them super straightforward. However, working with JSON is always going to be verbose, ARM templates have a lot of boilerplate required, making reusable templates is clunky and deployments can get tricky.

Aiming to address these (and other) issues, Azure is working on a project called Bicep. From the projects overview:

Bicep is a domain-specific language (DSL) that uses declarative syntax to deploy Azure resources. It provides concise syntax, reliable type safety, and support for code reuse. We believe Bicep offers the best authoring experience for your Azure infrastructure as code solutions.

Essentially it is a DSL built on-top of ARM that makes the development of templates simpler and promotes reuse through modules. Judging by the release rate on their GitHub the team is working hard to make it awesome very quickly and according to them, as of v0.3, Bicep is now supported by Microsoft Support Plans and Bicep has 100% parity with what can be accomplished with ARM Templates. So Bicep is definitely prod ready (at time of writing v0.4.63 was the latest release)!

Getting started

Ensure you have the latest version of Bicep installed (excellent installation guide here). I went with the az cli install option and all my examples will be using that variant. I also use Visual Studio Code and installed the Bicep VS Code Extension for enhanced editing.

Decompiling ARM to Bicep

First things first, it is possible to decompile an ARM template into Bicep by running the below command (all our ARM templates are in separate folders with an azuredeploy.json file for the template)

az bicep decompile --file azuredeploy.json

This creates a azuredeploy.bicep file in the same directory. Depending on your template, you will most likely get a stream of yellow warnings in the console, or potentially some red errors: don't panic! Even a single error will result in all the console output being red (at least on macOS) and at the very least you will get this warning message:

WARNING: Decompilation is a best-effort process, as there is no guaranteed mapping from ARM JSON to Bicep.
You may need to fix warnings and errors in the generated bicep file(s), or decompilation may fail entirely if an accurate conversion is not possible.
If you would like to report any issues or inaccurate conversions, please see https://github.com/Azure/bicep/issues.

It is important to note that the migration from ARM to Bicep will highlight issues in your ARM templates. ARM was more forgiving in certain aspects, after the migration, your templates will be in a better state.

Errors come in the form of Error BCPXXX: Description and the descriptions generally let you get to the root of the problem quickly. E.g. Error BCP037: The property "location" is not allowed on objects of type "Microsoft.EventHub/namespaces/eventhubs". Permissible properties include "dependsOn".

If the decompilation gives you any errors, my preferred approach is to fix the ARM template and then run the decompilation again until you get a "clean" decompilation (warnings are fine, just ensure you decompile with no errors).

Warnings come in two flavours, ones that have a code (Warning BCPXXX) and ones that don't have a code but have a link at the end. As of v0.4 there is a linter which helps you get your templates into great shape -- these are the warnings with the links at the end. Both kinds of warnings are generally quick to fix and are sensible updates to start getting the benefits from Bicep.

Common errors and how to fix them

User defined functions are not supported (BCP007, BCP057)

If you have created user defined functions in ARM, these will not be decompiled (follow the issue) and you will get two cryptic errors

Error BCP007: This declaration type is not recognized. Specify a parameter, variable, resource, or output declaration.

Error BCP057: The name "functionName" does not exist in the current context.

To fix, move the function logic into your template (this may require duplication, but can be neatened up in the Bicep template later)

Nested dependsOn (BCP034)

Sometimes the use of nested dependsOn properties in your ARM templates gets weird with Bicep (dependsOn can often be removed entirely in Bicep). The ARM version would have validated and deployed perfectly fine however you will get the following error when decompiling to Bicep.

Error BCP034: The enclosing array expected an item of type "module[] | (resource | module) | resource[]", but the provided item was of type "string".

Fortunately Bicep handles dependencies in a much smarter way than ARM meaning you can safely remove your nested dependencies and run again.

For Bicep, you can set an explicit dependency but this approach isn't recommended. Instead, rely on implicit dependencies. An implicit dependency is created when one resource declaration references the identifier of another resource.

To fix, delete your nested dependencies and validate (consider removing your dependsOn references entirely)

Strict schema validation (BCP037, BCP073)

Bicep validates the schema of each resource much more diligently than ARM. As a result you will probably find a bunch of places where you have added properties like location or tags to a resource that doesn't support them and ARM didn't mind this. These will be expressed as Error BCP037.

E.g. I had location on Microsoft.EventHub/namespaces/eventhubs

Error BCP037: The property "location" is not allowed on objects of type "Microsoft.EventHub/namespaces/eventhubs". Permissible properties include "dependsOn".

The one error I did run into was for Azure Logic Apps where the schema for Microsoft.Logic/workflows is missing the identity property needed for using Managed Identity with your Logic App:

Error BCP037: The property "identity" is not allowed on objects of type "Microsoft.Logic/workflows". Permissible properties include "dependsOn".

Another common error I encountered was having read-only parameters in my templates (these tend to come along if you have exported templates from the Azure Portal). E.g.

Error BCP073: The property "kind" is read-only. Expressions cannot be assigned to read-only properties.

To fix both types of errors, remove the properties that the error messages highlight and run the decompilation again.

Reserved words as parameters (BCP079)

In ARM templates you can have the same name for a parameter, variable, function etc as these are all addressed directly using parameters('name') or variables('name') etc. In Bicep the syntax is simplified and as such you will get errors if you have used a reserved word as a parameter. We had a couple of templates taking in a parameter called description resulting in a cryptic error message:

Error BCP079: This expression is referencing its own declaration, which is not allowed.

To fix, rename the parameter in your ARM template, update its usages and run the decompilation again.

Common warnings and how to fix them

Hopefully by now you are out of the error zone, for warnings you can now start editing the Bicep file itself. The Bicep VS Code Extension gives you great intellisense and highlights issues in the Bicep file.

To test an update on your Bicep file, run

az bicep build --file azuredeploy.bicep

Schema warnings, enums and types (BCP035, BCP036, BCP037, BCP073, BCP081, BCP174)

Data types in Bicep are stricter than in ARM. If you have used True or False for booleans these need to be updated to be true and false. When using enums, they need to match the enums in the schema exactly (case-sensitive). E.g. if you used ascending and the template schema defines Ascending this adjustment needs to be made. Examples of these warnings are shown below.

Warning BCP036: The property "kafkaEnabled" expected a value of type "bool | null" but the provided value is of type "'False' | 'True'".

Warning BCP036: The property "order" expected a value of type "'Ascending' | 'Descending' | null" but the provided value is of type "'ascending'".

Similar to the errors thrown by BCP037 and BCP073 you will get warnings on schema mismatches using codes BCP035, BCP037 and BCP073. These highlight schema mismatches, missing properties and read-only properties used. Examples below.

Warning BCP035: The specified "object" declaration is missing the following required properties: "options".

Warning BCP037: The property "apiVersion" is not allowed on objects of type "Output". No other properties are allowed.

Warning BCP073: The property "status" is read-only. Expressions cannot be assigned to read-only properties.

I did find an instance where the Bicep template matches the schema perfectly however warnings are still thrown. I think this is due to this issue and fortunately doesn't break anything.

To fix, update the offending property to match the schema or data type

String interpolation

concat gets to disappear in Bicep thanks to the lovely string interpolation option. I experienced two variants:

Warning prefer-interpolation: Use string interpolation instead of the concat function. [https://aka.ms/bicep/linter/prefer-interpolation]

To fix, search for all usages of concat and replace with the new syntax: 'string-${var}'.

Warning simplify-interpolation: Remove unnecessary string interpolation. [https://aka.ms/bicep/linter/simplify-interpolation]

To fix, remove the ${} and reference the variable directly. E.g. '${variable}' becomes variable.

Environment URLs

We had a lot of hardcoded URL's in our templates for storage suffixes, front door etc. There is a much better way to do this by using the environment() function. Caveat here, if you had environment as a parameter to your ARM template, update this to be something else like env for example otherwise you won't be able to use the function.

Warning no-hardcoded-env-urls: Environment URLs should not be hardcoded. Use the environment() function to ensure compatibility across clouds. Found this disallowed host: "core.windows.net" [https://aka.ms/bicep/linter/no-hardcoded-env-urls]

*To fix, have a look at all the URLs that the environment function provides and replace your hard-coded version with environment().<property>. *

E.g. for Azure Storage where core.windows.net had been hard coded, replacing with environment().suffixes.storage gives the desired result.

Scopes (BCP174)

Bicep introduces the concept of target scopes which dictates the scope that resources within that deployment are created in. This gives you a new way to define resources where you previously would have used the /providers/ syntax (role assignments, diagnostic settings etc). The warning you get looks as follows.

Warning BCP174: Type validation is not available for resource types declared containing a "/providers/" segment. Please instead use the "scope" property. [https://aka.ms/BicepScopes]

These are the most time-consuming to fix, essentially you need to

use the schema reference of the actual resource (so for diagnostic settings us microsoft.insights/diagnosticSettings@2017-05-01-preview) instead of the parent resource /providers/
add a scope referencing the parent resource
rename so as not to reference the parent resource
remove unnecessary properties and dependsOn

Before

resource functionAppLogAnalytics 'Microsoft.Web/sites/providers/diagnosticSettings@2017-05-01-preview' = {
  name: '${functionAppName}/Microsoft.Insights/LogAnalytics'
  tags: tagsVar
  properties: {
    name: 'LogAnalytics'
    workspaceId: resourceId(logAnalyticsResourceGroup, 'Microsoft.OperationalInsights/workspaces', logAnalyticsWsName)
    logs: [
      {
        category: 'FunctionAppLogs'
        enabled: true
      }
    ]
  }
  dependsOn: [
    functionApp
  ]
}

After

resource functionAppLogAnalytics 'microsoft.insights/diagnosticSettings@2017-05-01-preview' = {
  name: LogAnalytics
  scope: functionApp
  properties: {
    workspaceId: logAnalyticsResourceId
    logs: [
      {
        category: 'FunctionAppLogs'
        enabled: true
      }
    ]
  }
}

Should you migrate?

A resounding yes from me! The process above goes quite quickly, and I was on shiny new Bicep templates in less than an hour. Bicep is a much friendlier syntax to work with and the IDE support is great. What I also enjoyed is cleaning up all the errors that were present in my ARM templates that would have remained if not for the migration.

There is a neat comparison of ARM syntax vs Bicep syntax here which highlights a lot of the constructs that become simpler as well: https://docs.microsoft.com/en-us/azure/azure-resource-manager/bicep/compare-template-syntax

Enjoy the migration! I will be playing around with modules and reuse next and will share what I find.

Featured image background by Nick Fewings on Unsplash.

Azure Pipelines and Dependabot

Richard Roché — Mon, 31 May 2021 09:21:36 +0000

Keeping your dependencies up to date in a project is a really easy way to try and keep the software secure. New releases of a dependency often include

Patches for security vulnerabilities!
Performance improvements!
Awesome new features!
Bug fixes!

It can also be quite a boring activity and can be time-consuming for a team maintaining the project to run updates regularly into production. Fortunately tools like Dependabot exist!

Dependabot?

Dependabot creates pull requests on your repos with the dependencies you should update. You can read about how it works; but in a nutshell

it checks for updates of your dependencies
it opens up a pull request on your repo
you review the PR and merge

This is an awesome tool to save you time and I find it makes keeping your dependencies up to date really easy.

Integration with Azure Pipelines

Dependabot is baked into the GitHub ecosystem and really easy to use there. Recently I needed to solve this problem on a project in Azure DevOps using Azure Pipelines and thought I would share my solution.

If you search the Azure DevOps Extension Marketplace for Dependabot you will find this extension made by Tingle Software. It is always great to find extensions to speed up integration and I gave this one a whirl.

The extension is feature rich and works really well with little configuration required. Simply add the below to an Azure Pipeline and run it, and you'll end up with a host of PR's!

stages:
  - stage: CheckDependencies
    displayName: 'Check Dependencies'
    jobs:
      - job: Dependabot
        displayName: 'Run Dependabot'
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - task: dependabot@1
            displayName: 'Run Dependabot'
            inputs:
              packageManager: 'nuget' # Examples: nuget, maven, gradle, npm, etc. Add multiple tasks if multiple package managers are used in your solution
              targetBranch: 'main'
              openPullRequestsLimit: 10 # Limits the number of PR's you get
              setAutoComplete: true # Saves us one click, once our PR policies pass, the update will merge

Have a look at all the Task Parameters that the extension supports to fine tune your implementation. The above checks for nuget dependencies on the main branch, limits the number of PR's raise to be 10 and sets the PR to autocomplete (letting our branch policies and PR validation pipelines perform all their checks).

Work Item Linking

In the project I'm working in we have a policy set on all PR's to ensure they are linked to a work item. When I ran the above pipeline, I ended up with 10 PR's, none of which were linked to a work item so I couldn't quickly review and merge each one. The extension handles this by allowing you to pass in a workItemId parameter.

Changes in release 0.5
In the upcoming 0.5 release of the extension, the workItemId parameter has been renamed to milestone so please be on the look out for this change!

The Microsoft team have an extension for creating work items which is really configurable. For my teams workflow, we wanted to have a User Story on our board with all the PR's linked to it. We also didn't want the pipeline creating duplicate User Stories every time it runs if we already had one open that we are working on. After a bit of trial and error, adding the below to the pipeline gave us the desired result.

- task: CreateWorkItem@1
  displayName: 'Create User Story for Dependabot'
  inputs:
    workItemType: 'User Story' # We wanted a User Story created, but all work item types are available
    title: 'Update Dependencies' # The title of the work item
    # Adding some tags to the work item
    fieldMappings: |
      Tags=dependabot; dependencies 
    areaPath: 'your\area'
    iterationPath: 'your\iteration'
    # Adding duplicate detection, using the area path, iteration path and title to match
    preventDuplicates: true
    keyFields: |
      System.AreaPath
      System.IterationPath
      System.Title
    # Create output variables for the next task to be able use the work item ID
    createOutputs: true
    outputVariables: |
      workItemId=ID

- task: dependabot@1
  displayName: 'Run Dependabot'
  inputs:
    packageManager: 'nuget'
    targetBranch: 'main'
    openPullRequestsLimit: 10
    workItemId: $(workItemId) # This uses the output from the above as the work item to link the PR's to
    setAutoComplete: true

Docker Caching

The Dependabot extension depends on a Docker image hosted on Docker Hub. The image is around 4.4GB in size and can take some time to download in the pipeline.

In the docs it mentions

Since this task makes use of a docker image, it may take time to install the docker image. The user can choose to speed this up by using Caching for Docker in Azure Pipelines.

The caching tasks can look a bit confusing, breaking it down you need 3 parts:

- task: Cache@2
  inputs:
    key: docker | "${{ variables.imageToCache }}" # image to look for in the cache, e.g. tingle/dependabot-azure-devops:0.4
    path: $(Pipeline.Workspace)/docker # path of the cache
    cacheHitVar: DOCKER_CACHE_HIT # variable to set to true if a cache hit is found
  displayName: Cache Docker images

This checks if there is a cached version of the image to be used. If yes, DOCKER_CACHE_HIT is set to true, otherwise false.

- script: |
    docker load -i $(Pipeline.Workspace)/docker/cache.tar
  displayName: Restore Docker image
  condition: and(not(canceled()), eq(variables.DOCKER_CACHE_HIT, 'true'))

If we have a cached version of the image, we need to pipeline to download it. This step only runs if DOCKER_CACHE_HIT is set to true and will download the cached archive and load it into the docker context.

- script: |
    mkdir -p $(Pipeline.Workspace)/docker
    docker pull -q ${{ parameters.imageToCache }}
    docker save -o $(Pipeline.Workspace)/docker/cache.tar ${{ parameters.imageToCache }}
  displayName: Save Docker image
  condition: and(not(canceled()), or(failed(), ne(variables.DOCKER_CACHE_HIT, 'true')))

If we do not have a cached version of the image we need to pull it down from Docker Hub and save it as an archive to be cached.

Finishing up

We have a pipeline that runs on a schedule, running all the above steps together. A complete example can be found on GitHub here.

schedules:
  - cron: "0 4 * * 1"
    displayName: 'Weekly Run'
    always: true
    branches:
      include:
        - main

trigger: none

variables:
  - name: imageToCache
    value: tingle/dependabot-azure-devops:0.4

stages:
  - stage: CheckDependencies
    displayName: 'Check Dependencies'
    jobs:
      - job: Dependabot
        displayName: 'Run Dependabot'
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - task: CreateWorkItem@1
            displayName: 'Create User Story for Dependabot'
            inputs:
              workItemType: 'User Story'
              title: 'Update Dependencies'
              fieldMappings: |
                Tags=dependabot; dependencies
              areaPath: 'your\area'
              iterationPath: 'your\iteration'
              preventDuplicates: true
              keyFields: |
                System.AreaPath
                System.IterationPath
                System.Title
              createOutputs: true
              outputVariables: |
                workItemId=ID

          - task: Cache@2
            inputs:
              key: docker | "${{ variables.imageToCache }}"
              path: $(Pipeline.Workspace)/docker
              cacheHitVar: DOCKER_CACHE_HIT
            displayName: Cache Docker images
          - script: |
              docker load -i $(Pipeline.Workspace)/docker/cache.tar
            displayName: Restore Docker image
            condition: and(not(canceled()), eq(variables.DOCKER_CACHE_HIT, 'true'))
          - script: |
              mkdir -p $(Pipeline.Workspace)/docker
              docker pull -q ${{ variables.imageToCache }}
              docker save -o $(Pipeline.Workspace)/docker/cache.tar ${{ variables.imageToCache }}
            displayName: Save Docker image
            condition: and(not(canceled()), or(failed(), ne(variables.DOCKER_CACHE_HIT, 'true')))

          - task: dependabot@1
            displayName: 'Run Dependabot'
            inputs:
              packageManager: 'nuget'
              targetBranch: 'main'
              openPullRequestsLimit: 10
              workItemId: $(workItemId)
              setAutoComplete: true

In summary the pipeline does the following for you

Runs on a weekly schedule (update the cron to suite your needs)
Creates a new work item, with the tags we would like (avoiding duplicates)
Tries to use a cached version of the image for faster builds (creating a cached version if not found)
Run Dependabot, limiting the number of open PR's to 10, linking the PR's to the created work item and completing the PR once all the policies pass

Hopefully this will help you to get your projects running in Azure DevOps nice and up to date!