Automate Azure templateSpec deployment and update

Using Infrastructure as Code in a small team is an easy to moderate task, but things can be more complex when the enterprise scale and multiple teams need to work on the same recommendations and deploy the same resources in Azure. More, teams may not have the same maturity and may not advance at the same pace.
At the same time, you ask your teams to standardize their deployments and use shared configurations for security, network, or monitoring.
How can you share these components across teams knowing that you may have to deal with different versions? How to make sure everyone gains access to shared configuration?
A solution can be to use a special repository where teams can select what they need and integrate it into their solutions. The drawback is the manual work needed for each team. They have to clone the repository, copy files needed, add them into their ARM deployment scripts and then deploy the solution in Azure.
A shared resource should be a commodity, something you can easily integrate with a deployment just like you pick up a soda in a grocery store.
ARM template offers you the possibility to integrate code portions hosted outside your main template. The “Microsoft.Resources/deployments” let you integrate remote code. You can use a public repository like GitHub or a storage account. But there is some limitation, how to manage template versioning and life cycle and how to manage access control.
Azure Template Spec is the solution the achieve this goal in Azure.

Imagine the situation where your teams have full autonomy to deploy their solutions in their resource groups/subscriptions. But you want them to have standardized solutions for the network (spoke VNET for example), monitoring, and backup. Each solution has several versions to cover enterprise needs.

For network a template with a simple hub-spoke configuration and another with a nat gateway and several links.
For monitoring, a Log analytic workspace and another version with Azure Monitor for VM
For the backup solution with a configuration for dev or production workload.
In this hypothetical example, each configuration, network, backup, and monitoring have 2 different versions.

First, we need to find a name and a description for each templateSpec and their versions.

Network template spec, az-connection with two versions

• az-nethubspoke-simple01, simple connection to on-premises network connections
• az-nethubspoke-advance01, a connection to the on-premises network and several other network services.

Monitoring template spec, az-monitor with two version

• az-simple-monitoring, simple Log analytic workspace with a default configuration
• az-vm-monitoring, log analytic workspace with Azure Monitor for VMs solution

Backup template spec, az-backup with two version

• az-backup-dev, an Azure Backup vault with a simple policy
• az-backup-prod, an Azure Backup vault with a default policy aligned with the company backup policy.

A TemplateSpec object is like other resources in Azure, it needs a resource group. This resource group can be in any subscription as long this subscription is under the same tenant or users and service who will access these templateSpecs can log in the tenant. Template Spec uses Azure Active Directory and Azure RBAC to manage user access.

You may need an Azure Active Directory group to manage template spec access. Users, managed identity, and service principal will be added to this group. The reader role will be given to all template specs objects.

Now we need to find a solution to create templates files and implements versioning. We need 2 versions for each solution and possibly more later, so we need to enable the possibility to add more when needed. We also want to automate the process so each new version is automatically ready for users.
Each version should have its name and its description and each template spec object should have its name, description, and display name.
Each deployment process should redeploy only new and modified versions as well as update template data in the version files.
There is tow challenge here, how to get names and descriptions and discover new version and how to only update modified version and add new version wile living unmodified version untouched.
For the first one, we can create a folder for every templateSpec object to store each version JSON file. In each folder, a JSON named version.json containing the definition of the TemplateSpec, its name, description, and display name and an array for each version with the version name, description, and filename.

For example, the az-monitor version.json look like

{
    "templateSpecName": "az-monitor",
    "templateSpecDisplayName": "Template Spec for Azure monitoring",
    "templateSpecDescription": "This template spec contain several options to enable Azure monitoring",
    "versions": [
      {
        "versionName": "v-nethubspokesimple01",
        "versionDscription": "simple Log analylicit work space with default configuration",
        "versionFileName": "az-monitor-simple01.json"
      },
      {
        "versionName": "az-vm-monitoring",
        "versionDscription": "log analytic workspace with Azure Monitor for VMs solution",
        "versionFileName": "az-vm-monitoring01.json"
      }
    ]
  }

For the second challenge, we need to be sure to only update the existing version when there is a change and add a new one. We can use a PowerShell script but it will be complex to manage templateSpecs updates and new version creation. We need a solution that only changes templateSpec versions only when needed. And in Azure, a solution exists and this solution is an ARM Template (or Bicep) file.

We need to build a generic Azure ARM template to deploy the Azure templateSpec and the version from the version.json in each folder. We have the templateSpec name and its description and display name. For each templateSpec we also have version information, name description and the template JSON file to use for each version.
But how to integrate the version template file in an ARM template programmatically. This template is not mean to be deployed but to be stored. To upload a template for a templateSpec version two options are using the path or adding the version template data inside the deployment template.
The last option is easier when you think about the way to pass version template data as an ARM template parameter. To use JSON data as a parameter in an ARM template, you will not do need to use a string with the whole JSON file content, instead, you will need to use a PowerShell hashtable.

How to convert JSON data to a hashtable?

$versionTemplateData = Get-Content -Path pathToJsonFile | ConvertFrom-Json -AsHashtable

Now let’s try to build a tool to add and update templateSpec and their version in case of adding a new version, a new template, or an update in the version template file. We have a directory to host every templateSpec we need to deploy, each templateSpec is represented by a folder with a version file describing the templateSpec and the path to version files.

We need to list all these version files

$versionFilesList = Get-ChildItem -Path $templatesSpecFolder  -Recurse -Include version.json

Now for each version file, we need to extract, the current directory of the file, the content of the file converted from JSON. From this point, we can extract the templateSpec name, displayname, and description.
For the version, we need to parse the JSON array to collect the name, the description and the content of the ARM template converted from JSON to a hashable and put all this data an hashtable so we can call the new-azresourcegroupdeployment cmdlet for deploying each templateSpec by using inline parameters.

foreach ( $versionFile in $versionFilesList ) {

    $templateSpecDir = join-path -Path $versionFile.directoryName -ChildPath "version.json"


    $versionData = Get-Content $templateSpecDir  | ConvertFrom-Json -AsHashtable

    $templateSpecName = $versionData.templateSpecName

    $templateSpecDisplayName = $versionData.templateSpecDisplayName

    $templateSpecDescription = $versionData.templateSpecDescription

    $versionDataHashTable = $versionData.versions

    $versionArrayParam = @()

    foreach ($version in $versionDataHashTable) {

        $versionTemplateFile = Join-Path -Path $versionFile.directoryName -ChildPath $version.versionFileName

        if (Test-Path -Path $versionTemplateFile) {
            $versionTemplateData = Get-Content -Path $versionTemplateFile | ConvertFrom-Json -AsHashtable

            $versionHastable = @{
                "name"          = $version.versionName
                "description"   = $version.versionDscription
                "template"      = $versionTemplateData
            }
            $versionArrayParam += $versionHastable
        }

    }

    new-AzResourceGroupDeployment  -ResourceGroupName 01-templatespec-demo -TemplateFile $azDeployLocation -templateSpecName $templateSpecName -templateSpecDisplayName $templateSpecDisplayName -templateSpecDescription $templateSpecDescription -templateSpecVersionArray $versionArrayParam 
}

The ARM JSON deployment file look

    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "templateSpecName": {
            "type": "string", 
            "metadata": { 
                "description": "Name of the TemplateSpec object" 
              } 
        },
        "templateSpecDisplayName": {
            "type": "string"
        },
        "templateSpecDescription": {
            "type": "string", 
            "metadata": { 
                "description": "Description of the template Spec object" 
              } 
        },
        "templateSpecVersionArray": {
            "type": "array",
            "metadata": {
                "description": "Array of template spec version objects"
            }
        }
    },
    "functions": [],
    "variables": {
        "defaultLocation": "westeurope"
    },
    "resources": [
        {
            "type": "Microsoft.Resources/templateSpecs",
            "apiVersion": "2021-05-01",
            "name": "[parameters('templateSpecName')]",
            "location": "[variables('defaultLocation')]",
            "properties": {
                "description": "[parameters('templateSpecDescription')]",
                "displayName": "[parameters('templateSpecDisplayName')]",
                "metadata": {

                }
            }
        },
         {
            "name": "[concat(parameters('templateSpecName'),'/',parameters('templateSpecVersionArray')[copyIndex()].name )]", 
            "dependsOn": [
                "[resourceId('Microsoft.Resources/templateSpecs', parameters('templateSpecName'))]"
            ],
            "type": "Microsoft.Resources/templateSpecs/versions",
            "apiVersion": "2021-05-01",
            "location": "[variables('defaultLocation')]",
            "copy": {
                "name": "versionIteration ",
                "count": "[length(parameters('templateSpecVersionArray'))]"
            },
            "properties": {
                    "description": "[parameters('templateSpecVersionArray')[copyIndex()].description ]",
                    "mainTemplate": "[parameters('templateSpecVersionArray')[copyIndex()].template ]"
                    }
        }
    ],
    "outputs": {}
}

This is an example on you can dynamically manage templateSpec object and its version. You can start to create a pipeline to deploy them in your subscriptions. You can also extend this example by adding a testing task.