DEV Community: David Pazdera

vNext Management and Automation of cloud and hybrid workloads

David Pazdera — Mon, 13 Mar 2023 17:26:30 +0000

This post was created for the Azure Spring Clean 2023 event.

Introduction

If you happen to be responsible for running workloads on Azure in production, I bet you are familiar with Microsoft Monitoring Agent, Log Analytics, and Azure Automation services. They have been helping Azure customers with key management disciplines like monitoring, patching, inventory management, and process automation. We simply need to ensure our workloads are "well managed", protected, and watched for any 'abnormal' events that could impact the way our precious applications are available to their end-users.

With current trends like hybrid-cloud and multi-cloud and ever-changing business needs, our IT landscape is evolving with it, and what worked well and fit our needs a year ago might not be the best fit anymore.

Don't get me wrong, Azure Automation with its features is still a very robust and battle-tested service, but some specific design and implementation decisions make it difficult to stay relevant in the context of trends I mentioned.

Examples of such limitations are:

dependency on Microsoft Monitoring Agent (MMA) and its centralized workspace-level configuration,
Workspace mapping between Azure Automation and Log Analytics with specific regional pairs,
support for non-Azure servers (on-premises and Third-Party cloud virtual machines),
consistent onboarding experience.

One agent to rule them all

There were several major changes in Azure Management and Monitoring space, but I want to highlight two that are relevant to the topic.

The first one is Azure Monitor Agent (AMA) that aims (among other things) to consolidate the number of different agents that are available on Azure to monitor IaaS-based workloads and to decentralize its configuration by switching from workspace-level configuration to more distributed and granular Data Collection Rules (DCRs).

AMA collects monitoring data from the guest operating system of Azure and hybrid virtual machines and delivers it to Azure Monitor for use by features, insights, and other services, such as Microsoft Sentinel and Microsoft Defender for Cloud.
Azure Monitor Agent replaces all of Azure Monitor's legacy monitoring agents. There is a migration guide available for moving away from this 'legacy' agent.
With DCRs you can completely decouple what is being collected (what logs, metrics, and distributed traces), where is it being stored (or streamed out of Azure), and from what sources (VMs).

Tip: Use the DCR generator to generate DCR resources (rules) from MMA workspace configurations.

The second change is the VM extension model that is now common for Azure VMs (with Azure VM Agent) and Arch-connected machines (with Connected Machine Agent). VM extensions are small applications that provide post-deployment configuration and automation tasks.

Disclaimer: Several capabilities I will be describing are still in preview, so make sure you understand Azure terms and conditions before you decide to use any of these in your production environments.

How to keep your VMs patched

Current

Azure Automation Update Management solution has been around for a long time and it allows for both assessing and deploying missing patches and other types of updates across your Azure and non-Azure server fleet. Onboarding to this service was based on MMA agent, its configuration, and target Log Analytics workspace that was linked to Azure Automation account that had the Update Management solution enabled.

vNext

Say hello to 'Azure Update Management Center' (UMC). This new service - currently in Preview - has several key features:

it is a platform native functionality for Azure Compute and Azure Arc for Servers, globally available in all Azure Compute and Azure Arc regions
there is no dependency on Log Analytics and Azure Automation services
granular access control at resource level instead of access control at Automation account and Log Analytics workspace level
improved onboarding experience at scale via Azure Portal and Azure Policy
ability to enable on-demand and periodic assessments
configuration of scheduled maintenance (update deployments)

Azure UMC relies on new VM Extensions (LinuxPatchExtension, WindowsPatchExtension) that works the same way on Azure VMs and Azure Arc-enabled servers:

The extension is installed by either Azure VM agent or Connected Machine agent. No manual intervention required.
This extension is automatically installed when you initiate any UMC operations such as check for updates, install one time update, periodic assessment on your machine.

Tip: For unattended onboarding of Azure VMs, it is required to set the following VM properties:
patchSettings.patchMode: AutomaticByPlatform
patchSettings.assessmentMode: AutomaticByPlatform
enableAutomaticUpdates: false

When the management of a VM is initiated, UMC pushes an extension to the agent. The extensions run locally and interact with the OS. They retrieve assessment information about the status of system updates and initiate download & installation of approved updates.

Machines assigned to UMC report how up to date they're based on what source they're configured to synchronize with, e.g., WSUS, Microsoft Update, a local or public YUM or APT package repository.

All assessment information and update installation results are reported to UMC from the extension and are available for analysis with Azure Resource Graph. You can view up to the last seven days of assessment data, and up to the last 30 days of update installation results.

The following diagram shows all key components and their properties:

Important: Check what OS versions and distros are supported during the preview in the support matrix.

The easiest path for enabling automated onboarding is by leveraging three Azure Policies:

Configure periodic check for missing system updates on Azure VMs: with Modify effect, it updates patchSettings.assessmentMode property for the deployed VM to be AutomaticByPlatform.
Schedule recurring updates using UMC: this DINE policy updates pathcSettings.patchMode property of the deployed VM to be AutomaticByPlatform and it deploys a 'Configuration Assignment' resource that links the VM with an existing 'Maintenance Configuration' object.
Machines should be configured to periodically check for missing system updates: this policy checks VM's patchSettings.assessmentMode property if it has AutomaticByPlatform value. It could either Audit or Deny the request. You can pick the effect you prefer upon assignment.

A word of caution: UMC can manage machines currently managed by Azure Automation UM feature without causing any disruptions in your update management process. Microsoft does not recommend customers to migrate from Update Management until UMC goes GA!

Track your software inventory and changes

Current

When you onboard your VMs to Azure Automation, you are also getting Change Tracking and Inventory features as a bonus, you just need to ensure you enabled them on your Automation account.

There are several items being tracked, like Windows software, Linux software (packages), Windows and Linux files, Windows registry keys, Windows services, and Linux daemons.

While there are still some limitations and the supported OS matrix is derived from Log Analytics agent, it is a robust service that can help you with e.g., Software Asset Management and configuration drift detection.

vNext

I guess it won't surprise you that there is a preview available for Change Tracking and Inventory using AMA agent. After all, we want to get rid of the MMA agent and its dependencies.

The main benefits are derived from the use of AMA agent: multi-homing support, granular management of 'data sources' using DCRs, and compatibility with the Change tracking (CT) extension deployed through the Azure Policy on the client's virtual machine. Once you switch to AMA, the CT extension pushes the software, files, and registry to AMA.

During the preview, there is an extensive list of limitations you need to take into account.

If you plan to onboard several VMs at once, consider using built-in policy initiatives for VMs, VM Scale Sets, or Arc-enabled servers. Simply go to the Azure Portal, open Azure Policy page, and search for initiative definitions that begin with [Preview]: Enable ChangeTracking and Inventory for string.

Let's have a closer look at the initiative for Azure VMs and figure out, what individual policies do:

creates a User-Assigned Managed Identity and enables it for the VM (eventually you can bring your own / existing UAMI).
deploys AMA extension for the VM (if it isn't deployed already)
installs the ChangeTracking Extension (ChangeTracking-Windows or ChangeTracking-Linux) to enable File Integrity Monitoring (FIM) in Microsoft Defender.
deploys 'DCR Association' to link the VM to specified DCR to enable ChangeTracking and Inventory

The missing piece is the deployment of a ChangeTracking-specific DCR. You can find the deployment template here.

Automate your IT processes

Unlike some of the previous areas, this one provides two mature generally available solutions you could choose from:

Automation runbooks
Azure Function Apps with PowerShell runtime

A couple of years ago, I had a session at Nordic Infrastructure Conference called 'Mortal Compat - Azure Automation vs. Functions', where I compared both services on several levels. I published the slides from this talk at Speaker Deck. Some info is outdated but surprisingly a lot of key facts are still valid.

Mortal Compat - Azure Automation vs. Functions - Speaker Deck

In this combat, a new version of Azure Functions supporting PowerShell will challenge Azure Automation, a seasoned and widely adopted service for cloud …

speakerdeck.com

I was a huge fan of runbooks and several improvements (like Managed Identity and PowerShell 7 support) rolled out in recent months, but Azure Functions allow you to apply event-driven approach and "plug in" additional components like Logic Apps and Event Grid. For me, this was a game-changer and I switched from runbooks to functions for most scenarios.

Modern in-guest configuration management

Current

The last piece in the puzzle is about being able to configure virtual machines and servers from "the inside" by declaring their desired state.

Azure Automation has a feature called Automation State Configuration. It has been around for a long time, and it allows you to write, manage, and compile PowerShell DSC (Desired State Configuration) configurations.

You could either enforce the configuration you want or use it as a report-only tool. And it supports non-Azure VMs as well (running on-premises or in "other clouds").

vNext

As you might have guessed, there is a newer and better version of DSC (generally) available, managed by a feature of Azure Policy named guest configuration. The guest configuration service combines features of DSC Extension and Azure Automation State Configuration. Guest configuration also includes hybrid machine support through Arc-enabled servers.

But wait, there has been a change in the name of this new service. Azure Policy Guest Configuration is now called Azure Automanage Machine Configuration. You didn't see that coming, did you? :)

Describing in detail how it works would require a separate blog post. You can find more info in the documentation.

My own conclusion

All those new management capabilities are in different stages of 'production readiness'; some are in preview, some GA, some available in limited regions or for limited OS versions.

Is it a suitable time to make the switch? I would say: "Not yet"!
Is this a good time to test them out, build proof-of-concept and get ready to migrate? Absolutely!

Remember: the clock is ticking, and August 2024 deadline is coming sooner than you think.

Build your own templates for Azure Developer CLI

David Pazdera — Sat, 17 Dec 2022 17:36:37 +0000

This post was created for the Festive Tech Calendar 2022 event.

Introduction

Festive greetings eager learners,

I'm excited to contribute to this year's Festive Tech Calendar with this blog post. As you might have guessed, it will be about the "new CLI kid on the block", Azure Developer CLI.

Now, there is a pletora of great blog posts and videos about this very topic on the Internet, so why bother? Well, I was thinking about exploring a possibility for using azd not by Devs but by IT Engineers. Sounds like heresy, right? Well, considering how much a traditional IT Pro role has changed with 'infra as code' and 'config as code' practices, cloud adoption, and automation of processes, I don't think it's the case anymore.

pazdedav / azd-function-pwsh-itpro

Azure Developer CLI template for IT Process automation

azd-function-pwsh-itpro

Azure Developer CLI template for IT Process automation

Hope you will have fun with it! I certainly had when making this demo :)

View on GitHub

So, what are we trying to accomplish?

The mission

If you worked with Azure and tried to automate some IT processes for some time, you saw a gradual shift from Automation runbooks to PowerShell Functions and event-driven architecture. There are several patterns and use cases but the one we will focus on and try to "azdify" (yep, there is a term for this now) one particular Azure PowerShell Function that can automate shutdown of Azure VMs.

Our mission, if you choose to accept it, is to create a reusable azd template that will:

provision all required components (Azure Function App, storage account, etc.)
deploy the app (in this case a PowerShell code)
enable monitoring, so we could troubleshoot any errors
create a workflow that will provision & deploy for us automagically

The recipe

Our main cookbook will be this official guide on Microsoft Learn.

It doesn't make sense to start from scratch completely, so we could find an existing template (Microsoft or community provided) and cherry-pick elements we could re-use and refactor for our needs. You could either use azd-templates topic on GitHub or use this new Azwesome AZD web gallery.

After a long search, I decided to use this template and modify it (quite heavily) for our scenario.

A small snag

Since this article is being written while I am playing with the tool, and I don't know the outcome, there is one potential snag: PowerShell is not among officially supported languages. I am still hoping I'll be able to deploy the function with azd, but if not, I could still do it using VS Code or a CLI.

The kitchen

We need a solid dev environment with an IDE and tools that will help us on the way. Here are the specs of my Windows 11 "kitchen":

GitHub CLI v2.20.2
Azure CLI v2.43.0
Git v2.39.0
Windows Terminal v1.15.2875.0
Bicep CLI v0.12.40.16777
PowerShell v7.3.0.0
Visual Studio Code (stable) v1.74.0
Azure Functions Core Tools v4.x. Read this article to understand, what version you should install and from where.

If you like using winget, here is your "shopping list":

winget install --id GitHub.cli
winget install --id Microsoft.AzureCLI
winget install --id Microsoft.Bicep
winget install --id Microsoft.WindowsTerminal
winget install --id Microsoft.Bicep
winget install --id Microsoft.PowerShell
winget install --id Microsoft.VisualStudioCode
winget install -e --id Microsoft.AzureFunctionsCoreTools

We cannot forget about our main hero, azd, right? Since it is in Preview, you can't use winget yet. Pick a method based on your OS, I am doing:

powershell -ex AllSigned -c "Invoke-RestMethod 'https://aka.ms/install-azd.ps1' | Invoke-Expression"

We will also need several useful VS Code extensions on top:

code --install-extension ms-azuretools.vscode-azurefunctions
code --install-extension ms-azuretools.vscode-bicep
code --install-extension ms-vscode.powershell
code --install-extension ms-vscode.azure-account
code --install-extension ms-azuretools.azure-dev
code --install-extension ms-azuretools.vscode-azureresourcegroups
code --install-extension redhat.vscode-yaml

I also made a few things to ensure consistency between my defaults on GitHub and my local Git configuration:

git config --global init.defaultBranch main

Let's start baking

Since our goal is to publish our solution on GitHub, we want to start the development in the correct way by creating a project and cloning it locally. You could do it on github.com but I am big fan of "CLI first" approach:

gh repo create azd-function-pwsh-itpro --public --clone --add-readme --description 'Azure Developer CLI template for IT Process automation'
git pull origin main

Init

Now when we have the repo created, cloned, and in sync, let's start by scaffolding the repo with a basic "azd template" structure. For the sake of traceability, I will create an Issue and a branch first:

gh issue create -t "Bootstrap azd template" --body "Use azd init to scaffold an empty structure for the template." --assignee "@me"
gh issue develop 1 --name "pazdedav/issue1" --checkout

Now it's time to initialize the project by using an empty template and set a few parameters like Azure location, environment name (a prefix for the resource group that will be created), and your subscription Id.

azd init --location westeurope --subscription 00000000-0000-0000-0000-000000000000 --environment festcal --no-prompt

TIP: The documentation doesn't tell what value should be used for --template parameter for an empty template. What worked for me was to add --no-prompt argument instead of specifying a template. Then you won't get any prompt.

Notice, that there is a new .azure directory with some config files being generated by the tool. The content doesn't contain any secrets per se, but this directory is added to .gitignore anyway.

So far so good, so let's commit all files we got via az init to our branch, review the changes (ideally done by someone else 🌞) through PR, and merge our code:

git add .
git commit -m "Add azd project scaffold."
git push origin pazdedav/issue1
gh pr create --title "Add azd project scaffold." --body "Closes #1" --base main
gh pr merge --merge --subject "Add azd project scaffold." --delete-branch

Adding infrastructure code

In this step, we will add infra directory and cherry-pick some existing templates from our reference repository (see "The recipe" for more information).

Let's create a new issue and a branch:

gh issue create -t "Add Bicep IaC" --body "Add Bicep modules and main template." --assignee "@me"
gh issue develop 3 --name "pazdedav/issue3" --checkout

Note: If you wonder, why I skipped "issue 2", I discovered that GitHub includes PRs into the automatic numbering system, so the new issue that was created with gh issue create command returned "Creating issue in pazdedav/azd-function-pwsh-itpro. https://github.com/pazdedav/azd-function-pwsh-itpro/issues/3".

The azd project introduced with version 0.2.0-beta.2 a new structure of Bicep infra code based on modules and the template we are using as a reference has this new structure (with app and core subdirectories under the infra directory) aligned already. And since it contains all Azure resources we need for our project, I will simply download the codebase as a ZIP file and extract the content of the infra directory to my project. It should look like this:

The template includes some additional components we don't need (like a database), so we need to refactor the main.bicep and main.parameters.json files to ensure, we will only provision what is required for Azure PowerShell Function. At the same time, I would like to keep the monitoring part in place, so we can get more insights about how the function works (or doesn't work).

Before we start with refactoring, it's good practice to stage and commit all the copied files to the local repo.

git add .
git commit -m "Add infra code from the sample"

It would be difficult to write about all changes I have done, so I am sharing the state of three files I changed here:

main. Bicep:

targetScope = 'subscription'

@minLength(1)
@maxLength(64)
@description('Name of the the environment which is used to generate a short unique hash used in all resources.')
param environmentName string

@minLength(1)
@description('Primary location for all resources')
param location string

// Optional parameters to override the default __azd__ resource naming conventions. Update the main.parameters.json file to provide values. e.g.,:
// "resourceGroupName": {
//      "value": "myGroupName"
// }
param apiServiceName string = ''
param applicationInsightsDashboardName string = ''
param applicationInsightsName string = ''
param appServicePlanName string = ''
param keyVaultName string = ''
param logAnalyticsName string = ''
param resourceGroupName string = ''
param storageAccountName string = ''

@description('Id of the user or app to assign application roles')
param principalId string = ''

var abbrs = loadJsonContent('./abbreviations.json')
var resourceToken = toLower(uniqueString(subscription().id, environmentName, location))
var tags = { 'azd-env-name': environmentName }

// Organize resources in a resource group
resource rg 'Microsoft.Resources/resourceGroups@2021-04-01' = {
  name: !empty(resourceGroupName) ? resourceGroupName : '${abbrs.resourcesResourceGroups}${environmentName}'
  location: location
  tags: tags
}

// The application backend
module api './app/api.bicep' = {
  name: 'api'
  scope: rg
  params: {
    name: !empty(apiServiceName) ? apiServiceName : '${abbrs.webSitesFunctions}api-${resourceToken}'
    location: location
    tags: tags
    applicationInsightsName: monitoring.outputs.applicationInsightsName
    appServicePlanId: appServicePlan.outputs.id
    keyVaultName: keyVault.outputs.name
    storageAccountName: storage.outputs.name
    appSettings: {
    }
  }
}

// Give the API access to KeyVault
module apiKeyVaultAccess './core/security/keyvault-access.bicep' = {
  name: 'api-keyvault-access'
  scope: rg
  params: {
    keyVaultName: keyVault.outputs.name
    principalId: api.outputs.SERVICE_API_IDENTITY_PRINCIPAL_ID
  }
}

// Create an App Service Plan to group applications under the same payment plan and SKU
module appServicePlan './core/host/appserviceplan.bicep' = {
  name: 'appserviceplan'
  scope: rg
  params: {
    name: !empty(appServicePlanName) ? appServicePlanName : '${abbrs.webServerFarms}${resourceToken}'
    location: location
    tags: tags
    sku: {
      name: 'Y1'
      tier: 'Dynamic'
    }
  }
}

// Backing storage for Azure functions backend API
module storage './core/storage/storage-account.bicep' = {
  name: 'storage'
  scope: rg
  params: {
    name: !empty(storageAccountName) ? storageAccountName : '${abbrs.storageStorageAccounts}${resourceToken}'
    location: location
    tags: tags
  }
}

// Store secrets in a keyvault
module keyVault './core/security/keyvault.bicep' = {
  name: 'keyvault'
  scope: rg
  params: {
    name: !empty(keyVaultName) ? keyVaultName : '${abbrs.keyVaultVaults}${resourceToken}'
    location: location
    tags: tags
    principalId: principalId
  }
}

// Monitor application with Azure Monitor
module monitoring './core/monitor/monitoring.bicep' = {
  name: 'monitoring'
  scope: rg
  params: {
    location: location
    tags: tags
    logAnalyticsName: !empty(logAnalyticsName) ? logAnalyticsName : '${abbrs.operationalInsightsWorkspaces}${resourceToken}'
    applicationInsightsName: !empty(applicationInsightsName) ? applicationInsightsName : '${abbrs.insightsComponents}${resourceToken}'
    applicationInsightsDashboardName: !empty(applicationInsightsDashboardName) ? applicationInsightsDashboardName : '${abbrs.portalDashboards}${resourceToken}'
  }
}

// App outputs
output APPLICATIONINSIGHTS_CONNECTION_STRING string = monitoring.outputs.applicationInsightsConnectionString
output AZURE_KEY_VAULT_ENDPOINT string = keyVault.outputs.endpoint
output AZURE_KEY_VAULT_NAME string = keyVault.outputs.name
output AZURE_LOCATION string = location
output AZURE_TENANT_ID string = tenant().tenantId
output REACT_APP_API_BASE_URL string = api.outputs.SERVICE_API_URI
output REACT_APP_APPLICATIONINSIGHTS_CONNECTION_STRING string = monitoring.outputs.applicationInsightsConnectionString

main.parameters.json:

{
  "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "environmentName": {
      "value": "${AZURE_ENV_NAME}"
    },
    "location": {
      "value": "${AZURE_LOCATION}"
    },
    "principalId": {
      "value": "${AZURE_PRINCIPAL_ID}"
    }
  }
}

api.bicep:

param name string
param location string = resourceGroup().location
param tags object = {}

param allowedOrigins array = []
param applicationInsightsName string = ''
param appServicePlanId string
param appSettings object = {}
param keyVaultName string
param serviceName string = 'api'
param storageAccountName string

module api '../core/host/functions.bicep' = {
  name: '${serviceName}-functions-powershell-module'
  params: {
    name: name
    location: location
    tags: union(tags, { 'azd-service-name': serviceName })
    allowedOrigins: allowedOrigins
    alwaysOn: false
    appSettings: appSettings
    applicationInsightsName: applicationInsightsName
    appServicePlanId: appServicePlanId
    keyVaultName: keyVaultName
    runtimeName: 'powershell'
    runtimeVersion: '7.2'
    storageAccountName: storageAccountName
    scmDoBuildDuringDeployment: false
  }
}

output SERVICE_API_IDENTITY_PRINCIPAL_ID string = api.outputs.identityPrincipalId
output SERVICE_API_NAME string = api.outputs.name
output SERVICE_API_URI string = api.outputs.uri

I want to validate my infra code by running the following checks:

az bicep build --file infra/main.bicep
az deployment sub validate --location westeurope --template-file infra/main.bicep --parameters environmentName=festcal location=westeurope

The first one "transpiles" our Bicep templates into a single JSON ARM deployment template and displays all errors and warnings found by Bicep linter. The second command checks our infra code against ARM API (also known as pre-flight). You need to be signed in to Azure in the console for the second command to work.

Since our Bicep code creates a Resource Group and provisions all resources, we are using 'subscription' as a scope for the deployment.

I didn't get any warnings or errors, so we could try to create our environment with azd:

azd provision

Voilà. We have our environment completely provisioned up in Azure:

Note: the last __azd_ command also updates the .env file in .azure directory with all Bicep outputs we declared in the code but don't worry all files under the .azure directory are excluded from Git tracking, so they won't be committed and pushed to origin._

Now let's commit, push, and merge everything to GitHub:

git add .
git commit -t "Refactor infra code."
git push origin pazdedav/issue3
gh pr create --title "Add infrastructure code to the template." --body "Closes #3" --base main
gh pr merge --merge --subject "Add infra code to the template." --delete-branch

Adding PowerShell function

Now we can scaffold our functions project (using Azure Functions Core Tools), add some code, and see if azd can deploy it or not.

Let's create a new issue and a branch:

gh issue create -t "Add PowerShell Function" --body "Scaffold a new function project and add PowerShell code." --assignee "@me"
gh issue develop 5 --name "pazdedav/issue5" --checkout

We want to keep our project neat and structured, so the function code should be placed in this path: src/api. Let's create this path and initialize the project:

mkdir src\api
cd src\api
func init --worker-runtime powershell --managed-dependencies

We can see, what files were generated by func init in our editor:

For local functions development, you could either install a local storage emulator, or update local.settings.json file to use an existing storage account in Azure. I don't want to install even more tools to my development machine, so I will use the storage account we provisioned a while ago. Simply run the following command with the Function App name you got:

func azure functionapp fetch-app-settings func-api-c4dvlakzaax5o

Now we need some actual PowerShell function code. I didn't want to use the boilerplate code, so I was looking for a good example for our "IT Pros" scenario in the Serverless Library, and I chose this one created by Eamon O'Reilly. It's a timer-based function that can stop Azure VMs on a schedule.

First, we need to create a new function:

func new --name StopVMOnTimer --template "Timer trigger"

This function will need Az modules to work. Luckily, PowerShell Functions have a feature called "managed dependencies" that can download required modules for us. All we need is to:

check in the PowerShell Gallery what version we want to add. Currently, it is v9.2.0.
edit the requirements.psd1 file, so it looks like this:

# This file enables modules to be automatically managed by the Functions service.
# See https://aka.ms/functionsmanageddependency for additional information.
#
@{
    # For latest supported version, go to 'https://www.powershellgallery.com/packages/Az'. 
    # To use the Az module in your function app, please uncomment the line below.
    'Az' = '9.*'
}

Depending on how often we want this function to run, we might want to update the schedule key in function.json configuration file. I want to shutdown my VMs every day at 7 PM, so this is how it looks in my case:

{
  "bindings": [
    {
      "name": "Timer",
      "type": "timerTrigger",
      "direction": "in",
      "schedule": "0 0 19 * * *"
    }
  ]
}

Now let's replace the boilerplate code in run.ps1 file with the same file from Eamon's repo, so it looks like this:

# Input bindings are passed in via param block.
param($Timer)

# Specify the VMs that you want to stop. Modify or comment out below based on which VMs to check.
$VMResourceGroupName = "test-vms-rg"
#$VMName = "ContosoVM1"
$TagName = "AutomaticallyStop"

# Stop on error
$ErrorActionPreference = 'stop'

# Check if managed identity has been enabled and granted access to a subscription, resource group, or resource
$AzContext = Get-AzContext -ErrorAction SilentlyContinue
if (-not $AzContext.Subscription.Id) {
    Throw ("Managed identity is not enabled for this app or it has not been granted access to any Azure resources. Please see https://docs.microsoft.com/en-us/azure/app-service/overview-managed-identity for additional details.")
}

try {
    # Get a single vm, vms in a resource group, or all vms in the subscription
    if ($null -ne $VMResourceGroupName -and $null -ne $VMName) {
        Write-Information ("Getting VM in resource group " + $VMResourceGroupName + " and VMName " + $VMName)
        $VMs = Get-AzVM -ResourceGroupName $VMResourceGroupName -Name $VMName
    }
    elseif ($null -ne $VMResourceGroupName) {
        Write-Information("Getting all VMs in resource group " + $VMResourceGroupName)
        $VMs = Get-AzVM -ResourceGroupName $VMResourceGroupName
    }
    else {
        Write-Information ("Getting all VMs in the subscription")
        $VMs = Get-AzVM
    }

    # Check if VM has the specified tag on it and filter to those.
    If ($null -ne $TagName) {
        $VMs = $VMs | Where-Object { $_.Tags.Keys -eq $TagName }
    }

    # Stop the VM if it is running
    $ProcessedVMs = @()

    foreach ($VirtualMachine in $VMs) {
        $VM = Get-AzVM -ResourceGroupName $VirtualMachine.ResourceGroupName -Name $VirtualMachine.Name -Status
        if ($VM.Statuses.Code[1] -eq 'PowerState/running') {
            Write-Information ("Stopping VM " + $VirtualMachine.Id)
            $ProcessedVMs += $VirtualMachine.Id
            Stop-AzVM -Id $VirtualMachine.Id -Force -AsJob | Write-Information
        }
    }
    # Sleep here a few seconds to make sure that the command gets processed before the script ends
    if ($ProcessedVMs.Count -gt 0) {
        Start-Sleep 10
    } 
}
catch {
    throw $_.Exception.Message
}

I made two small modifications:

updated the $VMResourceGroupName variable to test-vms-rg
uncommented the $TagName variable, so only VMs with this resource tag will eventually be stopped

Test environment in Azure

If we want to test this function, we need to have an actual environment with a Resource Group (called test-vms-rg) and some VMs with that AutomaticallyStop tag.

We also need to make sure our function can perform "VM Stop" operations using PowerShell. Since we are using Managed Identity and our azd provision step created a system-assigned managed identity for us, it is only a matter of grabbing correct objectId and do a role assignment.

Something like this:

az group create --name test-vms-rg --location westeurope

$assigneeObjectId = az ad sp list --display-name func-api-c4dvlakzaax5o --query "[0].id" --output tsv

az role assignment create --assignee-object-id $assigneeObjectId --assignee-principal-type ServicePrincipal --role "Virtual Machine Contributor" --scope /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/test-vms-rg

Ideally, we should have at least one VM in our Resource Group to test the functionality. Let's create one quickly:

az vm create --resource-group test-vms-rg --name myVM01 --image UbuntuLTS --admin-username azureuser --generate-ssh-keys --tags AutomaticallyStop=true

Test locally

One of many Azure Functions benefits is the ability to test them locally before you publish them to Azure. For all functions other than HTTP and Event Grid triggers, you can test your functions locally using REST by calling a special endpoint called an administration endpoint. It has the following Uri format: http://localhost:{port}/admin/functions/{function_name}

Calling this endpoint with an HTTP POST request on the local server triggers the function.

Let's run the function and use Invoke-WebRequest cmdlet to trigger the function. We will need to open two windows in Windows Terminal for this:

First terminal session:

func start

Second terminal session:

$headers = @{
    'Content-Type' = 'application/json'
}
$uri = "http://localhost:7071/admin/functions/StopVMOnTimer"
Invoke-RestMethod -Uri $uri -Method Post -Headers $headers

For some reason, I got "400 (Bad Request)" response and the log running in the first terminal did not provide any clues, what I am missing.

We will still try to deploy this function to Azure.

Update azd manifest and deploy

Ok, let's update the azure.yaml file and check, if we can use azd deploy or not:

Remember, what I wrote earlier about a potential snag? It seems we need to wait for PowerShell language support for Functions.

For the time being, we will use func CLI to publish our function to Azure, more specifically with:

func azure functionapp publish func-api-c4dvlakzaax5o

This works well:

I feel good about the progress we made, so let's commit, push, and merge our code again:

git add .
git commit -m "Add function code."
git push origin pazdedav/issue5
gh pr create --title "Add PowerShell function code to the template." --body "Closes #5" --base main
gh pr merge --merge --subject "Add application code to the template." --delete-branch

Configure a GitHub workflow

One missing step is to enable integration with GitHub Actions, so we can automate both the provisioning and deployment of our solution.

Since we were tracking our work from GH issues, through PRs, to merges before, we won't skip this step for our last exercise:

gh issue create -t "Add GitHub workflow." --body "Add GitHub workflow yaml file." --assignee "@me"
gh issue develop 7 --name "pazdedav/issue7" --checkout

This is what we will do:

create a .github directory with a workflows subdirectory
copy the azure-dev.yml file from the sample repo, we have been working with

$uri = "https://github.com/Azure-Samples/todo-csharp-sql-swa-func/raw/main/.github/workflows/azure-dev.yml"
$workflowFile = (Invoke-WebRequest -Uri $uri).Content
mkdir .github\workflows
New-Item -Path ".github\workflows" -Name "azure-dev.yml" -ItemType "file" -Value $workflowFile

This workflow requires you to create a couple of secrets, before it can run successfully, as you can see in the workflow file. We could create them manually together with a Service Principal like this (don't do it like this, keep on reading):

gh secret set -f .\.azure\festcal\.env --app actions

az ad sp create-for-rbac --name azd-festcal-temp --role contributor --scopes /subscriptions/0c310ab4-e68f-45e7-b7e2-72fbbb32e891 --output json > gh_spn.json

$ghSpnValue = Get-Content -Path .\gh_spn.json -Raw
gh secret set AZURE_CREDENTIALS --body $ghSpnValue --app actions

Remove-Item .\gh_spn.json

Fortunatelly, azd can do everything for us, we simply need to run this command:

azd pipeline config

As you can see, everything is set:

We could say "Y" to get the pipeline started, but I'd like to do it manually, so I choose "n".

Let's close this last exercise with a couple of steps:

git add .
git commit -m "Add GitHub workflow file."
git push origin pazdedav/issue7
gh pr create --title "Add GitHub workflow to the template." --body "Closes #7" --base main
gh pr merge --merge --subject "Add azure-dev.yml to the template." --delete-branch

As expected, our pipeline run will fail due to the missing support for PowerShell in azd:

Summary

We could say this experiment wasn't strictly speaking successful. Our pipeline is failing, and we couldn't get the local test running.

On the other hand, we (hopefully) learned a lot about how azd works under the hood with a bunch of CLI examples using git, gh, func, PowerShell, and az.

If you managed to get to the end of this post, I hope it wasn't a waste of time for you.

Have a great Christmas holiday and all the best in 2023.

Cleanup

If you followed along and created resources in Azure, it's a good idea to clean things up to avoid an unexpected bill from Microsoft.

Azd has a simple command for it:

azd down

You might be prompted a few times to confirm your plan but in the end you should see something like this:

Other useful resources

For those of you who want to learn more about this awesome tool, here is a couple of resources I found especially useful:

Azure Developer CLI YouTube channel
Series of blog posts on Dev.to published by Christian Lechner

Azure Native Infrastructure as Code for Advanced Practitioners

David Pazdera — Thu, 26 Nov 2020 17:51:12 +0000

This post was created for the Festive Tech Calendar 2020 event and it is a narrative to the talk I gave at Infrastructure as Code User Group Oslo meetup.

The purpose is to demonstrate a complex end-to-end scenario for using advanced capabilities of Azure Resource Manager language (like template specs, deployment scopes, functions) and the new ARM DSL language called Bicep. It also tries to demystify a popular opinion that the ARM language is "just a JSON".

All code samples and other content could be found in my repo on GitHub:

pazdedav / iac-meetup-arm

Repository for Infrastructure as Code meetup, demonstrating a complex end-to-end scenario for using advanced capabilities of Azure Resource Manager language (like templateSpecs, deployment scopes, deploymentScripts) and the new ARM DSL called Bicep.

Story

Our story begins when Contoso Corp. (a fictious company) announces their new business strategy with digital transformation, customer obsession, and lean as key pillars. This had a profound impact on both IT and software development organizations at Contoso. Old habits and fixed mindset are no longer acceptable, there is a high pressure on everybody to invest their time to learn and grow.

Our first protagonist, Meghan is a senior engineer who has been with the company for five years. Mat on the other hand is part of a newly formed Central Cloud Team. In his new Cloud Engineer role, he is focusing on security, compliance, and building a 'secure and compliant' cloud platform that will be used by internal teams. He has a strong infrastructure background with two decades of experience managing networks and server infrastructure.

At the beginning of our story, a new cloud governance model has been defined by key stakeholders. This model is partly a re-write of Contoso's existing security policies (adopted for the cloud) that was combined with new regulatory requirements that were introduced to the industry.

Mat's team has an important task now. They must ensure there are reliable controls and audit capabilities enforced on their entire cloud environment, so all systems, apps, and services deployed to the cloud are compliant from Day 1.

Meghan's team has been developing a new application when this change of policies is announced. There is a growing fear that this new policy framework will slow down their progress and introduce many blockers. These concerns were escalated to VP of App Development and it was decided that the Central Cloud Team need to find out a way to support developers while maintaining the governance model as required by the business and industry regulators.

Apart from our two main protagonists, there are other heroes in this story :) :

Policy as Code

Mat's team was busy defining what controls and audit mechanisms they should use in the new governance model. They have done a few experiments, tested several technologies and platform features on Azure, and they decided to use Azure Policies together with Management Groups to apply them at scale, so they could ensure 'Day 1 compliance' for all new subscriptions as well as the existing ones.

They both defined and assigned all their policies in the Azure Portal but they wanted to adopt 'Policy as code' practice to automate definition and assignment workflows and improve change tracking.

The team learned about a new feature, allowing them to export Azure Policy objects (both definitions and assignments) from the Portal to GitHub repo. This feature also creates:

a workflow using GitHub Actions that can automate 'deployments' of any changes made in the repo
a GitHub secret for authenticating to Azure using a service principal
a service principal that was granted the 'Resource Policy Contributor' role, so it can change policies

Mat's team followed the process from the documentation and exported all their policies and assignments to src/policies/ folder in their repo. The workflow YML file can be found here. The workflow is by default configured to be triggered manually, but this can be changed.

They also published the list of applied policies in their Wiki to make it more readable for non-technical stakeholders.

Template Library

Being transparent about what policies are applied in the cloud environment (and how they can impact deployments done by internal teams) is a good start, but the Central Cloud Team wanted to increase productivity of app development teams even more, so they decided to introduce a new service called Template Library.

The goal was simple: provide a repository of reusable, dynamic, and security-hardened resource manager templates, fully compliant with the governance model, so they could be used as artifacts for building more complex deployments.

The idea of having reusable artifacts (single resource 'linked' templates that could be referenced from the main template) isn't new, but there are several challenges for adopting this technique.

Mat and his team heard about template specs, a new capability in Azure Resource Manager that allows developers to refer to these artifacts in form of ResourceId that could be referenced in the same way as linked templates but without the need to stage them first to e.g. Azure Blob storage or expose them in a public repository.

The team defined the following requirements:

all template specs will be stored in our GitHub repo
all changes (to already published versions or new versions) will be pushed to Azure using a workflow after a peer review
all developers will be granted a Reader role in the Resource Group where all specs are stored
develop an automated way to onboard 'service principals' to the Library for teams that use them in their workflows (pipelines)
publish a guide for our internal customers on how to use this Library, so they can start using it
if a spec uses a JSON object as a parameter, publish a definition on Wiki that explains the structure of the parameters.

Onboarding workflow

The team defined the following workflow, allowing dev teams to onboard their service principal (typically used in Azure DevOps, GitHub Actions, or any other CI/CD pipelines) to the Library:

Create a new branch in the Central Cloud Team's GitHub repo.
Modify contoso-template-library.prod.parameters.json file by adding objectId of your Service Principal as new array member of servicePrincipals parameter value. You can create your SP by using e.g. az ad sp create-for-rbac --name "{sp-name}" --sdk-auth command.
Create a Pull Request and specify your 'projectId' and 'costPollId' in the description.
When the request is reviewed and approved by our team, it will trigger a workflow that will assign a Reader role to your SP in the context of the Resource Group hosting the Template Library.

The template file - contoso-template-library.template.json - is a good demonstration of several ARM language features:

deployment scopes - allowing to create a Resource Group at the subscription level and assign an RBAC role (as a deployment on Resource Group level), all in one template
using comments in several places in the file (ARM allows both single-line and multi-line comments)
using functions like if, contains, concat, copy, resourceId, guid, etc.

Using template specs

Now we will switch to Meghan and learn how she used the Template Library provided by Mat's team in her workflow.

First, she used the onboarding procedure and added the service principle used by her team to the Library. Her Pull Request was approved, so she could modify her deployment template - serviceOne.template.json - and referenced 'contosoStorage' spec. Her team is using GitHub workflow to deploy the template to Azure.

Since her team uses parameter files (one for each environment), she used the information from the Wiki and correctly defined all required parameters.

Experimenting with Bicep

As part of their continuous improvement principle, Mat's team was eager to learn more about project Bicep, a new DSL language that could simplify authoring of ARM JSON templates.

They started with creating a bicep file that matches with the template spec for "contoso compliant storage account".

All bicep files need to be "transpiled" to ARM JSON templates using bicep build command. This requires Bicep CLI that can be downloaded from the project repo (releases).

The team wanted to build a pipeline (a GitHub workflow) that would on "pull" take all bicep files in the repo (and a specific path, e.g. src/bicep/*), transpile them to ARM JSON files, and deploy them as template specs to Azure to their 'Template Library'.

At that time, the template specs required a special version of Resources PowerShell module (stored in a private repo, they had access to), so they tried to develop a custom action that would contain all required binaries (bicep CLI, Az.Resources module), perform all needed steps, and deploy to Azure using AZ CLI.

They are still working on this improvement idea...

Note: There has been a lot of development in both template specs and Bicep. There are several Actions in GitHub Marketplace (from the community) for working with Bicep files. Also, it is no longer required to have a special version of Az.Resources PowerShell module to deploy template specs, and support for Azure CLI was also added.

Resources

Documentation

Template Specs concepts
Template Specs tutorial
Deployment scopes
Policy as Code with GitHub tutorial
Project Bicep repo
Bicep Playground

Meetup video on YouTube

If you want to watch a video, where I presented this story together with several demos, you can watch it from the recording (my part begins at 8:31 and ends at 1:03:38):

Please note, this wasn't a conference-like session, but a local meetup, so the presentation is not very polished!! As a bonus, you can learn about Farmer from Evgeny Borzenin (@EvgenyBorzenin).

Using Azure Express Route for online data transfers

David Pazdera — Sat, 07 Mar 2020 21:17:56 +0000

Introduction

Let's imagine this scenario: you have built a hybrid network between your corporate network and Azure using Azure Express Route and now you would like to leverage this private high-bandwidth connectivity for various purposes.

First scenario that comes to my mind is data migration, for example you want to move your 500TB archive to the cloud.

You have read all about Azure Blob Storage and its tiering model, and you think this can be the right solution.

Since you now have both your "source" and "destination", the next logical step is to figure out "how".

On a high level, you have two options:

online data transfer
offline data transfer

The first category is represented by tools (azcopy or Azure Storage SDK) and Azure services (like Azure Data Factory) and it requires a good network bandwidth (and time). The second category is dominated by the Azure Data Box product family.

Data Transfer Advisor

If you need some help with making the right choice, you could leverage quite new experience in the Azure portal. When you provision a new Storage Account (or open an existing one), you can see the "Data Transfer" blade, where you fill in three attributes - estimated data size, available network bandwidth, and transfer frequency - and based on your input you will get a recommendation similar to this one:

Now, you feel confident that using the online method with azcopy is the best option for you, especially since you have this nice 1Gbps private "pipe" to Azure.

Little catch

So far so good? Well, the moment you start planning your data transfer in details, you will (most likely) realize, that Azure Storage service exposes several public endpoints, and for Blob storage it will look something like this:
https://cs44deib9068be17x4267xa4b.blob.core.windows.net/

Unless you configured something called Microsoft peering on your Express Route circuit, all traffic from your network to those public endpoints will be routed over your Internet connection (router) and not via Express Route!

Private Link to the rescue

Do not despair. Microsoft launched a new service called Private Link that allows you to create a "private endpoint" that represents a specific instance of a PaaS service like Azure Storage and make it available (present it in form of a network interface card with a private IP) inside your Virtual Network.

This has several benefits like the ability to completely close public endpoints to your PaaS instance (block any access from the Internet) and make this instance available not only from within your VNet, but all peered VNets as well as your corporate network.

How is this helping in our data transfer scenario? A lot. That Private Endpoint that I will attach to my target Blob storage will have a private IP address belonging to the IP range that is advertised via BGP protocol and my Express Route connection to my corporate network. In other words, if I use a tool like azcopy or Azure Storage Explorer from a computer inside my network and target such private endpoint, this connection will utilize my Express Route connection and its bandwidth. This is exactly what we wanted :)

Solution design

This is how our solution will look like in the end.

This is what we need to do, assuming that all networking bits and pieces (Virtual Network, Express Route circuit with Private peering, ER Gateway) have been provisioned and configured in advance:

First, create a new Storage Account in the same region as your Virtual Network. In the 'Networking' part of the wizard, select Private endpoint option:

Populate all important parameters of the new private endpoint and link it with your VNet:

You can keep 'Integrate with Private DNS zone' option enabled, so the wizard also creates a private DNS zone (part of Azure DNS service offering). However, depending on your scenario, making this private zone available from your on-premises network requires additional configuration and careful planning. We will use a simpler option to make it working for our use case.

Depending what you provisioned in what Resource Group, the final result could look like this:

End-to-end testing

There is one more step you need to do to complete the setup. As I mentioned above, the name resolution is a complex topic in hybrid DNS scenarios, where you are in many cases bringing your existing DNS to Azure VNets while trying to utilize Azure DNS private zones. I will leave this topic for another article :).

For now, we will use the simplest option we have available, we will modify the hosts file in our source server (e.g. a machine, where we have our archive mounted) and add an entry that will resolve our blob endpoint to the private IP address of our Private endpoint. You can get both values from the Private endpoint resource:

There are many articles describing how to change hosts file, so I won't describe the process here.

After adding that entry, you should be able to test it using nslookup.

What is left is testing the actual upload of the data. Follow this article:

authenticate to Azure using AzCopy login command. Eventually you can use a SAS token (you need to create one in your target storage account and append it to the target Uri).
upload a directory with files (change both source drive/folder and destination Uri): azcopy copy 'C:\myDirectory' 'https://mystorageaccount.blob.core.windows.net/mycontainer' --recursive

Conclusion

This might look like a lot of work, but provisioning and configuring those handful of resources took me approx. 15 minutes (again assuming that all the "plumbing" was done beforehand), so it is definitely worth it.