DEV Community: Piotr Wachulec

Optimization of environmental costs in Azure. Disks. #01

Piotr Wachulec — Mon, 26 Apr 2021 14:50:02 +0000

The greater the environment, the more likely some resources are abandoned and unused. You know, someone requested a virtual machine, app service, but it turned out that it is unnecessary. Or some project or PoC finished and test resources are not removed. Due to this, it is important to look at your resources and check if can be removed or resized for example.

If you have virtual machines in your Azure environment, it's always good to check some things related to disks. In this post, you can see how easy to check if some disks are not attached or if on some virtual machines disk space is unallocated.

Checking if some disks are orphaned you can do with the below Powershell command:

$existingDisks = Get-AzDisk

Write-Output "`nUnattached disks:"
foreach ($disk in $existingDisks) {
    If ($disk.DiskState.Contains('Unattached')) {
        $disk.Name
    }
}

Get-AzDisk gets all disks in the current subscription. Each disk has the property DiskState which can tell you if the disk is attached to some virtual machine or not. The above script checks if the DiskState property has value Unattached which means that it isn't connected to any VM.

Another thing is that the disk can be connected to the virtual machine, but it isn't initialized on the OS level. How to check that? It's a more complex operation. You can't do that from the Azure perspective, you have to check disk statuses directly on the OS. But relax, you don't have to log into each virtual machine and check that manually. Let's use the Run Command interface. There are a few ways to run custom scripts directly on virtual machine, each of them has its own pros and cons. Run Command will be the best option for us.

I prepared a small script which will be run through Run Command interface on the virtual machine. Here it is:

$disksInfo = Get-Disk

foreach ($disk in $disksInfo) {
    If ($disk.PartitionStyle.Contains('RAW')) {
        Write-Output $true
    }
}

Important information is that the disk with the unallocated space has something called PartitionStyle with value RAW. The above script takes info about all physical disks in the machine and checks if any of those disks have PartitionStyle status equal RAW.

The process is: get info about all virtual machines in the subscription with the information if the virtual machine is running, run on each of them the custom script, aggregate info about the statuses of calling Run Command with the custom script, and show the list of virtual machines with unallocated spaces. The below scripts realize that process:

# List of vms
$vms = Get-AzVm -Status
$vmsToCheck = @()

# List of VMs with unallocated disks
foreach ($vm in $vms) {
    If ($vm.PowerState.Contains('VM running') -and $vm.StorageProfile.DataDisks.Length -gt 0) {
        $vmsToCheck += $vm
    }
}

foreach ($vm in $vmsToCheck) {
    $scriptBlock = {
        $runOutput = Invoke-AzVMRunCommand -ResourceId $args[0].Id -CommandId 'RunPowerShellScript' -ScriptPath '.\Test-IfVmHasUnallocatedDisks.ps1'
        If ($runOutput.Value[0].Message.Contains('True')) {
            return $args[0].Name
        }
    }

    $jobName = "$($vm.Name)-DisksCheck"

    $job = Start-Job -ScriptBlock $scriptBlock -ArgumentList $vm -Name $jobName
    $jobs += $job
}

Write-Host -NoNewline "`nChecking unallocated disks"
while ((Get-Job -Name "*-DisksCheck" | Where-Object State -eq Running)) {
    Write-Host -NoNewline "."
    Start-Sleep -Seconds 2
}

Write-Output "`nVMs with unallocated disks:"
Get-Job | Receive-Job

Small hack in the script above - calling Run Command takes quite a long time, so I'm running checking for chosen machines in parallel and after that, I'm waiting for when all jobs will be done.

Original post you can always find on my blog!

Slack notifications on Azure budget consumption

Piotr Wachulec — Wed, 21 Apr 2021 09:46:33 +0000

In my free time (I don't have too much free time, to be honest 😋) I help my colleagues in the open-source project where juniors can learn something useful for their career. I started to deliver some infra for that stuff on Azure, but due to the fact that I'm paying for resources from my pocket, I want to have some nice notifications if something consumes a big amount of money and burns the budget. For AWS costs we have a dedicated channel on Slack where daily money consumption is sent. I would try to add an Azure budget alarm there too.

How to integrate?

For achieving my aim I will use the Incoming WebHooks app on Slack. There I will able to send messages to the chosen channel via webhook.

We already have added that app and webhook for sending info about costs, so to avoid reaching the limits of the Incoming WebHooks app, I reused the existing webhook and channel for used for costs notification purposes.

Budget

The next step is to create the budget for the subscription.

In the next step, we have to configure notifications and alarms.

There I added a new action group where I added sending email in the Notifications section. In the Actions section, I added a webhook URI.

After that, I could add alarms:

The budget was created.

Original post you can always find on my blog!

How to manage resources in the external tenant in Azure?

Piotr Wachulec — Mon, 19 Apr 2021 11:52:18 +0000

The case

You want to manage resources in other tenants without additional workload around that like accounts creation for you and your teammates in the destination Azure tenant or disabling those accounts if someone leaves the team. Is there any solution for that?

The answer is: yes! And it is called Azure Lighthouse.

From docs:

Azure Lighthouse enables cross- and multi-tenant management, allowing for higher automation, scalability, and enhanced governance across resources and tenants. ~What is Azure Lighthouse?

How does it work?

Azure Lighthouse enables managing resources in another tenant directly from your 'home' tenant. You can define with that which users and/or Active Directory Security groups should have access to which resources and with which roles.

Requirements

To set up the Azure Lighthouse solution, you have to fulfill some requirements and gather a bunch of information. The full list you can find here:

Your tenant ID
Customer's tenant ID
List of subscription IDs and/or resource group IDs to which you have to have access
List of users and/or Active Directory groups which you want to assign to manage customer environments - IMPORTANT: AD groups have to be Security groups!
List of roles and permissions (with their IDs) which you want to assign
Someone with non-guest account in customer's tenant and Microsoft.Authorization/roleAssignments/write permission - it is required to create assignment (Deploy the Azure Resource Manager templates)
Something called mspOfferName and mspOfferDescription - the first one defines the connection between both tenants and their resources. IMPORTANT: You can't have multiple assignments at the same scope with the same mspOfferName.

How to onboard the customer?

If you gathered all of the above requirements, on Github you can find example ARM templates that allow you to onboard. You have to deploy prepared and fulfilled templates to each subscription separately.

Example

For preparing example I will use the template from the official docs:

{
    "$schema": "https://schema.management.azure.com/schemas/2018-05-01/subscriptionDeploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "mspOfferName": {
            "type": "string",
            "metadata": {
                "description": "Specify a unique name for your offer"
            },
            "defaultValue": "<to be filled out by MSP> Specify a title for your offer"

        },
        "mspOfferDescription": {
            "type": "string",
            "metadata": {
                "description": "Name of the Managed Service Provider offering"
            },
            "defaultValue": "<to be filled out by MSP> Provide a brief description of your offer"
        },
        "managedByTenantId": {
            "type": "string",
            "metadata": {
                "description": "Specify the tenant id of the Managed Service Provider"
            },
            "defaultValue": "<to be filled out by MSP> Provide your tenant id"
        },
        "authorizations": {
            "type": "array",
            "metadata": {
                "description": "Specify an array of objects, containing tuples of Azure Active Directory principalId, a Azure roleDefinitionId, and an optional principalIdDisplayName. The roleDefinition specified is granted to the principalId in the provider's Active Directory and the principalIdDisplayName is visible to customers."
            }
        }              
    },
    "variables": {
        "mspRegistrationName": "[guid(parameters('mspOfferName'))]",
        "mspAssignmentName": "[guid(parameters('mspOfferName'))]"
    },
    "resources": [
        {
            "type": "Microsoft.ManagedServices/registrationDefinitions",
            "apiVersion": "2019-09-01",
            "name": "[variables('mspRegistrationName')]",
            "properties": {
                "registrationDefinitionName": "[parameters('mspOfferName')]",
                "description": "[parameters('mspOfferDescription')]",
                "managedByTenantId": "[parameters('managedByTenantId')]",
                "authorizations": "[parameters('authorizations')]"
            }
        },
        {
            "type": "Microsoft.ManagedServices/registrationAssignments",
            "apiVersion": "2019-09-01",
            "name": "[variables('mspAssignmentName')]",
            "dependsOn": [
                "[resourceId('Microsoft.ManagedServices/registrationDefinitions/', variables('mspRegistrationName'))]"
            ],
            "properties": {
                "registrationDefinitionId": "[resourceId('Microsoft.ManagedServices/registrationDefinitions/', variables('mspRegistrationName'))]"
            }
        }
    ]
}

with parameters file:

{
    "$schema": "https://schema.management.azure.com/schemas/2018-05-01/subscriptionDeploymentParameters.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "mspOfferName": {
            "value": "Test PW Lighthouse"
        },
        "mspOfferDescription": {
            "value": "Test PW Lighthouse"
        },
        "managedByTenantId": {
            "value": "11111111-1111-1111-1111-111111111111"
        },
        "authorizations": {
            "value": [
                {
                    "principalId": "22222222-2222-2222-2222-222222222222",
                    "roleDefinitionId": "b24988ac-6180-42a0-ab88-20f7382dd24c",
                    "principalIdDisplayName": "PW MC"
                }
            ]
        }
    }
}

I translated the above template to Bicep - it's much easier to read and edit for me:

@description('Specify a unique name for your offer')
param mspOfferName string

@description('Name of the Managed Service Provider offering')
param mspOfferDescription string

@description('Specify the tenant id of the Managed Service Provider')
param managedByTenantId string

@description('Specify an array of objects, containing tuples of Azure Active Directory principalId, a Azure roleDefinitionId, and an optional principalIdDisplayName. The roleDefinition specified is granted to the principalId in the provider`s Active Directory and the principalIdDisplayName is visible to customers.')
param authorizations array

var mspRegistrationName = guid(mspOfferName)
var mspAssignmentName = guid(mspOfferName)

targetScope = 'subscription'

resource regDefinintion 'Microsoft.ManagedServices/registrationDefinitions@2019-09-01' = {
  name: mspRegistrationName
  properties: {
    registrationDefinitionName: mspOfferName
    description: mspOfferDescription
    managedByTenantId: managedByTenantId
    authorizations: authorizations
  }
}

resource regAssignment 'Microsoft.ManagedServices/registrationAssignments@2019-09-01' = {
  name: mspAssignmentName
  properties: {
    registrationDefinitionId: regDefinintion.id
  }
}

The Bicep file can be used with the same parameters file as the classic ARM template.

For test purposes, I used two different Azure tenants. I 'home' tenant I created AD Security group called 'PW MC'. I wanted to give to chosen subscription the Contributor rights for 'PW MC' AD group, so I deployed the above template with the Azure CLI:

az deployment sub create --location WestEurope --template-file ./lighthouse.bicep --parameters lighthouse.parameters.json

Previously I logged into the tenant with az login --tenant command.

Effect

The effect should be visible in two places, in both tenants. In the customer tenant when we will go to the Service providers page, we will able to see something like on the screenshot below:

and in the 'home' tenant when we will visit the My customers page:

If we click on 'Default directory':

Then you can click on the subscription name and do whatever you want with the resources.

Summary

You can see that implementing Azure Lighthouse is quite easy and reduce management concerns. And what is the most important - it is free!

Original post you can find on my website.