DEV Community: Carlos Mendible

Azure Quick Review (azqr) v2.9.0 is out!

Carlos Mendible — Fri, 19 Sep 2025 21:50:58 +0000

🚀 Azure Quick Review (azqr) v2.9.0 is out!
🛡️ New: Azure Policy scanning & reporting
📂 Fix: Nested management group retrieval
✅ Fix: Resource Group ID format validation

👉For sharper insights get the latest: https://github.com/Azure/azqr

AKS: Windows node pool with spot virtual machines and ephemeral disks

Carlos Mendible — Mon, 13 Sep 2021 10:00:00 +0000

Some months ago a customer asked me if there was a way to deploy a Windows node pool with spot virtual machines and ephemeral disks in Azure Kubernetes Service (AKS).

The idea was to create a cluster that could be used to run Windows batch workloads and minimize costs by deploying the following:

An AKS cluster with 2 linux nodes and ephemeral disks as the default node pool configuration.
A Windows node pool with Spot Virtual Machines, ephemeral disks and auto-scaling enabled.
Set the windows node pool minimum count and initial number of nodes set to 0.

To create a cluster with the desired configuration with terraform, follow the steps below:

Define the terraform providers to use

Create a providers.tf file with the following contents:

terraform {
  required_version = "> 0.12"
  required_providers {
    azurerm = {
      source  = "azurerm"
      version = "~> 2.26"
    }
  }
}

provider "azurerm" {
  features {}
}

Define the variables

Create a variables.tf file with the following contents:

variable "resource_group_name" {
  default = "aks-win"
}

variable "location" {
  default = "West Europe"
}

variable "cluster_name" {
  default = "aks-win"
}

variable "dns_prefix" {
  default = "aks-win"
}

Define the resource group

Create a main.tf file with the following contents:

# Create Resource Group
resource "azurerm_resource_group" "rg" {
  name     = var.resource_group_name
  location = var.location
}

Define the VNET for the cluster

Create a vnet-server.tf file with the following contents:

resource "azurerm_virtual_network" "vnet" {
  name                = "aks-vnet"
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  address_space       = ["10.0.0.0/16"]
}

resource "azurerm_subnet" "aks-subnet" {
  name                 = "aks-subnet"
  resource_group_name  = azurerm_resource_group.rg.name
  virtual_network_name = azurerm_virtual_network.vnet.name
  address_prefixes     = ["10.0.1.0/24"]
}

Define the AKS cluster

Create a aks-server.tf file with the following contents:

# Deploy Kubernetes
resource "azurerm_kubernetes_cluster" "k8s" {
  name                = var.cluster_name
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  dns_prefix          = var.dns_prefix

  default_node_pool {
    name                = "default"
    node_count          = 2
    vm_size             = "Standard_D2s_v3"
    os_disk_size_gb     = 30
    os_disk_type        = "Ephemeral"
    vnet_subnet_id      = azurerm_subnet.aks-subnet.id
    max_pods            = 15
    enable_auto_scaling = false
  }

  # Using Managed Identity
  identity {
    type = "SystemAssigned"
  }

  network_profile {
    # The --service-cidr is used to assign internal services in the AKS cluster an IP address. This IP address range should be an address space that isn't in use elsewhere in your network environment, including any on-premises network ranges if you connect, or plan to connect, your Azure virtual networks using Express Route or a Site-to-Site VPN connection.
    service_cidr = "172.0.0.0/16"
    # The --dns-service-ip address should be the .10 address of your service IP address range.
    dns_service_ip = "172.0.0.10"
    # The --docker-bridge-address lets the AKS nodes communicate with the underlying management platform. This IP address must not be within the virtual network IP address range of your cluster, and shouldn't overlap with other address ranges in use on your network.
    docker_bridge_cidr = "172.17.0.1/16"
    network_plugin     = "azure"
    network_policy     = "calico"
  }

  role_based_access_control {
    enabled = true
  }

  addon_profile {
    kube_dashboard {
      enabled = false
    }
  }
}

resource "azurerm_kubernetes_cluster_node_pool" "windows" {
  kubernetes_cluster_id = azurerm_kubernetes_cluster.k8s.id
  name                  = "win"
  priority        = "Spot"
  eviction_policy = "Delete"
  spot_max_price  = -1 # The VMs will not be evicted for pricing reasons.
  os_type = "Windows"
  # "The virtual machine size Standard_D2s_v3 has a cache size of 53687091200 bytes, but the OS disk requires 137438953472 bytes. Use a VM size with larger cache or disable ephemeral OS."
  # https://docs.microsoft.com/en-us/azure/virtual-machines/ephemeral-os-disks#size-requirements
  vm_size             = "Standard_DS3_v2"
  os_disk_type        = "Ephemeral"
  node_count          = 0
  enable_auto_scaling = true
  max_count           = 3
  min_count           = 0
}

data "azurerm_resource_group" "node_resource_group" {
  name = azurerm_kubernetes_cluster.k8s.node_resource_group
}

# Assign the Contributor role to the AKS kubelet identity
resource "azurerm_role_assignment" "kubelet_contributor" {
  scope                = data.azurerm_resource_group.node_resource_group.id
  role_definition_name = "Contributor" #"Virtual Machine Contributor"?
  principal_id         = azurerm_kubernetes_cluster.k8s.kubelet_identity[0].object_id
}

resource "azurerm_role_assignment" "kubelet_network_contributor" {
  scope                = azurerm_virtual_network.vnet.id
  role_definition_name = "Network Contributor"
  principal_id         = azurerm_kubernetes_cluster.k8s.identity[0].principal_id
}

Deploy the AKS cluster

Run:

terraform init
terraform apply

Get the credentials for the cluster:

RESOURCE_GROUP="aks-win"
CLUSTER_NAME="aks-win"
az aks get-credentials --resource-group $RESOURCE_GROUP --name $CLUSTER_NAME

To verify that there are no windows VMs running, execute:

kubectl get nodes

you should see something like:

NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
aks-default-36675761-vmss000000 Ready agent 80m v1.20.9 10.0.1.4 <none> Ubuntu 18.04.5 LTS 5.4.0-1056-azure containerd://1.4.8+azure
aks-default-36675761-vmss000001 Ready agent 80m v1.20.9 10.0.1.20 <none> Ubuntu 18.04.5 LTS 5.4.0-1056-azure containerd://1.4.8+azure

Deploy a Windows workload:

To deploy a Windows workload, create a windows_deployment.yaml file with the following contents:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: servercore
  labels:
    app: servercore
spec:
  replicas: 1
  template:
    metadata:
      name: servercore
      labels:
        app: servercore
    spec:
      nodeSelector:
        "kubernetes.azure.com/scalesetpriority": "spot"
      containers:
      - name: servercore
        image: mcr.microsoft.com/dotnet/framework/samples:aspnetapp
        resources:
          limits:
            cpu: 1
            memory: 800M
          requests:
            cpu: .1
            memory: 150M
        ports:
          - containerPort: 80
      tolerations:
        - key: "kubernetes.azure.com/scalesetpriority"
          operator: "Equal"
          value: "spot"
          effect: "NoSchedule"
  selector:
    matchLabels:
      app: servercore

and deploy it to your cluster:

kubectl apply -f windows_deployment.yaml

Note the following:

The kubernetes.azure.com/scalesetpriority label is used to ensure that the workload is scheduled on a spot node.
tolerations are used to ensure that the workload is scheduled on a spot node.
Deployment will take a while (> 5 minutes) since the windows pool must scale up to fullfill the request.

Now check the nodes again:

kubectl get nodes

this time you should see something like:

NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
aks-default-36675761-vmss000000 Ready agent 91m v1.20.9 10.0.1.4 <none> Ubuntu 18.04.5 LTS 5.4.0-1056-azure containerd://1.4.8+azure
aks-default-36675761-vmss000001 Ready agent 91m v1.20.9 10.0.1.20 <none> Ubuntu 18.04.5 LTS 5.4.0-1056-azure containerd://1.4.8+azure
akswin000000 Ready agent 102s v1.20.9 10.0.1.36 <none> Windows Server 2019 Datacenter 10.0.17763.2114 docker://20.10.6

If you check the pod events you’ll find that the workload triggered a scale up:

kubectl describe $(kubectl get po -l "app=servercore" -o name)

I’ll let you test what happens if you delete the deployment.

Hope it helps!!!

Please find the complete terraform configuration here

AKS: Persistent Volume Claim with an Azure File Storage protected with a Private Endpoint

Carlos Mendible — Mon, 02 Aug 2021 10:00:00 +0000

This post will show you the steps you’ll have to take to deploy an Azure Files Storage with a Private Endpoint and use it to create volumes for an Azure Kubernetes Service cluster:

Create a bicep file to declare the Azure resources

You’ll have to declare the following resources:

A VNET with 2 subnets. One for the private endpoint and the other for the AKS cluster.
An Azure Files storage.
A Private Endpoint for the storage.
A Private DNS Zone and Private DNS Zone Group.
A link between the Private DNS Zone to the VNET.
An AKS cluster.
A role assignment to add the kubelet identity of the cluster as a Contributor to the Storage Account.

in a main.bicep file with the following contents:

param sa_name string = 'akscsisa'
param aks_name string = 'akscsimsft'

// Create the VNET
resource vnet 'Microsoft.Network/virtualNetworks@2020-11-01' = {
  name: 'private-network'
  location: 'westeurope'
  properties: {
    addressSpace: {
      addressPrefixes: [
        '10.0.0.0/8'
      ]
    }
    subnets: [
      {
        name: 'endpoint'
        properties: {
          addressPrefix: '10.241.0.0/16'
          serviceEndpoints: []
          delegations: []
          privateEndpointNetworkPolicies: 'Disabled'
          privateLinkServiceNetworkPolicies: 'Enabled'
        }
      }
      {
        name: 'aks'
        properties: {
          addressPrefix: '10.240.0.0/16'
          serviceEndpoints: []
          delegations: []
          privateEndpointNetworkPolicies: 'Enabled'
          privateLinkServiceNetworkPolicies: 'Enabled'
        }
      }
    ]
    enableDdosProtection: false
  }
}

// Create the File Storage Account
resource sa 'Microsoft.Storage/storageAccounts@2021-01-01' = {
  name: sa_name
  location: 'westeurope'
  sku: {
    name: 'Premium_LRS'
    tier: 'Premium'
  }
  kind: 'FileStorage'
  properties: {
    minimumTlsVersion: 'TLS1_0'
    allowBlobPublicAccess: false
    isHnsEnabled: false
    networkAcls: {
      bypass: 'AzureServices'
      virtualNetworkRules: []
      ipRules: []
      defaultAction: 'Deny'
    }
    supportsHttpsTrafficOnly: true
    accessTier: 'Hot'
  }
}

// Create the Private Enpoint
resource private_endpoint 'Microsoft.Network/privateEndpoints@2020-11-01' = {
  name: 'sa-endpoint'
  location: 'westeurope'
  properties: {
    privateLinkServiceConnections: [
      {
        name: 'sa-privateserviceconnection'
        properties: {
          privateLinkServiceId: sa.id
          groupIds: [
            'file'
          ]
        }
      }
    ]
    subnet: {
      id: '${vnet.id}/subnets/endpoint'
    }
  }
}

// Create the Private DNS Zone
resource dns 'Microsoft.Network/privateDnsZones@2018-09-01' = {
  name: 'privatelink.file.core.windows.net'
  location: 'global'
}

// Link the Private DNS Zone with the VNET
resource vnet_dns_link 'Microsoft.Network/privateDnsZones/virtualNetworkLinks@2018-09-01' = {
  name: '${dns.name}/test'
  location: 'global'
  properties: {
    registrationEnabled: false
    virtualNetwork: {
      id: vnet.id
    }
  }
}

// Create Private DNS Zone Group 
resource dns_group 'Microsoft.Network/privateEndpoints/privateDnsZoneGroups@2020-11-01' = {
  name: '${private_endpoint.name}/default'
  properties: {
    privateDnsZoneConfigs: [
      {
        name: 'privatelink-file-core-windows-net'
        properties: {
          privateDnsZoneId: dns.id
        }
      }
    ]
  }
}

// Create AKS cluster
resource aks 'Microsoft.ContainerService/managedClusters@2021-02-01' = {
  name: aks_name
  location: 'westeurope'
  identity: {
    type: 'SystemAssigned'
  }
  properties: {
    kubernetesVersion: '1.19.9'
    dnsPrefix: aks_name
    agentPoolProfiles: [
      {
        name: 'default'
        count: 1
        vmSize: 'Standard_D2s_v3'
        osDiskSizeGB: 30
        osDiskType: 'Ephemeral'
        vnetSubnetID: '${vnet.id}/subnets/aks'
        type: 'VirtualMachineScaleSets'
        orchestratorVersion: '1.19.9'
        osType: 'Linux'
        mode: 'System'
      }
    ]
    servicePrincipalProfile: {
      clientId: 'msi'
    }
    addonProfiles: {
      kubeDashboard: {
        enabled: false
      }
    }
    enableRBAC: true
    networkProfile: {
      networkPlugin: 'kubenet'
      networkPolicy: 'calico'
      loadBalancerSku: 'standard'
      podCidr: '10.244.0.0/16'
      serviceCidr: '10.0.0.0/16'
      dnsServiceIP: '10.0.0.10'
      dockerBridgeCidr: '172.17.0.1/16'
    }
    apiServerAccessProfile: {
      enablePrivateCluster: false
    }
  }
}

// Built-in Role Definition IDs
// https://docs.microsoft.com/en-us/azure/role-based-access-control/built-in-roles
var contributor = '/subscriptions/${subscription().subscriptionId}/providers/Microsoft.Authorization/roleDefinitions/b24988ac-6180-42a0-ab88-20f7382dd24c'

// Set AKS kubelet Identity as SA Contributor
resource aks_kubelet_sa_contributor 'Microsoft.Authorization/roleAssignments@2020-04-01-preview' = {
  name: guid('${aks_name}_kubelet_sa_contributor')
  scope: sa
  properties: {
    principalId: reference(aks.id, '2021-02-01', 'Full').properties.identityProfile['kubeletidentity'].objectId
    roleDefinitionId: contributor
  }
}

Deploy the Azure resources

Run the following commands to deploy the Azure resources:

az group create -n private-pvc-test -l westeurope
az deployment group create -f ./main.bicep -g private-pvc-test

After 10 minutes or so you’ll have all resources up and running.

Install Azure CSI Driver

The Azure Files Container Storage Interface (CSI) driver can be installed on the cluster so Azure Kubernetes Service (AKS) can manage the lifecycle of Azure Files shares.

To install the driver run:

az aks get-credentials -n akscsimsft -g private-pvc-test --overwrite-existing
curl -skSL https://raw.githubusercontent.com/kubernetes-sigs/azurefile-csi-driver/v1.5.0/deploy/install-driver.sh | bash -s v1.5.0 --

and check the pods status:

kubectl -n kube-system get pod -o wide --watch -l app=csi-azurefile-controller
kubectl -n kube-system get pod -o wide --watch -l app=csi-azurefile-node

You should find instances of csi-azurefile-controller and csi-azurefile-node running without issues.

Create a Storage Class

Create a file named: storageclass-azurefile-csi.yaml with the following yaml:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: azurefile-csi
provisioner: file.csi.azure.com
allowVolumeExpansion: true
parameters:
  resourceGroup: <resourceGroup>  # optional, only set this when storage account is not in the same resource group as agent node
  storageAccount: <storageAccountName>
  # Check driver parameters here:
  # https://github.com/kubernetes-sigs/azurefile-csi-driver/blob/master/docs/driver-parameters.md
  server: <storageAccountName>.privatelink.file.core.windows.net 
reclaimPolicy: Delete
volumeBindingMode: Immediate
mountOptions:
  - dir_mode=0777
  - file_mode=0777
  - uid=0
  - gid=0
  - mfsymlinks
  - cache=strict  # https://linux.die.net/man/8/mount.cifs
  - nosharesock  # reduce probability of reconnect race
  - actimeo=30  # reduce latency for metadata-heavy workload

Remember to replace the values of <resourceGroup> and <storageAccountName> with the ones used in the previous deployment. (i.e. private-pvc-test and akscsisa)

Now deploy the Storage Class to the cluster:

kubectl apply -f storageclass-azurefile-csi.yaml

Create a Private Volume Claim

Create a Private Volume Claim that uses the storage class. Create pvc.yaml with the following contents:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-azurefile
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: azurefile-csi
  resources:
    requests:
      storage: 100Gi

Deploy the Private Volume Claim to the cluster and check that everything is ok:

kubectl apply -f pvc.yaml
kubectl get pvc

Now feel free to try and mount a volume using the Private Volume Claim: my-azurefile

Hope it helps!!!

Please find a bicep based sample here or if you prefer terraform here

References:

Plan IP addressing for AKS configured with Azure CNI Networking

Carlos Mendible — Fri, 09 Jul 2021 10:00:00 +0000

When configuring Azure Kubernetes Service with Azure Container Network Interface (CNI), every pod gets an IP address of the subnet you’ve configured.

So how do you plan you address space? What factors should you consider?

Each node consumes one IP.
Each pod consumes one IP.
Each internal LoadBalancer Service you anticipate consumes one IP.
Azure reserves 5 IP addresses within each subnet.
The Max pods per node is 250.
The Max pods per nodes lower limit is 10.
30 pods es the minimum per cluster.
Max nodes per cluster is 1000.
When a cluster is upgraded a new node is added as part of the process which requires a minimum of one additional block of IP addresses to be available. Your node count is then n + 1.
When you scale a cluster an additional node is added. Your node count is then n + number-of-additional-scaled-nodes-you-anticipate + 1.

With all that in mind the formula to calculate the number of IPs required for your cluster should look like this:

requiredIPs = (nodes + 1 + scale) + ((nodes + 1 + scale) * maxPods) + isvc

where:

nodes: Number of nodes (default 3)
maxPods: Max pods per node (default 30)
sacale: Number of expected scale nodes
isvc: Number of expected internal LoadBalancer services

To help you with this I’ve created a small console program written in golang: aksip which performs the necessary validations and calculations for you.

Let’s say you want a 50 node cluster with one internal load balancer that also includes provision to scale up an additional 10 nodes:

Just run:

aksip -n 50 -s 10

The output will show that you’ll need 1892 IP addresses and therefore a /21 subnet or larger:

{
"nodes": 50,
"scale": 10,
"maxPods": 30,
"isvc": 1,
"requiredIPs": 1892,
"cidr": "/21"
}

Hope it helps!!!

References:

Running k3s inside WSL2 on a Surface Pro X

Carlos Mendible — Sun, 09 May 2021 17:00:00 +0000

I’m a proud owner of a Surafe Pro X SQ2 which is an ARM64 device. If you’ve been reading me, you know I like to tinker with kubernetes and therefore I needed a solution for this device.

I remembered reading about k3s a lightweight kubernetes distro built for IoT & Edge computing, and decided to give it a try.

Installing k3s in WSL2

Download the binaries

wget https://github.com/k3s-io/k3s/releases/download/v1.19.10%2Bk3s1/k3s-arm64

Rename the file

mv k3s-arm64 k3s

Make the file executable

chmod +x k3s

Move the file to `/usr/local/bin`

sudo mv k3s /usr/local/bin

Copy to k3s config file to `$HOME/.kube`

sudo cp /etc/rancher/k3s/k3s.yaml $HOME/.kube

Make current user owner of the k3s config file

sudo chown $USER $HOME/.kube/k3s.yaml

Running k3s

Run the k3s kubernetes server

sudo k3s server

Test k3s

From another WSL2 console window:

Add the k3s config file to KUBECONFIG environment variable

export KUBECONFIG=$HOME/.kube/config:$HOME/.kube/k3s.yaml

Use k3s context

kubectl config use-context default

Check all running pods

kubectl get po --all-namespaces

you should get an output similar to this one:

NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system helm-install-traefik-mhhfz 0/1 Completed 0 20d
kube-system metrics-server-7b4f8b595-7cfsb 1/1 Running 1 20d
kube-system svclb-traefik-pqn56 2/2 Running 2 20d
kube-system coredns-66c464876b-pnpgm 1/1 Running 1 20d
kube-system local-path-provisioner-7ff9579c6-nhnbj 1/1 Running 6 20d
kube-system traefik-5dd496474-94lt5 1/1 Running 1 20d

Note: k3s uses: local-path-provisioner and saves volume data in the /var/lib/rancher/k3s/data folder

Hope it helps!!!

Deploy AKS + Kubecost with Terraform

Carlos Mendible — Fri, 30 Apr 2021 19:00:00 +0000

This morning I saw this tweet from Mr Brendan Burns:

AKS Cost Monitoring and Governance With Kubecost https://t.co/OStwIBsuPp

— brendandburns (@brendandburns) April 30, 2021

And I’m sure that once you also read through it, you’ll learn that you have to take several steps in order to achieve AKS Cost Monitoring and Governance With Kubecost.

I’m going to try and save you some time, providing you with a basic terraform configuration to help you get up and running in a breeze.

If you want to learn more about Kubecost in the context of AKS and Azure please read the Cost Governance section of the AKS enterprise-scale platform security governance and compliance guidelines.

Deploy AKS and Kubecost with Terraform

1. Create a `provider.tf` file with the following contents:

terraform {
  required_version = "> 0.14"
  required_providers {
    azurerm = {
      version = "= 2.57.0"
    }
    azuread = {
      version = "= 1.4.0"
    }
    kubernetes = {
      version = "= 2.1.0"
    }
    helm = {
      version = "= 2.1.2"
    }
  }
}

provider "azurerm" {
  features {}
}

# Configuring the kubernetes provider
# AKS resource name is aks: azurerm_kubernetes_cluster.aks
provider "kubernetes" {
  host                   = azurerm_kubernetes_cluster.aks.kube_config.0.host
  client_certificate     = base64decode(azurerm_kubernetes_cluster.aks.kube_config.0.client_certificate)
  client_key             = base64decode(azurerm_kubernetes_cluster.aks.kube_config.0.client_key)
  cluster_ca_certificate = base64decode(azurerm_kubernetes_cluster.aks.kube_config.0.cluster_ca_certificate)
}

# Configuring the helm provider
# AKS resource name is aks: azurerm_kubernetes_cluster.aks
provider "helm" {
  kubernetes {
    host                   = azurerm_kubernetes_cluster.aks.kube_config.0.host
    client_certificate     = base64decode(azurerm_kubernetes_cluster.aks.kube_config.0.client_certificate)
    client_key             = base64decode(azurerm_kubernetes_cluster.aks.kube_config.0.client_key)
    cluster_ca_certificate = base64decode(azurerm_kubernetes_cluster.aks.kube_config.0.cluster_ca_certificate)
  }
}

Note that you’ll be using azurerm to deploy Azure services, azuread to create a Service Principal information and the kubernetes and helm provider to install Kubecost.

2. Create a `variables.tf` file with the following contents:

# Location of the services
variable "location" {
  default = "west europe"
}

# Resource Group Name
variable "resource_group" {
  default = "aks-kubecost"
}

# Name of the AKS cluster
variable "aks_name" {
  default = "aksmsftkubecost"
}

# Name of the Servic Principal used by Kubecost
variable "kubecost_sp_name" {
  default = "kubecost"
}

Note: Replace the the default values with your desired location and names.

3. Create a `main.tf` file with the following contents:

# Create Resource Group
resource "azurerm_resource_group" "rg" {
  name     = var.resource_group
  location = var.location
}

# Create VNET for AKS
resource "azurerm_virtual_network" "vnet" {
  name                = "private-network"
  address_space       = ["10.0.0.0/8"]
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
}

# Create the Subnet for AKS.
resource "azurerm_subnet" "aks" {
  name                 = "aks"
  resource_group_name  = azurerm_resource_group.rg.name
  virtual_network_name = azurerm_virtual_network.vnet.name
  address_prefixes     = ["10.240.0.0/16"]
}

# Create the AKS cluster.
# Cause this is a test node_count is set to 1 
resource "azurerm_kubernetes_cluster" "aks" {
  name                = var.aks_name
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  dns_prefix          = var.aks_name

  default_node_pool {
    name            = "default"
    node_count      = 1
    vm_size         = "Standard_D2s_v3"
    os_disk_size_gb = 30
    os_disk_type    = "Ephemeral"
    vnet_subnet_id  = azurerm_subnet.aks.id
  }

  # Using Managed Identity
  identity {
    type = "SystemAssigned"
  }

  network_profile {
    network_plugin = "kubenet"
    network_policy = "calico"
  }

  role_based_access_control {
    enabled = true
  }

  addon_profile {
    kube_dashboard {
      enabled = false
    }
  }
}

# Create Application registration for Kubecost
resource "azuread_application" "kubecost" {
  display_name               = var.kubecost_sp_name
  identifier_uris            = ["http://${var.kubecost_sp_name}"]
}

# Create Service principal for kubecost
resource "azuread_service_principal" "kubecost" {
  application_id = azuread_application.kubecost.application_id
}

# Generate password for the Service Principal
resource "random_password" "passwd" {
  length      = 32
  min_upper   = 4
  min_lower   = 2
  min_numeric = 4
   keepers = {
    aks_app_id = azuread_application.kubecost.id
  }
}

# Create kubecost's Service principal password
resource "azuread_service_principal_password" "main" {
  service_principal_id = azuread_service_principal.kubecost.id
  value                = random_password.passwd.result
  end_date             = "2099-01-01T00:00:00Z"
}

# Get current Subscription
data "azurerm_subscription" "current" {
}

# Create kubecost custom role
resource "azurerm_role_definition" "kubecost" {
  name        = "kubecost_rate_card_query"
  scope       = data.azurerm_subscription.current.id
  description = "kubecost Rate Card query role"

  permissions {
    actions     = [
      "Microsoft.Compute/virtualMachines/vmSizes/read",
      "Microsoft.Resources/subscriptions/locations/read",
      "Microsoft.Resources/providers/read",
      "Microsoft.ContainerService/containerServices/read",
      "Microsoft.Commerce/RateCard/read",
    ]
    not_actions = []
  }

  assignable_scopes = [
    data.azurerm_subscription.current.id
  ]
}

# Assign kubecost's custom role at the subscription level
resource "azurerm_role_assignment" "kubecost" {
  scope                = data.azurerm_subscription.current.id
  role_definition_name = azurerm_role_definition.kubecost.name
  principal_id         = azuread_service_principal.kubecost.object_id
}

4. Create a `kubecost.tf` file with the following contents:

# Create the kubecost namespace
resource "kubernetes_namespace" "kubecost" {
  metadata {
    name = "kubecost"
  }
}

# Install kubecost using the hem chart
resource "helm_release" "kubecost" {
  name       = "kubecost"
  chart      = "cost-analyzer"
  namespace  = "kubecost"
  version    = "1.79.1"
  repository = "https://kubecost.github.io/cost-analyzer/"

  # Set the cluster name
  set {
    name  = "kubecostProductConfigs.clusterName"
    value = var.aks_name
  }

  # Set the currency
  set {
    name  = "kubecostProductConfigs.currencyCode"
    value = "EUR"
  }

  # Set the region
  set {
    name  = "kubecostProductConfigs.azureBillingRegion"
    value = "NL"
  }

  # Generate a secret based on the Azure configuration provided below
  set {
    name  = "kubecostProductConfigs.createServiceKeySecret"
    value = true
  }

  # Azure Subscription ID
  set {
    name  = "kubecostProductConfigs.azureSubscriptionID"
    value = data.azurerm_subscription.current.id
  }

  # Azure Client ID
  set {
    name  = "kubecostProductConfigs.azureClientID"
    value = azuread_application.kubecost.application_id
  }

  # Azure Client Password
  set {
    name  = "kubecostProductConfigs.azureClientPassword"
    value = random_password.passwd.result
  }

  # Azure Tenant ID
  set {
    name  = "kubecostProductConfigs.azureTenantID"
    value = data.azurerm_subscription.current.tenant_id
  }
}

The configuration in the previous file installs Kubecost in the AKS cluster. If you want to learn more about the available configuration options please check the following file: values.yaml

5. Deploy the solution:

Run the following commands:

terraform init
terraform plan -out tf.plan
terraform apply ./tf.plan

6. Test and browse Kubecost:

To check the status of the kubecost pods run:

az aks get-credentials -g aks-kubecost -n aksmsftkubecost
kubectl get pods -n kubecost

Then run:

kubectl port-forward -n kubecost svc/kubecost-cost-analyzer 9090:9090

and browse to http://localhost:9090 so you can start learning!

Hope it helps! Please find the complete code here

Deploy a Private Azure Cloud Shell with Terraform

Carlos Mendible — Mon, 12 Apr 2021 10:00:00 +0000

By default Cloud Shell sessions run inside a container inside a Microsoft network separate from any resources you may have deployed in Azure. So what happens when you want to access services you have deployed inside a Virtual Network such as a private AKS cluster, a Virtual Machine or Private Endpoint enabled services?

Well as you can imagine the solution is to deploy Cloud Shell into an Azure Virtual Network in such a way that the container, it runs on, can acces your private resources. This solution also protects the backing Storage Account that Cloud Shell uses for you the user profiles and data so that you end up with a locked down environmemt.

The following diagram shows the solution archictecture:

If you need more information about the solution, the services and any limitations please check the documentation here: Cloud Shell in an Azure Virtual Network.

Now let’s se how can we use terraform to make your Cloud Shell private!

Deploy a Private Azure Cloud Shell with Terraform

If Cloud Shell has been used in the past, the existing clouddrive must be unmounted. To do this run clouddrive unmount from an active Cloud Shell session.

1. Create a `provider.tf` file with the following contents:

terraform {
  required_version = ">= 0.13.5"
}

provider "azurerm" {
  version = "= 2.46.1"
  features {}
}

provider "azuread" {
  version = "= 1.3.0"
}

provider "http" {
  version = "= 2.0.0"
}

Note that we are using azurerm to deploy Azure services, azuread to get some Service Principal information and http to get your current public ip address so that only you can reach your Cloud Shell.

2. Create a variables.tf file with the following contents:

variable location {
  default = "west europe"
}

variable resource_group {
  default = "<resource group name>"
}

variable "vnet_name" {
  default = "<vnet name>"
}

variable sa_name {
  default = "<Backing Storage Account name>"
}

variable relay_name {
  default = "<Azure Relay name>"
}

Make sure you replace the placeholders with the default values you want to use.

3. Create a `main.tf` file with the following contents:

# Get Azure Container Instance Service Principal. Amazing right? Cloud Shell uses this Service Principal!
data "azuread_service_principal" "container" {
  display_name = "Azure Container Instance Service"
}

# Create Resource Groupto hold the resources.
resource "azurerm_resource_group" "rg" {
  name     = var.resource_group
  location = var.location
}

# Create a VNET.
resource "azurerm_virtual_network" "vnet" {
  name                = var.vnet_name
  address_space       = ["10.0.0.0/16"]
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
}

# Create a Containers Subnet. Here is where Cloud Shell will run.
resource "azurerm_subnet" "containers" {
  name                 = "cloudshell-containers"
  resource_group_name  = azurerm_resource_group.rg.name
  virtual_network_name = azurerm_virtual_network.vnet.name
  address_prefixes     = ["10.0.0.0/24"]

  # Delegate the subnet to "Microsoft.ContainerInstance/containerGroups".
  delegation {
    name = "cloudshell-delegation"

    service_delegation {
      name = "Microsoft.ContainerInstance/containerGroups"
    }
  }

  # Add service enpoint so Cloud Shell can reach Storage Accounts. At the moment the solution does not work with Private Enpoints for the Storage Account. 
  service_endpoints = ["Microsoft.Storage"]
}

# Create a subnet to host Azure Relay service.
resource "azurerm_subnet" "relay" {
  name                 = "relay"
  resource_group_name  = azurerm_resource_group.rg.name
  virtual_network_name = azurerm_virtual_network.vnet.name
  address_prefixes     = ["10.0.1.0/24"]

  enforce_private_link_endpoint_network_policies = true
}

# Create a network profile for the Cloud Shell containers.
resource "azurerm_network_profile" "networkprofile" {
  name                = "cloudshell"
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name

  container_network_interface {
    name = "cloudshell-containers"

    ip_configuration {
      name      = "ipconfig"
      subnet_id = azurerm_subnet.containers.id
    }
  }
}

# Assign Network Contributor to the Azure Container Instance Service Principal.
resource "azurerm_role_assignment" "network_contributor" {
  scope                = azurerm_network_profile.networkprofile.id
  role_definition_name = "Network Contributor"
  principal_id         = data.azuread_service_principal.container.object_id
}

# Create an Azure Relay namespace.
resource "azurerm_relay_namespace" "relay" {
  name                = var.relay_name
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name

  sku_name = "Standard"
}

# Add a private enpoint to the Azure Relay namespace.
resource "azurerm_private_endpoint" "endpoint" {
  name                = "cloudshell-privateendpoint"
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  subnet_id           = azurerm_subnet.relay.id

  private_service_connection {
    name                           = "privateendpoint"
    private_connection_resource_id = azurerm_relay_namespace.relay.id
    is_manual_connection           = false
    subresource_names              = ["namespace"]
  }
}

# Assign Contributor to the Azure Container Instance Service Principal.
resource "azurerm_role_assignment" "contributor" {
  scope                = azurerm_relay_namespace.relay.id
  role_definition_name = "Contributor"
  principal_id         = data.azuread_service_principal.container.object_id
}

# Create the Storage Account to hold the Cloud Shell profiles.
resource "azurerm_storage_account" "sa" {
  name                      = var.sa_name
  resource_group_name       = azurerm_resource_group.rg.name
  location                  = azurerm_resource_group.rg.location
  account_tier              = "Standard"
  account_replication_type  = "GRS"
  enable_https_traffic_only = true
}

# Create a file share to hold the user profiles.
resource "azurerm_storage_share" "share" {
  name                 = "profile"
  storage_account_name = azurerm_storage_account.sa.name
  quota                = 6
}

# Get your current public IP.
data "http" "current_public_ip" {
  url = "http://ipinfo.io/json"
  request_headers = {
    Accept = "application/json"
  }
}

# Protect the Storage Account setting the firewall.
# This is done only after the file share is created.
resource "azurerm_storage_account_network_rules" "sa_rules" {
  resource_group_name  = azurerm_resource_group.rg.name
  storage_account_name = azurerm_storage_account.sa.name

  default_action             = "Deny"
  virtual_network_subnet_ids = [azurerm_subnet.containers.id]

  # ip_rules = [
  #   jsondecode(data.http.current_public_ip.body).ip
  # ]

  depends_on = [
    azurerm_storage_share.share
  ]
}

# Create DNS Zone for Relay
resource "azurerm_private_dns_zone" "private" {
  name                = "privatelink.servicebus.windows.net"
  resource_group_name = azurerm_resource_group.rg.name
}

# Create A record for the Relay
resource "azurerm_private_dns_a_record" "relay" {
  name                = var.relay_name
  zone_name           = azurerm_private_dns_zone.private.name
  resource_group_name = azurerm_resource_group.rg.name
  ttl                 = 3600
  records             = [azurerm_private_endpoint.endpoint.private_service_connection[0].private_ip_address]
}

# Link the Private Zone with the VNet
resource "azurerm_private_dns_zone_virtual_network_link" "relay" {
  name                  = "relay"
  resource_group_name   = azurerm_resource_group.rg.name
  private_dns_zone_name = azurerm_private_dns_zone.private.name
  virtual_network_id    = azurerm_virtual_network.vnet.id
}

# Open the relay firewall to local IP
resource "null_resource" "open_relay_firewall" {
  provisioner "local-exec" {
    interpreter = ["powershell"]
    command = "az rest --method put --uri '${azurerm_relay_namespace.relay.id}/networkrulesets/default?api-version=2017-04-01' --body '{\"properties\":{\"defaultAction\":\\\"Deny\\\",\"ipRules\":[{\"ipMask\":\\\"${jsondecode(data.http.current_public_ip.body).ip}\\\"}],\"virtualNetworkRules\":[],\"trustedServiceAccessEnabled\":false}}'"
  }
  depends_on = [
    data.http.current_public_ip,
    azurerm_relay_namespace.relay
  ]
}

4. Deploy the solution:

Run the following commands:

terraform init
terraform plan -out tf.plan
terraform apply ./tf.plan

Hope it helps! Please find the complete code here

ASP.NET Core OpenTelemetry Logging

Carlos Mendible — Fri, 08 Jan 2021 10:00:00 +0000

As you may know I’ve been collaborating with Dapr and I’ve learned that one of the things it enables you to do is to collect traces with the use of the OpenTelemetry Collector and push the events to Azure Application Insights.

After some reading I went and check if I could also write my ASP.NET Core applications to log using the OpenTelemetry Log and Event record definition:

Field Name	Description
Timestamp	Time when the event occurred.
TraceId	Request trace id.
SpanId	Request span id.
TraceFlags	W3C trace flag.
SeverityText	The severity text (also known as log level).
SeverityNumber	Numerical value of the severity.
Name	Short event identifier.
Body	The body of the log record.
Resource	Describes the source of the log.
Attributes	Additional information about the event.

So here is how you can do it:

Create an ASP.NET Core application

dotnet new webapi -n aspnet.opentelemetry
cd aspnet.opentelemetry

Add a reference to the OpenTelemtry libraries

dotnet add package OpenTelemetry.Extensions.Hosting --prerelease
dotnet add package OpenTelemetry.Exporter.Console --prerelease

Modify the CreateHostBuilder method in Program.cs

public static IHostBuilder CreateHostBuilder(string[] args) =>
  Host.CreateDefaultBuilder(args)
      .ConfigureLogging(logging =>
      {
          logging.ClearProviders();
          logging.AddOpenTelemetry(options =>
          {
              options.AddProcessor(new SimpleExportProcessor<LogRecord>(new ConsoleExporter<LogRecord>(new ConsoleExporterOptions())));
          });
      })
      .ConfigureWebHostDefaults(webBuilder =>
      {
          webBuilder.UseStartup<Startup>();
      });

The code first clears all logging providers and then adds OpenTelemetry using the SimpleExportProcessor and ConsoleExporter.

Please check the OpenTelemetry .NET API repo to learn more.

Modify the log settings

Edit the appsettings.Development.json in order to configure the default log settings using the OpenTelemetry provider:

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft": "Warning",
      "Microsoft.Hosting.Lifetime": "Information"
    }
  },
  "AllowedHosts": "*"
}

Test the program

Run the following command to test the application.

dotnet run

You should see the logs, formatted as OpenTelemtry log records, written in the console:


LogRecord.TraceId:	00000000000000000000000000000000
LogRecord.SpanId:	0000000000000000
LogRecord.Timestamp:	2021-01-08T10:36:26.1338054Z
LogRecord.EventId:	0
LogRecord.CategoryName:	Microsoft.Hosting.Lifetime
LogRecord.LogLevel:	Information
LogRecord.TraceFlags:	None
LogRecord.State:	Application is shutting down…

Both TraceId and SpanId will get filled when a request is handled by your application and if the call is made by Dapr it will respect the values sent by it so you can correlate traces and logs improving the observability of your solution.

Hope it helps! and please find the complete code here

Dapr: Reading local secrets with .NET 5

Carlos Mendible — Wed, 18 Nov 2020 10:00:00 +0000

Now that Dapr is about to hit version 1.0.0 let me show you how easy is to read secrets with a .NET 5 console application.

Create a console application

dotnet new console -n DaprSecretSample
cd DaprSecretSample

Add a reference to the Dapr.Client library

dotnet add package Dapr.Client --prerelease

Create a Secret Store component

Create a components folder and inside place a file named secretstore.yaml with the following contents:

apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
  name: starwarssecrets
spec:
  type: secretstores.local.file
  metadata:
    - name: secretsFile
      value: ./secrets.json

This component will enable Dapr, and therefore your application, to read secrets from a secrets.json file.

Create a Secrets file

Create a secrets.json file with the following contents:

{
    "mandalorianSecret": "this is the way!"
}

Replace the contents of Program.cs

Replace the contents of Program.cs with the following code:

using System;
using Dapr.Client;

var client = new DaprClientBuilder().Build();
var secret = await client.GetSecretAsync("starwarssecrets", "mandalorianSecret");
Console.WriteLine($"Secret from local file: {secret["mandalorianSecret"]});

Test the program

Run the following command to test the application.

dapr run --app-id secretapp --components-path .\components\ -- dotnet run

Experiment

One of the amazing things about Dapr is that you code will be the same even if you change the underlying secret store.

In your local environment you can also try reading “secrets” from environment variables. In order to do so, replace the contents of the ./componentes/secrets.yaml file with:

apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
  name: starwarssecrets
spec:
  type: secretstores.local.env

Be sure to set, in your system, an environment variable named mandalorianSecret, for instance:

export mandalorianSecret="May The Force be with you"

and run the application again:

dapr run --app-id secretapp --components-path .\components\ -- dotnet run

Note: I recommend using the secret stores shown in this post only for development or test scenarios. For a complete list of supported secret stores check the following repo: https://github.com/dapr/components-contrib/tree/master/secretstores.

Hope it helps!

What I Learned From Hacktoberfest 2020

Carlos Mendible — Sun, 18 Oct 2020 10:00:00 +0000

Hacktoberfest® is an open global event where people all around de globe contribute to open source projects.

The idea behind Hacktoberfest® is great, in my opinion it encourages and motivates contributions specially from those who don’t know where to start with OSS, but saddly what we saw this year was many people, let’s call them trolls, spamming repos with useless pull requests in order to claim the nice tee. The Hacktoberfest® organization reacted quickly to fix the situation and the rules of the game have been changed: the event is now offically opt-in only for projects and mantainers.

Now let me tell you a bit of what I did:

What I Learned From Hacktoberfest

Azure Arc enabled Kubernetes:

I learned how to enable container monitoring on Azure Arc enabled Kubernetes and in the process I noticed that in the documentation a powershell command was using curl instead of Invoke-WebRequest so I sumbited the following PR: Replacing wget with iwr in the powershell sample

K8Spin

I continued learning about K8Spin: a nice OOS project which adds three new hierarchy concepts (Organizations, Tenants, and Spaces) to your kuberneets clusters in order to enable multi-tenancy.

Collaborating with the K8Spin team was great and I fixed two issues:

A broken link in one of their repos: Fixing Issue#1: oneinfra link
Added a Helm chart to install the K8Spin. I had created some charts in the past but for this one I had to learn about chart best practices and apply them.

As a bonus also added a PR to add a developer container to their repo in other to enable Codespaces and help any newcomer to start contributing avoiding python and other dependencies setup: Adding dev container definition

Bonus

I continued my journey learning and collaborating with Dapr and this time I submited a PR to remove a component from Dapr
I hade some SEO issues with my blog and it turns out the the theme I use was not adding [the post title as part of the correspoing tag(https://github.com/avianto/hugo-kiera/pull/41).

Glad to Help!

Managing Terraform Cloud with .NET Core

Carlos Mendible — Wed, 09 Sep 2020 10:00:00 +0000

Today I’m going to show you how to manage Terraform Cloud with .NET Core using the Tfe.NetClient library.

The idea is to create a simple console application that will:

Add GitHub as a VCS Provider.
Create a Workspace conected to a GitHub repo where your Terraform files live.
Create a variable in the workspace.
Create a Run (Plan) based on the Terraform files
Apply the Run.

Tfe.NetClient is still in alpha and not every Terraform Cloud API or feature is present. Please feel free to submit any issues, bugs or pull requests.

Prerequisites

A Terraform Cloud Account (You can create a Free Organization)
A Terraform Cloud Organization
A Terraform Cloud Organization Token
A Terraform Cloud Team Token
A GitHub Personal Access Token with the following permissions:
- repo: repo:status Access commit status
- repo: repo_deployment Access deployment status
- repo: public_repo Access public repositories
- repo: repo:invite Access repository invitations
- repo: security_events Read and write security events
- workflow
A GitHub Repo with the Terraform script you want to run.

1. Create a folder for your new project

Open a command prompt an run:

mkdir TerraformCloud

2. Create the project

cd TerraformCloud
dotnet new console

3. Add a reference to Tfe.NetClient

dotnet add package Tfe.NetClient -v 0.1.0
dotnet restore

4. Replace the contents of Program.cs with the following code

namespace TerraformCloud
{
    using System;
    using System.Net.Http;
    using System.Threading.Tasks;
    using Tfe.NetClient;
    using Tfe.NetClient.OAuthClients;
    using Tfe.NetClient.Runs;
    using Tfe.NetClient.Workspaces;
    using Tfe.NetClient.WorkspaceVariables;

    class Program
    {
        static async Task Main(string[] args)
        {
            // The values of this variables are hardcoded here just for simplicity and should be retrieved from configuration.
            var organizationName = "<organizationName>";
            var organizationToken = "<organizationToken>";
            var teamToken = "<teamToken>";
            var gitHubToken = "<GitHub Personal Access Token>";
            var gitHubRepo = "<github user or organization name>/<repo name>"; // i.e. cmendible/terraform-hello-world

            // Create an HttpClient
            var httpClient = new HttpClient();

            // Create the Configiration used by the TFE client.
            // For management tasks you'll need to connect to Terraform Cloud using an Organization Token.
            var config = new TfeConfig(organizationToken, httpClient);

            // Create the TFE client.
            var client = new TfeClient(config);

            // Connect Terraform Cloud and GitHub adding GitHub as a VCS Provider.
            var oauthClientsRequest = new OAuthClientsRequest();
            oauthClientsRequest.Data.Attributes.ServiceProvider = "github";
            oauthClientsRequest.Data.Attributes.HttpUrl = new Uri("https://github.com");
            oauthClientsRequest.Data.Attributes.ApiUrl = new Uri("https://api.github.com");
            oauthClientsRequest.Data.Attributes.OAuthTokenString = gitHubToken; // Use the GitHub Personal Access Token
            var oauthResult = await client.OAuthClient.CreateAsync(organizationName, oauthClientsRequest);

            // Get the OAuthToken.
            var oauthTokenId = oauthResult.Data.Relationships.OAuthTokens.Data[0].Id;

            // Create a Workspace connected to a GitHub repo.
            var workspacesRequest = new WorkspacesRequest();
            workspacesRequest.Data.Attributes.Name = "my-workspace";
            workspacesRequest.Data.Attributes.VcsRepo = new RequestVcsRepo();
            workspacesRequest.Data.Attributes.VcsRepo.Identifier = gitHubRepo; // Use the GitHub Repo
            workspacesRequest.Data.Attributes.VcsRepo.OauthTokenId = oauthTokenId;
            workspacesRequest.Data.Attributes.VcsRepo.Branch = "";
            workspacesRequest.Data.Attributes.VcsRepo.DefaultBranch = true;
            var workspaceResult = await client.Workspace.CreateAsync(organizationName, workspacesRequest);

            // Get the Workspace Id so we can add variales or request a plan or apply.
            var workspaceId = workspaceResult.Data.Id;

            // Create a variable in the workspace.
            // You can make the values invible setting the Sensitive attribute to true.
            // If you want to se an environement variable change the Category attribute to "env".
            // You'll have to create as any variables your script needs.
            var workspaceVariablesRequest = new WorkspaceVariablesRequest();
            workspaceVariablesRequest.Data.Attributes.Key = "variable_1";
            workspaceVariablesRequest.Data.Attributes.Value = "variable_1_value";
            workspaceVariablesRequest.Data.Attributes.Description = "variable_1 description";
            workspaceVariablesRequest.Data.Attributes.Category = "terraform";
            workspaceVariablesRequest.Data.Attributes.Hcl = false;
            workspaceVariablesRequest.Data.Attributes.Sensitive = false;
            var variableResult = await client.WorkspaceVariable.CreateAsync(workspaceId, workspaceVariablesRequest);

            // Get the workspace by name.
            var workspace = client.Workspace.ShowAsync(organizationName, "my-workspace");

            // To create Runs and Apply thme you need to use a Team Token.
            // So create a new TfeClient.
            var runsClient = new TfeClient(new TfeConfig(teamToken, new HttpClient()));

            // Create the Run.
            // This is th equivalent to running: terraform plan. 
            var runsRequest = new RunsRequest();
            runsRequest.Data.Attributes.IsDestroy = false;
            runsRequest.Data.Attributes.Message = "Triggered by .NET Core";
            runsRequest.Data.Relationships.Workspace.Data.Type = "workspaces";
            runsRequest.Data.Relationships.Workspace.Data.Id = workspace.Result.Data.Id;
            var runsResult = await runsClient.Run.CreateAsync(runsRequest);

            // Get the Run Id. You'll need it to check teh state of the run and Apply it if possible.
            var runId = runsResult.Data.Id;

            var ready = false;
            while (!ready)
            {
                // Wait for the Run to be planned .
                await Task.Delay(5000);
                var run = await client.Run.ShowAsync(runId);
                ready = run.Data.Attributes.Status == "planned";

                // Throw an exception if the Run status is: errored.
                if (run.Data.Attributes.Status == "errored") {
                    throw new Exception("Plan failed...");
                }
            }

            // If the Run is planned then Apply your configuration.
            if (ready)
            {
                await runsClient.Run.ApplyAsync(runId, null);
            }
        }
    }
}

Tfe.NetClient is still in early stages of development and the resulting code is very verbose and prone to errors. We will address this in a future relases introducing the use of enums and perhaps a fluent API.

5. Run the program

Run the following command:

dotnet run

6. Check the results

Hope it helps!

Azure Functions: use Blob Trigger with Private Endpoint

Carlos Mendible — Mon, 18 May 2020 00:00:00 +0000

The intent of this post is to help you understand how to connect an Azure Function to a Storage Account privately so all traffic flows through a VNet therefore enhancing the security of your solutions and blobs.

The Case:

Supose you have the following Azure Function written in C# which only copies a blob from one conatiner to another:

using System.IO;
using Microsoft.Azure.WebJobs;
using Microsoft.Extensions.Logging;

namespace Secured.Function
{
    public static class SecureCopy
    {
        [FunctionName("SecureCopy")]
        public static void Run(
        [BlobTrigger("input/{name}", Connection = "privatecfm_STORAGE")] Stream myBlob,
        [Blob("output/{name}", FileAccess.Write, Connection = "privatecfm_STORAGE")] Stream copy,
        ILogger log)
        {
            myBlob.CopyTo(copy);
        }
    }

In this case the storage account used, for the blob trigger and the output binding, has a public endpoint exposed to the internet, which you can secure using features such as the Storage Account Firewall and the new private endpoints which will allow clients on a virtual network (VNet) to securely access data over a Private Link. The private endpoint uses an IP address from the VNet address space for your storage account service. [1]

With those features we can lockdown all inbound traffic to the Storage Account to only accept calls from inside a VNet, so the next step is to enable a feature for Azure Functions that will give your function app access to the resources in the VNet: Azure App Service VNet Integration feature [2].

The following sketch shows how this works:

The Azure Function is integrated with a VNet using Regional VNet Integration (blue line).
The Storage Account (shown on the right) has a Private Endpoint which assigns a private IP to the Storage Account.
Traffic (red line) from the Azure Function flows through the VNet, the Private Endpoint and reaches the Storage Account.
The Storage Account, shown on the left, is used for the core services of the Azure Function and, at the time ow writing, can’t be protected using private enpoints.

But wait there is one more thing, you will need to add an Azure Private DNS Zone to enable the Azure Function to resolve the name of the Storage Account so it uses the private ip for communication.)

Note: The solution will require use of the PremiumV2, or Elastic Premium pricing plan for the Azure Function.

Deploying the Infrastructure with Terraform

We’ll be using Terraform (version > 0.12) to deploy the solution. Start creating a providers.tf file with the following contents:

terraform {
  required_version = "> 0.12"
}

provider "azurerm" {
  version = ">= 2.0"
  features {}
}

Define the following variables in a variables.tf file:

# Azure Resource Location
variable location {
  default = "west europe"
}

# Azure Resource Group Name
variable resource_group {
  default = "private-endpoint"
}

# Name of the Storage Account you´ll expose through the private endpoint
variable sa_name {
  default = "privatecfm"
}

# Name of the Storage Account backing the Azure Function
variable function_required_sa {
  default = "privatecfmfunc"
}

Create a mainty.tf file with the following contents (Make sure you read the comments to understand the manifest):

# Create Resource Group
resource "azurerm_resource_group" "rg" {
  name     = var.resource_group
  location = var.location
}

# Create VNet
resource "azurerm_virtual_network" "vnet" {
  name                = "private-network"
  address_space       = ["10.0.0.0/16"]
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  # Use Private DNS Zone. That's right we have to add this magical IP here.
  dns_servers = ["168.63.129.16"]
}

# Create the Subnet for the Azure Function. This is thge subnet where we'll enable Vnet Integration.
resource "azurerm_subnet" "service" {
  name                 = "service"
  resource_group_name  = azurerm_resource_group.rg.name
  virtual_network_name = azurerm_virtual_network.vnet.name
  address_prefixes     = ["10.0.1.0/24"]

  enforce_private_link_service_network_policies = true

  # Delegate the subnet to "Microsoft.Web/serverFarms"
  delegation {
    name = "acctestdelegation"

    service_delegation {
      name    = "Microsoft.Web/serverFarms"
      actions = ["Microsoft.Network/virtualNetworks/subnets/action"]
    }
  }
}

# Create the Subnet for the private endpoints. This is where the IP of the private enpoint will live.
resource "azurerm_subnet" "endpoint" {
  name                 = "endpoint"
  resource_group_name  = azurerm_resource_group.rg.name
  virtual_network_name = azurerm_virtual_network.vnet.name
  address_prefixes     = ["10.0.2.0/24"]

  enforce_private_link_endpoint_network_policies = true
}

# Get current public IP. We´ll need this so we can access the Storage Account from our PC.
data "http" "current_public_ip" {
  url = "http://ipinfo.io/json"
  request_headers = {
    Accept = "application/json"
  }
}

# Create the "private" Storage Account.
resource "azurerm_storage_account" "sa" {
  name                      = var.sa_name
  resource_group_name       = azurerm_resource_group.rg.name
  location                  = azurerm_resource_group.rg.location
  account_tier              = "Standard"
  account_replication_type  = "GRS"
  enable_https_traffic_only = true
  # We are enabling the firewall only allowing traffic from our PC's public IP.
  network_rules {
    default_action             = "Deny"
    virtual_network_subnet_ids = []
    ip_rules = [
      jsondecode(data.http.current_public_ip.body).ip
    ]
  }
}

# Create input container
resource "azurerm_storage_container" "input" {
  name                  = "input"
  container_access_type = "private"
  storage_account_name  = azurerm_storage_account.sa.name
}

# Create output container
resource "azurerm_storage_container" "output" {
  name                  = "output"
  container_access_type = "private"
  storage_account_name  = azurerm_storage_account.sa.name
}

# Create the Private endpoint. This is where the Storage account gets a private IP inside the VNet.
resource "azurerm_private_endpoint" "endpoint" {
  name                = "sa-endpoint"
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  subnet_id           = azurerm_subnet.endpoint.id

  private_service_connection {
    name                           = "sa-privateserviceconnection"
    private_connection_resource_id = azurerm_storage_account.sa.id
    is_manual_connection           = false
    subresource_names              = ["blob"]
  }
}

# Create the blob.core.windows.net Private DNS Zone
resource "azurerm_private_dns_zone" "private" {
  name                = "blob.core.windows.net"
  resource_group_name = azurerm_resource_group.rg.name
}

# Create an A record pointing to the Storage Account private endpoint
resource "azurerm_private_dns_a_record" "sa" {
  name                = var.sa_name
  zone_name           = azurerm_private_dns_zone.private.name
  resource_group_name = azurerm_resource_group.rg.name
  ttl                 = 3600
  records             = [azurerm_private_endpoint.endpoint.private_service_connection[0].private_ip_address]
}

# Link the Private Zone with the VNet
resource "azurerm_private_dns_zone_virtual_network_link" "sa" {
  name                  = "test"
  resource_group_name   = azurerm_resource_group.rg.name
  private_dns_zone_name = azurerm_private_dns_zone.private.name
  virtual_network_id    = azurerm_virtual_network.vnet.id
}

# Create the Storage Account required by Azure Functions.
resource "azurerm_storage_account" "function_required_sa" {
  name                      = var.function_required_sa
  resource_group_name       = azurerm_resource_group.rg.name
  location                  = azurerm_resource_group.rg.location
  account_tier              = "Standard"
  account_replication_type  = "GRS"
  enable_https_traffic_only = true
}

# Create a container to hold the Azure Function Zip
resource "azurerm_storage_container" "functions" {
  name                  = "function-releases"
  storage_account_name  = azurerm_storage_account.function_required_sa.name
  container_access_type = "private"
}

# Create a blob with the Azure Function zip
resource "azurerm_storage_blob" "function" {
  name                   = "securecopy.zip"
  storage_account_name   = azurerm_storage_account.function_required_sa.name
  storage_container_name = azurerm_storage_container.functions.name
  type                   = "Block"
  source                 = "./securecopy.zip"
}

# Create a SAS token so the Fucntion can access the blob and deploy the zip
data "azurerm_storage_account_sas" "sas" {
  connection_string = azurerm_storage_account.function_required_sa.primary_connection_string
  https_only        = false
  resource_types {
    service   = false
    container = false
    object    = true
  }
  services {
    blob  = true
    queue = false
    table = false
    file  = false
  }
  start  = "2020-05-18"
  expiry = "2025-05-18"
  permissions {
    read    = true
    write   = false
    delete  = false
    list    = false
    add     = false
    create  = false
    update  = false
    process = false
  }
}

# Create the Azure Function plan (Elastic Premium) 
resource "azurerm_app_service_plan" "plan" {
  name                = "azure-functions-test-service-plan"
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name

  kind = "elastic"
  sku {
    tier     = "ElasticPremium"
    size     = "EP1"
    capacity = 1
  }
}

# Create Application Insights
resource "azurerm_application_insights" "ai" {
  name                = "func-pe-test"
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  application_type    = "web"
  retention_in_days   = 90
}

# Create the Azure Function App
resource "azurerm_function_app" "func_app" {
  name                       = "func-pe-test"
  location                   = azurerm_resource_group.rg.location
  resource_group_name        = azurerm_resource_group.rg.name
  app_service_plan_id        = azurerm_app_service_plan.plan.id
  storage_account_name       = azurerm_storage_account.function_required_sa.name
  storage_account_access_key = azurerm_storage_account.function_required_sa.primary_access_key
  version                    = "~3"

  app_settings = {
    https_only                     = true
    APPINSIGHTS_INSTRUMENTATIONKEY = azurerm_application_insights.ai.instrumentation_key
    privatecfm_STORAGE             = azurerm_storage_account.sa.primary_connection_string
    # With this setting we'll force all outbound traffic through the VNet
    WEBSITE_VNET_ROUTE_ALL = "1"
    # Properties used to deploy the zip
    HASH            = filesha256("./securecopy.zip")
    WEBSITE_USE_ZIP = "https://${azurerm_storage_account.function_required_sa.name}.blob.core.windows.net/${azurerm_storage_container.functions.name}/${azurerm_storage_blob.function.name}${data.azurerm_storage_account_sas.sas.sas}"
  }
}

# Enable Regional VNet integration. Function --> service Subnet 
resource "azurerm_app_service_virtual_network_swift_connection" "vnet_integration" {
  app_service_id = azurerm_function_app.func_app.id
  subnet_id      = azurerm_subnet.service.id
}

Now download, into your working folder, the securecopy.zip file containing the sample Azure Function or create a zip, with the same name, containing your own code.

Deploy the solution running the following commands:

terraform init
terraform apply

Test the solution

Use the Azure portal or Storage Explorer to upload a file to the input container, after a few seconds you should find a copy of the file in the output container.

VNET Integration Name Resolution Test

If the previous test didn’t work, please connect through KUDU or the Console to the Azure Function and run the following command:

nameresolver <name of the storage account>.blob.core.windows.net

The output of the command should show 10.0.2.4 as the IP address. If that’s not the case you probably misconfigured something. [3]

Hope it helps!

References

[1] Use private endpoints for Azure Storage
[2] Integrate your app with an Azure virtual network: Regional VNET Integration
[3] Integrate your app with an Azure virtual network: Troubleshooting

DEV Community: Carlos Mendible

Azure Quick Review (azqr) v2.9.0 is out!

AKS: Windows node pool with spot virtual machines and ephemeral disks

Define the terraform providers to use

Define the variables

Define the resource group

Define the VNET for the cluster

Define the AKS cluster

Deploy the AKS cluster

Deploy a Windows workload:

AKS: Persistent Volume Claim with an Azure File Storage protected with a Private Endpoint

Create a bicep file to declare the Azure resources

Deploy the Azure resources

Install Azure CSI Driver

Create a Storage Class

Create a Private Volume Claim

Plan IP addressing for AKS configured with Azure CNI Networking

Running k3s inside WSL2 on a Surface Pro X

Installing k3s in WSL2

Download the binaries

Rename the file

Make the file executable

Move the file to /usr/local/bin

Copy to k3s config file to $HOME/.kube

Make current user owner of the k3s config file

Running k3s

Run the k3s kubernetes server

Test k3s

Add the k3s config file to KUBECONFIG environment variable

Use k3s context

Check all running pods

Deploy AKS + Kubecost with Terraform

Deploy AKS and Kubecost with Terraform

1. Create a provider.tf file with the following contents:

2. Create a variables.tf file with the following contents:

3. Create a main.tf file with the following contents:

4. Create a kubecost.tf file with the following contents:

5. Deploy the solution:

6. Test and browse Kubecost:

Deploy a Private Azure Cloud Shell with Terraform

Deploy a Private Azure Cloud Shell with Terraform

1. Create a provider.tf file with the following contents:

2. Create a variables.tf file with the following contents:

3. Create a main.tf file with the following contents:

4. Deploy the solution:

ASP.NET Core OpenTelemetry Logging

Create an ASP.NET Core application

Add a reference to the OpenTelemtry libraries

Modify the CreateHostBuilder method in Program.cs

Modify the log settings

Test the program

Dapr: Reading local secrets with .NET 5

Create a console application

Add a reference to the Dapr.Client library

Create a Secret Store component

Create a Secrets file

Replace the contents of Program.cs

Test the program

Experiment

What I Learned From Hacktoberfest 2020

What I Learned From Hacktoberfest

Azure Arc enabled Kubernetes:

K8Spin

Bonus

Managing Terraform Cloud with .NET Core

Prerequisites

1. Create a folder for your new project

2. Create the project

3. Add a reference to Tfe.NetClient

4. Replace the contents of Program.cs with the following code

5. Run the program

6. Check the results

Azure Functions: use Blob Trigger with Private Endpoint

The Case:

Deploying the Infrastructure with Terraform

Test the solution

VNET Integration Name Resolution Test

References

Move the file to `/usr/local/bin`

Copy to k3s config file to `$HOME/.kube`

1. Create a `provider.tf` file with the following contents:

2. Create a `variables.tf` file with the following contents:

3. Create a `main.tf` file with the following contents:

4. Create a `kubecost.tf` file with the following contents:

1. Create a `provider.tf` file with the following contents:

3. Create a `main.tf` file with the following contents: