Creating a Private Endpoint for Azure Storage Account using Terraform

#terraform #azure #devops #security

Enhancing the security of our infrastructure is paramount. Today, I'll guide you through the process of setting up a private endpoint for Azure Storage Account using Terraform, step by step, leveraging distinct services.
One of the best architectures is highly recommended from MS official as below

Deeping dive to the detail we can see how it goes behind the scenes

We can see that, all almost connections to the Azure services such as: Storage account, Container registry, Keyvault, etc. It should be used private endpoint for more secure.
So, Today we will focus on the Azure Storage Account first. After we creating private endpoint for Storage account it looks like

Let’s start
Step 1: Create a Resource Group

resource "azurerm_resource_group" "rg" {
  name = "rg-sd2488-non-prod-weu-infra"
  location = "westeurope"
  tags = {
    Owner="sd2488"
  }
}

Step 2: Create a Storage Account

resource "azurerm_storage_account" "st" {
  name                     = "stsd2488nonprodweu"
  resource_group_name      = azurerm_resource_group.rg.name
  location                 = azurerm_resource_group.rg.location
  account_tier             = "Standard"
  account_replication_type = "LRS"
  min_tls_version = "TLS1_2"
  network_rules {
    default_action             = "Deny"
    ip_rules                   = []
  }

  tags = {
    Owner="sd2488"
  }
}

Step 3: Create a Virtual Network with Subnets

resource "azurerm_virtual_network" "vnet" {
  name                = "vnet-sd2488-non-prod-weu"
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  address_space       = ["10.1.0.0/16"]  

  tags = {
    Owner="sd2488"
  }
}


resource "azurerm_subnet" "snet_endpoint" {
  name = "PrivateSubnet"
  virtual_network_name = azurerm_virtual_network.vnet.name
  resource_group_name = azurerm_resource_group.rg.name
  address_prefixes      = ["10.1.0.0/24"]  

}

resource "azurerm_subnet" "snet_bas" {
  name = "AzureBastionSubnet"
  virtual_network_name = azurerm_virtual_network.vnet.name
  resource_group_name = azurerm_resource_group.rg.name
  address_prefixes      = ["10.1.1.0/24"]
}

The Virtual Network comprises two subnets:
The PrivateSubnet: This is intended for creating the private endpoint. It also includes a Virtual Machine for testing purposes.
The AzureBastionSubnet: This subnet is designated for the Azure Bastion service, which enables secure connections to Virtual Machines.
Step 4: Set Up Public IP Address and Azure Bastion

resource "azurerm_public_ip" "pip" {
  name                = "pip-sd2488-non-prod-weu"
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  allocation_method   = "Static"
  sku                 = "Standard"
}

resource "azurerm_bastion_host" "bas" {
  name                = "bas-sd2488-non-prod-weu"
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name

  ip_configuration {
    name                 = "bas-configuration"
    subnet_id            = azurerm_subnet.snet_bas.id
    public_ip_address_id = azurerm_public_ip.pip.id
  }
}

Create a Public IP address and configure Azure Bastion.
Step 5: Establish a Private Endpoint
Before creating the private endpoint, generate a Private DNS Zone. Link the DNS Zone with the Virtual Network (VNet) and define the A record within the DNS Zone.

resource "azurerm_private_dns_zone" "pdns_st" {
  name                = "privatelink.blob.core.windows.net"
  resource_group_name = azurerm_resource_group.rg.name
}
Then,


resource "azurerm_private_endpoint" "pep_st" {
  name                = "pep-sd2488-st-non-prod-weu"
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  subnet_id           = azurerm_subnet.snet_endpoint.id

  private_service_connection {
    name                           = "sc-sta"
    private_connection_resource_id = azurerm_storage_account.st.id
    subresource_names              = ["blob"]
    is_manual_connection           = false
  }

  private_dns_zone_group {
    name                 = "dns-group-sta"
    private_dns_zone_ids = [azurerm_private_dns_zone.pdns_st.id]
  }
}

This step, we need to link the DNS Zone with Vnet and define A record in DNS Zone

resource "azurerm_private_dns_zone_virtual_network_link" "dns_vnet_lnk_sta" {
  name                  = "lnk-dns-vnet-sta"
  resource_group_name   = azurerm_resource_group.rg.name
  private_dns_zone_name = azurerm_private_dns_zone.pdns_st.name
  virtual_network_id    = azurerm_virtual_network.vnet.id
}

resource "azurerm_private_dns_a_record" "dns_a_sta" {
  name                = "sta_a_record"
  zone_name           = azurerm_private_dns_zone.pdns_st.name 
  resource_group_name = azurerm_resource_group.rg.name
  ttl                 = 300
  records             = [azurerm_private_endpoint.pep_st.private_service_connection.0.private_ip_address]
}

It seems almost thing done. Now, we are going to create an Virtual Machine and verify
Step 6: Create a Virtual Machine
Before creating Virtual machine, we need to create Network Interface and Network Security Group and link 2 services with each together

resource "azurerm_network_security_group" "nsg" {
  name                = "nsg-sd2488-non-prod-weu"
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name

  tags = {
    Owner="sd2488"
  }
}

resource "azurerm_network_interface" "nic" {
  name                = "nic-sd2488-non-prod-weu"
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name

  ip_configuration {
    name                          = "nic-configuration"
    subnet_id                     = azurerm_subnet.snet_endpoint.id
    private_ip_address_allocation = "Dynamic"
  }
}

resource "azurerm_network_interface_security_group_association" "nsgnic" {
  network_interface_id      = azurerm_network_interface.nic.id
  network_security_group_id = azurerm_network_security_group.nsg.id
}

resource "azurerm_windows_virtual_machine" "vm" {
  name                = "vm-sd2488-non"
  location              = azurerm_resource_group.rg.location
  resource_group_name   = azurerm_resource_group.rg.name
  size                = "Standard_F2"
  admin_username      = "adminuser"
  admin_password      = "P@$$w0rd1234!"  
  network_interface_ids = [
    azurerm_network_interface.nic.id,
  ]

  os_disk {
    caching              = "ReadWrite"
    storage_account_type = "Standard_LRS"
  }

  source_image_reference {
    publisher = "MicrosoftWindowsServer"
    offer     = "WindowsServer"
    sku       = "2019-Datacenter"
    version   = "latest"
  }
}

Creating a Virtual Machine involves setting up a Network Interface and a Network Security Group. These components need to be connected.
Execute the Terraform script with the "terraform apply" command.
After running the command successfully, the resources will be created on Azure portal. It looks like

Step 7: Now, we will verify the result.
Step 7.1: Accessing through Storage Explorer

As we can see the result. If trying to access the Storage Account through Storage Explorer. It should be evident that direct access is prohibited.
Step 7.2: Now, we will use virtual machine that is the same virtual network as Remote Access using Azure Bastion

Using Azure Bastion to remotely access the Azure Virtual Machine is the next step. This entails connecting to the VM through the Azure Portal and selecting the Bastion option.

Install Azure Storage Explorer to access the VM, retrieve the Access Key from the Storage Account, and establish a connection through the Explorer. Create a blob container as an additional validation.

Step 7.3: Using nslookup for FQDN Verification

To confirm the setup, we can use the nslookup command for the FQDN (fully qualified domain name) generated after creating the DNS A record. This step helps ensure that the DNS record is accurately resolving to the expected IP address.
By following these meticulous steps, we create a private endpoint for an Azure Storage Account, significantly bolstering the security of our infrastructure.