Automate Azure DevOps self-hosted agent installation using Terraform

You plan to deploy a virtual machine (VM) to Azure using Terraform, HashiCorp’s Infrastructure as Code (IaC) software tool. The VM will run pipeline builds, and to do this you need to add the VM to an agent pool within Azure DevOps. This post will cover how to use the Azure module for Terraform to automate the self-hosted agent installation for a Linux VM.

You need to make sure you have:

• Azure permissions to upload a file to an Azure Storage Account;
• Azure permissions to create virtual machines and extensions;
• A copy of Terraform to run the deployment;
• Azure DevOps owner or administrator permissions.

Before we start

I won’t be covering the code needed for the virtual machine creation in this post, for now refer to HashiCorp documentation (follow my blog for this future post). Make sure you complete the following before going further into the post:

• Using HashiCorp Configuration Language (HCL), write the specified code you need to create the VM with a Linux operating system (I used Ubuntu). Record the username of the local user you are going to create (record it as MyVmUser);
• Configure the backend to connect to Azure.

Do not deploy the virtual machine, we will do this as a single deployment.

I also expect you to have an Azure DevOps agent pool already created. If you haven't already, then create one following Microsoft's documentation.

What you need to do in Azure DevOps

We need to first record the Agent Pool name you are going to use (record it as MyDevOpsPool). We also need to record the organisation name; this will be within the url address (after dev.azure.com/) you are using to access Azure DevOps (record it as MyDevOpsOrg).

Now we are going to create a Personal Access Token (PAT) for use when running the installation script. Below are steps I took to create this but please check Microsoft's recommendations for any changes:

1. Within Azure DevOps, select user settings at the top right of the screen and select Personal access tokens;
2. Select New Token;
3. Select Show all scopes at the bottom of the windows and make sure only Agent Pools Read & Manage is selected;
4. Complete the rest of the configuration to your needs and select Save;
5. Record the PAT as we will use this for the installation script (record it as MyDevOpsToken)

Linux agent install script

Create a new file, copy the below script and replace the following values with what you have recorded from the previous sections:

• MyVmUser
• MyDevOpsOrg
• MyDevOpsToken
• MyDevOpsPool

I am also specifying the version of agent installation I want to deploy (line 3 & 4) so please make sure to amend if you require a different version.

sudo mkdir /myagent 
cd /myagent
sudo wget https://vstsagentpackage.azureedge.net/agent/2.179.0/vsts-agent-linux-x64-2.179.0.tar.gz
sudo tar zxvf ./vsts-agent-linux-x64-2.179.0.tar.gz
sudo chmod -R 777 /myagent
runuser -l MyVmUser -c '/myagent/config.sh --unattended  --url https://dev.azure.com/MyDevOpsOrg --auth pat --token MyDevOpsToken --pool MyDevOpsPool'
sudo /myagent/svc.sh install
sudo /myagent/svc.sh start
exit 0

Note: I would recommend changing some values into variables to be passed through during deployment. Values like token are sensitive and shouldn't be stored in clear text format. I will not be covering how to do this.

Save the above as extension.sh and upload the file to an Azure Storage Account Container (or an alternative public file storage area) and generate a Shared Access Signature (SAS) and copy the URL for usage within the next section.

HCL Azure Extension

We will now write the section of code needed within our Terraform files. Copy the following:

resource "azurerm_virtual_machine_extension" "devops_agent" {
  name                 = "ext-devops"
  virtual_machine_id   = azurerm_virtual_machine.devops_vm.id
  publisher            = "Microsoft.Azure.Extensions"
  type                 = "CustomScript"
  type_handler_version = "2.0"

  settings = <<SETTINGS
    {
        "fileUris": ["SAS TOKEN"],
        "commandToExecute": "sh extension.sh"
    }
SETTINGS

   depends_on = [
       azurerm_virtual_machine.devops_vm
       ]
}

Replace the value SAS TOKEN with the recorded SAS URL from the previous section. Also replace the following values as they will relate to your virtual machine configuration

• virtual_machine_id
• depends_on

Now deploy

Before you deploy, always run Terraform validate to check for errors in the files and Terraform plan to confirm what will happen as part of deployment. Once you are happy to proceed, then run Terraform apply.

During the deployment, you will see the virtual machine is deployed first and once deployed move onto the extension deployment which is where the DevOps self-hosted agent installs. After deployment completes successfully, you should be able to refresh your agent pool within Azure DevOps and see an agent appear with the same name as your virtual machine and the status as Online.

You now have an DevOps self-hosted agent deployed using Infrastructure as Code. There are many benefits using with this type of automation, something I will cover in future posts.

Tip before we end

If you're going to repeat the above to deploy more than one self-hosted agent, I would then recommend writing this code into a Terraform module. By entering the code into a module, you can reuse the module within the terraform configuration without having to write the code over and over again.

You can find examples of this code in my example's repository on GitHub .