Security plays an important role in most cloud projects. A central component of this is the so-called landing zone, which serves as the basis for all other workloads. An essential security aspect in such setups is to only grant public access in exceptional cases. But this is precisely where a challenge often arises: How can I continue to carry out deployments with Azure DevOps from a private network?
Without public access, external tools such as Azure DevOps have no access to the resources in the private network.
In this article, I will show you step by step how this problem can be solved. After reading it, you should be able to implement a similar architecture yourself.
Initial situation: No public access, no deployment?
If we set up a private virtual network (VNET) in Azure and block public access, this initially means that no connections from the Internet to the resources are permitted.
Publicly accessible endpoints always pose a security risk: They can potentially be reached by attackers from the internet, making attacks such as port scanning, brute force attacks or exploits against vulnerabilities possible. For business-critical applications or sensitive data in particular, it is therefore common practice to reduce the attack surface as much as possible by avoiding public access.
A virtual network (VNET) in Azure is a logical isolation within the Azure cloud, comparable to a separate network segment in a traditional data center. You can place resources such as virtual machines (VMs), databases, app services or other services in a VNET and allow them to communicate with each other via private IP addresses.
Network security groups (NSGs), private endpoints and your own routing rules can be used to precisely control which data traffic is permitted and which is not. The VNET acts like a security fence around all the resources it contains.
If we consistently do without public access, services such as Azure DevOps initially face an insurmountable hurdle: by default, pipelines use Microsoft-hosted build agents that run outside our Azure environment. These agents cannot simply “reach into” our private VNET, as they have no access to the private IP addresses and no route into the internal network.
As a result, deployments to private subnets, internal databases or protected API endpoints would fail - even though these resources are actually our target.
We therefore need a way to carry out deployments without compromising the security approach. The solution: We bring the agent into the private VNET, i.e. to where our resources are located. In this way, we can carry out deployments internally and at the same time take full advantage of the benefits of an isolated network.
Step 1: Set up self-hosted agent in the private VNET
Even if the agent is located in the private network, it must be able to connect to Azure DevOps in order to execute builds and download artifacts. To do this, specific exceptions must be allowed in the firewall.
Before you install the agent yourself, your VNET must be prepared accordingly. To do this, you should create your own subnet in which the VM for the agent will later be operated. This subnet should be configured in such a way that it has suitable security guidelines and is cleanly separated from the rest of the network. A Network Security Group (NSG) regulates incoming and outgoing traffic here. Outgoing traffic is primarily relevant for the self-hosted agent, as it has to communicate with Azure DevOps. Incoming traffic usually only requires management access (e.g. via SSH or Azure Bastion) so that you can reach and maintain the VM in the event of an error.
When creating the VM itself, it is important to choose a suitable size and image. In many cases, a medium-sized standard VM, such as the Standard_D2s_v3 type, is sufficient. You can choose between Windows Server or Linux (e.g. Ubuntu) as the operating system, depending on which tools and build environments you require. It is crucial that you mount the VM directly in the prepared subnet and do not assign a public IP address. This ensures that the VM is only accessible internally and does not offer any unnecessary attack surfaces to the outside world.
In order for the agent to be able to communicate with Azure DevOps, it must be allowed to reach certain destinations on the internet despite the private network. This requires specific releases in the firewall or in the NSG. The VM requires access to at least:
- https://dev.azure.com
- https://*.dev.azure.com
- https://aex.dev.azure.com
- weitere Domains
Only through these approvals can the agent register builds, download artifacts and send status messages back to Azure DevOps.
Step 2: Install agent on the VM
As soon as the VM and the firewall rules are in place, you can install the agent in the VM. To do this, download the agent package from Azure DevOps and register the agent via a personal access token (PAT) or via OAuth.
Before you start the actual installation, you should make sure that the VM has a working connection to the internet and that the URLs mentioned in step 1 are accessible. In addition, the operating system must be up-to-date and PowerShell should be installed on Windows systems and an up-to-date shell environment on Linux systems. In many cases, the .NET Core Runtime is also required, especially for newer agent versions.
To authenticate the agent to Azure DevOps, you first create a personal access token. This token is required later so that the agent can connect to your Azure DevOps tenant. You can find the creation in the Azure DevOps portal under your user account (top right), there under Personal access tokens. When creating the token, assign a name, select the desired validity period and specify the required authorizations, in this case mainly agent pools (read & manage). After creating the token, copy it and keep it safe, as it is only displayed once.
The next step is to download the appropriate agent package. To do this, go to the project settings in Azure DevOps, open the Agent Pools section and select the pool in which the new agent should appear. If no pool exists yet, you can also create a new one here. After clicking on New agent, select your operating system and download the ZIP archive provided. Unzip this package into a local directory on the VM, for example C:\azagent on Windows or /opt/azagent on Linux.
Then open a console with administrative rights on the VM, change to the unpacked directory and start the configuration script. Under Windows, this script is called config.cmd, under Linux or macOS config.sh. When you start it, you will be guided through a series of questions: First, enter the URL of your Azure DevOps server (e.g. https://dev.azure.com/mein-unternehmen). Then select the PAT authentication method and insert the previously created token. In the next step, select the agent pool in which the agent is to be registered and assign a name for the agent itself, such as private-vnet-agent-01. Optionally, you can define tags with which you can address the agent later in pipelines.
After the wizard has run, you will be asked whether the agent should be set up as a service. You should definitely confirm this option, as the agent will be started automatically each time the VM is restarted and will remain permanently available. After completing the configuration, you can activate the service on Windows with .\svc install and .\svc start, on Linux accordingly with sudo ./svc.sh install and sudo ./svc.sh start.
Once all steps have been successfully completed, you can check in the Azure DevOps Portal whether the agent is displayed as online in the selected pool. From this moment on, the agent is ready for use, can accept build and release jobs and has access to internal resources via the private VNET without these having to be publicly accessible.
Step 3: Create app registration and service connection
What we are still missing is authentication against Azure. So far, we have only created the agent that runs in the private network and executes our pipeline steps locally. However, in order for this agent to be able to create, change or delete resources in Azure - for example, deploy an app service instance or configure a database - it needs authorizations. This is exactly where the service connection comes into play.
At this point, it is important to clearly understand the difference between agent and service connection: The agent is the “worker” that does the actual work, i.e. performs builds, tests and deployments. The service connection, on the other hand, is the “key” that gives this worker access to Azure resources. Without this key, the agent could theoretically issue commands, but would not have any rights to change anything in Azure.
The service connection is based on an app registration in Microsoft Entra ID (formerly Azure AD). This app registration provides a technical identity that is secured by a secret. It enables Azure DevOps to log in securely to your Azure tenant without having to use a personal user or insecure access data.
To create the app registration, first navigate to Microsoft Entra ID in the Azure portal and select the App registrations section. There you create a new app registration, assign a descriptive name, for example devops-service-connection, and register the application. In most cases, you can leave the options for supported account types at “This organization directory only”; a redirect URI is not required for this use case.
After registering, you must create a secret that serves as the password for this technical identity. Under the menu item Certificates & secrets, you create a new secret, enter a description and select a suitable validity period. After creating it, you will receive a value, which you must copy immediately and save securely, as it will not be displayed again later.
In addition to this secret, you will need two other IDs from the app registration: the application ID (client ID) and the directory ID (tenant ID). Both can be found directly in the app registration overview. Together with the secret, they form the access data that is later stored in Azure DevOps.
In order for the App Registration to actually be allowed to manage resources in Azure, it must be assigned the appropriate authorizations. To do this, you can either go to the entire subscription in the Azure portal or - if you want more granular control - to the respective resource group. Under Access Control (IAM), assign the Contributor role to App Registration. This role allows you to create, change and delete resources. In some cases, it may make sense to choose a more restrictive role or even define a custom role, depending on how exactly you want to restrict the authorizations.
Once the app registration and authorizations have been prepared, the next step is to set up the service connection in Azure DevOps. To do this, open your project in Azure DevOps, go to the project settings, then to the Service connections section and create a new connection of the type Azure Resource Manager. Select the Service principal (manual) option here.
In the next step, enter all previously saved information: the subscription ID, the name of the subscription, the application ID (client ID), the secret and the directory ID (tenant ID). After saving, Azure DevOps automatically checks the connection. If everything is configured correctly, you can then name the service connection, for example Azure-Prod-Connection, and save it.
From this point on, your pipeline has an authorized connection to Azure.
Bottom line
By using a self-hosted agent in a private VNET, a secure and controlled deployment architecture can be set up that does not require public access. The combination with a cleanly configured service connection via an app registration offers the necessary flexibility for builds and deployments.
Even if the initial outlay is somewhat higher, a considerable increase in security and stability is achieved in the long term - a decisive advantage for companies with high security requirements or compliance specifications.
Top comments (0)