In this lab, you will learn how to configure a project and repository structure in Azure DevOps to support secure pipelines. This lab covers best practices for organizing projects and repositories, assigning permissions, and managing secure files.
Before you start
You’ll need an Azure subscription, Azure DevOps organization, and the eShopOnWeb application to follow the labs.
- Follow the steps to validate your lab environment.
Validate your lab environment
In preparation for the labs, it is crucial to have your environment correctly set up. This section will guide you through the setup process, ensuring all prerequisites are met.
The labs require Microsoft Edge or an Azure DevOps-supported browser.
Set up an Azure Subscription: If you don’t already have an Azure subscription, create one by following the instructions on this page or visit https://azure.microsoft.com/free to sign up for a free.
Set up an Azure DevOps organization: If you don’t already have an Azure DevOps organization that you can use for the labs, create one by following the instructions on this page, or at Create an organization or project collection.
Git for Windows download page`. This will be installed as part of prerequisites for this lab.
Visual Studio Code. This will be installed as part of prerequisites for this lab.
Azure CLI. Install the Azure CLI on the self-hosted agent machines.
.NET SDK - Latest version. Install the .NET SDK on the self-hosted agent machines.
Instructions to create an Azure DevOps Organization (you only have to do this once)
Note: Start at step 3, if you do already have a personal Microsoft Account setup and an active Azure Subscription linked to that account.
1.Use a private browser session to get a new personal Microsoft Account (MSA) at https://account.microsoft.com.
2.Using the same browser session, sign up for a free Azure subscription at https://azure.microsoft.com/free.
3.Open a browser and navigate to Azure portal at https://portal.azure.com, then search at the top of the Azure portal screen for Azure DevOps. In the resulting page, click Azure DevOps organizations.
4.Next, click on the link labelled My Azure DevOps Organizations or navigate directly to https://aex.dev.azure.com.
5.On the We need a few more details page, select Continue.
6.In the drop-down box on the left, choose Default Directory, instead of Microsoft Account.
7.If prompted (“We need a few more details”), provide your name, e-mail address, and location and click Continue.
8.Back at https://aex.dev.azure.com with Default Directory selected click the blue button Create new organization.
9.Accept the Terms of Service by clicking Continue.
10.If prompted (“Almost done”), leave the name for the Azure DevOps organization at default (it needs to be a globally unique name) and pick a hosting location close to you from the list.
11.Once the newly created organization opens in Azure DevOps, select Organization settings in the bottom left corner.
12.At the Organization settings screen select Billing (opening this screen takes a few seconds).
13.Select Setup billing and on the right-hand side of the screen, select your Azure Subscription and then select Save to link the subscription with the organization.
14.Once the screen shows the linked Azure Subscription ID at the top, change the number of Paid parallel jobs for MS Hosted CI/CD from 0 to 1. Then select SAVE button at the bottom.
Note: You may wait a couple of minutes before using the CI/CD capabilities so that the new settings are reflected in the backend. Otherwise, you will still see the message “No hosted parallelism has been purchased or granted”.
15.In Organization Settings, go to section Pipelines and click Settings.
16.Toggle the switch to Off for Disable creation of classic build pipelines and Disable creation of classic release pipelines
Note: The Disable creation of classic release pipelines switch sets to On hides classic release pipeline creation options such as the Release menu in the Pipeline section of DevOps projects.
17.In Organization Settings, go to section Security and click Policies.
18.Toggle the switch to On for Allow public projects
Note: Extensions used in some labs might require a public project to allow using the free version.
That option no longer exists.
Instructions to create and configure the Azure DevOps project (you only have to do this once)Note: make sure you completed the steps to create your Azure DevOps Organization before continuing with these steps.
To follow all lab instructions, you’ll need set up a new Azure DevOps project, create a repository that’s based on the eShopOnWeb application, and create a service connection to your Azure subscription.
Create the team project
First, you’ll create an eShopOnWeb Azure DevOps project to be used by several labs.
1.Open your browser and navigate to your Azure DevOps organization.
2.Select the New Project option and use the following settings:
- name: eShopOnWeb
- visibility: Private
- Advanced: Version Control: Git
- Advanced: Work Item Process: Scrum
3.Select Create.
Import eShopOnWeb git repository
Now, you’ll import the eShopOnWeb into your git repository.
1.Open your browser and navigate to your Azure DevOps organization.
2.Open the previously created eShopOnWeb project.
3.Select the Repos > Files, Import a Repository and then select Import.
4.On the Import a Git Repository window, paste the following URL https://github.com/MicrosoftLearning/eShopOnWeb and select Import:
5.The repository is organized the following way:
- .ado folder contains Azure DevOps YAML pipelines.
- .devcontainer folder container setup to develop using containers (either locally in VS Code or GitHub Codespaces).
- .azure folder contains Bicep & ARM infrastructure as code templates.
- .github folder container YAML GitHub workflow definitions.
- src folder contains the .NET 8 website used on the lab scenarios.
6.Leave the web browser window open.
7.Go to Repos > Branches.
8.Hover on the main branch then click the ellipsis on the right of the column.
9.Click on Set as default branch.
Create a service connection to access Azure resources
Next, you will create a service connection in Azure DevOps which will allow you to deploy and access resources in your Azure subscription.
1.Start a web browser, navigate to the Azure DevOps portal with the eShopOnWeb project open and select Project settings in the bottom left corner of the portal.
2.Select the Service connections under Pipelines, and then select Create service connection button.
3.On the New service connection blade, select Azure Resource Manager and Next (may need to scroll down).
4.Select Workload Identity federation (automatic) and Next.
Note: You can also use Workload identity federation (manual) if you prefer to manually configure the service connection. Follow the steps in the Azure DevOps documentation to create the service connection manually.
5.Fill in the empty fields using the information:
- Subscription: Select your Azure subscription. Then sign in to your azure account.
- Resource group: Select the resource group where you want to deploy resources.
-
Service connection name: Type
azure subs. This name will be referenced in YAML pipelines to access your Azure subscription.
6.Make sure the Grant access permission to all pipelines option is unchecked and select Save.
Note: The Grant access permission to all pipelines option is not recommended for production environments. It is only used in this lab to simplify the configuration of the pipeline.
Note: If you see an error message indicating you don’t have the necessary permissions to create a service connection, try again, or configure the service connection manually.
You have now completed the necessary prerequisite steps to continue with the labs.
Instructions
Exercise 1: Configure a secure project structure
In this exercise, you will configure a secure project structure by creating a new project and assigning it project permissions. Separating responsibilities and resources into different projects or repositories with specific permissions supports security.
Task 1: Create a new team project
1.Navigate to the Azure DevOps portal at https://aex.dev.azure.com and open your organization.
2.Open your organization settings at the bottom left corner of the portal and then Projects under the General section.
3.Select the New Project option and use the following settings:
- name: eShopSecurity
- visibility: Private
- Advanced: Version Control: Git
- Advanced: Work Item Process: Scrum
4.Select Create to create the new project.
5.You can now switch between the different projects by clicking on the Azure DevOps icon in the upper left corner of the Azure DevOps portal.
You can manage permissions and settings for each project separately by going to the Project settings menu and selecting the appropriate team project. If you have multiple users or teams working on different projects, you can also assign permissions to each project separately.
Task 2: Create a new repository and assign project permissions
1.Select the organization name in the upper left corner of the Azure DevOps portal and select the new eShopSecurity project.
2.Select the Repos menu.
3.Select the Initialize button to initialize the new repository by adding the README.md file.
4.Open the Project settings menu in the lower left corner of the portal and select Repositories under the Repos section.
5.Select the new eShopSecurity repository and select the Security tab.
Note: Ensure that you select the Security tab in the specific repository only, and not for all repositories in the project. If you select all repositories, you may lose access to other repositories in the project.
6.Remove the Inherit permissions from parent by unchecking the Inheritance toggle button.
7.Select the Contributors group and select the Deny dropdown for all permissions except Manage permissions and Read. This will prevent all users from the Contributors group from accessing the repository.
Note: In a real world scenario, you will deny the manage permissions to the Contributors group as well. For this lab, we are allowing the Contributors group to manage permissions to allow you to complete the lab.
8.Select your user under Users and select the Allow button to allow all permissions.
Note: If you don’t see your name in the Users section, enter your name in the Search for users or groups text box and select it in the list of results.
9.Your changes will be saved automatically.
Now only the user you assigned permissions and the administrators can access the repository. This is useful when you want to allow specific users to access the repository and run pipelines from the eShopOnWeb project.
Exercise 2: Configure a pipeline and template structure to support secure pipelines
Task 1: Import and run the CI pipeline
Let’s start by importing the CI pipeline named eshoponweb-ci.yml.
1.Navigate to the Azure DevOps portal at https://aex.dev.azure.com and open your organization.
2.Open the eShopOnWeb project in Azure DevOps.
3.Go to Pipelines > Pipelines.
4.Select the Create Pipeline button.
5.Select Azure Repos Git (Yaml).
6.Select the eShopOnWeb repository.
7.Select Existing Azure Pipelines YAML File.
8.Select the /.ado/eshoponweb-ci.yml file then click on Continue.
9.Select the Run button to run the pipeline.
Note: Your pipeline will take a name based on the project name. You will rename it to easier identify the pipeline.
10.Go to Pipelines > Pipelines and select the recently created pipeline. Select the ellipsis and then select Rename/move option.
11.Name it eshoponweb-ci and select Save.
Task 2: Import and run the CD pipeline
Note: In this task, you will import and run the CD pipeline named eshoponweb-cd-webapp-code.yml.
1.Go to Pipelines > Pipelines.
2.Select New pipeline button.
3.Select Azure Repos Git (Yaml).
4.Select the eShopOnWeb repository.
5.Select Existing Azure Pipelines YAML File.
6.Select the /.ado/eshoponweb-cd-webapp-code.yml file then select Continue.
7.In the YAML pipeline definition, set the variables section to:
yaml
variables:
resource-group: 'YOUR-RESOURCE-GROUP-NAME'
location: 'centralus'
templateFile: 'infra/webapp.bicep'
subscriptionid: 'YOUR-SUBSCRIPTION-ID'
azureserviceconnection: 'YOUR-AZURE-SERVICE-CONNECTION-NAME'
webappname: 'YOUR-WEB-APP-NAME'
8.Replace the values of the variables with the values of your environment:
- Replace YOUR-RESOURCE-GROUP-NAME with the name of the resource group you want to use in this lab, for example, rg-eshoponweb-secure.
- Set the value of the location variable to the name of the Azure region you want to deploy your resources, for example, centralus.
- Replace YOUR-SUBSCRIPTION-ID with your Azure subscription id.
- Replace YOUR-AZURE-SERVICE-CONNECTION-NAME with azure subs
- Replace YOUR-WEB-APP-NAME with a globally unique name of the web app to be deployed, for example, the string eshoponweb-lab-multi-123456 followed by a random six-digit number.
9.Select Save and Run and choose to commit directly to the main branch.
10.Select Save and Run again.
11.Open the pipeline run. If you receive the message “This pipeline needs permission to access a resource before this run can continue to Deploy to WebApp”, select View, Permit and Permit again. This is needed to allow the pipeline to create the Azure App Service resource.
12.The deployment may take a few minutes to complete, wait for the pipeline to execute. The pipeline is triggered following the completion of the CI pipeline and it includes the following tasks:
- AzureResourceManagerTemplateDeployment: Deploys the Azure App Service web app using bicep template.
- AzureRmWebAppDeployment: Publishes the Web site to the Azure App Service web app.
Note: In case the deployment fails, navigate to the pipeline run page and select Rerun failed jobs to invoke another pipeline run.
Note: Your pipeline will take a name based on the project name. Let’s rename it for identifying the pipeline better.
13.Go to Pipelines > Pipelines and select the recently created pipeline. Select the ellipsis and then select Rename/move option.
14.Name it eshoponweb-cd-webapp-code and click on Save.
Now you should have two pipelines running in your eShopOnWeb project.
After troubleshooting, the pipeline finally succeeded.
NOTE: Check the Troubleshooting section below to see how I resolved the errors.
Task 3: Move the CD pipeline variables to a YAML template
In this task, you will create a YAML template to store the variables used in the CD pipeline. This will allow you to reuse the template in other pipelines.
1.Go to Repos and then Files.
2.Expand the .ado folder and select New file.
3.Name the file eshoponweb-secure-variables.yml and select Create.
4.Add the variables section used in the CD pipeline to the new file. The file should look like the following:
yaml
variables:
resource-group: 'rg-eshoponweb-secure'
location: 'southcentralus' #the name of the Azure region you want to deploy your resources
templateFile: 'infra/webapp.bicep'
subscriptionid: 'YOUR-SUBSCRIPTION-ID'
azureserviceconnection: 'azure subs' #the name of the service connection to your Azure subscription
webappname: 'eshoponweb-lab-secure-XXXXXX' #the globally unique name of the web app
Important: Replace the values of the variables with the values of your environment (resource group, location, subscription ID, Azure service connection, and web app name).
5.Select Commit, in the commit comment text box, enter [skip ci], and then select Commit.
Note: By adding the
[skip ci]comment to the commit, you will prevent automatic pipeline execution, which, at this point, runs by default following every change to the repo.
6.From the list of files in the repo, open the eshoponweb-cd-webapp-code.yml pipeline definition, and replace the variables section with the following:
yaml
variables:
- template: eshoponweb-secure-variables.yml
7.Select Commit, accept the default comment, and then select Commit to run the pipeline again.
8.Verify that the pipeline run completed successfully.
Now you have a YAML template with the variables used in the CD pipeline. You can reuse this template in other pipelines in scenarios where you need to deploy the same resources. Also, your operations team can control the resource group and location where the resources are deployed and other information in your template values and you don’t need to make any changes to your pipeline definition.
Task 4: Move the YAML templates to a separate repository and project
In this task, you will move the YAML templates to a separate repository and project.
1.In your eShopSecurity project, go to Repos > Files.
2.Create a new file named eshoponweb-secure-variables.yml.
3.Copy the content of the file .ado/eshoponweb-secure-variables.yml from the eShopOnWeb repository to the new file.
4.Commit the changes.
5.Open the eshoponweb-cd-webapp-code.yml pipeline definition in the eShopOnWeb repo.
6.Add the following to the resources section before the variables section in the pipeline definition:
yaml
repositories:
- repository: eShopSecurity
type: git
name: eShopSecurity/eShopSecurity #name of the project and repository
7.Replace the variables section with the following:
yaml
variables:
- template: eshoponweb-secure-variables.yml@eShopSecurity #name of the template and repository
8.Select Commit, accept the default comment, and then select Commit to run the pipeline again.
9.Navigate to the pipeline run and verify that the pipeline is using the YAML file from the eShopSecurity repository.
Now you have the YAML file in a separate repository and project. You can reuse this file in other pipelines in scenarios where you need to deploy the same resources. Also, your operations team can control the resource group, location, security and where the resources are deployed and other information by modifying values in the YAML file and you don’t need to make any changes to your pipeline definition.
[!IMPORTANT] Remember to delete the resources created in the Azure portal to avoid unnecessary charges.
Review
In this lab, you learned how to configure and organize a secure project and repository structure in Azure DevOps. By managing permissions effectively, you can ensure that the right users have access to the resources they need while maintaining the security and integrity of your DevOps pipelines and processes.
Troubleshooting: Problems I Hit and How I Fixed Them
1. “Could not find any file matching the template file pattern”
Symptom: The “Deploy App Service Plan Bicep” step failed before reaching Azure.
Cause: I set templateFile: 'infra/webapp.bicep'. The task already builds the path as $(Pipeline.Workspace)/eshoponweb-ci/Bicep/$(templateFile), so it looked for .../Bicep/infra/webapp.bicep, which doesn’t exist. The CI pipeline publishes the file in a Bicep folder in the artifact, not in infra.
Fix: Set templateFile: 'webapp.bicep', with no folder prefix.
2. Quota error: SubscriptionIsOverQuotaForSku
Symptom:
Current Limit (S1 VMs): 0
Amount required for this deployment (S1 VMs): 1
Cause: The Bicep template defaults to the S1 SKU, and my subscription had a quota of 0 for it. This can happen even on pay-as-you-go subscriptions, because new subscriptions often start with no App Service quota for some tiers.
Fix attempt: I changed the SKU in infra/webapp.bicep to the Free tier, F1.
3. The change didn’t take effect, and F1 failed too
Symptom: The next run still reported S1 VMs.
Cause: The CD pipeline deploys the copy of the Bicep file inside the eshoponweb-ci build artifact, not the file in the repo. My edit only took effect after I re-ran eshoponweb-ci. After that, the error changed to Current Limit (F1 VMs): 0, so the subscription had no Free-tier quota in that region either.
Fix: Quota is per region, so I created a new resource group in West Europe and updated the resource-group and location variables.
4. 403 when checking the new resource group
Symptom:
Checking if the following resource group exists: RG.
Failed to check the resource group status. Error: {"statusCode":403}
Cause: The Azure DevOps service connection signs in as a service principal whose role assignment only covered my original resource group.
Fix:
In Azure DevOps, Project settings → Service connections → Manage App registration gave me the service principal’s display name.
In the Azure portal, on the new resource group: Access control (IAM) → Add → Add role assignment.
I picked the plain Contributor role (not Web Plan Contributor or Website Contributor) and selected the service principal as the member. Check that the Role tab doesn’t say “None” before clicking Review + assign.
After waiting a few minutes, the deployment succeeded.
5. TF401027: You need the Git 'CreateBranch' permission
Symptom: Clicking Initialize on the empty eShopSecurity repo failed.
Cause: My account lacked the Create branch permission on that repository, even though I was the organization owner and in the Project Collection Administrators group. An explicit Deny or a repo-level permission issue overrides group membership.
Fix: I checked the repo’s Security tab for a Deny, then created a fresh repo instead, because the creator of a repo gets full rights on it. To reuse the name eShopSecurity, I renamed the old repo first.
6. Pipeline run “Informational run” with a YAML error
Symptom: A run appeared with an error about YAML preventing Azure Pipelines from deciding whether to run, and clicking into it showed No plan found for identifier.
Cause: The YAML couldn’t be parsed, so the pipeline never started. There’s no execution plan to open or rerun.
Fix: Don’t rerun that entry. Use Edit → ⋮ → Validate on the pipeline to find the broken line, fix it, and start a new run.
7. The shared variables template
The lab moves variables into a template in a separate repo and references it from the pipeline:
`yaml
resources:
repositories:
- repository: eShopSecurity
type: git
name: eShopSecurity/eShopSecurity
variables:
- template: eshoponweb-secure-variables.yml@eShopSecurity
`
Things to check:
The alias after @ must match the repository: name in resources.
Put a space before any # comment, or YAML treats it as part of the value.
The template file holds the variable definitions (- name: / value:). It must not contain a template: line pointing to itself.
Because the pipeline no longer defines variables, the template must include every one it uses (resource-group, location, templateFile, subscriptionid, azureserviceconnection, webappname).
On the first run, Azure DevOps may ask you to permit access to the other repo.
Errors resolved while trying to view the App in a browser
8. Git: “Authentication failed” and PATs
Symptom: git pull and git push over HTTPS asked for a password and rejected it.
Cause: Azure Repos doesn’t accept your Microsoft account password. It needs a Personal Access Token with Code → Read & write, created for the right organization.
Fix: I created a new PAT (User settings → Personal access tokens), pasted it at the prompt (nothing shows as you type, and Shift+Insert or right-click pastes in WSL), and the push worked. I revoked the unused token afterwards.
After the push, I waited for the pipeline to run successfully.
9. Build warning NU1903 (known high severity vulnerability)
Symptom: The CI run and dotnet build both warned that Microsoft.AspNetCore.Components.WebAssembly.Server 8.0.15 had a known high severity vulnerability.
Fix:
dotnet list src/Web/Web.csproj package --outdated --highest-patch showed the newest 8.0.x patch (8.0.32).
Because the repo uses central package management, I changed the version in Directory.Packages.props.
Rebuilding with dotnet build eShopOnWeb.sln reported 0 warnings.
10. Running the app locally in WSL
MSB1011: more than one project or solution file: the repo root has both a .sln and a .dcproj. Name the target: dotnet build eShopOnWeb.sln and dotnet run --project src/Web.
LocalDB is not supported on this platform: LocalDB is Windows-only. Run with the in-memory database: UseOnlyInMemoryDatabase=true dotnet run --project src/Web, then open http://localhost:5000.
Key takeaways
A quota error is per SKU and per region, so changing either can clear it.
The CD pipeline deploys the CI artifact, so Bicep changes only apply after CI runs again.
A new resource group needs its own role assignment for the service connection’s service principal.
A failed YAML parse produces an informational run, so use Validate rather than rerunning it.
Azure Repos over HTTPS needs a PAT, not your account password.
Top comments (2)
If you only have a minute, here are the four things that cost me the most time:
Hope this saves someone an afternoon.
Curious how others handle this: do you keep your shared pipeline variables in a separate project like this, or use variable groups linked to Key Vault? I like the template approach because a separate team can control values like the resource group and region without touching the pipeline definition, but the cross-project permissions took some setup. What’s worked for your team?