Understanding the directory structure created by Azure DevOps tasks.

#azure #devops

As a beginner in all things Azure DevOps, understanding when folders are created and populated by the pipeline tasks is the first step to learning how to manipulate it to suit your needs.

The YAML

Using Visual Studio I created an ASP.NET based project (named TestApplication). The YAML file shown below was the template created by Azure DevOps, following this tutorial. However, the VSBuild@1 task was modified using this, as it resulted in a more suitable folder structure. I also added a PublishBuildArtifacts@1.



trigger:
- master

pool:
  vmImage: 'windows-latest'

variables:
  solution: '**/TestApplication.sln'
  buildPlatform: 'ANY CPU' # x86
  buildConfiguration: 'Release'
  buildversion: 1.0.0

steps:
- task: NuGetToolInstaller@1

- task: NuGetCommand@2
  inputs:
    restoreSolution: '$(solution)'

- task: VSBuild@1
  inputs:
    solution: '$(solution)'
    msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(build.artifactStagingDirectory)"'
    platform: '$(buildPlatform)'
    configuration: '$(buildConfiguration)'

- task: VSTest@2
  inputs:
    platform: '$(buildPlatform)'
    configuration: '$(buildConfiguration)'

- task: PublishBuildArtifacts@1
  inputs:
    PathtoPublish: '$(build.artifactStagingDirectory)'
    ArtifactName: '$(buildversion)'
    publishLocation: 'Container'

I ran this YAML file multiple times using PowerShell tasks to display the environment variables at different stages of the pipeline.

Overall look at the directory structure

This image shows a big-picture look at the directory structure after one complete run of the pipeline.

Summary of when each directory is used

Most of this information, though publicly available on the Microsoft docs, is not very intuitive to a beginner. I hope that this table summarizes the usage of the most relevant directories.

Directory	Uses	References
/a	This is the working director for agent a.	Agent.WorkFolder
/a/1/a	Artifact staging directory. This is where the VS Build task results are stored in. The publish build artifacts task creates an artifact of whatever is in this folder. Note - it gets purged before each new build.	Build.StagingDirectory, System.ArtifactsDirectory, Build.ArtifactStagingDirectory
/a/1/b	The output folder for compiled binaries.	Build.BinariesDirectory
/a/1/s	Source directory. This is the working directory and where your source code is stored.	Build.SourcesDirectory, System.DefaultWorkingDirectory

Conclusion

This information was very helpful to me when it came down to customizing a YAML for my project. I wasn't able to find any existing documentation or articles like this, so please feel free to add links in the comments to similar articles if they exist.
:)

Top comments (6)

Jesse Phillips • Jul 15 '21

The one I needed was: Agent.BuildDirectory
https://docs.microsoft.com/en-us/azure/devops/pipelines/build/variables?view=azure-devops&tabs=yaml

Here is a table that is what I was looking for, note that I'm going to change your use of /a to represent the agent working directory and instead anchor it to / noting that it technically could be anywhere.

Directory	Uses	Variable	Task Magic
/	This is the working director for agent	Agent.WorkFolder
/1	This is the working director for agent	Agent.BuildDirectory	Dowloaded Artifacts Are placed here
/1/a	Artifact staging directory. This is where the VS Build task results are stored in. Note - it gets purged before each new build.	Build.StagingDirectory, System.ArtifactsDirectory, Build.ArtifactStagingDirectory	The publish build artifacts task creates an artifact of whatever is in this folder.
/1/b	The output folder for compiled binaries.	Build.BinariesDirectory
/1/s	Source directory. This is where your source code is stored.	Build.SourcesDirectory, System.DefaultWorkingDirectory	This is the Working Directory for your tasks

Doug Parsons • Sep 30 '20 • Edited

Great article!

I think the one thing that should be pointed out is that none of these folders are guaranteed to have anything in them, depending on how your build is configured.

For example, if I have a set of steps in my YAML that look like this:

steps:

- task: NuGetCommand@2
  inputs:
    restoreSolution: '$(solution)'

- task: VSBuild@1
  inputs:
    solution: '$(solution)'
    platform: '$(buildPlatform)'
    configuration: '$(buildConfiguration)'

- task: VSTest@2
  inputs:
    platform: '$(buildPlatform)'
    configuration: '$(buildConfiguration)'

The only folder that is going to have anything in it is the 's' directory because that is where the build agent clones the source to. The build would then run inside the 's' folder and the output would be directed to whatever folder was configured in the projects (usually bin), so our output would live in /a/1/s/bin/myassembly.dll

You don't actually have to use any of the folders that the build agent provides variables for at all if you don't want to. If you wanted to send your build output to a directory called output you could do something like the following (some code omitted for brevity)

variables:
  base-output-directory: '$(Agent.BuildDirectory)\output'
...
steps:
- task: VSBuild@1
  inputs:
    solution: '$(solution)'
    platform: '$(build-platform)'
    configuration: '$(build-configuration)'
    msbuildArgs: /p:OutDir=$(base-output-directory)

This would send the output of our solution to /a/1/output.

Finally, a little of "the devil is in the details": if you find yourself needing to have different folders in your build directory, keep the folder names short and sweet. The folder names 'a', 'b', and 's' are done intentionally (probably for a number of reasons) but they help to reduce the chance of pathing errors. What I mean is that if your build agent is running on Windows, you can't have a file path longer than 259 characters. If the folder names were more descriptive (artifacts, binaries, source) those are additional characters that count against the total path. While this may seem like an edge case, one of the projects I maintain has this very problem because some developers decided to get overly descriptive in their class and folder naming structures.

Keep up the good work!