Did you know you can specify multiple repositories in one Azure DevOps YAML Pipeline and cause it to trigger by updates to any of the repositories? Let me show you how
Video
As usual, if you are a visual learner, or simply prefer to watch and listen instead of reading, here you have the video with the whole explanation and demo, which to be fair is much more complete than this post.
Link to the video: https://youtu.be/6CXaEDEZZRM
If you rather prefer reading, well... let's just continue :)
Step 1: Basic Pipeline
Let's start from a very simple pipeline definition:
trigger:
- main
pool:
vmImage: 'ubuntu-latest'
steps:
- checkout: self
- script: dir $(Build.SourcesDirectory)
Nothing much to see here, just a pipeline that uses it's own repo and prints out the content after checkout
Step 2: Add More Repos
Let's now make it a little more interesting, adding some additional repos to it. We will use the resources section for it.
resources:
repositories:
- repository: tools
type: git
name: MyProject/tools
ref: main
In this case we are adding a repository named tools that belongs to the Azure DevOps Project MyProject and we tell Azure Pipelines to "consider" the branch main (I'll explain in a moment what I mean by that) and to use tools as reference name for it.
We can also reference a GitHub repository:
- repository: MyGitHubRepo
type: github
endpoint: MyGitHubServiceConnection
name: MyGitHubOrgOrUser/MyGitHubRepo
ref: releases/123
Pretty similar to the other example, but to connect to GitHub we had to specify the service connection name (in the endpoint parameter). Note that here the reference name is MyGitRepo
Finally, we can also add repositories in another Azure DevOps Organization, in a similar way:
- repository: MyOtherAzureReposGitRepository
endpoint: OtherOrgAzureReposGitServiceConnection
type: git
name: OtherProject/MyAzureReposGitRepo
We need to use the service connection as well, because we are going to connect to another organization. Note that in this case we haven't specified the branch (which is indeed optional).
Step 3: The Triggers
At this point we have all our repositories referenced in the workflow, but the pipeline still triggers only on the local repo:
trigger:
- main
If we want it to be triggered also from another repo, we can add a trigger section to the repository definition:
- repository: tools
type: git
name: MyProject/tools
ref: main
trigger:
- dev
- release
For example, with the above snippet our pipeline will be triggered every time a change occurs in wither the dev or the release branches of our tools repo.
What about the main branch specified above too, you ask? well, that is not for the triggers... (I know, confusing... keep reading and you'll know ;) )
At the time of writing, triggers work only for additional repositories in Azure DevOps, in the same organization where the Pipeline is defined. GitHub repos and other repos defined as resources cannot be used for triggering.
Step 4: Using The Code
We now have our pipelines that is triggered by multiple repos. It's very likely that you would want to also use the code for those additional repos, right?
- checkout: MyGitHubRepo
- checkout: tools
- checkout: MyOtherAzureReposGitRepository
Just add the checkout then. As you can see, we use the reference name we've specified in the resources section to let the checkout step know what we want it to do.
And here is finally where the branch in the ref parameter comes into play, because it is the branch that will be checked out:
| Repo | Branch Checkout |
|---|---|
| self | default branch |
| tools | main |
| MyGitRepo | releases/123 |
| MyAzureReposGitRepo | default branch |
If you don't specify a path in the checkout step, Azure Pipelines will use the name of the repository to create the folder, not the repository value which is used to reference the repository in the checkout step.
All Together
With all we have added, our pipeline will now look like this:
resources:
repositories:
- repository: tools
type: git
name: MyProject/tools
ref: main
trigger:
- main
- release
- repository: MyGitHubRepo
type: github
endpoint: MyGitHubServiceConnection
name: MyGitHubOrgOrUser/MyGitHubRepo
ref: releases/123
- repository: MyOtherAzureReposGitRepository
endpoint: OtherOrgAzureReposGitServiceConnection
type: git
name: OtherProject/MyAzureReposGitRepo
trigger:
- main
pool:
vmImage: 'ubuntu-latest'
steps:
- checkout: self
- checkout: MyGitHubRepo
- checkout: tools
- checkout: MyOtherAzureReposGitRepository
- script: dir $(Build.SourcesDirectory)
You can try it out and you'll see (thanks to the last step) that all the code from the 4 repos is there for you to be used
Conclusions
This feature is useful, for example, if you want to consume a tool or a library from a different repository and you want to run tests for your application whenever the tool or library is updated.
It is also good if you keep your YAML file in a separate repository from the application code and you want to trigger the pipeline every time an update is pushed to the application repository.
Let me know in the comment section below how you think this feature can relate to your processes.
Also, checkout this video, where I talk about how to Run a job next in Azure Pipelines
Like, share and follow me 🚀 for more content:
📽 YouTube
☕ Buy me a coffee
💖 Patreon
📧 Newsletter
🌐 CoderDave.io Website
👕 Merch
👦🏻 Facebook page
🐱💻 GitHub
👲🏻 Twitter
👴🏻 LinkedIn
🔉 Podcast
Top comments (13)
hey dude! I separated my yaml from my main source repo for this exact reason. But now, my source repo doesn’t seem to get build statuses attached to commits—only the yaml repo does. Have you encountered this? Do you know of a solution?
That's actually a good point... Let me see if there is a workaround for that.
Hey, were you able to find a solution for this? Do you also know what the "Related" column is for in the Sources pane on a run page of yaml based pipeline. I tried to search whole documentation but couldn't find any reference. I just shows "none" for the repo in which I keep the yaml file but nothing other application code repo. Don't know if this can be helpful somehow.
No, I wasn't able to find any workaround. I also asked directly to the Azure DevOps product group and other MVPs but got no answers :(
On the "Related" it spells out Work-items and Artifact.
Work items will summarize the work items that are contained in the "build". How? It will analyze the commits you are builing and check the linked work items for those commits
Artifacts, instead, will summarize any build or pipeline artifact you publish in that pipeline
You can add a YAML file in your main source repository which extends from a template in your shared repository
@n3wt0n : Thanks for this lovely post. We have tried doing this on our side using our Azure DevOps Server collection / project.
We have 2 repositories A and B in same collection / project.
A has the application source code and B has the test scripts along with the yaml file for pipeline automation. When we do a code change to repo A, it does not trigger this pipeline. When we do code change to repo B, it does trigger the same pipeline. Below is snippet from Yaml file where "ExtenSURE_Data" is repo A and this yaml file sits in Repo B.
resources:
repositories:
trigger:
jobs:
We have checked all settings, cannot find anything wrong.
What should we check / do.
Uhm, it seems ok.
Can you try pasting the same snippet but with the right formatting?
Just paste it using the 3 backticks block
@n3wt0n : FYI - We tried this on Azure DevOps Services (cloud) and it worked. But does not work on Azure DevOps Server (On_premises). Seems we need to have something else done to make it work on ADO Server ?
'''
resources:
repositories:
trigger:
jobs:
'''
ExtenSURE_Data - Name of the repository A
ExtenSURE - Name of the project under which we have both A and B repositories
Name of yaml file under repository B is azure-pipelines.yml
"It is also good if you keep your YAML file in a separate repository from the application code and you want to trigger the pipeline every time an update is pushed to the application repository."
...I was actually looking at that! I still do not get how to checkout multiple resources and finally push to only one of them via GIT command line inside azure devops...may you help?
Thanks a lot for the good stuff
Hey, thanks.
If I understand correctly, you want to checkout from multiple repos and then push the whole thing to a single one?
Let's say you have 3 repo. Adding the 3 checkout instructions, one per repo, you will end up having the "3 parts" of code in:
/s/repo1
/s/repo2
/s/repo3
so what you'd need to do is simply make sure you are in the
/sfolder, and then do the "git magic" from there.or, even better, you can make sure you checkout the destination repo first, in let's say /s/destination folder
then, checkout the other 3 repos in the same folder and you are ready to do
git add ., commit, and push"GitHub repos and other repos defined as resources cannot be used for triggering" I am trying to run a trigger into GitHUb and it is really not working. Have you also tried again?