DEV Community

João Vitor
João Vitor

Posted on • Edited on

Clone azure repos in parallel

Azure repos are hosted inside devops projects.

You can list azure projects with

az devops project list --detect true
Enter fullscreen mode Exit fullscreen mode

To list repositories in azure from one specific az devops project

az repos list --project project_id
Enter fullscreen mode Exit fullscreen mode

To grab the repo list from multiple projects in parallel and save it to a file:

org=$(basename $(pwd)); az devops project list --detect true | jq -r '.value[]|.id' | parallel -j 10 "az repos list --project {} | jq -r '.[]|.sshUrl'" | tee ${org}.txt
Enter fullscreen mode Exit fullscreen mode

As the repo ssh url is different from the github one the clone-missing.sh script had to be changed in a not compatible way.

Difference

github ssh url: git@github.com:${githuborg}/${repo}.git
azure repo url: git@ssh.dev.azure.com:v3/${devopsorg}/${devopsproject}/${repo}

New azure-clone-missing.sh command added to the parallel_commands to handle this url difference.

Cloning azure repositories in parallel

Given that you created the org.txt file the command below clones only the missing repositories.

cd ~/azure-orgs/azuredevopsorg/
org=$(basename $(pwd)); cat ${org}.txt | parallel  -j 25 'azure-clone-missing.sh {}; echo job {#} completed {};'
Enter fullscreen mode Exit fullscreen mode

If you want to exclude some repositories from your clone.

PS change the regex that excludes the repo.
grep -Ev '(repo1|repo2)' ${org}.txt

cd ~/azure-orgs/azuredevopsorg/
org=$(basename $(pwd)); grep -Ev '(repo1|repo2)' ${org}.txt | parallel  -j 25 'azure-clone-missing.sh {}; echo job {#} completed {};'
Enter fullscreen mode Exit fullscreen mode

Documentation reference

Top comments (0)