DEV Community

João Vitor
João Vitor

Posted on • Edited on

Clone azure repos in parallel

Azure repos are hosted inside devops projects.

You can list azure projects with

az devops project list --detect true
Enter fullscreen mode Exit fullscreen mode

To list repositories in azure from one specific az devops project

az repos list --project project_id
Enter fullscreen mode Exit fullscreen mode

To grab the repo list from multiple projects in parallel and save it to a file:

org=$(basename $(pwd)); az devops project list --detect true | jq -r '.value[]|.id' | parallel -j 10 "az repos list --project {} | jq -r '.[]|.sshUrl'" | tee ${org}.txt
Enter fullscreen mode Exit fullscreen mode

As the repo ssh url is different from the github one the clone-missing.sh script had to be changed in a not compatible way.

Difference

github ssh url: git@github.com:${githuborg}/${repo}.git
azure repo url: git@ssh.dev.azure.com:v3/${devopsorg}/${devopsproject}/${repo}

New azure-clone-missing.sh command added to the parallel_commands to handle this url difference.

Cloning azure repositories in parallel

Given that you created the org.txt file the command below clones only the missing repositories.

cd ~/azure-orgs/azuredevopsorg/
org=$(basename $(pwd)); cat ${org}.txt | parallel  -j 25 'azure-clone-missing.sh {}; echo job {#} completed {};'
Enter fullscreen mode Exit fullscreen mode

If you want to exclude some repositories from your clone.

PS change the regex that excludes the repo.
grep -Ev '(repo1|repo2)' ${org}.txt

cd ~/azure-orgs/azuredevopsorg/
org=$(basename $(pwd)); grep -Ev '(repo1|repo2)' ${org}.txt | parallel  -j 25 'azure-clone-missing.sh {}; echo job {#} completed {};'
Enter fullscreen mode Exit fullscreen mode

Documentation reference

Image of Docusign

Bring your solution into Docusign. Reach over 1.6M customers.

Docusign is now extensible. Overcome challenges with disconnected products and inaccessible data by bringing your solutions into Docusign and publishing to 1.6M customers in the App Center.

Learn more

Top comments (0)

Billboard image

The Next Generation Developer Platform

Coherence is the first Platform-as-a-Service you can control. Unlike "black-box" platforms that are opinionated about the infra you can deploy, Coherence is powered by CNC, the open-source IaC framework, which offers limitless customization.

Learn more