Forem

rndmh3ro
rndmh3ro

Posted on • Originally published at zufallsheld.de on

1

TIL different ways to clone multiple git repos at once

As someone who frequently works with GitHub and GitLab repositories, I often need to get an overview of groups. At my workplace, we use GitLab’s group- and subgroup function to organize our team-specific and customer-specific structures.

Recently, I needed to clone all 27 repositories in our microservice-project. Doing this manually would require me to copy the URL for each repository and run “git clone \$URL” on the command line, which is time-consuming and tedious.

Fortunately, I discovered a tool called ghorg that simplifies this process. With just one command, I was able to clone entire GitHub organizations or GitLab (sub)groups. Here’s the command I used:

ghorg clone customergroup/services --base-url=https://git.example.com --scm=gitlab --token=$TOKEN --preserve-dir -p .

Enter fullscreen mode Exit fullscreen mode

This command clones all repositories in the group customergroup/services, connecting to our private GitLab instance, using the specified token $TOKEN to authenticate. The --preserve-dir option clones the repos in a directory structure that matches GitLab groups, so repository foo will be cloned into customergroup/services/foo.

ghorg clone customergroup/services --base-url=https://git.example.com --scm=gitlab --token=$TOKEN --preserve-dir -p .

 +-+-+-+-+ +-+-+ +-+-+-+-+-+
 |T|I|M|E| |T|O| |G|H|O|R|G|
 +-+-+-+-+ +-+-+ +-+-+-+-+-+

*************************************
* SCM : gitlab
* Type : org
* Protocol : https
* Location : ./
* Concurrency : 25
* Base URL : https://git.example.com
* Preserve Dir : true
* Config Used : none
* Ghorg version : v1.9.3
*************************************
27 repos found in customergroup/services

Success cloning repo: https://git.example.com/customergroup/services/foo.git -> branch: main
Success cloning repo: https://git.example.com/customergroup/services/foo-audit-report.git -> branch: master
Success cloning repo: https://git.example.com/customergroup/services/foo-tools.git -> branch: main
Success cloning repo: https://git.example.com/customergroup/services/foo-base.git -> branch: main
Success cloning repo: https://git.example.com/customergroup/services/foo-pages.git -> branch: main
Success cloning repo: https://git.example.com/customergroup/services/foo-helm.git -> branch: master
Success cloning repo: https://git.example.com/customergroup/services/foo-application-base.git -> branch: master

Enter fullscreen mode Exit fullscreen mode

But I’m also a great fan of the gitlab-cli and it has a similar feature, too!

Running the following command does the same as ghorg does:

GITLAB_HOST=git.example.com glab repo clone --group customergroup/services -p

Enter fullscreen mode Exit fullscreen mode

The syntax is slightly different. With --group $GROUP you define the group you want to clone; with -p you preserve the group-structure.

> GITLAB_HOST=git.example.com glab repo clone --group customergroup/services -p
Cloning into 'customergroup/services/foo'...
remote: Enumerating objects: 59, done.
remote: Counting objects: 100% (5/5), done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 59 (delta 0), reused 0 (delta 0), pack-reused 54
Receiving objects: 100% (59/59), 13.36 KiB | 1.03 MiB/s, done.

Enter fullscreen mode Exit fullscreen mode

Both tools produce the same folder structure with the cloned repositories:

> tree -d -L 3
.
└── customergroup
    └── services
        ├── foo
        ├── foo-audit-report
        ├── foo-tools
        ├── foo-base
        ├── foo-pages
        ├── foo-helm
        └── foo-application-base

Enter fullscreen mode Exit fullscreen mode

There are many more tools that do this, like myrepos, vcsh or gitlabber, but I didn’t try these.

Image of Timescale

Timescale – the developer's data platform for modern apps, built on PostgreSQL

Timescale Cloud is PostgreSQL optimized for speed, scale, and performance. Over 3 million IoT, AI, crypto, and dev tool apps are powered by Timescale. Try it free today! No credit card required.

Try free

Top comments (0)

Billboard image

The Next Generation Developer Platform

Coherence is the first Platform-as-a-Service you can control. Unlike "black-box" platforms that are opinionated about the infra you can deploy, Coherence is powered by CNC, the open-source IaC framework, which offers limitless customization.

Learn more

👋 Kindness is contagious

Immerse yourself in a wealth of knowledge with this piece, supported by the inclusive DEV Community—every developer, no matter where they are in their journey, is invited to contribute to our collective wisdom.

A simple “thank you” goes a long way—express your gratitude below in the comments!

Gathering insights enriches our journey on DEV and fortifies our community ties. Did you find this article valuable? Taking a moment to thank the author can have a significant impact.

Okay