Wen Ping

Posted on Oct 31 Edited on Oct 31 Originally published at anguskit.com

Move GitLab repositories to AngusGit: import public projects, push private and internal ones

#git #angusgit #gitlab

This article explains how to move public, internal, and private projects from an intranet GitLab to AngusGit. After reading it, you can choose import or push from the project's visibility (who can see the repository), map a multi-level subgroup to "team/repository name", push into an empty repository with git clone --bare plus git push --all and --tags, and compare both sides with git ls-remote.

Sign in to AngusGit first. For a public project, open Repositories in the sidebar and click Import. In Repository URL, paste the HTTPS address from Clone on the project page. Do not paste an SSH address. Owner is the team. Repository name follows the path rule in the next section. A name can use Chinese characters, letters, numbers, hyphens, and underscores. Turn on Make this repository private when only the team should see it. Begin import can be clicked after those three fields are filled in.

Before you push, open Settings → Profile, the Access Tokens tab, and click Generate New Token. Enter your current login password, select git:write, set the repository permission to WRITE, and copy the token immediately. The username is your login name, and the password is the token. Push one repository by hand first and save it in the system credential prompt, so the later batch script does not ask for a password on every line. For SSH instead, if you have no ~/.ssh/id_ed25519.pub, run ssh-keygen -t ed25519 -C "you@example.com" and register the line from cat ~/.ssh/id_ed25519.pub on the SSH Keys tab. The username is git, and the port is 2222.

The GitLab address below uses https://gitlab.example.com/platform/backend/order-service.git as the example. platform is the group, backend is the subgroup, and order-service is the project. Replace them with your own path. To see what a public import looks like, open the already imported xlliu/hello-world-import: it has only main and one commit. How the whole team runs both systems and how to roll back is in the GitLab migration checklist linked at the end.

1. Start with the GitLab project's visibility

Next to the project name, or under GitLab Settings → General → Visibility, the three levels differ like this:

GitLab visibility	Readable without signing in?	Path on AngusGit
Public	Yes	Current code only: Import repository. Keep history: push
Internal	No. A GitLab sign-in is required	Push
Private	No. You must be a project member	Push

The check is simple. Open a private browser window, do not sign in, and open the project page. Import works when you can see the code.

2. Public projects: import directly

In the sidebar, click Repositories, then Import at the top right. In Repository URL, paste the HTTPS address from Clone on the GitLab project page. Leave the SSH address out. The AngusGit server does not have your SSH key. The same form's result for a public repository is the GitHub Hello-World import, xlliu/hello-world-import.

The AngusGit server does the fetch, so it must be able to reach that GitLab. Import brings only the files that were on the default branch. When the source branch is master, the branch name on AngusGit is still main. Fill the form as at the start: Repository URL, Owner, Repository name, and Make this repository private when you need it.

3. Decide how paths map

A GitLab project can sit under several groups, for example platform/backend/order-service. An AngusGit repository URL has two levels: an owner (a person or a team) plus a repository name. Map the whole batch with one rule. Two common mappings:

GitLab path	Option 1: the top group becomes the team	Option 2: fold the subgroup into the repository name
`platform/backend/order-service`	`platform/order-service`	`platform/backend-order-service`
`platform/frontend/order-web`	`platform/order-web`	`platform/frontend-order-web`

Option 1 has a shorter address. Two subgroups with the same project name collide. Option 2 does not collide. The address is a bit longer. When you are unsure, use option 2.

Also check project names for dots. An AngusGit repository name can use Chinese characters, letters, numbers, hyphens, and underscores. A dot is removed. Rename docs.site to docs-site before you move it.

4. Private and internal projects: clone, then push

Create the team and an empty repository on AngusGit Using the rule from section 3, create the team on AngusGit, then create an empty repository with the matching name under that team. When you create the repository, leave Add a README file, Add .gitignore, and Choose a license unchecked. Any checked box means the repository is no longer empty, and the push is rejected.
Clone the whole project from GitLab Clone with your own GitLab account. When GitLab has two-factor authentication turned on, the HTTPS password field needs a GitLab personal access token. The read_repository scope is enough. --bare downloads every branch and tag from GitLab and does not create a working tree.
Rename the main branch when it is master A new empty repository on AngusGit uses main as the default branch. When the GitLab project's main branch is master, rename it locally first. Skip the rename when it is already main. After the rename, local habits and any branch name hardcoded in GitLab CI need to change to main as well.
Push every branch and tag The first command pushes every branch. The second pushes every tag. For HTTPS, the username is the AngusGit sign-in name, and the password is an access token with git:write. An access token is a secret string you generate and use in place of a password. An SSH URL works too.

The full commands are below. Replace gitlab.example.com and platform/backend-order-service with your addresses, and create the empty repository first. Skip git branch -m when the main branch is already main. Use the HTTPS URL of your empty repository. A private clone asks for an access token. Create the git:write token as at the start of this article, and do not put the token in an article or a screenshot.

git clone --bare https://gitlab.example.com/platform/backend/order-service.git
cd order-service.git
git branch -m master main
git push https://git.anguskit.com/platform/backend-order-service.git --all
git push https://git.anguskit.com/platform/backend-order-service.git --tags

--bare creates a bare repository: Git data only, with no working files to edit, which is what you want when you are forwarding the repository unchanged. Why not --mirror? --mirror downloads every remote ref, including refs/merge-requests/*, which GitLab stores for each merge request. A later git push --mirror copies those refs into the AngusGit repository. They do nothing useful there, and they clutter the branch list and the counts. --bare with --all and --tags pushes branches and tags, which is the part you need.

5. Move a batch

When there are many projects, write the mapping in a text file. Each line has two columns: the GitLab path, and "owner/repository name" on AngusGit. Create the empty repositories on AngusGit first, as in step 1 of section 4.

platform/backend/order-service platform/backend-order-service
platform/frontend/order-web platform/frontend-order-web

Save it as repos.txt, then run:

while read -r src dst; do
  git clone --bare "https://gitlab.example.com/${src}.git" tmp.git || continue
  git -C tmp.git branch -m master main 2>/dev/null
  git -C tmp.git push "https://git.anguskit.com/${dst}.git" --all
  git -C tmp.git push "https://git.anguskit.com/${dst}.git" --tags
  rm -rf tmp.git
done < repos.txt

The script clones, renames, and pushes one line at a time, then deletes the local copy. Before you run it, move one repository by hand, as at the start of this article, so the system credential manager remembers the GitLab token and the AngusGit token. Otherwise the script stops on every line to ask for a password.

Batch Import on AngusGit can take many addresses at once. Like a single import, it reads only public projects, brings only the current files, and every imported repository is public. For projects on an intranet GitLab, the script above is the better fit.

6. Check the result: branches and tags match on both sides

After each repository is pushed, compare branches and tags. The command lists every branch and tag ref on GitLab and on AngusGit, then compares them line by line:

diff \
  <(git ls-remote --heads --tags https://gitlab.example.com/platform/backend/order-service.git | sort) \
  <(git ls-remote --heads --tags https://git.anguskit.com/platform/backend-order-service.git | sort)

No output means the two sides match. If section 4 renamed master to main, you see that one pair of names differ. That is expected. Any other difference means a branch or a tag was not pushed. Run the push again.

Then open the repository on AngusGit. Check the count on the Branches tab, and check that authors and times in the commit history match GitLab.

The left side of the picture is an import, not a full push. Judge a push by git ls-remote showing no unexpected difference. When master was renamed to main, that one pair of names is expected to differ.

7. What is not inside the Git repository

A push moves only what is inside the Git repository. Recreate the rest on AngusGit:

What is on GitLab	Does git push bring it?	What to do on AngusGit
Project members and roles	No	Add people to the AngusGit team, or add collaborators under repository Settings → Access
Protected branches	No	Recreate the rules under repository Settings → Branch Protection
Merge requests, issues, and Wiki	No	Merge or close in-progress merge requests on GitLab before you move
`.gitlab-ci.yml`	The file arrives, and it does not run	In the console, use Import GitLab CI to turn it into `.anguscicd.yaml`
Webhooks	No	Add them again under repository Settings → Webhooks

8. When something fails

What you see	Cause	What to do
Source repository URL "…" is invalid or inaccessible during import	The project is internal or private, the URL is SSH, or the AngusGit server cannot reach GitLab	Push internal and private projects as in section 4. For a public project, confirm the URL is HTTPS, and run `git ls-remote <url>` on the AngusGit server
`HTTP Basic: Access denied` while cloning GitLab	GitLab has two-factor authentication on, or the password is wrong	Put a GitLab personal access token in the password field
`! [rejected] main -> main (fetch first)` during push	The AngusGit repository is not empty	Delete it and create an empty repository with all three boxes unchecked
`repository '…' not found` during push	The repository does not exist on AngusGit yet, or the owner and name do not follow the section 3 rule	Create the empty repository first, then check the team name and repository name in the URL
Only some tags differ in the check	The second command, `--tags`, was skipped, or the push dropped midway	Run `git push <url> --tags` again

Common questions

The GitLab project will keep changing. Can I move a copy now and catch up later?

Yes. The simplest way is to repeat section 4: git clone --bare again, rename if you need to, and push. Commits already on AngusGit are not duplicated. The push only adds commits that are new on GitLab. This holds when nobody has pushed straight to AngusGit in the meantime. If someone has, the two sides have diverged and the push is rejected. How to sync repeatedly before the cutover, then freeze GitLab and sync once more, is in the GitLab migration checklist.

Can I keep the review record inside a GitLab merge request?

No. Review records live in GitLab's database, not in the Git repository. Export them or save screenshots on GitLab when you need an archive.

What if the project uses Git LFS?

AngusGit does not support Git LFS. Git LFS (Git Large File Storage) stores large files outside normal Git history. After the push, those files are only pointer files. Commit them as normal files, or store them in an artifact repository.

Do GitLab submodule URLs need to change?

Yes. If .gitmodules still contains a GitLab URL, the submodule still fetches from GitLab. After the submodule's repository has also moved to AngusGit, change the URL in .gitmodules to the AngusGit address and commit that change.

Summary

To move repositories from GitLab, start with visibility. A public project can use Import repository, and that brings only the current files. Internal and private projects, and any project whose history you need, go into an empty repository on AngusGit: git clone --bare, then --all and --tags. Settle the subgroup mapping first, and rename dots in project names first. A batch is a mapping file plus the loop. After each push, compare both sides with git ls-remote --heads --tags. They match when the move is done.

Conclusion: For GitLab internal and private repositories, and for any repository whose history you need, map the multi-level subgroup to "team/repository name" first, then push into an empty repository with git clone --bare plus git push --all and --tags, and check the result with git ls-remote.

DEV Community