Github and Gitlab sync

#github #devops #tooling #git

Both Github and Gitlab are great ways to host your git repositories online. They both have great tools to manage your project, be it open source or private code.
I'm not going to compare both services in depth since many guides already do so. For me, the general difference between Github and Gitlab is Gitlab is focusing more effort on CI/CD pipelines and tools for companies whereas Github is focused more on the community and creating a platform to share open-source code.
Both can be used for either scenario... which makes it really hard to choose between them!

But what if I told you you didn't have to choose O_o?

Why I want to use both Gitlab and Github

Just a quick disclaimer, this is the method I use. It works for me but might not work for everyone. I recommend experimenting with things for yourself, do some tests.
I'm sharing my experience because I feel that it could be useful for someone in the same position as me.

I like to be able to organize my repositories into projects (or groups in Gitlab). Unfortunately, Github doesn't do this.
I also want to be able to reach as many people as possible with my repositories, Github has the largest community.
Damn it, I'm stuck, I want features from both providers.

Luckily for me, Gitlab has two things that make symbiosis between the two tools possible:

Github repository import This makes importing a repository from Github super easy
Repository synchronization from Gitlab -> Github (for free) This allows me to use Gitlab as the main source of truth but also have the repository on Github. I can use Github's issue tracker, wiki and forum (Github discussion)
Great CI/CD (ok, I lied when I said two things)

Sync Gitlab to Github (easy)

This one is simple. You could set up some CI/CD yourself. But Gitlab will automatically do this for you.

Go to "Settings > Repository > Mirroring repositories"
Enter your Github repo with your username in front https://<github username>@github.com/path/to/your/repo.git
In the password field, enter your Github token
Select push (this requires a subscription)
Press Mirror repository

From now on, changes to Gitlab will be mirrored to Github

Sync Github to Gitlab (intermediate unless you pay)

There are two ways to do this (both are equivalent just from opposite platforms)

Either you set up a CI/CD pipeline on Github that triggers using a webhook (on push and pull requests).
Either you set up a CI/CD pipeline on Gitlab that triggers using a webhook (on push and pull requests). Turns out this is actually a premium feature... You cannot use the pull webhook without paying now

I personally chose to do this from the Github side since the Github method is free

Method 1 Github CI/CD

Create the following secrets in your Github repository
1. TARGET_URL value: the URL of the Gitlab repository
2. TARGET_TOKEN value: Gitlab token
3. TARGET_USERNAME value: Gitlab username
Create a Github action for your repository with the following code:

name: GitlabSync

on:
  - push
  - delete

jobs:
  sync:
    runs-on: ubuntu-latest
    name: Git Repo Sync
    steps:
    - uses: actions/checkout@v2
      with:
        fetch-depth: 0
    - uses: wangchucheng/git-repo-sync@v0.1.0
      with:
        # Such as https://github.com/wangchucheng/git-repo-sync.git
        target-url: ${{ secrets.TARGET_URL }}
        # Such as wangchucheng
        target-username: ${{ secrets.TARGET_USERNAME }}
          # You can store token in your project's 'Setting > Secrets' and reference the name here. Such as ${{ secrets.ACCESS\_TOKEN }}
        target-token: ${{ secrets.TARGET_TOKEN }}

Make sure the branch you are pushing to in Gitlab is not protected or allows for force push

Method 2 Gitlab CI/CD (requires Gitlab premium)

Note that if you have a subscription with Gitlab, you can also do this in a few clicks in the same dashion as how we set up syncing from Gitlab to Github.

Go to "Settings > Repository > Mirroring repositories"

Enter your Github repo with your username in front https://<github username>@github.com/path/to/your/repo.git
In the password field, enter your Github token
Select pull (this requires a subscription)
Press Mirror repository

Manual Setup

Setup the webhooks

If you created the Gitlab project via the Github import tool then you can completely skip this step.

In your Github repository, go to "Settings > Webhooks"
Create a new webhook
- Payload path: https://gitlab.com/api/v4/projects/<gitlab project id>/mirror/pull (the project id can be found in "Settings > General")
- Content Type: application/json
- SSL Verification: off
- Only send for: push and pull
Activate
In the Gitlab project, go to "Settings > Webhooks"
select Push events from the repository and enter the Github repository (with a token if it's private)

Setup the CI/CD Variables

In the Gitlab project, go to "Settings > CI/CD > Variables"
Create a variable with key: ACCESS_TOKEN and value: (make it hidden because this is sensitive)
1. Create a variable with key: REMOTE_REPOSITORY_URL and value: <your Github token>@<your repository URL> (make it hidden because this is sensitive)

Setup the CI/CD

Create a pipeline by either going to "CI/CD > Editor" or by creating a file .gitlab-ci.yml in your project root and add the following content:

sync-with-github:
  before_script:
    - git config --global user.name "${GITLAB_USER_NAME}"
    - git config --global user.email "${GITLAB_USER_EMAIL}"
  script:
    - git remote add github $REMOTE_REPOSITORY_URL
    - git checkout master
    - git pull origin master
    - git pull github master
    - git status
    - git push https://root:$ACCESS_TOKEN@$CI_SERVER_HOST/$CI_PROJECT_PATH.git HEAD:master

Caveats

This doesn't come without downsides. Here are a few I have found while testing this setup.

Wikis, Issues, Discussions, boards, etc. will not sync between the two hosting providers. This will only sync what is in your repository. There are ways to synchronize the Wikis ( The Wiki is essentially a hidden .git repository and both Github and Gitlab use the same Wiki format )
It can get confusing to have two remote repositories at the same time
They can get out of sync if multiple commits happen at two locations at once and create conflicts (I only use this on repositories I and only I use)

Top comments (11)

hfgestaopublica • Sep 2 '21

If you want to use both GitHub and GitLab:

git remote add github <your-github-url> # create a remote for GitHub
git remote add gitlab <your-gitlab-url> # create another remote for GitLab
git push -u github <local_branch_name> # set the default upstream to GitHub

If you want to change your remote URL from GitLab to GitHub:

git remote set-url origin <your-github-url> # use GitHub as your (only) origin