Backup your repos via GitHub Actions

K8sCat — Sun, 27 Jun 2021 17:45:08 +0000

Anything may be lost unless you have a backup for them.

This article tell you how to backup your git repositories via GitHub Actions.

Our target

In this article, our final target is to backup git repositories under the user-a to another user named user-b .

Preparation

Before all, we need to create a new git repository used to run the backup workflow, and there will be some files in it and you can use the command tree to have a check:

.
├── .github
│   └── workflows
│       └── mirror.yaml

Normally, we wouldn’t to expose our secrets directly, so we need to create some secrets in the GitHub repository settings which will be used to clone, create and push git repo:

GH_TOKEN_A is the user-a ‘s GitHub AccessToken
GH_TOKEN_B is the user-b ‘s GitHub AccessToken
SLACK_WEBHOOK is used to notify us while the workflow started or ended

Mirror Workflow

List user-a ‘s repos

We’ll use the command-line tool gh to list user-a ‘s github repositories:

echo ${GH_TOKEN_A} > gh_token_a
gh auth login --with-token < gh_token_a
gh repo list user-a -L 1000 > a_repos
cat a_repos
cat a_repos | wc -l

Create repo with the same name

Before push the git repo, we need to make the repo existed under the user user-b , like list repos above, we also use gh to create a git repo with the same name:

gh repo create user-b/${repo_name} --private --description "${repo}" -y || true

Clone repo

While we make sure that the repo with the same name is already existed under the user user-b , we can clone the git repo from the user user-a first.

Unlike normal git clone, we should use the flag --bare to
clone the repo:

git clone --bare https://${GH_TOKEN_A}@github.com/user-a/${repo_name}.git ${repo_name}

Push repo

Finally, we can push the cloned repo to user-b ‘s repo which has the same name.

And we will use two flags to push them:

--all means push all branches
--mirror means push all refs, contains all branches and tags

cd ${repo_name}
mirror_repo="https://${GH_TOKEN_B}@github.com/user-b/${repo_name}.git"
git push --all -f ${mirror_repo} || true
git push --mirror -f ${mirror_repo} || true

Full workflow script

This is the full workflow script:

name: Mirror repos

on:
  schedule:
    - cron: "0 18 * * 0-4" # 设置定时任务，周一到周五的凌晨2点进行备份
  workflow_dispatch: # 手动触发构建

jobs:
  mirror:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2

      - name: GitHub CLI version
        run: gh --version

      - name: List repos
        env:
          GH_TOKEN_A: ${{ secrets.GH_TOKEN_A }}
        run: |
          echo ${GH_TOKEN_A} > gh_token_a
          gh auth login --with-token < gh_token_a
          gh repo list user-a -L 1000 > a_repos
          cat a_repos
          cat a_repos | wc -l

      - name: Mirror repos
        env:
          GH_TOKEN_A: ${{ secrets.GH_TOKEN_A }}
          GH_TOKEN_B: ${{ secrets.GH_TOKEN_B }}
          IGNORE_REPOS: "/repo_a/repo_b/"
          ONLY_BRANCH_REPOS: "/repo_c/repo_d/"
        run: |
          echo ${GH_TOKEN_B} > gh_token_b
          gh auth login --with-token < gh_token_b

          mkdir repos
          cd repos
          set -x
          cat ${GITHUB_WORKSPACE}/a_repos | while read repo; do
            repo_name=$(echo ${repo} | awk '{print $1}' | awk -F/ '{print $2}')
            [[ ${IGNORE_REPOS} =~ "/${repo_name}/" ]] && continue || true

            gh repo create user-b/${repo_name} --private --description "${repo}" -y || true
            rm -rf ${repo_name}

            git clone --bare https://${GH_TOKEN_A}@github.com/user-a/${repo_name}.git ${repo_name}

            cd ${repo_name}
            mirror_repo="https://${GH_TOKEN_B}@github.com/user-b/${repo_name}.git"
            git push --all -f ${mirror_repo} || true
            [[ ${ONLY_BRANCH_REPOS} =~ "/${repo_name}/" ]] && continue || true
            git push --mirror -f ${mirror_repo} || true
            cd -
          done

      - name: Slack Notification
        uses: rtCamp/action-slack-notify@v2
        env:
          SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK }}

DEV Community: K8sCat