Solved: Migrate Bitbucket Repositories to GitHub (Including Branches/Tags)

#devops #programming #tutorial #cloud

🚀 Executive Summary

TL;DR: Managing projects across both Bitbucket and GitHub creates workflow inefficiencies and context-switching for engineering teams. This guide provides a script-based solution to migrate Bitbucket repositories to GitHub, ensuring all branches, tags, and commit history are fully preserved.

🎯 Key Takeaways

The migration process leverages two core Git commands: git clone –bare to copy the entire repository database locally, and git push –mirror to push all references (branches, tags) to the new GitHub remote.
Essential prerequisites include admin access on both Bitbucket and GitHub, a GitHub Personal Access Token (PAT) with repo scopes, a Bitbucket App Password with repository:admin permissions, and local installations of Git and Python 3.
Destination repositories on GitHub must be created as completely empty, without any initial files like READMEs or licenses, for the git push –mirror command to function correctly.
A Python script automates the repetitive task of cloning and mirror-pushing multiple repositories, significantly reducing manual effort and the potential for human error.
For security, credentials such as GitHub PATs and Bitbucket App Passwords should be loaded from environment variables or a secure config system, never hard-coded directly into the script or committed to version control.

Migrate Bitbucket Repositories to GitHub (Including Branches/Tags)

Hey team, Darian here. A few quarters ago, our engineering org was split between Bitbucket for legacy projects and GitHub for new ones. I found myself constantly context-switching, managing two sets of credentials, and wrestling with CI/CD pipelines that had to pull from different sources. After a quick analysis, I realized my team was losing about three hours a week just managing this split. That’s when I scripted this migration process. Moving everything to GitHub consolidated our workflow, simplified permissions, and let our build systems breathe a little easier.

This guide is the refined version of that script and process. It’s designed to get you from A to B quickly and without losing any history. Let’s get it done.

Prerequisites

Before you start, make sure you have the following ready to go:

Admin Access: You’ll need admin permissions on your Bitbucket workspace and the target GitHub organization.
GitHub Personal Access Token (PAT): Generate a PAT with full repo scopes. This will be your password for automating Git operations.
Bitbucket App Password: Create an App Password in Bitbucket with repository:admin permissions. This is more secure than using your primary password.
Local Tools: Git and Python 3 must be installed on the machine you’re running the migration from.

The Step-by-Step Migration Guide

Step 1: Understand the “Bare Clone” and “Mirror Push”

The core of this entire process relies on two powerful Git commands. Instead of a normal clone, which checks out a working copy of a single branch, we’ll use git clone --bare. This command copies the entire Git database—every branch, every tag, every commit—without creating a working directory. It’s a perfect, self-contained backup of your repository’s history.

Then, we’ll use git push --mirror. This pushes everything from your local bare clone to the new remote on GitHub. The --mirror flag ensures that all references (branches, tags, etc.) in the bare repository are pushed exactly as they are. Any refs on the remote that aren’t in your local copy are even deleted, which is why it’s crucial to push to a brand new, empty GitHub repo.

Step 2: Prepare Your Destination on GitHub

For every Bitbucket repository you intend to migrate, you must first create a new, empty repository on GitHub. Do not initialize it with a README, license, or .gitignore file. The mirror push needs a clean slate to work with.

Pro Tip: I recommend creating the new GitHub repositories as private first. You can run the migration, verify everything looks correct, and then change the visibility to public or internal as needed. This prevents exposing a partially migrated or broken repository.

Step 3: The Automation Script

Manually cloning and pushing dozens of repositories is a recipe for error. We’re going to use a Python script to automate this. First, you’ll want to set up a new directory for this project and create a Python virtual environment. I’ll skip the standard virtualenv setup commands since you likely have your own workflow for that. Just make sure to install the libraries we’ll need, which in this case are built-in.

You should also create a config.env file to store your credentials and use a library like python-dotenv to load them, rather than hard-coding them in the script. For security, never commit credentials to version control.

Here’s the script. The logic is straightforward: it iterates through a list of repository names, constructs the correct URLs with credentials, and executes the clone and push commands in a subprocess for each one.

import os
import subprocess
import shutil

# --- Configuration ---
# Load these from environment variables or a secure config system.
GITHUB_USERNAME = os.environ.get('GITHUB_USER')
GITHUB_PAT = os.environ.get('GITHUB_PAT')
BITBUCKET_USERNAME = os.environ.get('BITBUCKET_USER')
BITBUCKET_APP_PASSWORD = os.environ.get('BITBUCKET_APP_PASSWORD')

GITHUB_ORG = 'TechResolve'
BITBUCKET_WORKSPACE = 'techresolve-workspace'

# A list of repositories to migrate.
REPOS_TO_MIGRATE = [
    'project-alpha',
    'legacy-api',
    'data-pipeline-utility'
]

def migrate_repository(repo_name):
    """Clones a repo from Bitbucket and mirror-pushes it to GitHub."""
    print(f"--- Starting migration for: {repo_name} ---")

    # 1. Construct authenticated URLs for the Git commands.
    bitbucket_url = f"https://{BITBUCKET_USERNAME}:{BITBUCKET_APP_PASSWORD}@bitbucket.org/{BITBUCKET_WORKSPACE}/{repo_name}.git"
    github_url = f"https://{GITHUB_USERNAME}:{GITHUB_PAT}@github.com/{GITHUB_ORG}/{repo_name}.git"

    # This will be the name of the local folder, e.g., "project-alpha.git"
    local_repo_path = f"{repo_name}.git"

    # Clean up any failed runs from before.
    if os.path.isdir(local_repo_path):
        print(f"Removing existing local directory: {local_repo_path}")
        shutil.rmtree(local_repo_path)

    try:
        # 2. Perform a bare clone. This downloads the entire Git object database.
        print(f"Cloning {repo_name} from Bitbucket...")
        clone_command = ['git', 'clone', '--bare', bitbucket_url]
        subprocess.run(clone_command, check=True, capture_output=True, text=True)
        print("Bare clone successful.")

        # 3. Perform a mirror push. This sends all branches and tags to the new remote.
        # We use 'cwd' to run the command from inside the newly cloned directory.
        print(f"Pushing {repo_name} to GitHub...")
        push_command = ['git', 'push', '--mirror', github_url]
        subprocess.run(push_command, check=True, capture_output=True, text=True, cwd=local_repo_path)
        print("Mirror push successful.")

    except subprocess.CalledProcessError as e:
        print(f"ERROR: A Git command failed for repository '{repo_name}'.")
        print(f"Stderr: {e.stderr}")
        return False
    finally:
        # 4. Clean up the local bare repository folder to save space.
        if os.path.isdir(local_repo_path):
            print(f"Cleaning up local clone: {local_repo_path}")
            shutil.rmtree(local_repo_path)

    print(f"--- Successfully migrated {repo_name}! ---\n")
    return True

def main():
    """Main function to loop through all specified repositories."""
    print("Starting Bitbucket to GitHub migration process...")
    successful_migrations = 0
    failed_migrations = []

    # IMPORTANT: This script assumes you have already created empty repos
    # on GitHub with names matching those in the REPOS_TO_MIGRATE list.

    for repo in REPOS_TO_MIGRATE:
        if migrate_repository(repo):
            successful_migrations += 1
        else:
            failed_migrations.append(repo)

    print("--- Migration Complete ---")
    print(f"Total Successful: {successful_migrations}")
    if failed_migrations:
        print(f"Total Failed: {len(failed_migrations)}")
        print(f"Failed Repositories: {', '.join(failed_migrations)}")
    else:
        print("All repositories migrated without errors.")

if __name__ == "__main__":
    main()

Step 4: Run the Script and Verify

After you’ve configured your credentials as environment variables and populated the REPOS\_TO\_MIGRATE list, you can run the script. It will provide a running log of its progress. Once it completes, head over to GitHub and check a few of the migrated repositories. Verify that all branches and tags are present by checking the branch/tag dropdown menus. The commit history should be fully intact.

Common Pitfalls (Where I Usually Mess Up)

Authentication Failure: The most common issue is a 403 Forbidden or Authentication failed error. This almost always means there’s an issue with your PAT or App Password. Double-check that they have the correct permissions and are embedded in the URL correctly.
Pushing to a Non-Empty Repo: If you accidentally initialized the GitHub repo with a README, the mirror-push will fail because it’s not a “fast-forward” push. The destination must be completely empty. Delete the repo on GitHub and create it again if this happens.
Typos in Names: A simple typo in the repository name, Bitbucket workspace, or GitHub organization in the script will cause a “repository not found” error. It sounds obvious, but it gets me every time when I’m moving fast.

Conclusion

And that’s it. This script-based approach gives you a repeatable, reliable, and fast way to migrate your entire codebase from Bitbucket to GitHub while preserving every last commit, branch, and tag. It turns a weekend of tedious manual labor into an automated process you can run in a few minutes. Now you can focus on what really matters: standardizing your tooling and building great software.

Best,

Darian Vance