Sync Git Repository from Github to AWS CodeCommit with Terraform

What is a Git Repository?

Git repository is like a data structure that VCS uses to store metadata for a set of files and directories. Contains a collection of files and a history of changes made to those files.

This git repository can then be a remote repository stored on a code hosting service like Github, BitBucket, etc.

The advantage of storing your git repository on a code hosting services like Github is to promote collaboration whereas it will be a common repository that all team members use to exchange their changes among the files.

Having a central code repository is essential part of development. Hence this needs to be planned well.

In our case, we have our private Github organization where we store our project codes and we have a requirement to synchronized it to another repository in AWS CodeCommit.

Getting Started

At first, we need to generate several tokens and an SSH Key.

• Generate a GitHub Token - provide workflow, organization, create and delete repository permissions
• Clone this repository.
• Generate an SSH Key.
• Substitute the appropriate values below (see description), create a terraform.tfvars file and replace the placeholders.

github_token          = "PUT YOUR GITHUB PERSONAL ACCESS TOKEN HERE"

aws_access_key_id     = "PUT AWS SECRET ACCESS KEY ID HERE"
aws_secret_access_key = "PUT AWS SECRET ACCESS KEY HERE"

github_repository_name     = "Github Repository Name e.g. 'samplegithubrepo'"
codecommit_repository_name = "CodeCommit Repository Name e.g. 'samplecodecommitrepo'"

ssh_private_key_path  = "PUT THE PATH WHERE SSH PRIVATE KEY IS STORED e.g. ~/.ssh/id_rsa"
ssh_public_key_path   = "PUT THE PATH WHERE SSH PUBLIC KEY IS STORED e.g. ~/.ssh/id_rsa.pub"

aws_region            = "PUT YOUR AWS REGION OF CHOICE"

Please take note that the values inside your terraform.tfvars file are not to be pushed to the Github repository. To prevent this, the Github repository itself has .gitignore file which *.tfvars are included.

Deployment

After doing the prerequisites above, you can start deploying the solution.

1. Run terraform init.
2. Run terraform apply.
3. Check the resources to be deployed and type "yes" to deploy the resources.

Troubleshooting

Here are some error's we encountered during testing of this solution.

Workflow Failed

> Run pixta-dev/repository-mirroring-action@v1
> fatal: no path specified; see 'git help pull' for valid url syntax

Resolution:

• Open the created GitHub Repository, Go to Settings > Secrets > Update CODECOMMIT_SSH_PRIVATE_KEY with your private SSH key.
• Rerun workflow.

Result

• All parts of the GitHub Repo are mirrored, i.e. branches, commits, etc.
• Setup CodePipeline to use CodeCommit as source.
• Pushing to Git auto-triggers a push up to CodeCommit, which in turn triggers a pipeline rerun.

Github Repository

See the Github repository here.

edwardmercado / mirror-repo-from-github-to-codecommit

Mirror Github Repository to CodeCommit

Prerequisites:

• Terraform
• AWS with appropriate credentials
• SSH Key Pair (public and private)
• GitHub Token

Getting Started

• Generate a GitHub Token - provide (workflow, create and delete repository permissions)
• Clone this repository.
• Generate an SSH Key - keep the default name if possible as it points to ~/.ssh/id_rsa and ~/.ssh/id_rsa.pub.
• Substitute values below, create a terraform.tfvars file and paste the below values.

github_token = "PUT YOUR GITHUB PERSONAL ACCESS TOKEN HERE"
aws_access_key_id     = "PUT AWS SECRET ACCESS KEY ID HERE"
aws_secret_access_key = "PUT AWS SECRET ACCESS KEY HERE"
repository_name = "PUT YOUR REPOSITORY NAME HERE (SAME ON BOTH GITHUB & CODECOMMIT)"

• Run terraform init
• Run terraform apply -auto-approve

Troubleshooting

Workflow Failed

> Run pixta-dev/repository-mirroring-action@v1
> fatal: no path specified; see 'git help pull' for valid url syntax

Resolution:

• Open the created GitHub Repository, Go to Settings > Secrets > Update CODECOMMIT_SSH_PRIVATE_KEY with your private SSH…

View on GitHub

For any queries, you can reach me at:

Dev.to
Twitter
LinkedIn