Removing Sensitive Data in Git

This guide explain how to remove sensitive text from your Git repo. It requires BFG Repo-Cleaner, which is endorsed by GitHub, and Java.

1. Close/merge all pull requests.
2. Make sure all developers' local branches are clean.
3. Clone your repo to create an emergency backup.
4. Download the bfg JAR file from BFG Repo-Cleaner, for example bfg-1.14.0.jar.
5. Go to a temporary working folder.
6. Clone a bare mirror of your repo, for example: git clone --mirror [e-mail]:SomeUser/myrepo.git
7. Create a text file named sensitive.txt with regular expressions to replace. For example, this text file will replace all occurrences of password123 with ***REMOVED*** and all occurrences of abc123 with samplePassword:

  password123
  abc123==>samplePassword

1. Copy in the bfg-1.14.0.jar file.
2. Execute this command to replace the sensitive text: java -jar bfg-1.14.0.jar --no-blob-protection --replace-text sensitive.txt myrepo.git
3. Go into the mirror repo: cd myrepo.git
4. Execute git reflog expire --expire=now --all && git gc --prune=now --aggressive
5. Push to your remote branch: git push . Note: If this fails, you may need to unprotect the branch in the remote Git server.
6. Ask all developers to re-clone the repo to get the rewritten Git histories.
7. Verify that the repo looks correct, then delete the local backup repo and temporary working folder.

Now all sensitive data is gone.

Thanks for reading!

Follow me on Twitter @realEdwinTorres for programming tips, software engineering content, and career advice. 😊

The content in this blog post is publicly available at Git, GitHub, GitLab, and BFG Repo-Cleaner.

Comments

[late_riser]

Is this process necessary if my PR (which contains sensitive data) is still open (or closed) and is not merged yet in main repo?
In my case, if I simply remove the commits that contain sensitive data, will it suffice?

[NIHA| Areon]

💡 Calling all developers and tech wizards! Areon Network presents a golden opportunity with its Hackathon. Register at hackathon.areon.network and compete for a share of the incredible $500,000 prize pool. Code your way to success! 🏆💻 #CodingCompetition #AreonNetwork