Table of Contents
Unintentional secrets leaks in git repositories or any version control for that matter, are one of the most prevalent root causes for initial access of threat actors.
According to the State of Secret Sprawls 2025 report1, GitGuardian detected almost 24 million secrets in public GitHub commits in 2024!
Examples of secrets are:
- AWS Access Keys,
- API tokens,
- a username:password combination,
- encryption keys,
- or anything you define as "secret" which should not be made public
In the following blog post I will explore different options of preventing leaking secrets in the first place and what to do when you were unable to prevent it.
Prevention
"An ounce of prevention is worth a pound of cure" - Benjamin Franklin, 1736
Although Benjamin Franklin famously used that proverb to advise fire-threatened Philadelphians, it holds true in the context of IT Security hygiene. When you can prevent your secrets from becoming public you don't have to deal with the aftermath of leaked secrets. The best way to prevent publishing your secrets is to detect them in your code - this is called secret detection.
There is an abundance of commercial and free secret detection tools out there. OWASP recommends2 the following Secret Detection tools:
- GitGuardian - Commercial product that also offers a free version
- gitleaks
- SAP's Credential Dig'ger
- Truffle Hog
- Yelp's detect-secrets
- Arnica
A "good" secret detection tool should offer the following capabilities:
- Extensive Secret Type Coverage: Look for tools that spare you the effort of writing custom RegEx for secret detection.
- Alerting workflow: When your tool does detect a leaked secret you want to be notified immediately via your preferred communication method. Think e-mail, Slack messages, JIRA tickets, etc.
-
Integration and Workflow Compatibility: It should integrate seamlessly into your Version Control Systems (VCS), CI/CD pipelines, and IDE. Wouldn't it be nice if your tool could scan for secrets before you
git pushor evengit commit? Check out the pre-commit framework. - Whitelisting and approval: Sometimes you may want to intentionally allow secrets to be made public. Your tool should be able to handle such edge-cases to allow and/or ignore a secret occurrence.
I've been using GitGuardian in both work-related as well as my own projects and can highly recommend it.
Reaction
Despite all of these tools and awareness for security best practices it can still happen that you accidentally leak a secret - we're all human after all. Here's what you should do in that case.
0. Don't panic
Discovering an incident where you've compromised secret information by publishing it publicly can and will be stressful. However, it's crucial not to panic but instead remain calm and take swift but planned action.
1. Rotate or revoke the secret
Once you've published a secret to a public git repository you should assume breach, meaning that you should treat the secret as compromised no matter if it's only been online for a few minutes. Today's threat actors use sophisticated tools and techniques that allow them to move at rapid speed once they've discovered valid credentials. Chris Farris has done some research3 on how long it takes for public AWS Access Keys to be discovered and actively used for nefarious activities.
That's why you should immediately rotate or revoke the secret. Your approach will vary depending on the type of secret involved in the compromise. Sometimes it's possible to "rotate" a secret (e.g. changing a password for a user). Other times you can only "revoke", "delete", or "invalidate" a secret.
The goal is to make the leaked secret unusable for potential attackers.
Clean up your git history (optional)
Leaking secrets in public repositories on GitHub and then removing them, is just like accidentally posting an embarrassing tweet, deleting it and just hoping no one saw it or took a screenshot.4
You may be tempted to simply "overwrite" a leaked secret by committing and pushing updated versions of your code to your git repository. However, Git is designed to keep an entire history of your work. And that also includes your commit history where attackers could still find an older version of your code that contains a leaked secret.
Once you've revoked your secret, it cannot be used anymore. Nonetheless, exposing secrets in your version control looks unprofessional and might raise concerns (even when they are expired). That's why you should consider rewriting your git history.
One tool to efficiently rewrite your git history is git-repo-filter. It is also what GitHub recommends 5 when removing sensitive data from a repository.
I've listed the steps to rewrite your git history below.
Step 0: Install git-filter-repo
If you haven't already, you can follow the instructions in the official installation guide.
Step 1: Backup your repository
Seriously, copy your entire Git repository to another location. If something goes wrong, you can always revert to this backup.
Step 2: Ensure a clean working directory
git-filter-repo requires a clean working directory and no uncommitted changes.
git status
If you have anything uncommitted, stash it (git stash) or commit it.
Step 3: Remove all remote origins (temporarily)
This prevents accidental pushing of the rewritten history to your remote before you're ready.
git remote remove origin
You'll add it back later.
Step 4: Constructing the git-filter-repo command
Let's say you want to replace a specific string (find) with another string (replace) from a specific file (file_name) in all commits:
git filter-repo --path file_name --replace-text <(echo 'find==>replace')
Let's break down this command6:
-
git filter-repo: The command to invoke the tool. -
--path file_name: This tellsgit-filter-repoto focus its operation only on thefile_namefile. This is crucial because it makes the operation faster and safer, as it won't touch other files. -
--replace-text: This is the core action. It instructs it to perform text replacement. It expects a file (or standard input) where each line defines afind==>replacepattern. -
<(echo 'find==>replace'): In essence, this dynamically creates a temporary "file" containing the single linefind==>replaceand passes its path to the--replace-textoption. This avoids having to create a separate text file (replace_rules.txt) just for this one rule.
Step 5: Re-add your remote origin
After you've run the command, your local history is rewritten. You need to re-establish the connection to your remote repository:
git remote add origin https://<URL_TO_YOUR_REMOTE_GIT>
Step 6: Force Push the rewritten history
🚨 CAUTION 🚨
Rewriting Git history is a powerful operation that fundamentally alters the project's commit log, leading to disrupted workflows for collaborators who will need to re-clone or force update their repositories, and potential data loss if not executed carefully. Inform any collaborators before performing this step, as they will need to re-clone or perform complex Git operations on their end. It can also complicate auditing and make it harder to trace the true lineage of changes, as commit hashes will change.
This is the most critical step and requires overwriting the remote history.
git push --force-with-lease --set-upstream origin main
-
--force-with-lease: This is a safer form of force push. It will only push if the remote branch has not changed since you last fetched it. This prevents overwriting changes made by someone else in the interim. -
--set-upstream origin main: This tells Git that your localmainbranch should track themainbranch on theoriginremote.
Step 7: Verify the changes
Finally, confirm that the sensitive information has been successfully removed from your repository's history.
- Online Verification: Go to your GitHub repository on the web, navigate to the file that contained the secret, and click on "History" or "Blame." You should see the line replaced in all commits.
- New Clone Test: Clone your repository into a fresh, empty directory and inspect older commits (e.g.,
git log --oneline, thengit show <old-commit-sha>:providers.tf) to ensure the token is no longer present anywhere in the history.
Monitor your logs for suspicious activities
As with any security incident, it's important to monitor for suspicious activities. Attackers frequently leverage compromised credentials to gain unauthorized access, move laterally within your systems, or exfiltrate sensitive data. By scrutinizing authentication logs, API activity, and access patterns for anomalies – such as logins from unusual locations, unexpected API calls, or resource creation outside of normal operations – you can quickly identify and respond to any post-compromise activity, limiting potential damage and preventing further breaches.
Personal experience
"By three methods we may learn wisdom: First, by reflection, which is noblest; Second, by imitation, which is easiest; and third by experience, which is the bitterest." - Confucius
I've recently chosen the bitterest option and learned wisdom by experience.
I had hardcoded (bad...) the api_token of my CloudFlare terraform provider configuration in my providers.tf and then pushed it to a public GitHub repository (worse...). Here's what I did in response to that:
I revoked my CloudFlare API Token
Luckily, Cloudflare offers the option to roll an existing API token. This essentially invalidates the "old" API token. It also has the benefit of keeping the existing permission scopes.
CloudFlare offers an API to verify whether a token works. After invalidating my leaked token I made sure that it would be unusable for anyone who may have discovered it:
curl "https://api.cloudflare.com/client/v4/accounts/<MY_ACCOUNT_ID>/tokens/verify" \
-H "Authorization: Bearer abcd-1234-efgh-56789"
The response I got validated that the API Token is no longer working:
{
"success": false,
"errors": [
{
"code": 1000,
"message": "Invalid API Token"
}
],
"messages": [],
"result": null
}
I could have stopped here, since the published API token was now rendered useless. However, I decided to go the extra mile and apply my lessons-learnt.
I changed my git history
As described in the previous section, I used git-filter-repo to replace my hardcoded API token throughout my entire git history.
For my example I used the following command to replace the API token (abcd-1234-efgh-56789) with a redacted version (REMOVED_FROM_GIT_HISTORY) in the providers.tf file in my entire git history:
git filter-repo --path providers.tf --replace-text <(echo 'abcd-1234-efgh-56789==>REMOVED_FROM_GIT_HISTORY')
After performing all the necessary steps I was able to verify that my commit history no longer showed the hardcoded API token, but a redacted value instead:
I set up ggshield, pre-commit & pre-push secret scanning, and installed the GitGuardian Visual Studio Code extension
To avoid accidentally leaking secrets in the future, I went ahead and implemented what's called "Shift Left Security".
Shift-left security is the practice of performing code and software security assurance processes as early as possible in the software development lifecycle (SDLC). - definition from the Wiz Academy7
Setting up ggshield locally
ggshield8 is the command line tool for the GitGuardian secret detection engine. You can use it to search for secrets in files, repositories, Docker images and more.
Below is an example of scanning a specific file (providers.tf) for secrets:
ggshield secret scan path providers.tf
Scanning... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 1 / 1
> providers.tf: 1 secret detected
>> Secret detected: Generic High Entropy Secret
Validity: No Checker
Occurrences: 1
Known by GitGuardian dashboard: NO
Incident URL: N/A
Secret SHA: 3e56b8b21a7e2846a37c797a5115918092442c161e47c067dc8894dd29868e58
Detector documentation: https://docs.gitguardian.com/secrets-detection/secrets-detection-engine/detectors/generics/generic_high_entropy_secret
12 | # Hardcoded credentials for initial test
13 | provider "cloudflare" {
14 | api_token = "abcd*********************-****6789"
|_____________apikey_____________|
15 | }
This shows that ggshield was able to detect my hardcoded API token and alert me on it. That's a good start! However, having to manually run the secret detection tool is not a great "Developer Experience".
💡 TIP 💡
Runningggshieldwill create a local cache of the detected secrets (.cache_ggshield). You may want to add this to your.gitignoreto prevent it from being accidentally committed to your version control system.
Setting up pre-commit and pre-push scans
GitGuardian offers a way to automatically scan your code for secrets before your commits and/or pushes. That means you don't have to explicitly invoke ggshield manually to find the secrets in your code.
Pre-commit hook
A pre-commit hook is a client-side git hook that runs right before the commit is created.
To enable pre-commit scanning, run:
ggshield install --mode global
GitGuardian will now warn and prevent you from commiting code that contains secrets. In the example below, I tried to commit the providers.tf file which contained the hardcoded API Token:
git commit -m"Hardcoded the api_token"
Scanning... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 1 / 1
> commit://staged/providers.tf: 1 secret detected
>> Secret detected: Generic High Entropy Secret
Validity: No Checker
Occurrences: 1
Known by GitGuardian dashboard: NO
Incident URL: N/A
Secret SHA: 3e56b8b21a7e2846a37c797a5115918092442c161e47c067dc8894dd29868e58
Secret in Secrets Manager: False
Detector documentation: https://docs.gitguardian.com/secrets-detection/secrets-detection-engine/detectors/generics/generic_high_entropy_secret
| \ No newline at end of file
12 | +# Hardcoded credentials for initial test
13 | +provider "cloudflare" {
14 | + api_token = "abcd*********************-****6789"
|_____________apikey_____________|
15 | +}
> How to remediate
Since the secret was detected before the commit was made:
1. replace the secret with its reference (e.g. environment variable).
2. commit again.
> [Apply with caution] If you want to bypass ggshield (false positive or other reason), run:
- if you use the pre-commit framework:
SKIP=ggshield git commit -m "<your message>"
Let's examine this in more detail:
- GitGuardian rejected my commit because it found a secret
- It highlighted where the secret was found in my code (
Line 14) - It suggested remediation steps (
replce the secret with environment variable) - It showed how to ignore the secret finding (by adding
SKIP=ggshieldbefore thegit commitcommand)
Pre-push hook
A pre-push hook is a client-side git hook that runs right before a reference is pushed to a remote (git push).
To enable pre-push scanning, run:
ggshield install --mode global -t pre-push
GitGuardian will now warn and prevent you from pushing code that contains secrets. In the example below, I tried to push the providers.tf file which contained the hardcoded API Token:
git push
Scanning... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 1 / 1
commit d6803caac9883af4ab4108fe037553313afa3922
Date: Fri May 30 10:53:00 2025 +1000
> commit://d6803caac9883af4ab4108fe037553313afa3922/providers.tf: 1 secret detected
>> Secret detected: Generic High Entropy Secret
Validity: No Checker
Occurrences: 1
Known by GitGuardian dashboard: NO
Incident URL: N/A
Secret SHA: 3e56b8b21a7e2846a37c797a5115918092442c161e47c067dc8894dd29868e58
Secret in Secrets Manager: False
Detector documentation: https://docs.gitguardian.com/secrets-detection/secrets-detection-engine/detectors/generics/generic_high_entropy_secret
| \ No newline at end of file
12 | +# Hardcoded credentials for initial test
13 | +provider "cloudflare" {
14 | + api_token = "abcd*********************-****6789"
|_____________apikey_____________|
15 | +}
> How to remediate
Since the secret was detected before the push BUT after the commit, you need to:
1. rewrite the git history making sure to replace the secret with its reference (e.g. environment variable).
2. push again.
To prevent having to rewrite git history in the future, setup ggshield as a pre-commit hook:
https://docs.gitguardian.com/ggshield-docs/integrations/git-hooks/pre-commit
> [Apply with caution] If you want to bypass ggshield (false positive or other reason), run:
- if you use the pre-commit framework:
SKIP=ggshield-push git push
Let's examine this in more detail:
- GitGuardian rejected my push because it found a secret.
- It highlighted where the secret was found in my code (in commit
d6803caac9883af4ab4108fe037553313afa3922onLine 14). - It suggested remediation steps (
replce the secret with environment variable). - It showed how to ignore the secret finding (by adding
SKIP=ggshieldbefore thegit pushcommand).
Setting up the Visual Studio Code integration
GitGuardian also offers an integration for Visual Studio Code (VSC)9.
Once you've installed the VSC integration, GitGuardian will highlight it in your editor:
The easiest method of learning wisdom
My recent experience was a bitter lesson, but one that ultimately strengthened my security posture and reinforced the importance of proactive defense.
You don't have to learn this wisdom the "bitter" way yourself. You can use Confucius's easiest method of learning wisdom ("imitation") by taking action today:
Integrate a secret detection tool like GitGuardian into your workflow and set up those crucial pre-commit and pre-push hooks. It’s a small effort that yields monumental security benefits, ensuring your secrets stay secret.
-
Exposing secrets on GitHub: What to do after leaking credentials and API keys ↩
-
Check out Removing sensitive data from a repository ↩
-
Check out the git-repo-filter manual for more details. ↩
-
Check out Shift Left Explained: What It Means to Shift Security Left ↩
-
You can install it from the Visual Studio Marketplace ↩
Top comments (2)
Hah, I also learned this the "bitter" way. This was a good read. And thanks for sharing the OWASP security tools page, I was not aware of it. I'm happy to see TruffleHog is in there since I already use that :)
This guide is super helpful! Keeping secrets out of git is so important, and the tips here on detecting and removing them quickly are really practical.