How I Purged 121 Leaked Secrets From 10 Years of Git History With Claude Code

TL;DR

I inherited a 10-year-old repo with ~28,000 commits and a nasty surprise: credentials hardcoded all over its git history. Using a secret scanner for detection and Claude Code for triage and remediation, I classified 3,400 raw hits down to 121 real secrets, rotated everything, rewrote history with git filter-repo, and shipped guardrails so it never happens again. Here's the full playbook, including the mistakes. 🔐

The Problem

Last quarter my team took ownership of a legacy service. The codebase itself was fine — tests passed, CI was green. Then I ran a routine secret scan before wiring it into our deployment pipeline:

$ gitleaks detect --source . --report-path report.json
...
WRN leaks found: 3407

Three. Thousand. Four. Hundred. Hits. Across a decade of git history.

Now, most of those were noise — test fixtures, example configs, high-entropy strings that were just UUIDs. But buried in there were real database passwords, cloud provider keys, and third-party API tokens. Some belonged to systems still running in production.

Here's what makes this problem genuinely hard:

• Deleting a secret from HEAD does nothing. It lives forever in history. Anyone who clones the repo gets every version of every file ever committed.
• You can't just rewrite history first. If the key is still valid, rewriting is theater — the secret was already exposed. Rotation has to come first.
• Triage at this scale doesn't fit in a human brain. Nobody is manually reviewing 3,407 findings with any accuracy. By hit #200 you're rubber-stamping.

The third point is where an AI agent earned its keep.

How I Solved It

The whole effort took about a week. Here's the flow:

flowchart LR
    A[Scan: gitleaks] --> B[Triage: Claude Code]
    B --> C[Rotate live credentials]
    C --> D[Rewrite: git filter-repo]
    D --> E[Coordinated force-push]
    E --> F[Guardrails: pre-commit + CI]

Step 1: Scan wide, don't tune yet

My first instinct was to tune gitleaks rules to cut the noise. That was a mistake — I burned half a day and every rule I tightened risked hiding a real leak. Lesson: let the scanner be paranoid, and make triage cheap instead.

I ran gitleaks (v8.x) across full history and dumped everything to JSON:

gitleaks detect --source . \
  --report-format json \
  --report-path /tmp/leaks-raw.json
# 3,407 findings, ~4 minutes on 28k commits

Step 2: Let Claude Code triage all 3,407 findings

This is the part I'd never do by hand again. I pointed Claude Code (v2.x) at the report and had it classify every finding with the surrounding context — not just the matched string, but the file, the commit, and what the code around it was doing.

The prompt boiled down to:

For each finding, output a JSON verdict: real_secret | test_fixture | false_positive, with a one-line justification and a confidence score. A finding is real_secret only if the value could plausibly authenticate against a live system. When uncertain, escalate to real_secret — false negatives are expensive here, false positives are cheap.

That last sentence matters. For secret triage you want the agent biased toward over-reporting, which is the opposite of how most people prompt for code review.

Claude Code chunked through the report in batches, reading the actual file contents at the offending commits (git show <sha>:<path>) to judge context. A password in docker-compose.test.yml with the value password123 is a fixture. A 40-character token in deploy/prod.env committed by a human at 2am is not. That contextual judgment is exactly what regex can't do and humans can't sustain for 3,407 rows.

Each verdict came back as structured JSON, which meant the downstream steps (rotation checklist, replacements file) could be generated mechanically instead of hand-copied:

{
  "finding_id": 2214,
  "file": "deploy/prod.env",
  "commit": "9f3ac1e",
  "verdict": "real_secret",
  "confidence": 0.94,
  "reason": "40-char token assigned to STRIPE_SECRET_KEY in a deploy config; matches live key format, not a test prefix."
}

One practical tip: batch size matters more than you'd think. I started with 500 findings per pass and the verdicts got noticeably lazier toward the end of each batch — the same "attention fade" a human reviewer has. Dropping to 50 findings per batch, each with fresh context, kept quality flat from the first verdict to the last. It cost more in tokens; it was worth every cent.

The result:

Verdict	Count
False positive (entropy noise, UUIDs, hashes)	2,890
Test fixtures / dummy values	396
Real secrets	121

I spot-checked ~150 verdicts by hand across all three buckets. I found zero misclassified real secrets and only a handful of fixtures marked "real" out of caution — exactly the failure direction I asked for. ✅

Step 3: Rotate before you rewrite

This ordering is non-negotiable, and it's the step people skip because it's boring:

1. Inventory — for each of the 121 secrets: what system is it for, is it still valid, who owns it?
2. Test validity — Claude Code wrote small probe scripts (read-only calls) to check which credentials were still live. 38 of 121 still worked. One database password from 2019 still opened a production replica. 😱
3. Rotate — new credentials issued into our secrets manager, old ones revoked. Humans held the credentials; the agent generated the checklist and the migration snippets, and never saw the new values.

Only after every live credential was dead did I touch history. Rewriting first just destroys your evidence while the keys keep working.

Step 4: Rewrite history with git filter-repo

git filter-repo (v2.x) replaces matched strings across every commit. Claude Code generated the replacements file from the triage output:

# replacements.txt — lines are literal matches unless prefixed with regex:
AKIA****************==>***REMOVED-AWS-KEY-17***
regex:postgres://[^@ ]+@==>postgres://***REMOVED***@

# fresh clone, never your working copy
git clone --mirror git@internal:legacy-service.git scrub.git
cd scrub.git
git filter-repo --replace-text ../replacements.txt

Two practical notes:

• Run it on a mirror clone, verify, then push. Keep an encrypted archive of the original mirror for audit purposes — legal may need the pre-rewrite state.
• Re-run gitleaks on the rewritten mirror before pushing. My first pass missed 4 secrets that appeared only in deleted files' blobs; the second pass caught them.

Step 5: The force-push is a social problem, not a technical one

Rewriting history changes every commit SHA after the first affected commit. Every open PR, every local clone, every CI cache pointing at old SHAs breaks. Treat it like a small migration:

• Announced a 2-hour freeze window a week ahead
• Merged or closed all open PRs beforehand
• Pushed the rewritten history, then had everyone re-clone (not pull — pulling across a rewrite creates merge chaos)
• Contacted the git host to clear cached views, since old commits stay fetchable by SHA until the server garbage-collects them

Step 6: Guardrails so this never recurs

The fix isn't complete until the next leak is blocked at commit time:

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/gitleaks/gitleaks
    rev: v8.18.4
    hooks:
      - id: gitleaks

Plus the same scan as a required CI check (for people who bypass hooks with --no-verify — you know who you are), and a quarterly full-history scan on a schedule. Since rollout: 6 attempted commits with live-looking secrets, all blocked before they ever reached the remote. 🚀

Lessons Learned

1. Rotate first, rewrite second. Always. A rewritten history with live keys is a false sense of security. The 38 still-valid credentials were the actual incident; the history rewrite was cleanup.
2. Bias your AI triage toward false positives, and say so explicitly in the prompt. Default LLM behavior optimizes for "probably fine." For security work you must invert that, and it's one sentence of prompting.
3. Context beats patterns for triage. The scanner's regexes produced 3,407 candidates; reading the surrounding code at the offending commit is what got to 121. That reading step is mechanical, unbounded, and exactly what agents are for.
4. Spot-check the buckets you trust least. I audited samples from all three verdict classes, not just the "real" pile. The scary failure mode is a real secret sitting in the false-positive bucket where nobody looks.
5. History rewrites are 20% git, 80% coordination. The filter-repo command took minutes. The freeze window, PR cleanup, re-clone instructions, and host-side cache clearing took days. Budget accordingly.

What's Next

Two things I'm building on top of this:

• Pre-merge triage: running the same context-aware classification on every PR's diff, so a leaked key gets a "this looks live, here's why" comment within minutes instead of surfacing in next quarter's audit.
• Validity probing as a scheduled job: the "is this credential still live?" check was the single highest-signal step, so it's becoming a recurring read-only job against anything the scanner flags.

Wrap-up

If you take one thing from this post: run a full-history secret scan on your oldest repo this week. Not HEAD — history. You will probably not like what you find, but you'll like it a lot more than an attacker finding it first.

If you found this useful, follow me here on Dev.to — I write about practical AI-agent engineering: real workflows, real numbers, and the failure modes nobody puts in the launch post. Got a git-history horror story of your own? I'd genuinely love to hear it in the comments. 💬