Secret scanning on public code is the one breach-monitoring technique where a solo analyst with a laptop and a free CI runner genuinely competes with vendor tooling. The vendors sell enterprise dashboards; the physics of the problem is simple enough that a well-tuned script wins on latency. Here is the operational detail that separates a working scanner from a noise machine.
Pattern layer: three tiers, not one regex list.
-
Tier A - format-checkable keys. Cloud and SaaS tokens have checksum-able formats:
AKIA[0-9A-Z]{16}with Luhn validation on AWS keys,gh[pousr]_[A-Za-z0-9]{36,},AIza[0-9A-Za-z\-_]{35}(Google, 39 chars, reject on length),xox[baprs]-[0-9a-zA-Z-]{10,}, GitLabglpat-[A-Za-z0-9_\-]{20,}, Slack app tokens, Stripesk_live_/rk_live_. Validate the format and the checksum; Luhn on AWS keys alone kills 90% of false positives from documentation. -
Tier B - structural secrets. Private keys (
-----BEGIN (RSA|EC|OPENSSH) PRIVATE KEY-----), JWTs (eyJ[A-Za-z0-9_\-]+\.eyJ), PEM blocks, self-hosted tokens with known prefixes. Decode the JWT payload (it's base64 JSON, no signature needed) and checkiss/audclaims against your target list - a JWT naming your target's auth server is a hit even if you can't verify the signature. -
Tier C - entropy + context. High-entropy strings (
[A-Za-z0-9+/]{40,}with Shannon entropy > 4.2) near assignment keywords (secret,key,token,password). Tier C alone is a spam cannon; Tier C restricted to file paths (config/,.env,settings, CI YAML) is a decent net.
Scope layer: search target strings, not the whole planet. Code-search APIs let you query for target identifiers (domains, org names, product names) and run the pattern layer over the result set. Polling the whole code index for Tier C is impossible; polling target-scoped queries every 6 hours is free.
Dedup layer: hash the secret, key by (repo, secret-hash). A leaked key in 300 forks = one finding. A key rotated (same format, new value) = a new finding, and the rotation timestamp is a remediation signal.
Latency is the product. The vendor tools run scheduled enterprise scans; your 6-hour cadence on target-scoped queries usually sees a new public exposure first. The metric to log per finding: exposure timestamp, first-detection timestamp, removal timestamp. Median detection-to-remediation is the number you report.
Scope discipline again: in bug-bounty programs, an exposed key is a reportable finding, not an access opportunity. Report through the program. The watching is the product; the touching is the lawsuit.
The full pattern list (60+ formats with validators), the JWT-claim trick, and the dedup schema are in the Telegram & Web OSINT Bundle ($5). Free sample brief shows the output format.
Runs free on GitHub Actions - no server, no paid APIs.
Top comments (1)
tr.ee/dev-to