A leaked API key is a bad day. A leaked seed phrase is your funds gone, permanently, with nobody to call.
If you're building on Ethereum or any EVM chain with an AI coding agent, that second scenario is closer than
it looks. Claude Code and Codex CLI write every tool call and every bit of tool output into local session
transcripts by design, so sessions can resume. That means anything the agent ever cat'd, printed for
debugging, or was told — including a .env file with a mnemonic in it, or "here's my test wallet, use this
key" — is sitting in plaintext on disk indefinitely.
I added Web3-specific detection to agent-audit, a tool I'd already
built for general secret scanning of these transcripts:
- BIP-39 seed phrase detection — checked word-by-word against the actual 2048-word spec wordlist, not a fuzzy guess. A valid mnemonic is, by definition, 100% real wordlist words, so there's no tolerance to tune.
-
Ethereum private keys — gated on context like
privateKey:, since a bare 64-hex-char string is the exact same shape as an ordinary transaction hash, which is everywhere in any Web3-adjacent log. - Infura/Alchemy RPC URLs with embedded project keys.
- Recognizes Hardhat's well-known public default test mnemonic (
test test ... junk) and reports it as informational, not critical — it controls no real funds by design and is in nearly every local Hardhat setup, so flagging it as a leak would just be noise.
Dogfooding caught two real bugs before I shipped this
First version used a 90%-match tolerance for the mnemonic check and extracted every lowercase substring in
the whole document before sliding a window over it. Against real logs, this flagged decompiled game-code
identifiers as "leaked wallets" — short common words like item, room, time getting stitched together
across unrelated code punctuation into a fake phrase that was never actually contiguous text.
Fixed both: mnemonics now require an exact 100% match (loses zero real detections, since that's literally
the spec), and word extraction only follows genuine space-separated runs, which code naturally breaks and
real text — prose or an actual pasted seed phrase — naturally satisfies. 10 new tests, including regression
tests for both bugs.
Free, MIT, zero dependencies, 100% local: https://github.com/CrypLed/agent-audit
npx github:CrypLed/agent-audit
Curious whether other Web3 devs have run something like this on their own agent logs and found something —
or whether this is a non-issue for people who are already careful never to paste a real key into a chat.
Top comments (0)