I opened HERALD this morning and two stories sat next to each other on the same digest: a Hacker News thread about a RodeCaster Duo audio interface shipping with SSH enabled by default, and a Zenn post from a Japanese dev who realized his .zshrc was exporting raw API keys straight into Claude Code's environment.
Different products. Same shape of bug.
The default state leaks.
I've been building CodeHeal, a security scanner for AI-generated code, for about two months now. And every time I read stories like these, the same uncomfortable thought hits: the people in the comments aren't careless. They just trusted the defaults.
The pattern that won't go away
Here's the Zenn author's story, paraphrased: "I had export OPENAI_API_KEY=sk-... in my zshrc. I started Claude Code. The agent saw it. I didn't realize until I checked the session log."
That's not a misuse. That's the tool working exactly as designed. Shells inherit environment variables. Agents inherit shells. There's no vault between them.
The RodeCaster story has the same skeleton in hardware: the firmware ships with sshd listening, default creds, no on-device warning. The user did nothing wrong. The default did everything wrong.
Writing the secret detector was the worst week
When I started writing CodeHeal's secret-detection rules, I assumed the hard part would be regexes. It wasn't. The hard part was deciding which "looks like a secret" to actually flag.
-
.env.examplewithOPENAI_API_KEY=your-key-here→ don't flag -
config.tswith// TODO replace laternext to a real-looking JWT → flag hard - A README snippet showing
Authorization: Bearer abc123→ depends on entropy - A test fixture with
password: "test123"→ context-dependent
I rewrote that detector four times before I trusted it. The first version was a regex spam factory — every variable named *_key lit up red. The second version overcorrected and missed real leaks because they didn't have "key" in the name. The fourth version is the one I ship: entropy + naming heuristics + neighborhood scan for placeholder markers.
It's not glamorous code. It's the code that took the longest.
What I actually see when I scan raw AI output
I have a habit now: every time a new model drops, I paste the same prompt — "build me a Stripe webhook handler" — and run CodeHeal on the output untouched.
In the last three runs:
- Hardcoded
STRIPE_WEBHOOK_SECRETas a string literal (twice) - A
STRIPE_SECRET_KEYleft in a comment block (// for testing) - One file with no secrets but a SQL query built via template-literal concatenation — different category, same vibe of "the model didn't think this was wrong"
None of these would survive a code review. All of them would survive not running a code review, which is increasingly what's happening when developers paste AI output straight into a PR and click merge.
Why static analysis, specifically
I get asked: "Why not just have an LLM review the LLM's output?"
Because the LLM that wrote the code didn't think the hardcoded secret was a problem. It put it there. Asking another LLM, trained on roughly the same distribution, to flag it is asking a fish to notice water.
Static analysis doesn't have an opinion. It pattern-matches against known-bad shapes. sk_live_ followed by 24+ chars of base58 → flag, full stop. No "well, in this context maybe it's a placeholder." That bluntness is the feature, not a limitation.
CodeHeal currently runs 93 rules across 14 categories. Secret leakage is one of those categories. The boring categories — SQL injection patterns, unsafe deserialization, missing CSRF tokens — actually fire more often. But secrets are the ones that make developers audibly swear when the report comes back. Nothing else has that effect.
The boring takeaway
Two stories on one morning's HERALD digest, both about defaults that leak. The fix isn't "be more careful." Humans aren't more careful. The fix is a check that runs whether you remember to run it or not.
Pre-commit hook. CI step. Whatever. Just something between your AI's output and your git push.
If this hit a nerve: CodeHeal's free tier (5 scans/day, no signup) is at scanner-saas.vercel.app/scan. Paste the next thing your AI coding assistant generates. See what it caught that you didn't.
Top comments (0)