DEV Community

Cover image for 65-75% of Claude's Code Shipped a Vuln. Hardcoded Secrets Are the One AI Agents Can Auto-Fix.
Ofri Peretz
Ofri Peretz

Posted on • Edited on • Originally published at ofriperetz.dev

65-75% of Claude's Code Shipped a Vuln. Hardcoded Secrets Are the One AI Agents Can Auto-Fix.

#security #eslint #node #ai

Ask an AI assistant to "wire up Stripe" or "connect to the database" and watch
what it produces:

const stripe = new Stripe("sk_live_51H8xY2eZvKf..."); // demo key it left in
const db = new Pool({ password: "changeme" }); // placeholder it forgot to remove
const JWT_SECRET = "your-secret-key"; // the classic
Enter fullscreen mode Exit fullscreen mode

Coding assistants optimize for "runs on the first try," and the fastest path to
runnable code is a literal in place. So they hardcode demo keys,
placeholder credentials, and bare config literals — at the speed they
generate everything else. That's CWE-798 (Use of Hard-coded Credentials),
and it now enters codebases faster than any human ever added it. When I had
Claude generate 80 common Node.js functions with no security context,
65–75% shipped with a vulnerability
— and a hardcoded credential was one of the most repeated patterns. The model
isn't being careless; it's being fast, and a literal is the fastest thing to
type.

Here's the twist that makes this fixable rather than just alarming: the same
property that makes AI a prolific source of these bugs — it reads and writes
structured text — makes it a capable fixer. eslint-plugin-secure-coding's
no-hardcoded-credentials rule emits a finding that carries the CWE, CVSS,
compliance tags, and the exact fix. Feed that back to the assistant and it
remediates its own output. This is the agentic-CI loop: AI writes → linter
flags in machine-readable form → AI fixes.

Hardcoded credentials are the rare case where that loop is genuinely closed.
For most vulnerability classes, "ask the model to fix it again" is a gamble — I
measured a fix-one-bug-get-two-more failure mode across three remediation
rounds
where prompt-only feedback regressed. CWE-798 is different: the remediation is a
single deterministic rewrite — hoist the literal to process.env — with no
behavioral branches for the model to get creative in. That's why this is the
one AI-introduced vulnerability worth wiring into an autonomous loop first.

Series — Hardening AI Agents: I Let Claude Write 80 Functions (65–75% had a vuln) → Hardcoded secrets: the one AI can auto-fix (you are here) → Claude wrote a NestJS service — ESLint found 6 holes. The thread: AI writes the bug, a machine-readable lint finding closes the loop.

TL;DR

  • AI assistants introduce hardcoded secrets (CWE-798) at scale — bare demo keys, placeholder passwords, and config literals left in source.
  • no-hardcoded-credentials (in eslint-plugin-secure-coding) catches them and emits a structured, CWE-tagged finding an AI agent can parse and auto-fix.
  • The detector is two-mode (registered key prefixes fire anywhere; generic secrets need a credential-named identifier) so it's quiet enough to run as a CI error. Full mechanism in the secure-coding deep-dive.

Why the lint error is written for the machine

A human reads error: hardcoded credential and sighs. An AI agent reads the
structure and acts. Point the rule at the literal Stripe key an assistant
left behind:

// src/payments.ts
import Stripe from "stripe";

const STRIPE_SECRET_KEY = "sk_live_51H8xY2eZvKfABCD1234";
export const stripe = new Stripe(STRIPE_SECRET_KEY);
Enter fullscreen mode Exit fullscreen mode

Run npx eslint . and that's the exact, unedited finding it prints — line,
column, and all (paste the file above into a fresh repo with the rule enabled
and you'll get it character-for-character):

src/payments.ts
  4:27  error  🔒 CWE-798 OWASP:A04-Cryptographic CVSS:9.8 | Hard-coded API key detected | CRITICAL [SOC2,PCI-DSS,HIPAA,GDPR]
   Fix: Use environment variable: process.env.STRIPE_SECRET_KEY or secret management service | https://cwe.mitre.org/data/definitions/798.html
Enter fullscreen mode Exit fullscreen mode

Every token in that line is a machine signal — and the rule derives them, it
doesn't hand-type them: the CWE drives an auto-enrichment table
(@interlace/eslint-devkit) that fills in the OWASP category, the CVSS score,
and the compliance set, so the finding can't drift out of sync with the
vulnerability class. Read left to right:

  • CWE-798 — a stable, machine-readable vulnerability class the model has seen thousands of times in training; it knows the remediation pattern.
  • OWASP:A04-Cryptographic — the OWASP Top 10 (2025) bucket CWE-798 maps to (A04 is Cryptographic Failures in the 2025 list — secrets in source are a key-management failure), so the finding slots straight into an OWASP report.
  • CVSS:9.8 + CRITICAL — the severity the agent uses to prioritize this over a style nit.
  • [SOC2,PCI-DSS,HIPAA,GDPR] — the compliance frameworks the finding maps to, for an audit trail the agent can cite.
  • Fix: — the exact transformation (→ process.env.STRIPE_SECRET_KEY, derived from the variable name), so the edit is deterministic, not a guess.

Drop that into Cursor/Copilot/Claude (or an autonomous CI agent) and the fix is
mechanical: hoist the literal to an environment variable or a secret manager.
The rule turns a vague "be secure" instruction into a closed, verifiable loop.

Want to see it fire on your own repo right now? Two lines:

npm install --save-dev eslint-plugin-secure-coding
npx eslint .   # any hardcoded sk_live_… / password: "…" lights up as an error
Enter fullscreen mode Exit fullscreen mode

(Full config and per-repo tuning — allowing test fixtures, etc. — is in
Install below.)

The fix the rule wants 

// ✅ no literal in source; the secret comes from the environment
const stripe = new Stripe(process.env.STRIPE_SECRET_KEY);
const db = new Pool({ password: process.env.DATABASE_PASSWORD });
// for higher assurance: fetch from AWS Secrets Manager / Vault at runtime
Enter fullscreen mode Exit fullscreen mode

The rule flags the bare literal — new Stripe("sk_live_…"),
password: "changeme", const JWT_SECRET = "…" — and its suggested fix hoists
it to a process.env reference, the exact shape the agent should have produced.
(One deliberate nuance: a value that's already an env read with a literal
fallback — process.env.X || "dev-default" — is treated as already
remediated, since the real secret lives in the environment; that form is the
rule's accepted output, not a finding. So the thing it catches is the bare,
env-less literal.)

Why this survives code review

If hardcoded secrets are so obvious, why do they keep reaching main? Because
the failure isn't ignorance — it's the review process itself. I've watched all
three of these wave a sk_live_… straight through:

  • It reads as a placeholder. password: "changeme" and JWT_SECRET = "your-secret-key" look like scaffolding the author will swap before merge. The reviewer pattern-matches "obvious dummy value" and moves on — and "before merge" never arrives.
  • It's buried in a green diff. The line lands inside a 400-line PR that adds a feature, passes CI, and does what the ticket asked. A reviewer scanning for logic bugs is not entropy-scoring every string literal; the secret rides in on the back of working code.
  • Nobody owns "is this a real key?" Telling a revoked test key from a live one isn't a judgment a human makes at review speed, so the question quietly doesn't get asked. With AI-generated PRs this is worse: the volume is higher, the author can't vouch for any individual line, and the literal looks exactly like the thousands of legitimate ones the model emitted.

A blocking lint rule fixes the one thing humans are structurally bad at here:
applying the same boring check to every literal, on every PR, without
fatigue. That's the case for making it an error, not a warning — a warning
gets the same "I'll fix it later" treatment as the placeholder did. Put bluntly:
the credential didn't survive review because nobody looked — it survived
because looking at every string literal isn't a job a human can do, and it is
the only job a linter does.

How it stays quiet enough to be an error

A naive secret scanner drowns you in false positives, which trains everyone
(human and agent) to ignore it. no-hardcoded-credentials makes two
different decisions: registered vendor key prefixes (sk_live_, AKIA…)
fire anywhere because they're unambiguous, while a generic high-entropy string
is only flagged when the surrounding identifier names a credential
(apiKey, password, token) and clears a length floor. That context check is
load-bearing, and I have the receipt: an early, context-blind version of this
rule fired 842 times on the vercel/ai codebase — and the real count of
hardcoded credentials was zero. 807 of those "findings" were TypeScript
union-type literals, error class names, and the bare string "test". I walk
that whole false-positive autopsy in
what ground truth caught that unit tests missed.
The context gate is what dropped that to zero — and that low false-positive rate
is what lets you run it as a blocking CI error, and what makes an agent trust
the signal instead of suppressing it. The
secure-coding getting-started
walks the full two-mode mechanism and the rest of the security rules in the
plugin.

Here's the part that matters if you point this at AI output: vercel/ai is a
hand-written human library, and it still buried a context-blind rule under
807 false positives — because it names things experimental_onToolExecutionStart
and AI_ToolCallNotFoundForApprovalError. That long, underscore-laced,
type-literal-heavy texture is exactly what an LLM emits when it generates
TypeScript. Run a naive credential regex over a folder of Claude- or
Gemini-written code and you don't get a security report — you get noise
proportional to how verbosely the model named its symbols. So the two-mode
design isn't a nicety; it's the only reason this rule survives contact with the
code AI is now writing fastest.

And the "run it on AI output" part isn't hypothetical — I keep doing it. When I
gave Claude and Gemini the
same NestJS prompt, Claude shipped 6 security findings and Gemini 2,
and when I
ranked five models by the security of what they generate
the "best coder" wasn't the safest one. The throughline: which model you pick
changes how many of these literals land, but not whether they land — every
model leaves some. The two-mode rule is the constant that catches them
regardless of which assistant wrote the diff. (Want to reproduce the precision
claim on your own model? Generate the corpus with Gemini, run the
structural-only pass against the context-tiered pass, and report the delta —
same rule, swap the corpus.)

Install 

# npm
npm install --save-dev eslint-plugin-secure-coding
# yarn
yarn add --dev eslint-plugin-secure-coding
# pnpm
pnpm add --save-dev eslint-plugin-secure-coding
# bun
bun add --dev eslint-plugin-secure-coding
Enter fullscreen mode Exit fullscreen mode 
// eslint.config.js — `configs` is a NAMED export
import { configs } from "eslint-plugin-secure-coding";

export default [configs.recommended];
Enter fullscreen mode Exit fullscreen mode

Tune it for your repo (e.g. allow fixtures in tests):

import { configs } from "eslint-plugin-secure-coding";

export default [
  configs.recommended,
  {
    rules: {
      "secure-coding/no-hardcoded-credentials": [
        "error",
        { allowInTests: true },
      ],
    },
  },
];
Enter fullscreen mode Exit fullscreen mode

Compatibility

Surface Support
Package managers npm, yarn, pnpm, bun
Node >= 18.0.0
ESLint `^8.0.0 \
Module system CommonJS — {% raw %}eslint.config.js or .mjs
AI assistants the CWE/CVSS/compliance/fix message is plain text in the lint output — consumable by Cursor, Copilot, Claude Code, or an autonomous CI agent with no extra integration

Honest scope

  • It catches the literal in source, not key validity. It flags sk_live_…; it can't tell a revoked test key from a live one. Rotate anything that was ever committed.
  • Auto-fix needs a human gate for the secret value. The agent can hoist the literal to process.env.X deterministically, but where the real secret lives (env, Secrets Manager, Vault) is an architectural decision — the rule points at it, you choose it.
  • One rule, not a secret-scanning platform. Pair it with a history/secret scanner (commits already pushed) and rotation; this is the pre-merge gate that stops new ones — including the ones your AI just wrote.

Where this sits

This is one rule in eslint-plugin-secure-coding — a set of framework-agnostic
"pure coding security" rules (see the
full getting-started).
It's part of the Interlace ecosystem —
domain-specific static analysis whose findings are deliberately structured for
both humans and the agents now writing most of the code.

This piece is part of my Hardening AI Agents series. The
65–75% experiment
is where the headline number comes from; if you want the same
machine-readable-finding loop applied to a whole AI-written service, see
Claude wrote a NestJS service — ESLint found 6 security holes,
where hardcoded credentials were one finding among several in real generated
code. And for the framework-aware version of the same loop on agent code, see
securing AI agents in the Vercel AI SDK.

What's the worst hardcoded secret you've caught in an AI-generated PR — a live
sk_live_…, a real DB password, an internal token? And did it get caught by a
human, a scanner, or only after it shipped? Drop the story in the comments.

⭐ Star on GitHub if your AI assistant has ever left a "your-secret-key" literal in your source.

I'm Ofri Peretz, a security engineering leader and the author of the
Interlace ESLint ecosystem — domain-specific static analysis for security,
reliability, and performance on the Node.js stack, with findings structured for
the AI agents now writing the code.

ofriperetz.dev · LinkedIn · GitHub

Top comments (0)