Two days ago Fafa flagged a problem with our outreach pitch: we tell prospective users getdebug catches AI-app security bugs, but our own benchmark was a handful of one-bug micro-fixtures. Useful for unit testing the detector. Useless for the question a real developer asks: does this catch the bugs my app actually has?
So we built CodeSecBench Tier C — a public corpus of six deliberately-vulnerable AI apps, each ~40 files, each carrying 12–18 labeled bugs across all six AI-app security categories. The truth lives in a separate public repository so scanners never see the labels at scan time. The first two repos are live; this post is the first calibration cycle.
The benchmark, in one paragraph
Six target repos under getdebug-ai/cst-* (cst- = CodeSecBench Tier C), each a different stack: Next.js + Vercel AI SDK, Vite + Express + LangChain.js, SvelteKit + Anthropic, Express + tool-calling agent, FastAPI + Python, CrewAI multi-agent. Total corpus: ~89 vulnerable rows + 33 safe near-misses + 27 borderline cases = 149 labeled lines. Each repo has a known-safe.ts hallucination control file — any scanner finding inside it is a guaranteed false positive. The truth lives at getdebug-ai/codesecbench-truth; the "don't peek" norm is documented in the README, same as every honest public benchmark.
File paths in each repo are randomized — domain-appropriate, not template-matched. A vendor allowlisting lib/user.ts won't generalize from repo #1 to repo #2 (server/services/personalization.ts). The benchmark measures detection skill, not memorization.
Running getdebug 0.5.1 against the first two repos
We ran getdebug analyze . --quiet --json against the two completed repos and scored against the truth file using a span+tolerance JOIN scorer (any finding whose line span overlaps a truth row's span, ±5 lines, credits the row).
23% recall. Far short of where we'd need to be for an outreach pitch. But the data is useful — the misses cluster. Both repos missed the same canonical CWE patterns:
- Shell & SQL injection via args.X — the canonical SDK tool-callable shape. Repo #1's execAsync(args.command) and repo #2's sql.unsafe(args.query) are both classic CWE-78 / CWE-89 sinks. The detector's existing regex only matched exec(tool.input.X) — the SDK's typed-tool-ref form. Real code uses args.X, where args is the typed function parameter. API key returned in JSON response body — the second-most-common Next.js / Express leak after NEXT_PUBLIC_. Pattern is Response.json({apiKey: process.env.X_API_KEY}). The detector had no rule for this shape at all. The fix: two new regexes, sixty minutes Both gaps are regex-detectable. Pattern A — the args.X form — needed only to add args as a valid identifier prefix alongside tool, block, toolUse, etc., plus extend the sink list to include SQL: sql.unsafe, db.unsafe, pool.unsafe, db.prepare:
var unsafeToolOutputArgsRe = regexp.MustCompile(
\b(?:exec|execSync|execAsync|spawn|spawnSync|eval|run|runCommand|,
runSync|sql\.unsafe|db\.unsafe|db\.query|pool\.unsafe|pool\.query|
client\.unsafe|client\.query|db\.prepare)
\s*\(\s*[^)]{0,160}?\bargs\.\w+
)
Pattern B — the key-in-response form — is a fresh detector with a response-context anchor:
var keyInResponseRe = regexp.MustCompile(
(?s)(?:Response\.json|res\.json|res\.send|return\s+json|,
return\s+Response\.json)
\s*\(\s*\{[^}]{0,400}?
(?:apiKey|api_key|secret|token|key)\s*:\s*
process\.env\.[A-Z][A-Z0-9_]*(?:KEY|TOKEN|SECRET)
)
Both new patterns ship with explicit negative tests. Parameterized SQL via the tagged template (sqlSELECT * FROM users WHERE id = ${userId}) doesn't fire. Legitimate SDK construction (new OpenAI({apiKey: process.env.OPENAI_API_KEY})) doesn't fire. The point is to catch new shapes, not over-fire on safe ones.
The numbers after 0.5.2
The 50pp jump on client-side-llm-key means both repos' #2 carrier (server route returning the key in a response body) is now caught. The 33pp jump on unsafe-tool-output means the canonical CWE-78 and CWE-89 sinks — execAsync(args.command) and sql.unsafe(args.query) — are caught. These are real-world patterns, not contrived; we saw both in the wild while building the corpus.
What didn't move: unsafe-role-merge, prompt-injection, and one of the unbounded-stream rows in repo #1. The unbounded-stream miss is a label issue — the detector hit at line 42 (stream: true) while the truth label is at line 48 (the for-await loop), 6 lines apart, outside the ±5 tolerance. Widening that label to a span fixes it; we'll do that in the v0.1.1 truth release. The role-merge and prompt-injection misses are real detector gaps, and they're the next calibration target.
The loop
Each repo becomes a learning artifact. Add a repo → score with the current tool → identify the gaps → ship detector fixes → re-score all earlier repos with the new version → build the next repo → repeat. The /bench page tracks the time-series. Each calibration cycle gets a blog post.
Next: cst-sveltekit-stream (#3) is in author-mode now. SvelteKit puts the system message in a separate top-level parameter (anthropic.messages.create({system: "...", messages: [...]})), not as a role inside the messages array. The existing role: "system" detector won't see it. That's the kind of stack-specific blind spot the corpus exists to surface.
Try it yourself
If you're building a SAST tool that targets AI-app categories, CodeSecBench is for you. The corpus + truth file are MIT licensed, and there's a vendor-side scorer at codesecbench-truth/score.js (zero deps, ~120 lines). Run your tool against the public targets, JOIN against the truth, open a PR with your results. The don't-peek norm is the only ask.
If you're a developer wondering whether your own AI app has any of these patterns today: npm i -g @getdebug/cli and getdebug analyze .. 0.5.2 lands on npm with the next release; the LLM-augmented pass (--local-llm) is already there.
Top comments (0)