DEV Community

Varun Poojari
Varun Poojari

Posted on

One command, four minutes, $3.73: what an AI security review found in OWASP NodeGoat

My last two posts were about AI security scanners getting things wrong: one reported bugs in code it never read and one showed $0.00 for a cost nobody knew.

Fair question after all that: what does a run look like when it works and how do you check it?

So here's a real one. No edits, no cherry-picking. secfoo, the open-source (MIT) tool our team at RakFort builds, pointed at OWASP NodeGoat, with the same checks I used to catch those earlier bugs.

The command

uvx secfoo run --skill sast --skill secret-scanning \
  --target https://github.com/OWASP/NodeGoat --fail-on high
Enter fullscreen mode Exit fullscreen mode

That's the whole setup. No config file. secfoo picks whichever coding agent you already have installed (this run used Claude Code), clones the repo, hands the agent a fixed security brief and collects a report in a fixed format.

NodeGoat is a deliberately vulnerable Node.js/Express/MongoDB app that OWASP uses for training. That makes it a good test: its bugs are planted on purpose and documented, so you can check the scanner against a known answer.

The result

The run is from 14 November, using quick depth, with both skills running at the same time.

Skill Critical High Medium Low Time Cost
Static code analysis 0 6 5 1 4 min 12 s $2.34
Secret scanning 0 3 2 1 2 min 15 s $1.39

That's 18 findings, 9 of them High, in about 4 minutes, for $3.73. The run ended with exit code 2, because --fail-on high did its job.

The Highs:

  • eval() on a POST body, which lets a logged-in user run code on the server (CWE-95)
  • SSRF, where the user controls the whole URL the server fetches, and the response comes back as HTML (CWE-918)
  • NoSQL injection through a MongoDB $where string (CWE-943)
  • IDOR, where allocations are looked up by the user ID in the URL instead of the session (CWE-639)
  • A missing admin check on /benefits (CWE-862)
  • Template autoescaping turned off globally, so every field is an XSS risk (CWE-79)
  • An RSA private key, the session-signing secret and an AES key, all committed to the repo

What one finding looks like

Here's the first one, trimmed a little:

[HIGH] F1: Server-side JS injection via eval() on POST body
Location: app/routes/contributions.js:32-34
The contributions update handler passes three fields from req.body straight into eval(). Any authenticated user (self-registration is open at /signup) can POST preTax=require('child_process').execSync('id').toString() ... The isNaN validation runs after the eval, so it does not constrain the payload.

const preTax = eval(req.body.preTax);
const afterTax = eval(req.body.afterTax);
const roth = eval(req.body.roth);
Enter fullscreen mode Exit fullscreen mode

Recommendation: Replace eval() with parseInt(value, 10) (or Number()), reject NaN, negative and out-of-range values, and validate the body with a schema before use.

Every finding has the same shape: CWE, OWASP category, a CVSS vector, the file and line, the vulnerable code and a fix. I like the detail that the isNaN check runs after the eval. That's the kind of thing a quick pattern match would miss.

How I checked it wasn't lying

These are the same checks from my earlier posts, applied to this run.

1. Did it find the planted bugs? NodeGoat ships tutorial pages that list its vulnerabilities. The report lines up with them, including the isAdmin middleware that's deliberately commented out.

2. Did it read the real code? Every finding names a real file and line range (app/routes/research.js:15-26, app/data/allocations-dao.js:77-79 and so on). The report from my earlier post, the one where the agent never saw the code, couldn't name a single real function. This one names them all.

3. Is the cost real, or a default? $3.73 is spend reported by the agent itself, not a fallback zero.

4. Did "success" mean success? Exit code 2 means "findings broke your policy". secfoo now uses separate exit codes for the outcomes that matter: 0 clean, 1 the scan itself broke, 2 findings gate, 3 cost gate. A broken scan can't pass as a clean one.

What I also liked: it says when it's unsure

Two findings aren't marked "Confirmed":

  • Hardcoded secrets are marked Conditional, since they only matter if those values reach production.
  • A ReDoS on the bank routing validator has its OWASP category marked unverified rather than forced into a box.

The report also lists what it didn't review (node_modules, vendored front-end bundles, tests, the static tutorial pages). A scanner that tells you its blind spots is easier to trust than one that sounds sure about everything.

Honest limits

  • One run, quick depth. Model output varies, so if you run it yourself the counts and wording will differ a little.
  • A known target. NodeGoat is a fair test of "does it find real bugs", but it's easier than a real codebase, where nobody has written down the answers.
  • $3.73 is not nothing. That's fine for a test or a release check, but you'd want --max-cost in CI. That's the cost gate behind exit code 3.

Try it yourself

The full reports, the SARIF file (18 results, ready for GitHub code scanning) and the terminal output are all in the repo:
examples/sample-output/nodegoat

Or run it:

pip install secfoo
secfoo run --skill sast --skill secret-scanning --target https://github.com/OWASP/NodeGoat --fail-on high
secfoo serve   # browse the results in a local dashboard
Enter fullscreen mode Exit fullscreen mode

secfoo is open source: github.com/secfoo-com/secfoo. If it's useful, a star helps a lot. If you run it and get different results, I'd really like to see them in the comments.

What deliberately vulnerable app should I point it at next?

Top comments (0)