What is Secfoo?
Secfoo is an open-source CLI that turns an AI coding agent into a disciplined security reviewer. Instead of building its own scanner, it hands your codebase and a specific review checklist to an AI agent you already have like Claude Code, Cursor, Gemini CLI, Codex, and few others - lets it actually read the code the way a human security reviewer would.
You pick a target (a local folder or a GitHub repo), pick one or more checks to run, and Secfoo does the rest: renders the right prompt, runs the agent, and stores the results in a local dashboard you can browse in your browser. It can run 8 different kinds of review — architecture, threat modeling, SAST, SCA, secret scanning, prompt review, deployment readiness, and responsible AI compliance.
Part of my internship on Secfoo was real simple: install it, use it like a real user would, and see if it actually does what it claims.
One of its agent options broke that promise on the very first real test I ran. Here's exactly how I proved it, because "the AI made something up" is a big claim, and I wanted receipts, not a hunch.
The setup
Secfoo has a "no coding-agent CLI needed" option — give it an API key, and it talks straight to a model instead of needing Claude Code or Cursor installed. I pointed it at a small folder in Secfoo's own codebase and asked for a SAST (static code analysis) scan:
secfoo run --skill sast --target src/secfoo/report --agent secfoo
It came back successful, with 3 findings — 1 High, 1 High, 1 Medium. Confident-looking report, proper CWE IDs, CVSS vectors, even illustrative fix diffs.
The first red flag
Secfoo saves the exact prompt it sends to the model, alongside the report. So before trusting the output, I checked what the AI had actually been shown:
grep -c "def \|class \|import " prompt.md
# 0
Zero. Out of 161 lines. The entire prompt was instructions and a single line:
## **Target**
Inspect the codebase at: `/path/to/src/secfoo/report`
That's it. No file contents. Nothing. The model was told to review code it was never actually shown.
The proof it made things up
The report cited three "High/Medium" vulnerabilities, each with a specific file path:
-
db/query_handler.py:15— SQL injection -
template/render.py:45— XSS -
file_upload/upload.py:30— path traversal
None of those files exist. Anywhere in the project. I checked with find. The folder I scanned only has 7 files, and not one of them is named any of those. The AI had nothing real to look at, so it generated the textbook example anyone would expect from "write me a SAST report" with zero actual input — and reported it with full confidence, including the line "Confidence in this assessment is moderate to high."
Ruling out "maybe the AI is just unreliable"
Before calling this a tool bug rather than a model limitation, I ran the exact same skill, on the exact same folder, through a different built-in option in Secfoo — one that actually reads files first. That one:
- Correctly named all 7 real files in the folder
- Cited a real, specific line of code accurately — down to the exact syntax
- Found 0 issues and said so honestly, instead of padding the report to look thorough
Same tool, same folder, same day. One option told the truth. The other one guessed and called it confidence.
Why this matters more than a normal bug??
A security tool's entire value is that you can trust what it tells you. A tool that invents vulnerabilities is worse than one that finds nothing — it burns your time chasing ghosts, or worse, gives you false confidence that a real issue doesn't exist.
The good news: this is now fixed — the team removed the broken implementation. But it's a good reminder for anyone building on top of an LLM: "the run succeeded and returned a well-formatted answer" is not the same claim as "the answer is true." If your pipeline can check its own inputs the way I did here — just read the prompt you're about to log — do it. It's a five-second check that would have caught this before it ever shipped.
Top comments (0)