I Built a Security Scanner That Uses Your Python AST — Not an LLM, Not a Cloud Service

#security #python #opensource #devtools

A project called shannon hit GitHub trending this week with a 96% success rate on penetration testing benchmarks. Autonomous AI that reads your source code, finds vulnerabilities, and executes working exploits to prove they’re real.

I’ve been watching the AI security space and that number is legitimately impressive. But it also raised a question I couldn’t stop thinking about: how many basic security issues are sitting in codebases right now that don’t need a 96%-accurate AI pentester to find?

Pattern matching. AST analysis. Zero LLM. Things Python has been able to do for years.

So I built vuln-grep to answer that question on my own code.

What it detects

vuln-grep scan ./src

Sample output:

Scanned 43 files — 3 finding(s):

[HIGH] SQL_INJECTION  api/views.py:47
  SQL query built with f-string — use parameterized queries
  → query = f"SELECT * FROM users WHERE id = {user_id}"

[HIGH] UNSAFE_EVAL  utils/sandbox.py:12
  eval() called with dynamic argument — potential code injection
  → result = eval(user_input)

[MEDIUM] UNSAFE_YAML  config/loader.py:8
  yaml.load() without Loader= is unsafe — use yaml.safe_load()
  → data = yaml.load(f)

File path, line number, what’s wrong, and the actual line.

The full rule set

Rule	Severity
`eval()`/`exec()` with dynamic input	HIGH
`subprocess` with `shell=True`	HIGH
`pickle.load()` on untrusted data	HIGH
SQL queries via f-strings or `%` formatting	HIGH
Hardcoded passwords, API keys, tokens	HIGH
`DEBUG = True` in source	HIGH
`yaml.load()` without `Loader=`	MEDIUM
`assert` used for security checks	MEDIUM
`Access-Control-Allow-Origin: *`	MEDIUM
`hashlib.md5` / `hashlib.sha1`	LOW

Why AST instead of regex

Regex on source code misses context. It’ll flag every eval including eval inside a string literal, miss exec stored in a variable, and can’t distinguish subprocess.run([cmd]) from subprocess.run(cmd, shell=True).

AST analysis understands the code structure. vuln-grep knows when eval() is called with a constant (safe) vs. a variable (potentially dangerous). It knows when subprocess has shell=True as a keyword argument. That distinction matters.

The SQL injection rule even covers f-strings specifically:

# This gets caught:
query = f"SELECT * FROM users WHERE name = {name}"

# This too:
query = "SELECT * FROM users WHERE name = " + name

# This doesn’t (it’s parameterized, which is correct):
cursor.execute("SELECT * FROM users WHERE name = %s", (name,))

CI integration

- name: Security scan
  run: vuln-grep scan . --severity MEDIUM --fail-on HIGH

Exits non-zero if HIGH findings are found. Plug it into any CI pipeline.

What I found on my own code

I ran it on 5 of my repos. Found:

2 yaml.load() calls without Loader= (MEDIUM)
1 subprocess call with shell=True that didn’t need to be (HIGH)
A variable named api_key set to a placeholder string that the rule flagged as suspicious (LOW — a false positive, but a useful one to review)

None of these needed an AI to find. They needed a tool that would actually look.