DEV Community

mattybellx
mattybellx

Posted on

I Built an Offline SAST Scanner — Try It on Your Code and Tell Me Where It Fails

#security #python #secdev #devsec

https://github.com/mattybellx/Ansede

I've been working on this on and off for about a year. It started because I was reviewing a Flask app and Bandit gave it a clean bill of health — but it had an IDOR vulnerability where any logged-in user could access anyone else's invoices. Bandit saw the @login_required decorator and moved on. Semgrep OSS did the same thing.

So I started writing checks for the stuff they miss — not just "is there an auth decorator" but "does this route verify that the current user owns the resource they're accessing." That turned into a whole scanner.

What it does:

  • Finds IDOR (CWE-639), missing auth (CWE-862), ownership bypass (CWE-285) — the access control stuff
  • Also the standard injection/XSS/path traversal stuff
  • Works on Python, JavaScript/TypeScript, Go, Java, C#, Ruby, PHP
  • Fully offline, no dependencies beyond Python itself, no telemetry

Numbers (take with salt, I ran these myself):

  • 100% recall on a corpus of 164 CVEs I put together (Semgrep OSS got ~23%, CodeQL ~34% on the same set)
  • Scanned 33 real GitHub repos — found 1,255 things vs CodeQL's 167 (but I'm the first to admit some of those are probably noise)

Where I think it's weak and would love help:

  • The corpus I benchmarked against is only 164 CVEs. I need more real-world test cases
  • Java support is structural but doesn't have the deep Spring Security modeling yet
  • I'm sure there are false positives I haven't caught — I've been staring at this code too long to see them
  • The Go and Ruby analyzers are pretty basic

If anyone wants to try it on their own code and tell me what it gets wrong, I'd genuinely appreciate it. Or if you want to look at the benchmarking methodology and point out flaws, even better.

pip install ansede-static

Repo has the full methodology for the CVE recall test and the 3-tool comparison. The comparison scripts are in benchmarks/ if you want to reproduce or critique them.

2. Reddit r/Python

Where: https://www.reddit.com/r/Python/submit
Rules: Must be Python-related. Self-promo is okay if you're upfront about it and engage in comments. Don't just drop a link and leave.
How not to get removed: Use the "Discussion" or "Showcase" flair. Write a real post, not just a link. Python community appreciates "I built a tool, here's what I learned."

Title: I built a Python SAST scanner and would love feedback — especially on what it gets wrong

Flair: Discussion

Text:

Been working on a static analysis tool for about a year. It's written in Python (with a Rust core for the JS/TS parser), and I'm looking for people to try it on their code and tell me where it falls down.

The short version: it's like Bandit or Semgrep but focused on access control vulnerabilities — IDOR, missing authorization, ownership bypass. The kind of stuff where just checking for a decorator isn't enough.

You can install it with pip:

pip install ansede-static
Enter fullscreen mode Exit fullscreen mode

Then just point it at your code:

ansede-static your_project/ --verbose
Enter fullscreen mode Exit fullscreen mode

It supports Python, JS/TS, Go, Java, C#, Ruby, and PHP, but Python and JS are the most mature.

What I'm looking for:

  • False positives on your real code. I've tested it on 33 repos but that's nothing
  • Languages or frameworks where it's useless — be honest, I need to know where not to spend time
  • The benchmarking methodology: does it hold up? The comparison scripts are all in the repo

If you try it and it's terrible on your codebase, please tell me. That's more useful than "nice project" comments.

GitHub: https://github.com/mattybellx/Ansede

3. Reddit r/netsec

Where: https://www.reddit.com/r/netsec/submit
Rules: Technical security content only. No vendor marketing. Self-posts are preferred over direct links. They're picky — low-effort posts get removed.
How not to get removed: Write a technical self-post explaining the methodology. The comparison data is interesting to this crowd. Don't just link the repo.

Title: I benchmarked 3 free SAST tools against 164 CVEs — would appreciate a sanity check on my methodology

Text:

I've been building a SAST scanner and wanted to know if it actually works, so I put together a corpus of 164 CVEs across Python, JavaScript, Go, Java, and C# and ran my tool against Semgrep OSS and CodeQL on the same set.

Results (my tool / Semgrep OSS / CodeQL):

  • Python: 100% / 18.8% / 31.3%
  • JavaScript: 100% / 25.0% / 37.5%
  • Go: 100% / 22.2% / 33.3%
  • Java: 100% / 20.0% / 30.0%
  • C#: 100% / 30.0% / 35.0%

Before anyone asks: yes, I built the tool being tested. Yes, that's a conflict of interest. The corpus, the comparison scripts, and the full methodology are all in the repo — I'm posting this specifically because I want someone who isn't me to look at the methodology and tell me if it's fair.

Things I'm worried about:

  • Corpus selection bias: I may have unconsciously picked CVEs my tool handles well
  • The CodeQL setup might not be optimal — I used the default query packs
  • Sample size: 164 CVEs isn't huge

If anyone wants to audit the methodology or run the comparison themselves, everything is in benchmarks/. The CVE corpus is in benchmarks/fixtures/. I'd genuinely appreciate criticism.

The tool itself is at https://github.com/mattybellx/Ansede if you want context, but the benchmarking methodology is what I'm hoping for feedback on.

Top comments (0)