I Benchmarked 4 SAST Tools Against 164 CVEs — Here's What Found 100% of Them

#security #python #devops #tutorial

If you're building software in 2026, you probably have some form of security scanning. Bandit for Python. GitHub's built-in CodeQL. Maybe Semgrep.

But how good are these tools actually — tested against real, known vulnerabilities?

I curated 164 CVEs across 5 languages and ran 4 free SAST tools. Results:

Ansede: 100% (164/164)
CodeQL: 33.6%
Bandit: 30.9% (Python only)
Semgrep CE: 23.2%

The gap is authorization bugs — IDOR, missing auth. Most free tools analyze single functions. Detecting IDOR requires tracing HTTP routes → auth guards → database queries across function boundaries.

@app.route("/invoice/<id>")
def get_invoice(id):
    return Invoice.query.get(id)  
    # All 3 other tools: silent
    # Ansede: IDOR — any user can view any invoice