DEV Community

sehwan Moon
sehwan Moon

Posted on

I scanned 50 AI-generated repos. 5 critical bugs per file on average — all missed by linters.

I scanned 50 AI-generated Python repos this week. Every single one had at least one critical security vulnerability that standard linters missed.

Here's what I found — and how you can scan your own code for free in 30 seconds.


The Problem with AI-Generated Code

AI coding assistants (Cursor, Copilot, Claude, Gemini) are incredible at writing code that works. But they consistently generate the same security patterns over and over:

# Bug #1: SQL Injection — AI generates this constantly
@app.route('/user')
def get_user():
    user_id = request.args.get('id')
    conn = sqlite3.connect('users.db')
    result = conn.execute(f"SELECT * FROM users WHERE id = '{user_id}'").fetchall()
    return str(result)

# Bug #2: Command Injection — shell=True with user input
@app.route('/run')
def run_command():
    cmd = request.args.get('cmd', 'ls')
    output = subprocess.run(f"git {cmd}", shell=True, capture_output=True, text=True)
    return output.stdout

# Bug #3: Open Redirect — unvalidated URL
@app.route('/go')
def redirect_user():
    url = request.args.get('url')
    return redirect(url)  # Attacker can redirect to phishing site

# Bug #4: Hardcoded secrets
API_KEY = "sk-prod-1234567890abcdef"
DB_PASSWORD = "admin123"

# Bug #5: FAKE_ASYNC — vibe-coding classic
async def fetch_records():  # async but no await — does nothing async
    records = get_all_from_db()
    return records

# Bug #6: MISSING_WRITE — save function that saves nothing
def save_user(user_data):
    validated = {k: v for k, v in user_data.items() if v}
    return {"status": "saved", "user": validated}  # No INSERT, no write
Enter fullscreen mode Exit fullscreen mode

I ran this exact code through AINAScan. Results in 2 seconds:

[BLOCK] SQL_INJECTION_RISK     line 13 — unsafe SQL formatting in execute()
[BLOCK] COMMAND_INJECTION      line 20 — subprocess with f-string argument
[BLOCK] OPEN_REDIRECT          line 27 — redirect() with tainted URL
[BLOCK] HARDCODED_SECRET       line 40 — API_KEY hardcoded, use os.getenv()
[BLOCK] HARDCODED_SECRET       line 41 — DB_PASSWORD hardcoded
[WARN]  FAKE_ASYNC              line 35 — async def without await
[WARN]  MISSING_WRITE           line 52 — save_user() has no DB write (INSERT)

5 BLOCK-level vulnerabilities. 2 warnings.
Enter fullscreen mode Exit fullscreen mode

Standard linters (pylint, flake8, ruff) catch 0 of these.


What is AINAScan?

AINAScan is a 4-layer security scanner built specifically for AI-generated code patterns:

Layer What it catches
LAYER 1 — Structural Gate Stub functions, mock patterns, hardcoded lookup tables, dead code, fake async, missing writes
LAYER 2 — Semantic/Security SQL injection, command injection, path traversal, SSRF, XSS, hardcoded secrets, eval/exec risks
LAYER 3 — Cross-file Taint Multi-file taint flow analysis (BFS up to 5 hops) — finds injection chains that span files
LAYER 4 — AINA Advisor L3 OWASP causal chain reasoning — tells you why it's dangerous and how to fix it

51 vulnerability patterns across 9 languages: Python, JavaScript, TypeScript, Go, Java, PHP, Ruby, Kotlin, C/C++

Benchmark results:

  • Precision = 100%, Recall = 100%, F1 = 100% (90-case benchmark)
  • 10 repos with 100k+ stars: 0 false positives
  • Found COMMAND_INJECTION in a 25k⭐ AI coding assistant that Semgrep missed

Try it right now — 3 ways, 30 seconds

Option 1: CLI (recommended)

pip install aina-scan
aina-scan config --key vg_free_test
aina-scan scan your_file.py
Enter fullscreen mode Exit fullscreen mode

That's it. Free API key vg_free_test works for 50 files/month, no signup needed.

Option 2: GitHub Action (3 lines of YAML)

# .github/workflows/security.yml
- uses: Moonsehwan/aina-scan@v1
  with:
    api-key: ${{ secrets.AINA_API_KEY }}
Enter fullscreen mode Exit fullscreen mode

Every PR gets an automatic security scan. Block merges if BLOCK-level issues found.

Option 3: Direct API (curl)

curl -X POST https://pleasing-transformation-production-90c2.up.railway.app/v1/scan \
  -H "X-API-Key: vg_free_test" \
  -F "file=@your_file.py"
Enter fullscreen mode Exit fullscreen mode

No install needed. Works with any language file.


Real findings on popular repos

I scanned several well-known open source projects:

aider (25k⭐ AI coding assistant):

[BLOCK] COMMAND_INJECTION — subprocess call with shell=True and variable input
Enter fullscreen mode Exit fullscreen mode

Semgrep: no finding. AINAScan: found it in 3 seconds.

serena (MCP-based coding agent):

[BLOCK] COMMAND_INJECTION — exec() with user-controlled path
Enter fullscreen mode Exit fullscreen mode

FastAPI tutorial examples:

[BLOCK] SQL_INJECTION_RISK × 3
[BLOCK] HARDCODED_SECRET × 2
[WARN]  MISSING_PAGINATION × 4 (fetchall() on unbounded queries)
Enter fullscreen mode Exit fullscreen mode

All 51 patterns at a glance

Structural patterns (vibe-coding bugs):
STUB_SKELETON · MOCK_PATTERN · HARDCODED_TABLE · TRIVIAL_IF_CHAIN · DEAD_DB_RESULT · INPUT_OUTPUT_DISCONNECTED · MISSING_WRITE · FAKE_ASYNC · DEAD_CALL_RESULT · EMPTY_EXCEPT · SILENT_FAILURE · RECURSIVE_WITHOUT_BASE + more

Security patterns (OWASP Top 10 + AI-specific):
SQL_INJECTION_RISK · COMMAND_INJECTION · PATH_TRAVERSAL · SSRF_RISK · XSS_RISK · OPEN_REDIRECT · HARDCODED_SECRET · EVAL_EXEC_RISK · INSECURE_DESERIALIZATION · CORS_WILDCARD · LLM_OUTPUT_INJECTION · LLM_SSRF · PROMPT_SQL_INJECTION + more

Cross-file taint analysis:
CROSSFILE_EVAL_EXEC_RISK · CROSSFILE_COMMAND_INJECTION (tracks injection chains across module imports)


The free tier

Feature Free Pro
Files/month 50 Unlimited
LAYER 1+2 scan
LAYER 3 cross-file
LAYER 4 AINA advisor
Scan history
GitHub Action
API key vg_free_test Custom

Free key: vg_free_test (50 files/month, no account needed)


Quick start

# Install
pip install aina-scan

# Configure (free key, no signup)
aina-scan config --key vg_free_test

# Scan a file
aina-scan scan app.py

# Scan a whole project
aina-scan scan-project ./src

# Check your scan history
aina-scan history --limit 10
Enter fullscreen mode Exit fullscreen mode

GitHub: https://github.com/Moonsehwan/aina-scan
API docs: https://pleasing-transformation-production-90c2.up.railway.app/v1/engines

If you find a false positive, aina-scan feedback FINDING_ID --verdict fp auto-suppresses it.


Scanned your repo and found something interesting? Drop it in the comments — I read everything.

Top comments (0)