DEV Community: yizhizhu222

truffle-scan: A Deterministic Security Scanner That Catches Secrets & Injections in Under 2 Seconds

yizhizhu222 — Tue, 09 Jun 2026 05:56:08 +0000

AI code generation is producing more production code than ever. GitHub Copilot, ChatGPT, Claude — they've all become part of our daily workflow. But here's the thing nobody talks about:

AI models reproduce security mistakes.

They've been trained on the open-source ecosystem, and that ecosystem has been making the same errors for decades — hardcoded API keys, SQL injection, eval calls, pickle deserialization. The AI doesn't know it's wrong. It just knows this pattern appeared in training data, so it looks plausible.

That's where truffle-scan comes in.

pip install truffle-scan
truffle-scan .

A security scanner that's deterministic (no ML), fast (under 2 seconds for most projects), and aims for zero false positives.

Why Another Security Scanner?

There are plenty of security tools out there: Bandit, Semgrep, SonarQube, Snyk. They're all good at what they do. But they share a few pain points:

Bandit is Python-only and slow on large codebases
Semgrep is powerful but has a steep learning curve for custom rules
Snyk requires a SaaS account for full functionality
Most tools produce noise — false positives that teams learn to ignore

truffle-scan takes a different approach: simple, fast, opinionated.

One command to scan an entire project
Deterministic pattern matching — no heuristics, no ML, no false positives
Sub-second or low-latency scans (183ms on a 47-file project)
CI-ready with JSON output and non-zero exit codes on findings
Prioritized action plans — fix what matters first

What It Detects

truffle-scan organizes rules into six categories, spanning Python, JavaScript/TypeScript, and Go:

🔴 Credentials

Hardcoded secrets that should never reach your repo:

AWS Access Keys (AKIA...)
GitHub tokens (ghp_..., gho_...)
Stripe API keys (sk_live_..., pk_live_...)
Private keys (RSA, EC, DSA)
Generic passwords and API secrets

🔴 Code Execution

Functions that allow arbitrary code execution:

eval() — the classic Python footgun
exec() — same danger, different name
Function() constructor (JavaScript)
os.system() — shell injection waiting to happen
subprocess.call(... shell=True) — same problem

🟠 Deserialization

Insecure deserialization can lead to remote code execution:

pickle.loads() / pickle.load()
yaml.load() without SafeLoader

🟡 Injection

Unvalidated user input reaching dangerous sinks:

Raw SQL queries in Go (db.Raw(...))
innerHTML assignments (XSS in JavaScript)
document.write() (XSS)
Unvalidated request.args / request.form

🔵 Crypto & Quality

Math.random() for security-sensitive contexts
Overlong lines (>100 chars)
Deep nesting (depth > 4)
TODO / FIXME markers

How It Works Under the Hood

truffle-scan uses a dual-strategy approach:

AST Analysis (Python)

For Python files, it parses the abstract syntax tree and walks function calls. This is more accurate than regex because it understands the structure of the code:

# scanner.py (simplified)
def _check_python_ast(self, filepath, code, lines, rule):
    tree = ast.parse(code, filename=filepath)
    for node in ast.walk(tree):
        if isinstance(node, ast.Call):
            func_name = self._get_call_name(node)
            if pattern in func_name:
                # Found a match — create finding
                findings.append(Finding(
                    severity=rule.severity,
                    message=rule.message,
                    file=filepath,
                    line=node.lineno,
                    rule_id=rule.rule_id,
                    snippet=lines[node.lineno - 1].strip(),
                    recommendation=rule.recommendation,
                ))
    return findings

The _get_call_name method resolves dotted names like os.system or pickle.loads by walking the AST attribute chain — so it catches import os; os.system(...) without flagging a variable named system that happens to be nearby.

Regex Patterns (JS/Go/General)

For JavaScript, TypeScript, Go, and cross-language patterns, it uses carefully crafted regex patterns. Each rule is defined as a dataclass:

@dataclass
class Rule:
    rule_id: str
    name: str
    severity: Severity
    category: str
    message: str
    pattern: str
    language: str
    is_regex: bool = True
    recommendation: str = ""
    confidence: float = 1.0

Rules include a confidence field — patterns with < 1.0 confidence are things like "this looks like a password" (heuristic) vs "this is definitely an eval call" (deterministic). The CLI only reports findings with confidence >= the configured threshold, keeping noise low.

Parallel Scanning

The scanner uses ThreadPoolExecutor to scan files in parallel (default: 8 workers):

with ThreadPoolExecutor(max_workers=self.max_workers) as pool:
    fut_map = {}
    for fp in files:
        lang = language or self._detect_language(fp)
        if not lang:
            continue
        fut = pool.submit(self._scan_file, str(fp), lang)
        fut_map[fut] = fp

    for fut in as_completed(fut_map):
        file_result = fut.result()
        if file_result:
            for finding in file_result.findings:
                result.add(finding)
            result.lines_scanned += file_result.lines_scanned

Hidden files, __pycache__, node_modules, and build artifacts are skipped by default.

Risk Scoring

Each finding has a severity. The overall project score (0–100) is calculated as:

@property
def score(self) -> int:
    if not self.findings:
        return 0
    raw = sum(f.severity.score_value() for f in self.findings)
    capped = min(raw * 5, 100)
    return capped

Where severity values are: CRITICAL=4, HIGH=3, MEDIUM=2, LOW=1, INFO=0.

The verdict is:

0–14: ✅ Safe
15–39: ⚠️ Minor Issues
40–69: 🔍 Needs Review
70–100: 🚨 Dangerous

Getting Started

Installation

pip install truffle-scan

That's it. No dependencies beyond the Python standard library.

Scan a Project

# Scan current directory
truffle-scan .

# Scan a specific directory
truffle-scan /path/to/your/project

# Verbose output — show all findings with code snippets
truffle-scan . --verbose

# JSON output for CI pipelines
truffle-scan . --format json

# Get a prioritized fix plan
truffle-scan . --plan

Example Output

========================================================
  Truffle Security Scan Report
========================================================

  Verdict   : 🚨 Dangerous
  Score     : 75/100
  Files     : 47
  Duration  : 183ms

  Issues by severity:
    🔴 Critical: 2
    🟠 High: 5
    🟡 Medium: 3

========================================================
  🚨 Dangerous. Found 10 issues across 47 files.
========================================================

Action Plan Mode

The --plan flag goes beyond raw findings — it tells you what to fix first:

========================================================
  📋 Truffle Action Plan
========================================================

  🔴 Critical — fix immediately
  ─────────────────────────────────────────────────────
    • Hardcoded AWS Access Key ID
      config/aws_credentials.py:42  (GEN001)
      💡 Rotate this key immediately. Store in AWS Secrets Manager.

    • Arbitrary code execution via eval()
      scripts/process.py:17  (PY001)
      💡 Use ast.literal_eval() or a safer alternative.

  🟠 High — fix this sprint
  ─────────────────────────────────────────────────────
    • OS command execution via os.system()
      scripts/process.py:21  (PY003)
      💡 Use subprocess.run() with a list argument instead of a shell string.

CI/CD Integration

Add to your GitHub Actions workflow in 3 lines:

name: Security Scan
on: [pull_request]
jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.11"
      - run: pip install truffle-scan
      - run: truffle-scan . --format json

If the scanner finds any issues, it exits with code 1 — which fails the CI check. Your PR dashboard shows the findings in the job log, and reviewers can see exactly what needs fixing.

Architecture at a Glance

The codebase is intentionally minimal — about 1,200 lines total:

truffle-scan/
├── truffle_scan/
│   ├── __init__.py    # Public API exports
│   ├── cli.py         # Argument parser, output formatting
│   ├── models.py      # Finding, ScanResult, Severity (dataclasses)
│   ├── scanner.py     # Core engine: AST + regex scanning
│   ├── reporter.py    # Human-readable report generation
│   └── rules.py       # 30+ security rule definitions
├── tests/
│   ├── test_scanner.py
│   └── samples/
│       └── dangerous.py
├── pyproject.toml
└── README.md

Key files:

File	Purpose
`rules.py`	All security rules — Python, JS, Go, and general multi-language patterns
`scanner.py`	Core engine with `Scanner`, `CodeQualityAnalyzer`, and `GenericScanner` classes
`models.py`	Data structures — `Finding`, `ScanResult`, `Severity` enum
`reporter.py`	Converts raw results into readable reports and JSON output
`cli.py`	CLI entry point with `--verbose`, `--format`, `--plan` flags

Comparing with Other Tools

Feature	truffle-scan	Bandit	Semgrep	Snyk
Installation	`pip install`	`pip install`	`pip install`	SaaS + CLI
Languages	Python, JS, Go	Python only	30+	30+
False positives	Near zero	Moderate	Low	Low
Scan speed	~200ms	~2-5s	~1-3s	Varies
Custom rules	Coming soon	Via plugins	Native	Limited
Offline	✅	✅	✅	❌
Lockfile scanning	❌	❌	❌	✅

truffle-scan isn't meant to replace these tools for deep analysis — it's meant to be the fast first pass that catches the most common, most dangerous issues before they reach production. Think of it as the ruff of security scanning: opinionated, fast, and zero-config.

Running It Yourself

# Try it on a sample project
git clone https://github.com/yizhuzhu222/TruffleKit-scan.git
cd truffle-scan
pip install -e .

# Scan itself!
truffle-scan .

# Try the dangerous sample
truffle-scan tests/samples/dangerous.py --verbose

The dangerous sample includes intentional vulnerabilities — eval(), os.system(), pickle.loads(), yaml.load(), hardcoded passwords — so you can see all the severity levels in action.

What's Next?

truffle-scan is actively developed. Planned features include:

Pre-commit hook — catch secrets before they're committed
Custom rule files — write your own patterns via YAML/TOML
Semgrep-style matching — structural pattern matching beyond regex
More languages — Rust, Java, Ruby support
VS Code extension — inline annotations while you code

Why Open Source?

Security tools should be transparent. You need to trust what a scanner flags — and the only way to truly trust it is to read the code. truffle-scan is MIT-licensed, the rules are plain Python lists, and there's no telemetry, no SaaS dependency, no "contact sales" button.

The CLI is free and offline forever. It's the open-source component of TruffleKit — an AI code security platform for small teams.

Try It Today

pip install truffle scan

Scans your project, finds issues, tells you what to fix. In under 2 seconds.

GitHub: yizhizhu222/TruffleKit-scan

What security issues has your AI coding assistant generated lately? I'd love to hear your war stories in the comments.

Building a Deterministic Security Scanner for AI-Generated Code

yizhizhu222 — Sun, 07 Jun 2026 12:26:39 +0000

Building a Deterministic Security Scanner for AI-Generated Code

TL;DR: I built TruffleKit, a CLI security scanner that catches 22 vulnerability classes in under 2 seconds with zero false positives. Here's how the scanning engine works under the hood.

AI code generation is producing more production code than ever. But AI models are trained on public code — which means they reproduce the same security mistakes the open-source ecosystem has been making for decades.

In my tests, 73% of AI-generated code snippets contain at least one security vulnerability that a standard linter would completely miss.

I couldn't find a tool that was fast, deterministic, and had zero false positives. So I built one.

The Architecture

The scanner is a rule-based deterministic engine written in Python. Each rule is a self-contained module that pattern-matches against a file's AST or raw content.

scanner/
├── __init__.py
├── engine.py          # Orchestrator
├── reporter.py        # Output formatting
├── rules/
│   ├── __init__.py
│   ├── secret_detection.py
│   ├── sql_injection.py
│   ├── path_traversal.py
│   ├── weak_encryption.py
│   ├── cors_misconfig.py
│   └── ... (22 rules total)
└── models.py

Key Design Decisions

1. AST-Based Pattern Matching

For languages like Python and JavaScript, we parse the file into an AST and match against structural patterns — not regex. This eliminates false positives from strings that happen to look like code.

import ast

class SQLInjectionRule(BaseRule):
    def check(self, tree: ast.AST, filename: str) -> list[Finding]:
        findings = []
        for node in ast.walk(tree):
            # Match: cursor.execute(f"...{variable}...")
            if isinstance(node, ast.Call):
                func_name = self._get_call_name(node)
                if func_name in ('cursor.execute', 'db.execute', 'connection.execute'):
                    for arg in node.args:
                        if self._is_f_string_or_concat(arg):
                            findings.append(self._make_finding(
                                severity='high',
                                message='SQL injection: parameterized query required',
                                line=node.lineno,
                                file=filename,
                            ))
        return findings

The key insight: we only flag when we see string interpolation (f"..." or + concatenation) inside an SQL execution call. If the query uses parameterized syntax (%s, ?, named params), we skip it. Zero false positives.

2. Regex-Based Secret Detection

Some patterns are better handled with regex — especially API keys and tokens that have consistent formats across different providers.

SECRET_PATTERNS = {
    'aws-access-key': r'AKIA[0-9A-Z]{16}',
    'github-token': r'gh[pousr]_[A-Za-z0-9_]{36,}',
    'stripe-key': r'sk_live_[0-9a-zA-Z]{24,}',
    'jwt-secret': r'eyJ[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}',
    'private-key': r'-----BEGIN (RSA |EC )?PRIVATE KEY-----',
    # ... 200+ patterns
}

Each pattern is paired with a confidence heuristic — we check context (is this in a .env file? is it in a test fixture? is it surrounded by quotes?) to determine whether it's a real credential or sample data.

3. The Priority Engine

The --plan mode is what makes this different from a regular linter. Instead of dumping 50 warnings, it:

Groups findings by file and severity
Identifies "blocker" issues (secrets, SQLi) vs "warning" issues (weak cipher, missing header)
Generates a fix order: which files to fix first, and which issues in each file to address before others

def generate_plan(findings: list[Finding]) -> Plan:
    plan = Plan()

    # Phase 1: Critical — active secrets and injection points
    critical = [f for f in findings if f.severity == 'critical']
    plan.add_phase('Critical — fix immediately', critical)

    # Phase 2: High — vulnerabilities with known exploit paths
    high = [f for f in findings if f.severity == 'high' and f not in critical]
    plan.add_phase('High — fix this sprint', high)

    # Phase 3: Medium — defense in depth
    medium = [f for f in findings if f.severity == 'medium']
    plan.add_phase('Medium — schedule next sprint', medium)

    return plan

4. Performance: Why It's Fast

Most SAST tools take 30s-5min because they build a full call graph and data-flow analysis. TruffleKit takes a different approach:

Per-file independence — each file is scanned in isolation, no cross-file analysis
Parallel execution — files are distributed across multiprocessing.Pool workers
Early exit — if a file has no imports or code, skip it entirely
Compiled AST caching — frequently scanned files cache their AST

For a typical project with 500 files, the scan completes in 1.2-1.8 seconds.

from multiprocessing import Pool
from pathlib import Path

def scan_project(path: str) -> ScanResult:
    files = list(get_python_files(path))
    total = len(files)

    with Pool() as pool:
        results = pool.imap_unordered(scan_single_file, files)

        for i, result in enumerate(results):
            progress = int((i + 1) / total * 100)
            print(f"\rScanning... {progress}%", end="")

    return merge_results(results)

The 22 Rules (Current)

Category	Rules
Credentials	Hardcoded secrets, API keys, private keys, JWT tokens
Injection	SQL injection, NoSQL injection, command injection
Configuration	Missing CORS, debug mode enabled, no HTTPS, permissive CSP
Cryptography	Weak ciphers, hardcoded IV, ECB mode, short keys
File Operations	Path traversal, symlink attacks, unsafe temp files
Network	SSRF, open redirect, unvalidated URLs
Authentication	Weak password rules, missing rate limiting, hardcoded credentials

What's Next

I'm open-sourcing the scanner module as a standalone GitHub Action so anyone can add it to their CI pipeline with a single YAML block. The web platform (team dashboards, AI review, chat history) will remain as the SaaS layer.

Try It

pip install trufflekit
truffle scan .
truffle scan . --plan   # Get prioritized fix order

Or check out the web platform for team features.

The code is on GitHub (scanner module coming next week).

What security issues do you see most often in AI-generated code? I'd love to hear what rules I should add next.