Static Application Security Testing (SAST) tools are a staple of any mature AppSec programme. Tools like Semgrep, Bandit, and SonarQube are used daily by security engineers to catch vulnerabilities before code ships to production. But how do they actually work under the hood?
As part of my transition from 13 years of software engineering into application security, I built my own SAST scanner from scratch in Python and ran it against four of the most well-known intentionally vulnerable applications in the OWASP ecosystem. This post covers what I built, how I tested it, and what the results tell us about real-world vulnerability patterns.
What I Built
The tool is a language-agnostic, regex-based static analysis scanner with a YAML-driven rule engine. The core design decisions were:
YAML rules over hardcoded logic. Every detection is a YAML file — no code changes required to add new vulnerability patterns. This mirrors how production tools like Semgrep work, and means a security team could extend the ruleset without touching the engine.
Language-agnostic by design. Rather than building AST parsers per language (which is how deeper tools work), the scanner uses regex patterns that fire across any file type. This trades some precision for breadth — it can scan Java, TypeScript, PHP, Python, Kotlin, Go, and more in a single pass.
Three output modes. Terminal output for quick scans, JSON for CI/CD pipeline integration (with configurable exit codes to fail builds), and an HTML report for sharing findings.
Docker-first. The entire tool runs as a container — no local Python environment needed. Mount your codebase, get a report.
The scanner covers 27 rules across five categories mapped to CWE identifiers and OWASP Top 10:
- Injection (INJ): SQL injection, command injection, XSS, SSTI, LDAP injection
- Secrets (SEC): AWS keys, hardcoded passwords, private keys, JWT secrets, database connection strings
- Cryptography (CRYPTO): MD5/SHA-1 usage, insecure random, ECB mode, disabled TLS verification
- Authentication (AUTHN): JWT algorithm confusion, insecure session cookies, timing attacks, IDOR
- Misconfiguration (MISC): Path traversal, debug mode, XXE, unrestricted file upload, CORS wildcards, insecure deserialization
The full source is on GitHub: github.com/pgmpofu/sast-tool
The Test Targets
I chose four intentionally vulnerable applications maintained by OWASP — each written in a different language, each targeting a different slice of the vulnerability landscape:
| App | Language | Files Scanned | Scan Duration |
|---|---|---|---|
| WebGoat | Java | 562 | 6.70s |
| OWASP Juice Shop | TypeScript/Node.js | 805 | 12.62s |
| DVWA | PHP | 189 | 1.17s |
| NodeGoat | JavaScript/Node.js | 34 | 1.49s |
Results at a Glance
| App | Critical | High | Medium | Low | Total |
|---|---|---|---|---|---|
| WebGoat | 31 | 61 | 0 | 0 | 92 |
| Juice Shop | 3 | 63 | 1 | 0 | 67 |
| DVWA | 0 | 12 | 2 | 0 | 14 |
| NodeGoat | 0 | 9 | 0 | 0 | 9 |
| Total | 34 | 145 | 3 | 0 | 182 |
182 findings across four codebases in under 23 seconds of total scan time.
WebGoat — 92 Findings (31 Critical)
WebGoat is a Spring Boot Java application built by OWASP specifically to teach developers about security vulnerabilities. It had the highest finding count by far, which makes sense — it's a structured learning platform with one lesson per vulnerability type.
SQL Injection was the dominant finding. The scanner flagged string concatenation in SQL queries across multiple lesson files. A representative example from SqlInjectionLesson6a.java:
query = "SELECT * FROM user_data WHERE last_name = '" + accountName + "'";
This is textbook SQLi — user input concatenated directly into a query string with no parameterization. The fix is straightforward: use PreparedStatement with ? placeholders. What's notable here is that the same pattern appeared in 8 separate lesson files, each demonstrating a different variant of the attack (blind injection, union-based, error-based).
Insecure deserialization showed up as Critical. In InsecureDeserializationTask.java:
new ObjectInputStream(new ByteArrayInputStream(Base64.getDecoder().decode(b64token)))
Java's ObjectInputStream is one of the most dangerous APIs in the language. Deserializing untrusted data with it can lead to remote code execution — this class of vulnerability was at the heart of the Apache Commons Collections exploit chain that affected thousands of Java applications. The scanner correctly flagged both the task file and the SerializationHelper utility class.
Command injection via Runtime.exec() was caught in VulnerableTaskHolder.java:
Process p = Runtime.getRuntime().exec(taskAction);
Passing unsanitized user input to exec() is equivalent to os.system() in Python — an attacker who controls taskAction can run arbitrary OS commands on the server.
Private key material hardcoded in source. The scanner found PEM key headers in CryptoUtil.java — a reminder that even educational codebases can demonstrate the exact mistakes they're teaching against.
Coverage verdict: Partial. WebGoat covers lessons across the full OWASP Top 10. The scanner did well on the categories it has rules for — SQL injection, deserialization, command injection, and cryptographic failures. However, several vulnerability classes present in WebGoat went undetected:
| Vulnerability | In WebGoat | Scanner Detected | Gap Reason |
|---|---|---|---|
| SQL Injection | ✅ | ✅ | Regex matched string concatenation patterns |
| Command Injection | ✅ | ✅ |
Runtime.exec() pattern matched |
| Insecure Deserialization | ✅ | ✅ |
ObjectInputStream pattern matched |
| Hardcoded Private Key | ✅ | ✅ | PEM header pattern matched |
| XXE (XML External Entity) | ✅ | ❌ | Scanner has XXE rule but WebGoat's parser config is in XML files, not Java — rule needs expanding |
| Path Traversal | ✅ | ❌ | WebGoat's path traversal uses Spring's @RequestParam binding, not direct open() calls — SAST missed it |
| Broken Access Control / IDOR | ✅ | ❌ | Requires understanding of authorization logic — not detectable by regex alone |
| CSRF | ✅ | ❌ | No CSRF token rule exists in current ruleset |
| JWT Vulnerabilities | ✅ | Partial |
JWTHeaderKIDEndpoint SQLi was caught; algorithm confusion attack was not |
| Insecure HTTP Communication | ✅ | ❌ | Runtime/config issue, not detectable in source code |
OWASP Juice Shop — 67 Findings (3 Critical)
Juice Shop is a modern Node.js/TypeScript e-commerce application and the most widely used security training platform in existence. Its 805 files took the scanner 12.6 seconds — the largest codebase of the four.
The most interesting Critical finding was a private RSA key hardcoded in lib/insecurity.ts:
-----BEGIN RSA PRIVATE KEY-----
MIICXAIBAAKBgQDNwqLEe9wgTXCbC7+RPdDbBbeqjdbs4kOPOIGzqLpXvJXlxxW8...
This is intentional in Juice Shop's case — it's used to sign JWTs as part of a challenge. But in a real application, committing an RSA private key to a public repository means every JWT the application has ever issued can now be forged by anyone who cloned the repo. This is exactly the kind of finding that would be Severity 1 in a real penetration test.
SQL injection appeared in the challenge fix files — specifically in codefixes/dbSchemaChallenge_1.ts. This is a meta-finding: Juice Shop includes intentionally wrong "fix" options as part of its challenge mechanic, and our scanner correctly identified that one of the proposed fixes still contained vulnerable code:
models.sequelize.query("SELECT * FROM Products WHERE ((name LIKE '%" + criteria + "%'...")
The bulk of findings (63 HIGH) were insecure random number generation — Math.random() used throughout data generation scripts. These are mostly false positives in context (seeding test data doesn't require cryptographic randomness), which is an important lesson about SAST tuning in practice. A production deployment of this tool would add # sast-ignore annotations or exclude data generation files from scans. SAST tools always require triage — raw finding counts are never the whole story.
Coverage verdict: Low — but not because the scanner is weak. Juice Shop encompasses vulnerabilities from the entire OWASP Top Ten and beyond, including categories that are fundamentally undetectable by static analysis:
| Vulnerability | In Juice Shop | Detected | Gap Reason |
|---|---|---|---|
| SQL Injection | ✅ | ✅ | String concatenation in challenge files caught |
| Hardcoded Private Key | ✅ | ✅ | PEM header in insecurity.ts caught |
| Insecure Random | ✅ | ✅ (mostly FP) | Correct pattern, wrong context — test data seeding |
| Broken Access Control / IDOR | ✅ | ❌ | Requires understanding which basket belongs to which user — dataflow, not regex |
| Broken Authentication | ✅ | ❌ | Logic flaw enforced at runtime, not visible in source |
| XSS (DOM-based) | ✅ | Partial |
innerHTML rule fires on some cases; Angular template bindings missed |
| Security Misconfiguration | ✅ | ❌ | Express security headers are a runtime/config concern |
| NoSQL Injection | ✅ | ❌ | No NoSQL injection rules in current ruleset |
| Prototype Pollution | ✅ | ❌ | Requires AST-level analysis of object property assignments |
| SSRF | ✅ | ❌ | Requires taint tracking from user input to HTTP call |
The most important gap here is the entire class of logic vulnerabilities — IDOR, broken auth flows, business logic abuse. These are the vulnerabilities that cause real breaches and they are essentially invisible to regex-based static analysis. They require DAST (dynamic testing against a running application) or manual review.
DVWA — 14 Findings (0 Critical, 12 High)
DVWA (Damn Vulnerable Web Application) is a classic PHP application. Its smaller codebase (189 files) and simpler architecture produced a tighter, more focused set of findings.
XSS via innerHTML assignment was the most frequent pattern, appearing 5 times in vulnerabilities/authbypass/authbypass.js. Each instance directly assigned user-controlled data to innerHTML — the canonical XSS pattern:
cell0.innerHTML = user['user_id'] + '<input type="hidden" ...>';
An attacker who can influence user['user_id'] can inject arbitrary HTML and JavaScript. The fix is to use textContent for plain text or sanitize with DOMPurify before any HTML rendering.
CORS wildcard was flagged twice in the API vulnerability module:
header("Access-Control-Allow-Origin: *");
A wildcard CORS policy allows any website to make cross-origin requests to the API. When combined with sensitive endpoints, this enables cross-site request forgery at scale.
One genuinely interesting false positive: the scanner flagged $password = "password"; in vulnerabilities/sqli/test.php as a hardcoded credential. It is technically a hardcoded password — but it's a test fixture, not an application secret. This illustrates a core challenge in SAST: the tool can't understand intent, only pattern. A mature AppSec workflow would suppress this with an inline comment and document the reasoning.
Coverage verdict: Moderate. DVWA has 14 named vulnerability categories. The scanner caught XSS and CORS-related issues, but missed the majority:
| Vulnerability | In DVWA | Detected | Gap Reason |
|---|---|---|---|
| XSS (DOM, Reflected, Stored) | ✅ | Partial |
innerHTML caught in JS files; PHP echo XSS patterns not in ruleset |
| CORS Wildcard | ✅ | ✅ | Wildcard header pattern matched |
| SQL Injection | ✅ | ❌ | DVWA's SQLi is in PHP — our PHP SQL injection pattern ($_GET, $_POST concatenation) missing from ruleset |
| Command Injection | ✅ | ❌ | DVWA uses shell_exec() and system() in PHP — no PHP command injection rule exists |
| CSRF | ✅ | ❌ | No CSRF token validation rule exists |
| File Inclusion (LFI/RFI) | ✅ | ❌ | PHP include($_GET['page']) pattern not in ruleset |
| File Upload | ✅ | ❌ |
move_uploaded_file() without MIME validation — no PHP upload rule |
| Brute Force / Weak Session IDs | ✅ | ❌ | Runtime behaviour, not detectable statically |
| Blind SQL Injection | ✅ | ❌ | Same as above — PHP SQL patterns missing |
| CSP Bypass | ✅ | ❌ | Runtime/header concern |
The DVWA results reveal a specific gap: the current ruleset underserves PHP. Most of DVWA's vulnerabilities are in PHP server-side code using functions like shell_exec(), include(), mysql_query(), and move_uploaded_file(). Adding PHP-specific rules for these functions would significantly improve coverage.
NodeGoat — 9 Findings (0 Critical, 9 High)
NodeGoat is a smaller Node.js application (34 files) that maps directly to the OWASP Top 10. Its low finding count reflects its size rather than its security posture.
All 8 code findings were Math.random() usage in financial contexts — generating stock quantities and fund amounts:
const stocks = Math.floor((Math.random() * 40) + 1);
const funds = Math.floor((Math.random() * 40) + 1);
In a real financial application, predictable random number generation for account values would be a significant finding. An attacker who can predict the seed could game the system. Here it's test data seeding, but the scanner correctly flags the pattern.
The ninth finding was particularly interesting — it came from package-lock.json, where a deprecated dependency's own warning message mentioned Math.random():
"deprecated": "Please upgrade to version 7 or higher. Older versions may use Math.random()..."
The scanner flagged the deprecation warning itself as a finding. This is a genuine false positive — useful as a demonstration that SAST tools scan all files indiscriminately unless you configure exclusions. In practice, package-lock.json and similar lockfiles should be excluded from scans.
What This Tells Us About SAST in Practice
Running the tool across four codebases surfaced some clear patterns worth highlighting for any engineer new to application security:
Injection vulnerabilities are everywhere, even in educational apps. SQL injection via string concatenation is one of the oldest vulnerabilities in existence — it was in the OWASP Top 10 when it was first published in 2003 and it's still there today. The fact that it appears across Java, PHP, and TypeScript codebases in the same scan demonstrates that it's a language-agnostic problem rooted in developer habit, not language design.
False positives are inevitable and manageable. Of the 182 findings, a meaningful portion are context-dependent: Math.random() seeding test data, hardcoded values in test fixtures, private keys that are intentionally public for training purposes. A SAST tool's job is to surface candidates for human review, not to replace it. The value is in the signal-to-noise ratio and the speed — 182 candidates across 1,590 files in 23 seconds.
SAST is most powerful as a prevention tool, not a detection tool. Every finding in these codebases was already known — they're intentionally vulnerable. The real value of SAST is catching these patterns before they reach a code review, not after they've been running in production. Embedding this kind of scanner as a pre-commit hook or CI/CD gate means developers get feedback at the moment they write the code.
Regex-based scanning has limits. Several real vulnerabilities in these apps weren't caught — NoSQL injection in NodeGoat's MongoDB queries, prototype pollution patterns, and some of the more subtle authentication bypass issues in DVWA. These require either AST-level analysis or semantic understanding of data flow that regex alone can't provide. Tools like Semgrep's taint tracking address this, and it's the natural next step for evolving this scanner.
What Would Full Coverage Require?
Across all four apps, a pattern emerges around three distinct gaps. Each requires a different technical approach to close.
1. Missing Rules (Fixable Now)
The simplest gaps — vulnerabilities the scanner could detect with regex but currently has no rule for. These are straightforward YAML additions:
| Missing Rule | Target Apps | Example Pattern |
|---|---|---|
| PHP SQL Injection | DVWA | mysql_query("SELECT * FROM users WHERE id=" . $_GET['id']) |
| PHP Command Injection | DVWA |
shell_exec($_GET['cmd']) or system($_POST['input'])
|
| PHP File Inclusion | DVWA | include($_GET['page']) |
| PHP File Upload | DVWA |
move_uploaded_file() without MIME validation |
| NoSQL Injection | NodeGoat, Juice Shop | db.collection.find({$where: req.query.input}) |
| CSRF Token Absence | DVWA, WebGoat | State-changing forms/endpoints without token validation |
| Open Redirect | DVWA, Juice Shop |
res.redirect(req.query.url) without validation |
| Server-Side Template Injection (PHP) | DVWA |
eval() or preg_replace() with /e modifier |
These could be added to the ruleset in an afternoon and would immediately improve detection on PHP codebases.
2. Taint Tracking (Requires Architecture Change)
Several missed vulnerabilities involve user input flowing through multiple functions before reaching a dangerous sink. For example, in NodeGoat:
// Input enters here (source)
router.get('/profile', function(req, res) {
var userId = req.params.userId;
// ...flows through several functions...
// ...reaches the sink here
db.collection('users').findOne({_id: ObjectId(userId)}, callback);
});
Regex can detect patterns at the sink (findOne with a variable), but can't verify whether userId actually came from user input without following the data flow across function boundaries. This is the core limitation of regex-based scanning.
What's needed: A taint analysis engine that tracks data from sources (HTTP request parameters, headers, cookies) to sinks (SQL queries, shell commands, file paths, HTML output). This is how tools like Semgrep's Pro engine, CodeQL, and Checkmarx work. Implementing this would require moving from regex to an AST-based approach using a library like tree-sitter for cross-language parsing.
3. Runtime/Logic Vulnerabilities (DAST Territory)
Some vulnerabilities simply cannot be found by reading source code. They require observing the application's behaviour at runtime:
- Broken Access Control / IDOR — can only be confirmed by making requests as User A and observing whether User B's data is returned
- Brute Force / Rate Limiting — requires actually sending repeated requests and observing the response
- Weak Session IDs — requires generating multiple sessions and analysing their entropy statistically
- Security Headers — requires making an HTTP request and inspecting the response headers
- Business Logic Flaws — requires understanding the intended workflow and deviating from it
These are the domain of DAST tools (project #10 and #11 on the portfolio list) and manual penetration testing. No amount of SAST improvement will close this gap — it's a fundamental constraint of static analysis.
The Takeaway: SAST + DAST + Manual = Defence in Depth
The industry consensus is that no single tool type provides complete coverage. The standard approach in mature AppSec programmes is layered:
SAST (this tool) → Catches code-level flaws early, in the IDE or CI pipeline
SCA (project #3) → Catches vulnerable dependencies
DAST (projects #10, #11) → Catches runtime behaviour, logic flaws, config issues
Manual Review → Catches everything that requires human judgement
Each layer has different strengths and blind spots. The goal isn't a single perfect tool — it's defence in depth across the entire SDLC.
Running It Yourself
The tool is open source. To scan any codebase:
# Clone the scanner
git clone https://github.com/pgmpofu/sast-tool
cd sast-tool
docker build -t sast-tool .
# Scan any project
docker run --rm \
-v /path/to/your/project:/src \
-v $(pwd)/reports:/reports \
sast-tool /src \
--exclude "node_modules" "build" "dist" \
--format html \
--output /reports/report.html
The HTML report includes severity filtering, CWE/OWASP references, and remediation guidance for every finding.
Contributions and additional rule PRs are welcome — particularly for Terraform misconfiguration patterns, Kubernetes YAML security issues, and deeper Java deserialization gadget detection.
This is part of a series documenting my transition from software engineering into application security. Next up: building a dependency vulnerability auditor that cross-references your pom.xml, build.gradle, and package.json against the NVD and OSV databases.
Top comments (0)