Introduction
"A coding-agent skill for multi-phase security audits with independently verified, machine-readable findings."
This is article #186 in the "One Open Source Project a Day" series. Today's project is security-audit — Cloudflare's coding-agent skill, 13,825 Stars, MIT license.
security-audit addresses a critical question: how do you make AI-generated security audit results trustworthy enough to hand to a security team? The answer isn't "a smarter model" — it's a structured process: six audit phases, adversarial validation, and machine-readable finding records. It turns "the model says there's a problem here" into "here's a confirmed vulnerability with source evidence, a reproducible path, and a priority ranking."
What You'll Learn
- The six-phase audit workflow, from reconnaissance to target-neutral reporting
- The adversarial validation principle (the checker is never the finder)
- The difference between three verdicts:
confirmed/needs_validation/rejected - How the coverage ledger makes multi-run audits additive
- The sandbox requirement: why "can't execute target code" means
needs_validation
Prerequisites
- Experience with Claude Code or similar coding agents and the skill mechanism
- Basic understanding of security audit concepts (attack surface, trust boundaries, vulnerability confirmation)
- Node.js fundamentals
Project Background
Overview
This skill is the single-repo starting point of Cloudflare's vulnerability discovery harness. Cloudflare's official blog Build your own vulnerability harness describes how that system evolved into a multi-stage, fleet-wide vulnerability discovery platform — and security-audit is the single-repo version it evolved from.
It's not a "generate a security report" prompt template — it's an orchestration system: isolated sub-agents run reconnaissance, hunting, validation, and verification, and every step produces structured records with independent validators.
Author / Team
- Organization: Cloudflare
- Primary language: JavaScript (zero-dependency validators, written for Node.js)
- License: MIT
- Created: 2026-06-18
Project Stats
- ⭐ GitHub Stars: 13,825+
- 🍴 Forks: 740+
- 📄 License: MIT
- 📅 Created: 2026-06-18
Quick Start
Installation
# Install with the Skills CLI
npx skills add https://github.com/cloudflare/security-audit-skill \
--skill security-audit
# User-level installation
npx skills add https://github.com/cloudflare/security-audit-skill \
--skill security-audit \
--global
Usage
Start your coding agent pointed at the codebase you want to audit, then say:
security audit this codebase
Or:
find security vulnerabilities in ./src
do a security review, output to ~/audits/my-project
The skill activates automatically when the request matches its trigger (security audit / find vulnerabilities / pen-test, etc.).
Two Modes
| Mode | Trigger | Behavior |
|---|---|---|
| guidance | Security questions, focused reviews, methodology | Use only relevant parts; don't run the full workflow or write files |
| full audit | Explicit audit/pen-test request, end-to-end review | Run all six phases and write report files |
Core: The Six-Phase Audit Workflow
Phase 1: Reconnaissance
Launch multiple research agents in parallel, each returning structured source facts with file:line references:
- Agent 1a: product type, tech stack, build commands, subsystem boundaries
- Agent 1b: principals, authority, trust boundaries, control locations
- Agent 1c: entry surfaces, copies, and sinks
Outputs architecture.md and coverage-ledger.json. Reconnaissance is read-only — no external services are contacted.
Phase 2: Coverage-Led Hunting
Assigns ledger "coverage units" to isolated general agents. Each hunter reads only its assigned source blocks, writes only to its own scratch/, and returns one structured result.
Key: coverage critics find gaps — which units were missed, which assignments overlap.
Phase 3: Independent Candidate Validation
Every unique candidate (after deduplication) goes to a fresh verifier that did not hunt it, tasked with trying to refute it.
The verifier's prompt says explicitly: "You did not write this candidate. Try to refute it from repository source and bounded local evidence."
Phase 4: Structured Output
Writes three verdict records to findings.json and validates them with validate-findings.cjs.
Phase 5: Independent Record Verification
Fresh agents verify final source claims. Material replacements receive another independent verifier.
Phase 6: Target-Neutral Reporting
Derives REPORT.md, FINDINGS-DETAIL.md, and NEEDS-VALIDATION.md from the verified records and coverage ledger.
Three Verdicts: Clear Semantics
This is the most instructive design point — verdicts aren't "high/medium/low risk"; they're "evidence completeness":
| Verdict | Meaning |
|---|---|
confirmed |
Complete source trace with a bounded, reproducible observation |
needs_validation |
An exact unresolved fact, but no severity assigned |
rejected |
A disproved candidate (records why it was rejected) |
The key principle: "a source-grounded suspicion" ≠ "a confirmed vulnerability." When a lead can't be validated because of sandbox constraints, it stays needs_validation — never hastily marked confirmed, never silently dropped.
Design Principles
1. Adversarial Validation
The agent that checks a finding is never the agent that found it. This prevents model self-confirmation — when a model validates its own finding, it tends to confirm it.
2. Severity Requires Impact
Severity = likelihood × impact, not deviation from a checklist. A problem that matches a checklist but has no actual impact is not a vulnerability.
3. Defense-in-Depth Gaps Are Not Vulnerabilities
If Layer A already prevents the attack, the absence of Layer B is a hardening note, not a vulnerability.
4. Multiple Runs Improve Coverage
In Cloudflare's test runs, a single run found roughly half the vulnerabilities that repeated runs found in total. So the skill is designed for additive multi-run coverage — each run uses prior ledgers and findings to target gaps.
The Coverage Ledger: Making Audits Additive
coverage-ledger.json is the skill's core data asset.
A normal security audit is one-shot — run once, produce a report, and running again means starting from zero. security-audit's ledger makes audits incremental:
First audit run:
→ generates coverage-ledger.json (records which units are covered, confirmed, or pending)
→ finds N vulnerabilities
Second audit run (same repo):
→ reads the prior ledger and findings
→ only re-hunts "uncovered gaps" and "changed source"
→ carries forward prior findings still backed by current source
→ does NOT treat stale or unresolved prior work as covered
Each unit has a state: planned → in_progress → completed / deferred. If a unit can't be assigned because of agent-count limits, it's explicitly marked deferred with a reason — never silently dropped.
Machine-Readable Finding Records
findings.json follows the schema in report-schema.json, paired with zero-dependency validators:
| File | Purpose |
|---|---|
report-schema.json |
JSON schema for all three verdicts |
validate-findings.cjs |
Zero-dependency validator (used in Phases 4/5) |
validate-coverage-ledger.cjs |
Coverage ledger validator (used in Phases 1–5) |
The parent runs validate-coverage-ledger.cjs after creating the ledger and after each update, and validate-findings.cjs in Phase 4 and after each Phase 5 replacement.
This "machine-readable + independently validated" design lets audit output feed downstream tooling, rather than being a PDF only humans can read.
Sandbox Requirements: An Honest Boundary
security-audit explicitly requires: executing target-controlled code must happen inside an OS-enforced sandbox.
The sandbox must:
- Disable external networking
- Use a sanitized allowlisted environment
- Enforce resource limits
- Allow writes only to assigned scratch paths
If these controls are unavailable, the workflow does not execute target code and keeps the lead as needs_validation.
This is an honest engineering decision: better to leave something unconfirmed than to run potentially malicious code without a safe boundary. In AI security tooling, this explicit stance on the "validation boundary" is more trustworthy than "I can auto-run the PoC."
Resources
- 🌟 GitHub: cloudflare/security-audit-skill
- 📝 Background blog: Build your own vulnerability harness
- 🔧 Skills CLI: skills.sh
- ✉️ Contact: security-ai-research@cloudflare.com
Summary
security-audit represents a specific judgment: the bottleneck in AI security auditing isn't "can the model find vulnerabilities" — it's "can the findings be trusted."
Three things worth noting:
Adversarial validation is the core of credibility. The problem with many AI audit tools is that discovery and validation are done by the same model — the model "finds" a vulnerability, then validates its own finding, naturally biased toward confirmation. security-audit cuts this bias at the process level with the "checker ≠ finder" structural constraint.
Verdict semantics = evidence completeness, not risk level. confirmed / needs_validation / rejected describe "where the evidence chain breaks," not "how severe this is." Severity (likelihood × impact) is a field inside confirmed. This separation makes audit results machine-processable and trusted by security teams.
Additive coverage solves the "audits are one-shot" problem. Traditional audits go stale the moment they're done. security-audit's coverage ledger makes audits incremental: each run builds on prior coverage, only filling gaps and revalidating changes. This is closer to "continuous security" than "periodic audit."
If you're doing security work with a coding agent, or want to understand how to make AI-generated security conclusions trustworthy, security-audit is the most complete open-source reference available.
Explore PrimeSkills — a curated marketplace of AI agents and skills, each validated against real enterprise workflows. No hype, just what actually works.
Visit my personal site for more insights and interesting products.
Top comments (0)