Scanners are wonderful at finding candidates and terrible at finding truth. Every tool in this space — including our own BugTraceAI — produces output that looks like findings and sometimes isn’t.
Our rule is simple: nothing enters a report until a human has reproduced it and captured evidence. Three reasons, learned from shipping this platform:
1. False positives cost more than missed findings. A reported-but-wrong finding burns the client’s trust and their engineer’s afternoon. A finding we catch manually next week costs one sprint. Asymmetric — so we bias hard toward verification.
2. Severity is a business judgement, not a calculator. CVSS gives a number; it cannot know that the “critical” SSRF is unreachable in your network topology. Human context decides ranking, and the report says why.
3. The report is the product. If we ship unverified output, we’ve shipped a scan, not an assessment.
We practice what we preach: automated breadth (crawling, fuzzing, CVE matching) covers the full URL tree, then manual expert testing confirms every finding with step-by-step reproduction and HTTP evidence before it ships. Roughly 60% of engagement hours are manual.
If you’re building security tooling, we argue about this exact stuff in our community forums — disagreement welcome.
Repo: https://github.com/BugTraceAI/BugTraceAI
Top comments (0)