We're not naming the platforms yet. Several findings are still in responsible disclosure, some without a confirmed patch. But we can share the methodology, the numbers, and enough about the findings to make this worth reading.
Over the past several months we ran our system against real open-source projects in production, used by thousands of organizations, across 7 programming languages. Result: 30+ confirmed real vulnerabilities, with at least one independently corroborated by a security researcher who has no connection to us. Here's how it works and what we actually found.
The architecture
The system has two main components.
Structural code modeling. Not text pattern matching, a graph that combines syntax, control flow, and data flow across the entire codebase. This is what lets the system reason about how files interact with each other, not just what's inside each file in isolation. Most vulnerabilities in real codebases don't live in a single function; they emerge from the interaction between components.
AI agents that reason like an attacker. For each candidate location flagged by the structural model, an agent generates a concrete exploitation hypothesis, then actively searches the code for evidence to disprove it. A finding only gets reported if the agent can't rule it out.
The hallucination problem in AI-based vulnerability detection is real and well-documented. Our answer to it is a three-layer independent verification pipeline:
- Layer 1: Agent generates the attack hypothesis.
- Layer 2: An independent analyst agent — with more context and access to surrounding code — whose sole job is to attempt to disprove Layer 1's finding.
- Layer 3: Literal factual verification using a model from a different provider than the one that generated the original finding. It checks every specific claim, line numbers, function names, actual values against the source code, word for word.
Layer 3 already caught a real false positive in production that Layers 1 and 2 had passed as confirmed. The false positive was caused by a variable reassignment several function calls up the chain that neither earlier pass had followed far enough. That's the kind of thing that makes AI vulnerability reports unreliable in practice and the reason we built the verification architecture the way we did.
The prioritization engine: why the cost is so low
The $6–7 figure per full codebase scan comes from a specific architectural decision: the expensive AI reasoning only runs on a small fraction of the codebase.
The first pass is a deterministic prioritization engine, no LLM / no API cost, that scores every function in the repo based on structural signals: data flow complexity, external input handling, memory management patterns, interaction with authentication or cryptographic primitives, and similar heuristics. This runs fast and cheap. Its job is to filter out the 98–99% of code that almost certainly isn't worth deep analysis.
Only the functions that score above a threshold go to the AI reasoning layers. Here's what that looked like across three real runs:
Project B's higher percentage (17.4%) is an outlier, that codebase had a higher density of functions touching external input directly. The 0.4–0.9% range on the other two is more representative.
For comparison: a thorough manual security review runs at roughly 100–150 LOC/hour per specialist, by standard industry estimates. Project A would take approximately 1,900 human-hours at that rate around 48 weeks of full-time work. We're not claiming this is a controlled equivalence; a human expert prioritizes differently and wouldn't attempt 100% coverage of a repo that size. But the system does run 100% of the code through the prioritization engine before deciding what to go deep on, the coverage is systematic, not intuition-based.
Blind CVE reproduction as a reliability test
Before running against unknown targets, we validated the system's reliability by giving it codebases with known vulnerabilities it hadn't seen without telling it where the vulnerability was.
Test 1: Pre-patch version of a widely-used JWT/authentication library, with a public CVE for an algorithm confusion vulnerability. The system found it.
Test 2: Pre-disclosure version of infrastructure used across a large part of the AI application industry, with a critical pre-authentication SQL injection. The system found it. According to public reports, this same vulnerability was actively exploited within 36 hours of its official public disclosure.
These weren't needle-in-a-haystack scenarios designed to make the system look good — they were real codebases with the full complexity of production code, and the system found the right thing without being pointed at it.
What we found in the wild
Running against the latest public versions of several critical projects with no known CVE going in:
Heap buffer overflow in a widely-deployed, low-level infrastructure component. Found autonomously, without a prior hint. An independent security researcher, with no connection to us, reported the exact same bug at nearly the same time. The affected platform cross-validated both reports in their triage process. We can't inflate that validation, it was an external third party reaching the same conclusion independently.
JWT signature verification vulnerability without a prior CVE, in the latest public version of a widely-used piece of AI infrastructure.
Multiple vulnerabilities in a widely-adopted local AI runtime, memory exhaustion DoS, cryptographic integrity bypass, and command injection — currently in responsible disclosure.
Total across all runs and languages: 30+ confirmed vulnerabilities, all passing through the three-layer verification pipeline before being reported.
What this doesn't do
The system doesn't replace the security specialist making the final call. Every finding comes with a concrete attack hypothesis and specific evidence, but a human still decides what's real, what's in scope, and what gets reported. What changes is what that person spends their time on: they get a short list of findings with context, not thousands of functions to read through hoping something stands out.
We're not claiming zero false positives, the three-layer architecture exists precisely because we expect some to get through, and we're still refining based on patterns we've identified in production. What we are claiming is that the verification pipeline makes false positives significantly harder to sustain through all three layers than in a single-pass system.
Happy to answer questions about the architecture, the prioritization heuristics, or specifics on any of the finding categories. We'll share more on the individual vulnerabilities as disclosures are confirmed.
Top comments (0)