DEV Community

Kame|local-first tools
Kame|local-first tools

Posted on

"Before you let an AI say \"I found a vulnerability\" — on changing my aim, and throwing away my own finding"

AI-assisted vulnerability research has a quiet but decisive step before the flashy "I found a 0-day": confirming that the thing is actually novel, and actually a defect. This is the record of aiming an authorization-scoped testing harness at three mature open-source projects, black-box-rediscovering a real CVE-class bug on a non-obvious surface — and then declining to report it, because it turned out to be an already-fixed public CVE. No exploit here. Only the method, and public CVE numbers.

1. What I built — an authorization-scoped harness

Three parts: (a) a deny-by-default egress scope-lock that mechanically refuses traffic to anything not explicitly allowed; (b) an append-only ledger that enforces the human sign-off at the data layer (a vulnerability is "confirmed" by a human gate, never by the LLM); (c) a coordinated-vulnerability-disclosure (CVD) pipeline that separates the steps you may automate from the steps that stay human. On top of that, I black-box test self-hosted OSS using two of my own accounts (self-A / self-B) and compare. No offensive reconnaissance (no unauthorized scanning, no hunting for victims). Only a self-hosted lab I solely occupy, with synthetic data.

2. First, an honest non-finding

I began by systematically attacking the "horizontal IDOR on primary CRUD" surface of three mature targets (two task managers and a no-code DB): "can B read A's object by id?" The result was 403 across the board, zero findings. This is not failure — it is the correct result, because the front door is exactly what popular OSS hardens most. What matters is being able to record "there was nothing here" honestly. Green does not mean working; I front-loaded an A-control (legitimate access returning 200 + a known marker) every time, so the "non-finding" is one where false positives were already eliminated — which is what makes the later claim trustworthy.

3. Changing the aim — judgment over target selection

The hypothesis: real authorization bugs live in code paths that don't pass the per-object permission check — aggregation (group by), relation expansion (linked records / lookups), derived reads (export, notifications, feeds), and the leaks in "intended" sharing (shared views). If the front door is hard, don't switch targets; switch the surface you aim at.

4. The re-aimed run hit on the first try

Aimed at a no-code DB's public shared-view, one cycle of black-box differential testing reproduced two over-exposures. First, a column hidden in the view returned its raw values through the aggregation endpoint. Second, records from an unshared related table returned through the relation-expansion endpoint. The decisive clue was an inconsistency: the main row-fetch API strictly enforced the column-hiding, while the auxiliary endpoints did not. "The main path enforces it but the auxiliary path doesn't" is not decoration — it is the signature of an ACL bypass.

5. And then I threw away my finding

This is the point. Before shouting "0-day," I checked public advisories (touching nothing on the target, reading only public information). The two reproductions were already published and fixed as CVE-2026-47378 (hidden-column leak via aggregation, cross-table related-data) and CVE-2026-47279 (the public shared-view relation endpoint not checking column visibility). I further confirmed, deterministically from a version/build diff, that the instance I tested was running pre-fix code due to a distribution-channel lag. Therefore this is not a novel 0-day — it is a reproduction of known CVEs on an out-of-date build. So I do not report it: sending an already-public, already-fixed CVE as a "new discovery" is duplicate noise and a breach of discipline. I recorded it as dismissed, honestly.

6. The lesson — judgment beats bug count

Two things are true at once. First, the method rediscovered real CVE-class authorization bugs, black-box, on exactly the non-obvious surfaces theory predicted (three "front-door" cycles found nothing; one "re-aimed" cycle found two CVE-class issues). Aim mattered. Second — and just as important — the discipline to recognize "reproduced != novel" and stop myself from over-claiming. AI-assisted research earns trust not through flash, but by verifying before it shouts.

7. What I do NOT include (explicit)

No exploit, no PoC. I never touched the target's production (a self-hosted synthetic lab only). What I publish is the method and public CVE numbers. No unfixed or undisclosed vulnerability appears in this article.


Stories about flashy discoveries are everywhere. What's rare is the story of "I thought I found something, verified it, and threw it away." The more powerful automation gets, the more this discipline is worth.

Top comments (0)