Security teams are trained to treat a new critical CVE as a fire alarm. A recent JFrog investigation shows what happens when the alarm itself may have been generated from fiction.
JFrog examined a newly created GitHub repository that published more than 50 vulnerability advisories, many targeting SQLite. According to the researchers, 54 of the 55 advisories they reviewed appeared to be LLM-generated fabrications.
The strongest evidence was not an AI detector. It was the code.
Some advisories cited functions that did not exist in the affected SQLite version. Others pointed to unrelated lines, described fixes that were never committed, or supplied proof-of-concept inputs that failed before reaching the alleged vulnerable code.
Yet several of those reports still acquired CVE identifiers and high severity metadata. One was initially scored as critical before being downgraded.
That makes this more than another story about a model hallucinating.
An LLM can invent a plausible-looking vulnerability report in seconds. For the reports that passed downstream checks, polished structure was treated as evidence:
- a repository accepts the report
- an identifier is assigned
- severity metadata propagates
- scanners alert customers
- security teams spend time investigating reports that do not reproduce
The hallucination mattered because downstream systems accepted its shape without checking its substance.
For developers and security teams, the practical response is not to ignore CVEs. It is to preserve a verification step for reports that arrive without a trustworthy disclosure trail. Check the affected version, inspect the referenced code, reproduce the PoC in isolation, and compare the claim with the vendor's own advisory history.
The uncomfortable lesson is that structured data can still be synthetic noise. A CVE-shaped object is not automatically a vulnerability.
Source: JFrog Security Research, “SQLite Critical CVEs or LLM Slop?”
https://research.jfrog.com/post/sqlite-critical-cves-or-llm-slops/
Disclosure: AI tools assisted with drafting and source organization. The factual claims were checked against the linked investigation before publication.
Top comments (0)