Your AI coding assistant just generated a clean import block. The packages look right — right naming convention, right category, right style. One of them doesn't exist.
That's not a hypothetical. Language models generate package names from pattern completion, not registry lookups. A name that fits the context gets generated regardless of whether it exists on PyPI. The code compiles and may pass review before the import fails at deployment — or before a malicious actor registers the name first.
BrassCoders catches it before either happens.
The Check That Only AI Code Needs
BrassCoders flags imports of packages that don't exist on PyPI — a finding category unique to AI-generated code, where an assistant confidently recommends a dependency it invented. The check is opt-in (--check-package-hallucination) because it requires outbound network calls to the registry, unlike the rest of BrassCoders's offline-first scans. Every third-party import in the project gets validated; any name that returns a 404 gets flagged at HIGH severity.
HIGH severity, not informational. An unregistered name is a live attack surface the moment someone else decides to register it.
Why Language Models Invent Package Names
A language model completes patterns — it doesn't query PyPI before generating an import. BrassCoders exists because this distinction matters: a name that fits the semantic context of the code gets generated regardless of whether it has ever existed in a registry. The model has no mechanism to check, and no reason to distinguish between a real package and a plausible-sounding one.
Security researcher Bar Lanyado documented this in research published by Vulcan Cyber in February 2023: ChatGPT consistently produced non-existent package names that fit the context of the prompt. The names were formatted correctly, categorized plausibly, and returned with apparent confidence. Lanyado registered several of them on PyPI. They became available for any install that followed the AI's recommendation.
This is different from a wrong API or a misremembered function signature. Those are recall failures — the package exists, the usage is wrong. A hallucinated package name is a generation artifact: the model produced a name the registry has never seen.
From Hallucination to Hijack
BrassCoders treats hallucinated package names as a supply chain finding because the attack path is short. An AI assistant generates a non-existent package name. A developer copies the import into the codebase. A malicious actor scans for package names that AI models hallucinate consistently — names that sound right, follow naming conventions, and get suggested repeatedly — and registers one on PyPI with a setup.py that executes arbitrary code on install.
The developer runs pip install, and the supply chain attack completes.
This mirrors the dependency confusion pattern that OWASP A08:2021 (Software and Data Integrity Failures) covers. The AI-generated variant has one additional property: the hallucinated name is consistent. The model generates it every time someone asks the same question, which makes the set of high-value names to squat discoverable by querying AI assistants at scale.
What the Check Catches — and What It Doesn't
BrassCoders validates each import name against PyPI at scan time. A name that returns a 404 is flagged; a name with a live registry entry is not flagged, regardless of what that package does.
Two things follow from that scope. A hallucinated name that someone has already registered as a malicious package will not be caught — the name exists, so the check passes. The check is also a point-in-time snapshot: a package that doesn't exist today might exist tomorrow. The value is in catching names before registration, not after.
That's the actual scope. Most hallucinated names sit unregistered for months or indefinitely — the check catches real exposure, just not every possible exposure.
Running the Check
BrassCoders validates package names in one flag added to a normal scan. The rest of the command is unchanged.
# Opt in to the hallucination check (requires network access to PyPI)
brasscoders scan --check-package-hallucination /path/to/your/project
# --offline always overrides it back off
brasscoders scan --offline /path/to/your/project
Findings appear in the YAML output as package_hallucination__ with HIGH severity. Each finding names the package and the file and line where the import appears.
Why This Risk Is Specific to AI-Generated Code
A static scanner running on human-written code rarely sees this finding — developers don't generally import packages they invented. BrassCoders added the check because AI coding assistants do, at a rate high enough to warrant dedicated detection. When a model hallucinates a package name, it tends to hallucinate the same name consistently, across every developer who gives it a similar prompt. The hallucinated name becomes predictable, which makes the attack precomputable.
A typosquatting attack requires a developer to make a mistake. A hallucinated-package attack requires a developer to trust their AI assistant — which is the entire premise of using one.
pip install brasscoders
brasscoders scan --check-package-hallucination /path/to/your/project
Top comments (0)