Every developer who has used an AI coding assistant for more than a week has hit this moment: the assistant confidently suggests array.uniqueBy() or client.retryWithBackoff(), you go looking for the method in the docs, and it does not exist. Not in this version, not in any version. The model invented a method name that sounds exactly like something the library should have, and it wrote code around it as if it were real.
This is not a bug in the sense of broken software. It is a predictable consequence of how these models generate text, and understanding why it happens changes how you write prompts and structure context to avoid it.
Pattern Completion, Not Lookup
A language model does not have a database of your library's actual API surface sitting in memory that it queries before writing a line of code. It generates the next most probable token given everything it has seen, including the patterns of thousands of similar libraries during training. If most ORMs have a .find_or_create() convenience method, the model will happily produce one for your ORM too, whether or not your specific library actually implements it.
The method name it invents is not random. It is a statistically reasonable guess based on naming conventions across the ecosystem. That is exactly what makes it dangerous: the hallucinated method looks completely plausible, uses the right casing convention, and fits the surrounding code so naturally that it can slip past a quick read-through.
Why This Gets Worse With Popular But Inconsistent Libraries
Hallucination rates are not uniform across libraries. A library with a small, well-documented, consistent API tends to get hallucinated against less, because there is less ambiguous pattern space for the model to draw from. A large, sprawling library with multiple versions, deprecated methods still floating around in training data, and inconsistent naming across modules gives the model far more plausible-sounding options to choose from incorrectly.
This is one reason older or heavily-forked open source projects tend to produce more hallucinated suggestions than newer, tightly scoped ones. The training data contains years of blog posts, Stack Overflow answers, and abandoned pull requests referencing methods that existed in one version, were renamed in another, or were proposed but never merged.
Context Starvation Makes It Worse
Hallucination rates climb sharply when the assistant does not have the actual library source or type definitions in its context window. Without that grounding, the model falls back entirely on its training-time impression of what the library's API probably looks like, which is exactly the scenario that produces confident, wrong answers. This is the same underlying issue we cover in our guide on managing an AI coding assistant's context window without blowing the token budget: what the model can see directly changes what it's willing to guess at.
Including the actual type definitions, interface files, or a relevant excerpt of the library's public API in context measurably reduces hallucinated method calls, because the model has something concrete to pattern-match against instead of relying purely on memory of similar libraries.
Verification Habits That Catch It Early
The fastest catch is compiling or running the code before trusting it. Statically typed languages surface a hallucinated method immediately as a type error. Dynamically typed languages are more forgiving at write time and more dangerous at runtime, since a hallucinated method might not fail until a code path actually executes it in production, sometimes weeks later. A quick check against Python's own documentation or the equivalent for whatever language you're in is often faster than debugging the eventual runtime failure.
A second habit worth building: when an AI assistant suggests a method you don't immediately recognize, treat unfamiliarity as a signal to check the official documentation rather than as a sign you've simply forgotten the API. Experienced developers get burned by this more than beginners precisely because they trust their instinct that the suggestion "looks right," and a well-hallucinated method name is specifically engineered by pattern-matching to look right.
Prompting Around the Problem
Explicitly asking the assistant to only use methods it can point to in the provided context, rather than methods it believes should exist, changes its behavior meaningfully. This doesn't eliminate hallucination, since the model can still misread or misremember something genuinely in context, but it shifts the failure mode from "confidently invented" to "confidently misread," which is generally easier to catch during review.
Asking for a citation of where in the provided context a suggested method appears is a surprisingly effective technique. If the model cannot point to it, that's a strong signal the method is generated from training-time pattern completion rather than grounded in anything you actually gave it.
When Hallucination Signals a Deeper Context Problem
Persistent hallucination on a specific library across many sessions usually means the assistant is never being given grounding for that library at all, not that the model is simply bad at that particular API. Teams that add the library's type stubs or a condensed API reference to their standard context bundle for that codebase see the hallucination rate for that specific dependency drop substantially, because the model finally has something real to check itself against instead of guessing from training data.
This connects directly to the broader discipline of deciding what belongs in an assistant's context for a given task, rather than treating context as an afterthought. A narrower, deliberately assembled context consistently produces fewer hallucinated APIs than a broad, unfocused one, for the same reason a person who has actually read the docs makes fewer mistakes than one guessing from memory of similar tools.
Hallucination Isn't Unique to Obscure Libraries
There's a common assumption that hallucination mostly happens with obscure, poorly documented libraries where training data would naturally be thin. In practice, some of the most confidently wrong suggestions happen with extremely popular libraries that have gone through several major versions, precisely because training data spans multiple incompatible versions of the same API. A method that existed in version 2, was renamed in version 3, and got a slightly different signature in version 4 gives the model a genuinely ambiguous pattern to draw from, and it will often produce a plausible-sounding blend of all three rather than the one that's actually correct for the version you're using.
This is worth internalizing specifically because it cuts against intuition. Developers tend to trust suggestions more for well-known, heavily-used libraries, on the reasonable assumption that more training data means more reliable output. For hallucination specifically, more training data spanning more incompatible versions can produce the opposite effect: more plausible-sounding wrong answers, not fewer.
Team-Level Habits That Reduce the Rate
Beyond individual verification habits, a few team-level practices meaningfully reduce how often hallucinated methods make it into a codebase at all. Pinning and documenting the specific version of a library in project documentation, and making that version visible in whatever context gets assembled for the assistant, removes a lot of the version-ambiguity problem described above. Code review specifically flagging unfamiliar method calls for a documentation check, rather than assuming the reviewer would already know if something didn't exist, catches a category of bug that's otherwise easy to miss, since a hallucinated method often looks completely idiomatic to someone skimming a diff.
The Practical Takeaway
Hallucinated methods are not a sign the model is broken. They are a sign the model was asked to answer a question it did not have grounding to answer accurately, and it did what these systems always do when grounding is missing: it produced the most statistically plausible continuation available. Give it the real API surface, ask it to point to where a suggestion comes from, and verify anything it can't cite. That combination catches the overwhelming majority of hallucinated calls before they ever reach a pull request.
If you want the fuller picture on managing what an assistant sees and why that changes the quality of what it produces, the breakdown on context and token budgets covers the mechanics in more depth than fits in one companion post. That article comes out of AI-assisted development work done by 137Foundry, which spends a lot of time in exactly this kind of context-grounding problem on client codebases.
Top comments (0)