DEV Community

Cover image for Hallucinated Packages & Dependency Drift in AI-Assisted Code
Muhammad Ali Mughal
Muhammad Ali Mughal

Posted on

Hallucinated Packages & Dependency Drift in AI-Assisted Code

#security #php #node #ai

During a routine review on a client's Laravel and Vue project, one pull request added a Composer package with a perfectly reasonable name for parsing VAT numbers. The code looked fine and the tests passed locally for the author, because he had copied a helper from an assistant's answer and installed the package by hand. The CI build failed with "package not found". The package had never existed. The assistant had invented a plausible name. That time the failure was harmless. The worrying part is that anyone can register a missing name on a public registry, and attackers do watch for names that assistants tend to invent.

I wrote this with help from AI tools and checked every command against Composer 2.8, npm 10 and pnpm 9. I am a Principal Software Engineer in Karachi with 9+ years across PHP/Laravel, Vue, Node and .NET microservices, and through Loop Origin I maintain production SaaS codebases where a growing share of code is AI-assisted.

📚 Series Navigation — Maintaining AI-Written Code in Production

Related earlier posts: Subresource Integrity & CDN Script Pinning · Cron Jobs That Never Ran: Heartbeats & Dead-Man Switches

How do you stop AI coding assistants from adding fake or risky dependencies?

Direct answer: Treat every new dependency in an AI-assisted PR as a review item. Verify the package exists on the official registry, check its age, maintainers and downloads, require lockfile changes to be reviewed, block unapproved packages with an allowlist in CI, run composer audit and npm audit, and pin versions so drift only happens through deliberate update PRs.

Warehouse shelves with labelled boxes and one empty gap, AI-generated

Two different problems

It helps to separate them because the fixes differ.

Hallucinated packages are names that do not exist, or did not exist when the model was trained. Researchers have measured this: a 2024 study of code-generating models ("We Have a Package for You!", Spracklen et al.) found a meaningful share of suggested packages did not exist, and many invented names repeated across prompts. Repeated names are what make "slopsquatting" possible: someone registers the invented name and waits.

Dependency drift is quieter. The assistant suggests "^2.0" where you had "2.4.1", adds a second HTTP client because it did not notice the one you already use, or pulls an old major version from its training data. Nothing breaks today. Six months later you have three date libraries and a security advisory on one of them.

Risk signals compared

Signal Hallucinated / typo package Legit but risky Healthy dependency
Exists on Packagist / npm No, or created very recently Yes Yes
Age of first release Days or weeks Years, but last release long ago Years, steady releases
Maintainers One new account One inactive maintainer Several, known org
Weekly downloads Near zero Declining Stable or growing
Install scripts (postinstall) Often present Sometimes Rare, documented
Source repo link Missing or mismatched Archived Active, matches package

No single row decides it. A brand-new package with one maintainer, an install script and no repository link should stop the PR until someone checks it by hand.

Step 1: make dependency changes visible in review

Most teams collapse composer.lock and package-lock.json in the diff view, which is exactly where surprises hide. I add a small CI step that prints a readable summary of what changed:

#!/usr/bin/env bash
# ci/dependency-diff.sh — summarise added/removed packages vs main
set -euo pipefail
git fetch origin main --depth=1

echo "== Composer packages added =="
diff <(git show origin/main:composer.lock | jq -r '.packages[].name' | sort) \
     <(jq -r '.packages[].name' composer.lock | sort) | grep '^>' || echo "none"

echo "== npm packages added (top level) =="
diff <(git show origin/main:package.json | jq -r '.dependencies + .devDependencies | keys[]' | sort) \
     <(jq -r '.dependencies + .devDependencies | keys[]' package.json | sort) | grep '^>' || echo "none"
Enter fullscreen mode Exit fullscreen mode

Post that output as a PR comment and reviewers see "added: acme/vat-parser" in plain text instead of a 3,000-line lockfile.

Step 2: verify each new package exists and looks healthy

For Composer, the Packagist API gives you creation date and download counts:

pkg="acme/vat-parser"
curl -sf "https://repo.packagist.org/p2/${pkg}.json" > /dev/null \
  || { echo "BLOCK: ${pkg} not found on Packagist"; exit 1; }
curl -s "https://packagist.org/packages/${pkg}/stats.json" | jq '.downloads.total'
Enter fullscreen mode Exit fullscreen mode

For npm:

npm view some-package time.created maintainers repository.url --json
npm view some-package scripts.postinstall
Enter fullscreen mode Exit fullscreen mode

I fail the build when a package is missing, and flag for manual review when it is younger than 90 days or has fewer than a few thousand total downloads.

Step 3: an allowlist for production dependencies

For client projects with strict needs, an explicit allowlist works well. It is just a file in the repo:

{
  "composer": ["laravel/framework", "guzzlehttp/guzzle", "spatie/laravel-permission"],
  "npm": ["vue", "axios", "pinia", "dayjs"]
}
Enter fullscreen mode Exit fullscreen mode

CI compares top-level dependencies against it. Adding a package means a one-line change to the allowlist that a senior reviewer approves, which forces the conversation "do we need this, and do we already have something that does it?". That question alone has removed duplicate HTTP and date libraries from several projects we maintain.

Step 4: pin versions and control drift 

// composer.json — prefer exact or tight ranges for production apps
"require": {
  "php": "^8.3",
  "laravel/framework": "11.31.*",
  "guzzlehttp/guzzle": "7.9.2"
}
Enter fullscreen mode Exit fullscreen mode 
# .npmrc
save-exact=true
ignore-scripts=true   # enable per-package only when needed
Enter fullscreen mode Exit fullscreen mode

Then let a bot (Renovate or Dependabot) open grouped update PRs on a schedule. Updates become deliberate and reviewable, not a side effect of someone pasting an assistant's snippet. ignore-scripts=true blocks install scripts by default, which removes one of the most common ways a malicious package runs code on a developer laptop or CI runner.

Step 5: audit continuously 

composer audit --locked --format=summary
npm audit --omit=dev --audit-level=high
Enter fullscreen mode Exit fullscreen mode

Run both on every PR and nightly on main. Nightly matters because advisories are published after you merge.

Teaching the assistant your stack

Prevention is cheaper than detection. In the repo's assistant instructions file (whatever your tool reads, such as a rules or AGENTS.md file) list the approved libraries for common jobs: "HTTP: Guzzle. Dates: Carbon. Validation: Laravel validator. Never add a new dependency without saying so explicitly." In our experience this cuts surprise packages sharply, though it never replaces the CI checks.

Diagnostic checklist

  • [ ] CI prints added/removed packages as a readable PR comment.
  • [ ] Builds fail when a new package does not exist on the official registry.
  • [ ] Packages younger than 90 days or with very low downloads need manual approval.
  • [ ] Production dependencies are on an allowlist with senior approval for changes.
  • [ ] Versions are pinned (save-exact, tight Composer constraints) and updated via bot PRs.
  • [ ] Install scripts are disabled by default.
  • [ ] composer audit and npm audit run on every PR and nightly.
  • [ ] Assistant instruction files list approved libraries for common tasks.

FAQ

How common are hallucinated packages really?

Common enough to plan for. Published research found that a noticeable percentage of package names suggested by code models did not exist, with open models doing worse than commercial ones. Even a small rate matters when a team merges hundreds of suggestions a month.

Isn't npm audit enough?

No. Audit tools only know about published advisories. A freshly registered malicious package has no advisory yet, which is why existence, age and allowlist checks come first.

Should we ban AI assistants from adding dependencies?

Banning tends to fail quietly. Requiring dependency changes to be explicit, summarised and approved works better and still lets the assistant save time.

What about private registries?

A private mirror (Private Packagist, Verdaccio, Artifactory) that only proxies approved packages is the strongest control. Make sure scoped names cannot fall through to the public registry, or you invite dependency confusion.

How do we clean up existing drift?

Start with composer show --direct and npm ls --depth=0, find duplicates doing the same job, pick one, and migrate in small PRs with characterization tests from Part 2 guarding behaviour.

Can Loop Origin help set this up?

Yes. We add these gates as part of our custom web application and maintenance work, usually in a day or two for a typical Laravel and Vue codebase.

Next in this series, Part 4 covers static analysis gates: PHPStan, TypeScript strict mode and coverage in CI.

Follow & Connect

Top comments (0)