A reviewer on my team approved a pull request in four minutes last month. The diff was small: an AI assistant had suggested swapping a hand-rolled date parser for a package with a tidy name. The build passed. The tests passed. Two weeks later we learned the package had been published eleven days before the suggestion was made, had a single maintainer, and existed almost entirely because AI assistants kept recommending it. Nothing in our review interface had surfaced any of that. The reviewer made a consequential decision with missing evidence, the decision owner was nominally a human, and the point of reversibility — before the package entered our lockfile and shipped — had already passed.
This is the failure mode behind what security researchers now call slopsquatting: attackers publish packages with names that language models plausibly hallucinate, then wait for the suggestions to arrive. Most coverage of this problem focuses on detection tooling. I want to focus on the interface moment instead: what evidence must be visible at the exact point of approval, and what should stop the flow entirely.
The pattern: a Provenance Card before the approval button
When an AI assistant proposes adding or swapping a dependency, the approval control should be gated by a card that answers four questions the model cannot be trusted to answer about itself:
| Field | Evidence source | Stop condition |
|---|---|---|
| Package age | Registry publish timestamp | Younger than your threshold (e.g., 90 days) → hard stop, require senior review |
| Adoption signal | Download history, dependent count, not a single-week spike | Below threshold or anomalous spike → flag, do not auto-approve |
| Maintainer identity | Registry account age, other packages, linked org | Single fresh account, no history → hard stop |
| Name similarity | Edit distance from the package you actually asked for | Suggested name ≠ requested name → show both side by side |
The last row matters more than it looks. The classic slopsquatting move is that you asked for nothing at all — the model invented a plausible name. So the card must distinguish three states explicitly: requested by the human, suggested as an alternative to a requested package, and originated entirely by the model. That third state should render the approval button disabled until the reviewer checks a box that says, in plain language: I verified this package exists independently of this suggestion.
A reproducible verification workflow
Here is the concrete loop I now run before approving any AI-suggested dependency. It is deliberately boring, because boring is auditable:
1. CAPTURE — Record the exact suggestion: package name, version,
the prompt that produced it, and the model/session ID.
2. INDEPENDENT LOOKUP — Query the registry directly (never through
the same assistant) for publish date, maintainers,
and download history.
3. CROSS-CHECK — Ask a *second* model session, with no context from
the first, to evaluate the package's legitimacy.
Agreement between two sessions is weak evidence but
cheap; disagreement is a strong stop signal.
4. DECIDE — Approve, reject, or escalate, and write the reason
into the decision log next to the captured suggestion.
5. PRESERVE — Keep rejected suggestions in the log. A name that
gets suggested twice by different sessions is exactly
the pattern worth reporting.
Cost is the usual objection to steps 2–3, and it is a fair one: teams skip verification when every extra query burns budget. This is where I have been using MonkeyCode in my own workflow — its free model access and free server option mean the cross-check session and the registry-lookup scaffolding run without a per-query bill, which removes the most common excuse for collapsing steps 2–4 into a shrug. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The workflow above does not depend on any specific tool, though — any setup where a second, context-isolated model session is affordable will do.
If you want a low-friction place to try this loop, standing the cross-check step up on a free tier is a reasonable weekend experiment before you argue for it in a team budget review.
Researching the card, not just shipping it
A provenance card is a design hypothesis until you test it. The protocol I would run:
- Scenario-based review sessions. Give reviewers seeded PRs: one slopsquatted package, one legitimate young package, one well-established package. Half see the card, half see a standard diff.
- Success measure: reviewers catch the slopsquatted package and do not over-reject the legitimate young package. A card that makes people reject everything has failed differently, not succeeded.
- Stop measure for the research itself: if the card increases average review time by more than ~2 minutes on legitimate PRs, the evidence density is wrong — cut fields before cutting the pattern.
- Accessibility checks: the three provenance states must not be encoded by color alone; the disabled approve button needs an accessible description explaining why it is disabled and what action enables it; registry data must render in text, not only in sparklines.
Limitations, honestly
Provenance evidence reduces one attack surface; it does not establish that a package is good. An old, popular package can still be compromised. The two-session cross-check is weak evidence — models share training data and can share blind spots, which is why step 2 queries the registry directly rather than trusting any model. And this pattern adds friction precisely where teams are under pressure to move fast; if your organization will not honor the stop conditions, the card becomes theater.
Who should not use this approach: teams that pin and vendor every dependency already have a stronger control and do not need a review-time card. And if your AI assistant never proposes dependencies — only edits code you wrote — the provenance moment does not exist in your flow; spend the effort on evidence cards for whatever decision does exist.
The generalizable claim is narrower than "verify AI output." It is: identify the single most consequential approval in your flow, ask what evidence was missing when it last went wrong, and put that evidence — with a real stop condition behind it — in the exact moment where a human still has the power to say no.
Top comments (0)