This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine
What I Built
I asked a model running o...
For further actions, you may consider blocking this person and/or reporting abuse
The state vocabulary is the part I'd keep. Most tooling collapses "registry didn't answer" into "fine", and treating unknown as clean is how dashboards end up making claims the data never made. Keeping unknown as its own state, and scoping "clean" differently for a manifest (every dependency checked) versus a pasted answer (every name found is real), is the honest version: the report says what was verified and what it wasn't.
The temporal angle is the one I'd push on. A GHOST verdict is a verified-at-T claim, and slopsquatting is precisely the attack that invalidates it on a delay: the name is empty today, the attacker registers it next week, and the PR-time check from last month is now making a stale claim about a name that exists and is malicious. POSSESSED catches that, but only if the check re-runs at install time rather than resting on a cached verdict.
And the census frequency is load-bearing in both directions. "Seen 7x" is a priority queue for the defender, but it's also exactly the list an attacker registers first, since those names have proven paste-and-install traffic. The census doesn't just measure the phenomenon; publishing the hits changes which ghosts get possessed. Curious whether you weight the check order or alerting by census frequency for that reason.
Good point about staleness. A GHOST result is only valid at the time of the check.
checkqueries the registries every time it runs and doesn't cache verdicts, so if someone registers that package later, the next run will catch it. It'll then show up as POSSESSED, regardless of how many downloads it has.The limitation is that
checkis just a CLI command, not an install hook. So you'd need to run it before installing dependencies, ideally as part of CI.Also, no, frequency isn't weighted. "Seen 7x" is just informational. Even if a ghost appears only once, it can still become dangerous if someone registers that name.
On publishing the names, you're right that hashing and masking aren't foolproof. The counts are still published alongside the hashes, and since the model weights are open, someone determined enough could regenerate most of the names. It only makes things harder, not impossible.
One thing I'm still not entirely happy with is
watch. It currently works only on my machine because a fresh clone doesn't have the original names to check. So by hiding the list from attackers, I'm also making it harder for other defenders to monitor those same ghosts.The CLI-not-hook limitation reads to me as the right boundary, not a missing feature. An install hook is arbitrary code running at install time, which is the attack surface itself; a CI gate on lockfile changes gets the same coverage without becoming the thing it's checking for. And the freshness then comes free from the no-cache design: every CI run is a new verified-at-T claim, and POSSESSED-regardless-of-downloads is the right trigger because download counts lag registration by construction. Gating on popularity would blind you to exactly the attack window.
The watch distribution problem is the interesting one. If the weights are open and the names are regenerable, the names aren't the secret - the census is a convenience, not a barrier. So instead of shipping the list, ship the recipe: model version, prompt set, extraction code. Every defender regenerates the same census locally, watch works from a fresh clone, and nothing gets published that an attacker can't already produce for the same compute. You don't gain secrecy, but you stop subsidizing the attacker: today the published counts do their curation work for free. The asymmetry worth fighting isn't information, it's cost - a published ranked list is maximum defender convenience at zero attacker cost, while a published recipe is equal cost on both sides.
On frequency weighting I'd land slightly differently. You're right that a once-seen ghost can be dangerous, but the attacker still has to choose which names to register, and they choose by expected paste-and-install traffic, which correlates with how often models invent the name. Weighting watch order by census frequency doesn't create that signal; it acknowledges the signal already exists on the attacker's side.
Agreed on the CLI limitation. I don't think adding a hook that runs arbitrary code during installation is necessarily the right approach for a tool that's supposed to protect against that kind of risk.
One thing I should correct from my earlier reply: the repo already includes everything needed to reproduce the census. The task list, prompts, extraction code and seeds are all there, along with the model digests. Running
census sweepfollowed bycensus resolvebuilds the local list thatwatchuses. So a fresh clone won't have anything to watch until those steps are run.That said, everyone's results won't be exactly the same. In my repeat test, 26 of 29 answers named the same package, but only 6 of the 8 ghosts came back. So defenders will likely get overlapping lists rather than identical ones.
There's another limitation with POSSESSED worth mentioning. It ignores download counts only for names already identified by the census. Everything else still goes through the normal age and usage checks. So if someone registers a name that isn't on the list and artificially boosts its numbers, it could still be classified as alive. That's a blind spot I hadn't considered enough.
Also,
checkdoesn't currently read lockfiles. CI could trigger a check whenever a lockfile changes, but the actual validation would still be against the manifest. That means transitive dependencies aren't covered.I agree with your point about publishing the counts, too. They don't actually influence any decisions in the tool, so exposing them probably helps an attacker more than it helps defenders.
I'd probably leave the watch order as it is for now since all 93 names are checked in a single pass. But I agree that prioritisation becomes more important as the list grows.
Checked the repo before answering, and you're right - the recipe was already the design. census sweep and census resolve are documented, src/census.js and data/tasks.json are there. Consider that point conceded.
The 6-of-8 reproduction rate is the interesting wrinkle. It weakens the fresh-clone symmetry I claimed (defenders get overlapping lists, not identical ones), but the overlap is itself a property worth measuring. The union across independent sweeps beats any single published list, and an attacker can't know which ghosts a given defender's sweep caught. The reproduction rate becomes a coverage metric in its own right: a sweep that re-finds only 75% of its own ghosts is telling you how much of the ghost space one pass even samples, and that number belongs in the report next to the counts.
Your POSSESSED blind spot reads to me as a label problem more than a heuristic problem. Right now "alive" means two different things: "passed the age and usage checks" for most names, and "not currently empty" for census-listed ones. Every other state in the vocabulary is bounded - unknown says nothing was proven, dead names its evidence - but alive is an unbounded claim that hides which path verified it. The boosted-downloads squat passes because any heuristic is gameable with enough effort; the durable fix is making the label say "passed the checks we ran" instead of "is trustworthy", so the report never claims more than the path supports.
Lockfile reading sounds like the highest-value extension for the same reason: names are exact, no English-parsing ambiguity, and every CI run re-verifies the entire resolved tree at check time. A hit there is also the more urgent signal - a POSSESSED direct dependency is probably a paste, but a POSSESSED transitive one means something upstream already got eaten.
I think you're onto something with the union approach, but I'd be careful with the 75% figure. That came from repeating the same run at temperature 0 with the same seed, using just 8 ghosts. It tells us more about consistency than actual coverage.
The more interesting result was the haunt round. I reran the same questions five times with different seeds and got 63 distinct ghosts, compared to just 29 in the first sweep. So a single run caught less than half of what those questions could produce. And since six runs probably still didn't catch everything, even that isn't the full picture.
On the labels, a name already found by the census can't be classified as alive. It goes straight to POSSESSED. So alive basically means the package exists and doesn't look suspiciously new.
POSSESSED is where things get a bit tricky. It can mean a name the model previously hallucinated that now exists, which is a pretty strong signal. But it can also mean a package that's less than a year old with very little usage, which could easily be a legitimate small project. The detail line tells you which case it is, but the main label doesn't make that distinction.
I agree that lockfile support is probably the most useful next step because it gives us exact dependency names. One concern, though: if a transitive dependency triggers the weaker POSSESSED condition, it's more likely to be a new helper package than an actual compromise. And checking the entire dependency tree would mean quite a few registry requests on every CI run.
I'd want to separate those two cases before treating every POSSESSED result as a security warning.