The npm supply-chain conversation tends to happen without a denominator. "Install scripts are dangerous" — fine, but how many packages even have one? I kept not finding the number, so I went and measured it.
The number
I took 658 packages from one corner of npm (the MCP-server ecosystem — that's what I happened to be auditing) and counted how many declare an install-lifecycle script: the preinstall / install / postinstall hooks that run code the moment you npm install.
Twenty do. That's 3.0% of the sample.
Two honest caveats before anyone runs with that figure:
It's 3% of this sample, not "3% of npm." 658 packages is the union of a handful of registry search queries — a search result, not a census frame. Run it on a different slice and you'll get a different number.
It counts the top-level package's own declared hooks. npm runs install scripts for every package in the resolved dependency tree, so a package with a clean manifest can still execute code at install through a dependency three levels down. This measures the surface a package is responsible for itself.
With those stated: the number is calmer than the panic implies, and it costs nothing to reproduce for any package:
curl -s https://registry.npmjs.org//latest | jq '.scripts'
The part where the manifest lied to me
Reading the manifest tells you a script exists. It does not tell you what the script does — and on four of the twenty, that gap bit me.
Four declared preinstall: npx only-allow pnpm. I filed those as a benign package-manager guard: only-allow just prints an error if you used the wrong package manager. In intent, that's exactly what it is.
But npx fetches only-allow from the registry if it isn't already cached. So a "guard" whose whole job is to print a message makes a network call on your machine at install time. Nothing sinister — it's a completely standard idiom — but it is not what "benign guard" implied to me, and I could not have learned it by reading the text.
That's the general lesson, and the only reason this is worth writing up:
A manifest tells you what a script is. Only running it tells you what it needs.
So I made the "running it" part reusable
Reading manifests is cheap and misleading; running the scripts is where the truth is. So I wrapped that in something you can point at any package.
It fetches the exact tarball npm install would, extracts it inside a container with no network, all Linux capabilities dropped, and a read-only filesystem, runs the declared install scripts, and returns a verdict — plus the sha256 of the exact bytes it read, the exit code, and the sandbox's own output.
npx -y @kenwea/mcp check debug
Resolved debug → debug@4.4.3
https://registry.npmjs.org/debug/-/debug-4.4.3.tgz
verdict MANUAL_REVIEW
why this package declares no preinstall, install or postinstall script, so
nothing of its own runs when it is installed. That is worth knowing and
it is not a pass: we executed nothing, and an unrun artifact is not a
passing one
sha256 89c1ac9c946ee8905a875837114528e97eeae35e03be3190584b2216af43e4a7
size 13449 bytes
executed no
signed ed25519, key 00f55dd04da212b3
Note the verdict logic, because it's the part I care most about getting right: "nothing ran" is information, not a pass. A package with no install script isn't "safe," it just moved the interesting behavior to import-time. Reporting that as approved would be the convenient lie.
Why it's signed
The output is signed under a published Ed25519 key and bound to the sha256 of the bytes. That matters for one reason: "I ran it and it's fine" from the party that wrote the code is circular — you cannot vouch for your own artifact, and every reviewer knows it. A signature from something that isn't the author is a third-party record you can forward, and one that outlives the version: when a registry pulls a compromised release (as npm did with the recent chalk / debug tombstones), the bytes are gone and the incident becomes unauditable. A signature minted while the version was live is the thing that survives.
You can verify any signature yourself — the payload names the artifact, the hash, the verdict, and the constraints it ran under. (Verification is client-side on purpose: a page where we confirmed our own signature would prove nothing.)
In CI
Default is report-only — a verdict is a successful result even when it's rejected. To make a pipeline block, ask for it:
npx -y @kenwea/mcp check express@4.18.2 --fail-on rejected
Or as a GitHub Action:
- uses: kenwea-protocol/kenwea-notary-action@v1 with: package: ${{ matrix.package }} fail-on: rejected --fail-on rejected blocks only on rejected; --fail-on manual_review also blocks on manual_review. An honest non-answer — a package it couldn't fetch — never fails the gate. Turning "we couldn't read it" into a red build is the one thing this refuses.
Honest limits
So you find out here rather than by being surprised:
Dependencies are not installed. This measures a package's own declared install surface, not its full transitive closure. That's a deliberate scope, not an oversight — but it's a real limit.
Node and Python only, and 20 checks/hour on the free anonymous key (minted on first use to rate-limit — no signup, no email).
A clean install script says nothing about import-time. Which is the honest edge of the whole thing…
The question I can't answer
Install-time is the surface everyone worries about, and it turns out to be nearly empty — 3% in my sample, and half of those are package-manager guards. Meanwhile the surface that's actually large — what a module does the first time you require() it — I can't find anyone measuring, and I don't have a design for it either.
For a module whose whole job is to sit there until a function call arrives, what would "run it and see" even mean? Import it and watch — for how long, against what expectation of normal? That's the part I'd genuinely take feedback on.
Disclosure: I built this. The check (@kenwea/mcp) is one piece of kenwea.com, which is early and quiet — the check runs standalone with no account, which is why I'm writing about it rather than the site.
Top comments (0)