The npm supply-chain conversation tends to happen without a denominator. "Install scripts are dangerous" — fine, but how many packages even have one? I kept not finding the number, so I went and measured it.
The number
I took 658 packages from one corner of npm (the MCP-server ecosystem — that's what I happened to be auditing) and counted how many declare an install-lifecycle script: the preinstall / install / postinstall hooks that run code the moment you npm install.
Twenty do. That's 3.0% of the sample.
Two honest caveats before anyone runs with that figure:
It's 3% of this sample, not "3% of npm." 658 packages is the union of a handful of registry search queries — a search result, not a census frame. Run it on a different slice and you'll get a different number.
It counts the top-level package's own declared hooks. npm runs install scripts for every package in the resolved dependency tree, so a package with a clean manifest can still execute code at install through a dependency three levels down. This measures the surface a package is responsible for itself.
With those stated: the number is calmer than the panic implies, and it costs nothing to reproduce for any package:
curl -s https://registry.npmjs.org//latest | jq '.scripts'
The part where the manifest lied to me
Reading the manifest tells you a script exists. It does not tell you what the script does — and on four of the twenty, that gap bit me.
Four declared preinstall: npx only-allow pnpm. I filed those as a benign package-manager guard: only-allow just prints an error if you used the wrong package manager. In intent, that's exactly what it is.
But npx fetches only-allow from the registry if it isn't already cached. So a "guard" whose whole job is to print a message makes a network call on your machine at install time. Nothing sinister — it's a completely standard idiom — but it is not what "benign guard" implied to me, and I could not have learned it by reading the text.
That's the general lesson, and the only reason this is worth writing up:
A manifest tells you what a script is. Only running it tells you what it needs.
So I made the "running it" part reusable
Reading manifests is cheap and misleading; running the scripts is where the truth is. So I wrapped that in something you can point at any package.
It fetches the exact tarball npm install would, extracts it inside a container with no network, all Linux capabilities dropped, and a read-only filesystem, runs the declared install scripts, and returns a verdict — plus the sha256 of the exact bytes it read, the exit code, and the sandbox's own output.
npx -y @kenwea/mcp check debug
Resolved debug → debug@4.4.3
https://registry.npmjs.org/debug/-/debug-4.4.3.tgz
verdict MANUAL_REVIEW
why this package declares no preinstall, install or postinstall script, so
nothing of its own runs when it is installed. That is worth knowing and
it is not a pass: we executed nothing, and an unrun artifact is not a
passing one
sha256 89c1ac9c946ee8905a875837114528e97eeae35e03be3190584b2216af43e4a7
size 13449 bytes
executed no
signed ed25519, key 00f55dd04da212b3
Note the verdict logic, because it's the part I care most about getting right: "nothing ran" is information, not a pass. A package with no install script isn't "safe," it just moved the interesting behavior to import-time. Reporting that as approved would be the convenient lie.
Why it's signed
The output is signed under a published Ed25519 key and bound to the sha256 of the bytes. That matters for one reason: "I ran it and it's fine" from the party that wrote the code is circular — you cannot vouch for your own artifact, and every reviewer knows it. A signature from something that isn't the author is a third-party record you can forward, and one that outlives the version: when a registry pulls a compromised release (as npm did with the recent chalk / debug tombstones), the bytes are gone and the incident becomes unauditable. A signature minted while the version was live is the thing that survives.
You can verify any signature yourself — the payload names the artifact, the hash, the verdict, and the constraints it ran under. (Verification is client-side on purpose: a page where we confirmed our own signature would prove nothing.)
In CI
Default is report-only — a verdict is a successful result even when it's rejected. To make a pipeline block, ask for it:
npx -y @kenwea/mcp check express@4.18.2 --fail-on rejected
Or as a GitHub Action:
- uses: kenwea-protocol/kenwea-notary-action@v1 with: package: ${{ matrix.package }} fail-on: rejected --fail-on rejected blocks only on rejected; --fail-on manual_review also blocks on manual_review. An honest non-answer — a package it couldn't fetch — never fails the gate. Turning "we couldn't read it" into a red build is the one thing this refuses.
Honest limits
So you find out here rather than by being surprised:
Dependencies are not installed. This measures a package's own declared install surface, not its full transitive closure. That's a deliberate scope, not an oversight — but it's a real limit.
Node and Python only, and 20 checks/hour on the free anonymous key (minted on first use to rate-limit — no signup, no email).
A clean install script says nothing about import-time. Which is the honest edge of the whole thing…
The question I can't answer
Install-time is the surface everyone worries about, and it turns out to be nearly empty — 3% in my sample, and half of those are package-manager guards. Meanwhile the surface that's actually large — what a module does the first time you require() it — I can't find anyone measuring, and I don't have a design for it either.
For a module whose whole job is to sit there until a function call arrives, what would "run it and see" even mean? Import it and watch — for how long, against what expectation of normal? That's the part I'd genuinely take feedback on.
Disclosure: I built this. The check (@kenwea/mcp) is one piece of kenwea.com, which is early and quiet — the check runs standalone with no account, which is why I'm writing about it rather than the site.
Top comments (5)
Install-time execution is one of the easiest supply-chain risks to forget because it happens before the app even exists. Measuring how often packages run scripts is useful evidence for policy, sandboxing, and CI network restrictions.
Thanks and "before the app even exists" is the part that makes it slippery: there's nothing running yet to observe, so the install itself is the only window.
The npx-guard case in the post is the sharpest argument for your CI-network-restrictions point. Four of the twenty were npx only-allow pnpm guards that reach the network at install time (npx fetches only-allow from the registry), so a blanket network-deny at install would break legitimate package-manager guards, not just malicious postinstalls. That pushes toward "flag and separate" rather than "ban" network deny by default with a small allowlist for the handful that genuinely need egress. And the 3% number is what makes that allowlist maintainable: the surface is small enough to enumerate.
That "flag and separate" conclusion feels right. I would rather have CI split installs into network-free, network-declared, and postinstall-executing lanes than pretend one global rule fits all packages. The dangerous case is not only malicious code; it is invisible behavior during dependency resolution that nobody meant to grant.
That three-lane split is the right shape, and I think it gets sharper if the lane is something a package DECLARES rather than something CI discovers. "network-free", "network-declared", and "postinstall-executing" become a contract: a package in the network-free lane that reaches out at install is a violation you fail on, not noise you triage. The deviation from the declared lane is the signal, which is cleaner than scanning for danger in the abstract.
Your second point is the harder one, and it is the part my post left open. "Invisible behavior during dependency resolution that nobody meant to grant" is the transitive closure: the install-time surface is the union of scripts across the whole resolved tree, and authority accrues through deps nobody reviewed. Which means the lane a package runs in is not a property of the package at all, it is a property of the resolved tree at that moment. So the enforceable artifact is the lockfile: pin it and the lanes become auditable per resolved version; without a pin, "which lane does this install run in" has no stable answer, because tomorrow's resolution can quietly move it.
Underneath both is one flip: install-time authority is grant-by-default today. Nothing requests the network, the filesystem, or the environment; it is all just there. The three lanes are really a way to make the default deny and force a package to request what it needs, which is the only version of this where "nobody meant to grant it" stops being the normal case.
Declared lanes are the cleaner contract. Discovery-only CI turns every surprise into a triage debate; declared behavior lets you fail on drift. I would still keep a quarantine lane for legacy packages, but the valuable signal is exactly what you said: the package promised one install behavior and performed another.