Your AI agent doesn't read READMEs — we scored 95,000 packages to see what it's installing anyway

Marcelo — Tue, 25 Aug 2026 14:33:47 +0000

By Marcelo, builder of Package Rating

Over the past few months I've been crawling npm, PyPI, crates.io, and RubyGems — pulling real
GitHub activity, dependency health, and known vulnerability data for every package I can find —
and scoring each one on a 0-100 scale. The goal was simple: before you npm install something,
you should be able to see, in one number, whether it's actually worth trusting.

95,338 packages in, here's what the data actually shows.

Almost nothing scores well

Band	Packages	Share
Excellent (90–100)	34	0.04%
Good (70–89)	1,893	2.0%
Fair (50–69)	49,707	52.1%
Poor (25–49)	43,717	45.8%
Critical (0–24)	36	0.04%

Almost half the open-source ecosystem sits in "Poor." Barely 2% clears "Good." This isn't a
judgment on any individual maintainer — most of these projects are unpaid, unfunded, and doing
fine by the standards their authors set for themselves. But it does mean the median package you
depend on is, by objective signals (maintenance activity, dependency health, versioning
discipline, known vulnerabilities), mediocre — not because open source is bad, but because most
of it was never resourced to be excellent.

Popularity doesn't buy you much safety

I expected the most-downloaded packages to score meaningfully better than obscure ones — more
users, more scrutiny, more pressure to keep things solid. The data doesn't really back that up:

Weekly downloads	Avg. score	Sample size
>1M	54.6	11,148
100k–1M	54.3	9,983
10k–100k	52.6	11,610
<10k	48.4	62,646

A 6-point spread on a 100-point scale, between the most- and least-downloaded tiers. Heavy usage
correlates with slightly better scores, not dramatically better ones. The package with 40
million weekly downloads sitting in your node_modules right now is not obviously safer than one
with 4,000 — you have to actually check.

412 popular packages carry a known critical vulnerability right now

This is the number that should make you go check your own lockfile. Across the packages we've
scored, 412 currently carry at least one critical-severity finding, and another 942 carry a
high-severity one — not hypothetical risk, but flagged, known issues in versions people are
installing today. Some examples, picked from packages with over 100 million weekly downloads:

pyyaml — 279M weekly downloads, 4 critical findings, score 33.5
litellm — 190M weekly downloads, 6 critical / 17 high findings, score 56.3
pillow — 125M weekly downloads, 10 critical / 50 high findings, score 38
numpy — 260M weekly downloads, 1 critical / 4 high findings, score 38
js-yaml — 279M weekly downloads, 1 critical / 3 high findings, score 37.5

None of these are obscure. All five are foundational, near-universal dependencies across their
respective ecosystems — the kind of package nobody thinks to double-check because "everyone uses
it." That's exactly the assumption worth re-examining.

Your AI agent doesn't read READMEs

Here's the part that changes the calculus on all of the above: a growing share of the packages
landing in package.json and requirements.txt today were never chosen by a person. An AI coding
agent hit a missing-import error, picked the first plausible package, and installed it — no
README, no GitHub tab, no "huh, last commit was three years ago" gut check. That gut check was
never a great safety net to begin with, but at least it existed. For agent-driven installs, it
usually doesn't.

That's not a hypothetical shift — it's already how a lot of code gets written. And it means the
412-critical-finding numbers above aren't just a "read more carefully" problem anymore; they're a
"the thing choosing your dependencies can't read at all" problem. The fix isn't asking agents to
be more careful — it's giving the build a signal an agent (or its CI pipeline) can act on
without human judgment in the loop. That's the actual reason we built this as a score and an
API, not a blog post: audit-dependencies
can fail a build the moment an agent adds something with a critical finding, before it ever
reaches a PR a human skims and approves.

Check yours

Every score above comes from data anyone can pull for free: packagerating.com
scores any npm, PyPI, crates.io, or RubyGems package on demand, and the
audit-dependencies GitHub Action
drops the same scoring straight into your CI, with an optional threshold gate if you want your
build to fail on packages below a bar you set.

I'll be publishing this as a recurring monthly report as the dataset grows — if there's a specific
ecosystem or angle (governance/compliance, a particular language, "most improved" packages) you'd
want to see next, drop a comment and let me know.

Methodology note: scores are computed from six weighted signals — liveness (recent commit
activity), community health, dependency risk, known-vulnerability findings, versioning discipline,
and general dependency hygiene. Full breakdown at
packagerating.com/github-action. Figures in this post
reflect the catalog as of August 2026; the "abandoned popular packages" angle we originally
scoped for this piece is being held back for a follow-up — a recent bug-fix recrawl of a subset of
the catalog is still working through GitHub's API rate limits, and we didn't want to publish
maintenance-activity claims about specific packages before that backlog fully clears.

DEV Community: Marcelo

Your AI agent doesn't read READMEs — we scored 95,000 packages to see what it's installing anyway

Almost nothing scores well

Popularity doesn't buy you much safety

412 popular packages carry a known critical vulnerability right now

Your AI agent doesn't read READMEs

Check yours