DEV Community: Ismail Khouya

We Scored 675 MCP Servers on Security. 89% Failed.

Ismail Khouya — Fri, 04 Sep 2026 10:44:42 +0000

The Model Context Protocol (MCP) has become the default way to connect AI assistants to real tools — databases, file systems, SaaS APIs, dev environments. In under two years, the ecosystem has grown to thousands of servers. Almost none of them limit what an AI agent can do to them.

We run RepoAI, a directory that scores every MCP server on 15 structural security signals — not a penetration test, but a trust score built from real, verifiable facts: does it offer a read-only mode, does it authenticate requests, is it actively maintained, who published it. We recently audited our own scoring methodology end to end, fixed two real bugs that were under-counting risk, and re-scored the entire directory. The numbers that came back are worth sharing.

The headline numbers

Across all 675 published MCP servers in our directory:

Score	Risk level	Servers	Share
80–100	Safe	6	0.9%
60–79	Medium	66	9.8%
0–59	High risk	603	89.3%

Fewer than 1 in 100 servers scores as genuinely safe by any reasonable structural standard. Nearly 9 in 10 land in our "high risk" band.

Three signals drive most of that:

92.1% of servers offer no read-only mode at all — every install gets full read/write access, with no safer way to run it.
35.9% have at least one tool we classify as dangerous (it can execute code, delete data, or write files); of those, 43.1% combine that with having no read-only escape hatch — the single worst combination in our scoring model.
1.9% of servers have no authentication mechanism whatsoever — anyone who has the config file can invoke every tool the server exposes. Only 14.7% support OAuth, the strongest of the auth methods we track.

Only 12% of servers are published by the actual vendor rather than a third party.

This isn't a fluke of which servers happened to get listed first

Before publishing this, we wanted to rule out selection bias — maybe our directory just happened to accumulate the worst-scoring servers first. So we searched GitHub directly for MCP servers we hadn't imported yet (sorted by star count, so the most established, most-used candidates came first) and scored 20 new ones with the exact same methodology, cold.

The most popular candidate had 1,860 GitHub stars. Not one of the 20 reached even our "medium" tier. The highest score was 55 out of 100.

The pattern holds outside our own dataset. This looks like the honest current state of the ecosystem, not an artifact of our sample.

A concrete example

mcp-server-trello, a community-maintained MCP that connects an assistant to Trello boards, exposes 35 tools — including delete_checklist_item and delete_comment, both flagged as dangerous operations. It offers no read-only mode. There is no way to run it in a mode where the assistant can look but not touch. It scores 20/100 in our system, and that's a fair, mechanical read of what the repository itself documents — not a judgment of the maintainer, who has built something genuinely useful. It's simply typical: a small, popular, actively-used integration that ships full write access with no safety valve, because until now almost nothing in this space has asked for one.

Why "nothing bad has happened yet" doesn't mean much here

The natural objection: this ecosystem is used constantly, by a lot of people, and it isn't visibly on fire. Doesn't that mean the risk is overstated?

It's worth remembering that Log4Shell — one of the most severe vulnerabilities in a decade — sat unnoticed inside a library used by a meaningful share of enterprise Java applications for eight years before anyone found it. Absence of a headline is not evidence of absence of risk; often it's evidence that nobody has looked yet, or that when something does go wrong, nobody can tell.

MCP servers raise a version of this that's specific to agentic AI: prompt injection. An assistant doesn't need a bug in the server to misuse it — it needs to be tricked, by a webpage, a document, or an email it was asked to read, into calling a tool it shouldn't. When that tool has no read-only mode and no authentication boundary, the blast radius is whatever the tool can do. This failure mode produces no crash, no alert, and no log line that looks unusual. Researchers at Anthropic and elsewhere have written about exactly this "lethal trifecta" — private data, untrusted content, and a way to communicate externally — as the shape of risk that's specific to giving language models real tools to call. It's a quiet risk by construction, which is exactly why it doesn't show up as a wave of public incidents even where it is real.

None of this means MCP is unsafe to use. It means the safety work — read-only modes, scoped authentication, tool-level risk disclosure — is largely still ahead of the ecosystem, not behind it. That's normal for infrastructure this young. npm, early cloud IAM, and the first wave of consumer IoT devices went through the same phase, at similar or lower adoption numbers, before better defaults became standard.

What we'd ask of anyone shipping an MCP server

Three of our 15 signals matter more than the rest combined, and all three are achievable in an afternoon, not a rewrite:

Ship a read-only mode. If your tools can write, add a flag or config option that restricts the assistant to reads only. This one change had the single biggest effect on scores in our entire dataset.
Require authentication. Even a static API key is enough to clear our bar — it's the complete absence of one that costs points.
Document what your tools can do. We classify tool risk from what's actually described in your README and tool schemas. A tool named delete_x that says so plainly scores more fairly than one whose real capability is only discoverable by reading the source.

Methodology, briefly

Every score is a plain sum of 15 signals — repo health (maintenance activity, license, contributor count, community adoption) plus MCP-specific behavior (read-only mode, authentication strength, tool risk) — clamped to 0–100. It's a trust signal built from public, verifiable facts, not a security audit or vulnerability scan; a high score means a server looks well-maintained and conservatively scoped, not that its code has been proven free of bugs. The full breakdown, including exact point values for every check, is public on our methodology page.

Every score referenced in this piece reflects the directory at the time of writing, and is a live number — it moves as servers add real safety features or new ones get reviewed. Look up any specific server's score and full check breakdown on its RepoAI page, or browse the full directory.

RepoAI is an independent directory of MCP servers, Claude Skills, and the wider AI tooling ecosystem — not affiliated with Anthropic or the maintainers of any server we list unless a listing explicitly says "Official."

What does your own MCP server score? Curious where the gaps are in your specific setup — drop the repo link below.

I Built a Security Scorer for 213 MCP Servers — Here's What Made the Difference

Ismail Khouya — Sun, 26 Jul 2026 16:52:35 +0000

Every time I wanted to add an MCP server to Claude Desktop or Cursor, I ended up
with a dozen browser tabs open, trying to answer one question the README never
answers: is this actually safe to give tool-call access to my machine?

MCP (Model Context Protocol) directories exist, but most of them just mirror
GitHub — stars, a README excerpt, a copy-paste config. None of them score
risk. So I built RepoAI, and I want to share the exact
methodology, because I think it's more interesting than the product pitch.

The problem with "just check the stars"

Stars measure popularity, not safety. A server with 5,000 stars can still:

have no read-only mode (every tool call can write/delete)
ship with no auth method at all
be an unmaintained fork nobody's patched in a year
be archived — meaning whatever's wrong with it stays wrong, forever

None of that shows up in a star count.

The 15 signals

computeSecurityScore() runs 15 checks against each server, sums signed point
values, and clamps the result to 0–100. Two categories:

Repo health (max +101):

Check	Points	Why
Official maintainer	+18 / 0	Vendor-published > random fork
Editorially verified	+5 / 0	A human actually reviewed it
Actively maintained	+12 / +6 / 0	Last commit ≤90d / ≤365d / older
Established contributor base	+10 / +5 / 0	Bus factor: ≥5 / ≥2 / 1
Community adoption	+10 / +5 / 0	Stars: ≥1000 / ≥100 / fewer
Has license	+8 / 0	Unlicensed = unclear usage terms
Reasonable dependency footprint	+5 / +2 / 0	Smaller supply-chain surface
Has homepage/docs	+3 / 0
Substantial README	+5 / 0	≥500 chars
Archived	0 / −40	No future fixes. Ever.

MCP-specific:

Check	Points	Why
Read-only mode available	+15 / 0	A safer way to run it should exist
Supports OAuth	+10 / 0	Stronger than static tokens
Dangerous tools present	0 / −10 / −25	Share of tools flagged execute/delete/write
Dangerous tools, no safe mode	0 / −20	The double penalty — risky and no escape hatch
Weak or missing auth	0 / −15	Anyone with the config file can use it

Scores land in three bands: 80–100 safe, 60–79 medium, 0–59 high risk.

The part I almost got wrong

My first instinct was to let an LLM write the verdict and set the score.
That's a mistake — an AI can be convinced by good marketing copy in a README
just as easily as a human can. So the pipeline splits it:

The score comes only from verifiable facts: GitHub metadata, license, commit history, tool risk classification. No AI in that loop at all.
The verdict (the human-readable "should you use this" paragraph) starts as an AI draft grounded strictly in those same facts — then a human reads it before it ever goes live. Nothing gets published on the strength of an AI's opinion alone.

What it looks like end to end

For a server like qdrant/mcp-server-qdrant, the pipeline:

Scrapes the repo (README, package.json, releases, commit history)
Runs the 15 checks → a score (58, in this case — medium risk, mostly held back by no OAuth support and a moderate dependency footprint)
Generates a ready-to-paste config for Claude Desktop / Cursor / VS Code / Windsurf
Extracts its actual tool list and FAQs from the README
Waits for a human to confirm the AI's draft summary before anything ships

Where it stands today

213 servers, 36 categories, all re-scraped and re-scored on a rolling basis
so a score reflects last week's commit activity, not last year's.

It's open at repoai.io — MCP is the first layer, Claude
Skills and the rest of the AI-tool ecosystem are next, same trust model.

Curious what other people would want a security scorer to check for —
what's bitten you when installing an MCP server?