We scanned 900+ MCP configurations on GitHub. 75% failed basic security checks. Nobody pins versions. The most popular MCP server is a bare shell wrapper.
I expected to find maybe a dozen hardcoded API keys and a handful of overly permissive configurations scattered across the results. The usual negligence you stumble on when you go digging through public repositories looking for things people probably shouldn't have committed.
What I didn't expect was that three out of four configuration files would fail basic security checks, and that the single most popular "MCP package" in the entire dataset wouldn't actually be a package at all.
This is the full account of how I got to those numbers, what the raw data revealed along the way, and where I think the whole MCP configuration ecosystem is quietly heading.
AI creates faster than it can be verified. MCP servers multiply this problem: every tool your agent calls is a new unverified input. The runtime layer, the proxy between your agent and the API, is where verification actually happens, because it's the only place that sees everything.
Why I started poking around in the first place
I've been building AI agent security tooling for the past few months, mostly focused on runtime enforcement — basically making sure autonomous agents don't do things they shouldn't be doing when they're making calls to LLM APIs behind your back.
MCP kept surfacing in that work. For anyone who hasn't encountered it yet: MCP is the protocol that Claude Desktop, Cursor, and a growing number of similar tools rely on to connect AI agents to external servers. You define which servers the agent talks to in a JSON config, and then it just... has those capabilities. Reading files, querying databases, calling APIs, running shell commands, whatever those servers decide to expose.
That configuration file is basically the permission boundary for everything the agent can do. Get it wrong and every misconfiguration flows directly into the agent's behavior, which gets uncomfortable when you consider that agents process untrusted input from users, tool outputs, and scraped web content.
I kept running across theoretical discussions of MCP vulnerabilities. Prompt injection through tool results, malicious MCP servers, data exfiltration via crafted tool calls. Plenty of hypothetical attack scenarios had been written up, but I couldn't find anyone who had actually gone and looked at what real developers are configuring in practice.
I figured the fastest way to answer those questions was to just go look.
How the scanner was built
The core approach was deliberately unsophisticated. GitHub's Code Search API, looking for specific filenames and content patterns across public repos. The scanner grabs claude_desktop_config.json, .cursor/mcp.json, and anything with mcpServers in it, pulls down the raw file, tries to make sense of the JSON, and if it parses okay, runs its 52 checks against it.
GitHub's Code Search caps results at roughly 1000 per query pattern, which I partially worked around by splitting queries using date ranges and file size qualifiers. Some file paths on GitHub contain spaces, parentheses, and unicode characters (one particularly memorable path included Portuguese text about "creating your second brain with AI"), and the scanner kept crashing on URL encoding issues. Three separate rounds of fixes before the thing could crawl reliably.
After approximately 40 minutes of crawling, the scanner had collected 900 configuration files from 839 unique repositories. Every repository identifier was SHA256 hashed before being stored. No owner names, no repository URLs, and no actual credential values exist anywhere in the dataset.
The initial results were surprisingly bad
75% of the collected configuration files contained at least one security finding.
I had gone into this expecting something around 30%, maybe 40%. Not three quarters.
The severity split: 1.6% critical (actual credential exposure), 76.2% high, 21% medium. I couldn't figure out why high severity was so dominant until I drilled into the individual check results.
Turns out one specific check was responsible for almost all of it.
Nobody pins versions (43.6%)
Nearly half of all scanned configuration files reference MCP server packages without specifying which version should be installed. This single check accounts for the vast majority of high-severity findings in the entire dataset.
The pattern I encountered over and over:
{
"mcpServers": {
"filesystem": {
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-filesystem", "/home/user"]
}
}
}
That -y flag is the problem. It tells npm to just grab whatever happens to be the latest version right now and run it. If someone pushes a bad update to that package tonight, or if the maintainer account gets compromised, your agent loads the new code next time it starts. Nobody reviews it.
The fix is trivial:
"args": ["-y", "@modelcontextprotocol/server-filesystem@1.2.3", "/home/user"]
The JavaScript ecosystem already went through precisely this lesson. The left-pad incident in 2016 was supposed to have permanently established the principle that you pin your dependencies. That was ten years ago. And now we're doing the exact same thing, except the packages involved don't just pad strings. They read your filesystem and execute shell commands.
The shell access problem is worse than it sounds
Roughly one in eleven configuration files grants the AI agent direct access to command execution. The most frequently appearing entry across the entire dataset is not a recognized package with documented behavior and scope controls. It's run. Just that. A bare shell command wrapper.
| What developers put in their configs | Count | What it gives the agent |
|---|---|---|
run (bare shell wrapper) |
136 | Can execute any command |
server-filesystem |
51 | Reads and writes files |
mcp-remote |
34 | Connects to remote MCP servers |
server-github |
16 | GitHub API access |
server-sequential-thinking |
11 | Reasoning chain stuff |
server-puppeteer |
11 | Controls a headless browser |
server-memory |
9 | Stores data persistently |
server-playwright |
9 | Also controls a browser |
So the bare unrestricted shell executor beat the official Anthropic-maintained scoped package by almost 3 to 1. I had to recount that because it seemed wrong.
Many of the server-filesystem entries were pointed at absurdly broad paths. Not /home/user/project/data but just /. Or C:\. Or the user's entire home directory — SSH keys, cloud credentials, browser profiles, your whole digital identity sitting there for the agent to browse through.
What kept me scrolling: the combinations
The scanner evaluates individual findings in isolation. But the thing that proved most concerning was how frequently multiple issues appeared stacked together:
{
"mcpServers": {
"shell": {
"command": "run",
"args": []
},
"files": {
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-filesystem", "/"]
},
"github": {
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-github"],
"env": {
"GITHUB_TOKEN": "ghp_xxxxxxxxxxxxxxxxxxxx"
}
}
}
}
Nothing pinned. Shell wide open. Filesystem pointed at root. GitHub token just sitting right there. One file, committed together, probably in about 30 seconds.
One bad npm update and an attacker can read everything on disk, run whatever commands they want, and push code to your GitHub. The agent handles the whole chain by itself with no human anywhere in the process.
This isn't hypothetical. In February 2026, an autonomous AI agent operating under the GitHub account hackerbot-claw systematically exploited misconfigured CI/CD workflows across seven major open-source repositories, including projects from Microsoft, DataDog, and the CNCF. The agent achieved remote code execution in five of the seven targets. Every attack relied on the same root cause: overly permissive configs that nobody audited.
What I changed in my own configuration
Four changes:
1. Every package version pinned explicitly. Manual updates and occasional breakage, but that friction is the point.
2. Shell access removed entirely. After seeing run in 136 configurations with zero restrictions, "convenient" stopped being a justification.
3. All credentials moved into .env files, .gitignore verified.
4. Filesystem paths scoped to the specific project directory. Not home folder. Not root.
That's it. Four changes that take about two minutes.
Where I think this is heading
75% of public configs failing basic checks isn't an individual negligence problem. When three quarters of your users get it wrong, the defaults are wrong. If the quick path through the documentation gives you an insecure config and the secure version requires you to know about version pinning and go look up the latest tag number, the insecure version is going to win every time.
The MCP ecosystem has maybe a year before one of two things happens. Either the tooling catches up — built-in config validation, automatic version locking, permission audit integrated into Claude Desktop and Cursor. Or there's a big enough supply chain incident that the conversation gets forced from the outside.
The EU AI Act enforcement begins in August 2026, five months away. Audit trails for AI agent behavior are about to become a legal requirement.
Looking at what I found in these 900 configs, my money is on the incident coming first. I hope I'm wrong.
The scanning tool, all 100+ checks across 9 categories, and the full analysis pipeline are open source. Run it yourself at orchesis.ai/scan. Everything runs in your browser, no data sent anywhere.
The runtime proxy that catches these issues in production: github.com/poushwell/orchesis. MIT license, zero dependencies, pip install orchesis.
Top comments (0)