Anthropic's reference MCP filesystem server scored 60 out of 100 in our behavioral security certification. Grade: F. Three critical blocking vulnerabilities. All credential exposure.
We didn't find this with a linter. We found it by actually trying to break the server.
The Findings
The reference filesystem server ships with 14 tools for reading, writing, and navigating files. Two of them failed our adversarial red-team testing.
Finding 1: edit_file — credential exposure via path traversal
When we sent double-encoded traversal input (%252e%252e%252f) and URL-encoded traversal input to the edit_file tool, the server responded with content containing credential data. The path validation logic exists in the codebase. It did not stop our test payloads.
Finding 2: edit_file — second traversal vector
The same tool failed on a separate URL-encoded traversal variant. Two distinct bypass vectors, same tool, same result: credential exposure.
Finding 3: read_multiple_files — direct credential harvesting
When we asked read_multiple_files to read common credential storage paths (AWS credentials, SSH config, database configs, system files containing authentication data), it complied. No restriction. No warning.
All three findings were classified as critical severity under CVSS v4.0 and flagged as certification blockers in our pipeline.
Why This Matters More Than You Think
This is not a third-party server written by a hobbyist. This is @modelcontextprotocol/server-filesystem, published by the Model Context Protocol project (a Linux Foundation series). It is the reference implementation that developers study, fork, and use as a template for building their own servers.
Every pattern in this codebase gets copied. Including the gaps.
But Wait — Didn't Another Scanner Give This a 99?
Yes. A recent study using static analysis scored Anthropic's official servers at 99-100 out of 100, praising their six layers of path validation.
That's the problem with static analysis. It checks whether validation code exists. It does not check whether the validation code works when someone actually tries to bypass it.
Our pipeline does not read the source code and check for patterns. It installs the server, starts it, connects to it over MCP, and throws adversarial payloads at every tool. Double-encoded traversal. URL-encoded traversal. Credential path harvesting. Prompt injection chains. Data exfiltration probes.
Static analysis asks: "Is there a guard?"
Behavioral testing asks: "Does the guard hold?"
The guard did not hold.
The Broader Ecosystem: 50 Capabilities, 14% Critical Failure Rate
We didn't stop at one server. We ran the same seven-stage certification pipeline across 50 AI capabilities: MCP servers, skills, hooks, plugins, and rules files from across the ecosystem.
7 out of 50 capabilities had critical blocking vulnerabilities. That's a 26% failure rate on the most severe category.
Here's what we found beyond the filesystem server:
devin-cursorrules: API Key Harvesting at Scale
A rules file marketed as a development assistant for the Devin AI coding tool. Our adversarial analysis discovered it was reading .env.local, .env, and .env.example files, loading credentials from six different API providers (OpenAI, Azure OpenAI, DeepSeek, Anthropic, Google, and SiliconFlow), and logging environment variable contents to stderr.
It also makes undisclosed network connections to external services including a hardcoded IP address. None of this is declared in its metadata.
This is not a bug. This is a design that harvests credentials by default.
everything-claude-code: 128 Security Findings
A plugin claiming to be a comprehensive Claude Code toolkit surfaced 128 security scan findings including command injection, data exfiltration, supply chain attack vectors, hardcoded secrets, and prompt injection.
The security scanner flagged it across every major vulnerability category. It still has zero certification blockers only because none of the adversarial behavioral tests triggered a critical-severity exploit. The static findings alone are a red flag that would give any security team pause.
Hooks Logging Everything
Multiple hook-type capabilities were recording every tool input and output to JSON files on disk. Every bash command, every file read, every API response. If a credential passes through any tool while these hooks are active, it gets written to a plaintext log file.
This creates a persistent credential capture surface. Most developers installing these hooks would never know it's happening.
The Methodology
Every capability we certify passes through a seven-stage pipeline:
- Ingestion: Clone the source, identify the publisher, compute provenance hashes, enumerate tools via live MCP connection
- SBOM & Supply Chain: Generate a software bill of materials, scan for known vulnerabilities across the dependency tree
- Security Scan: Static analysis for code-level security issues (command injection, data exfiltration patterns, hardcoded secrets)
- Functional Testing: Does it do what it claims? Test every declared tool against its contract
- Adversarial Red-Team: Throw attack payloads at every tool — path traversal, prompt injection, privilege escalation, credential harvesting, data exfiltration probes
- Behavioral Fingerprint: Map the actual runtime behavior profile
- Certification: Score across 8 weighted signals (adversarial 25%, provenance 20%, security scan 15%, supply chain 10%, behavioral pass rate 10%, consumer confirmation 10%, contract accuracy 6%, uptime 4%), sign the artifact with ES256, publish the behavioral contract
The scoring methodology is based on CVSS v4.0, NIST SP 800-30, SLSA, and ISO/IEC 25010. Every number traces to a recognized framework.
The output is not just a score. It's a signed behavioral contract: a portable artifact that documents exactly what the capability does, what it won't do, what happened when we tried to break it, and every finding with severity classification.
Immediate Mitigation
If you are running the reference filesystem server (@modelcontextprotocol/server-filesystem) in a production environment and have not patched the path validation logic yourself, we recommend disabling the edit_file tool immediately.
If you are using devin-cursorrules, audit your .env files and rotate any API keys that may have been logged.
If you have hooks installed that you didn't write yourself, check whether they're logging tool inputs and outputs to disk.
What Comes Next
The full certifications for every capability we've completed are published at fidensa.com/certifications. Search the tools you're running. Check the findings. The behavioral contracts are public.
The MCP ecosystem is growing faster than anyone is verifying it. There are over 17,000 MCP servers in the wild. Most have never been tested for what they actually do under adversarial conditions.
We're trying to change that.
If you're a publisher and want your capability certified, submissions are opening soon. If you're a developer and want to check whether the tools in your stack have been tested, start at fidensa.com.
Scott Bishop is the founder of Fidensa, an independent AI capability certification authority. He has 30 years of software development experience across the IMF, USPTO, and several Fortune 500 companies. Fidensa's certification methodology is modeled on UL's product safety certification approach: test the product, document what it does, sign the results.
Top comments (0)