Not affiliated with CData Software. Numbers come from CData’s public “Build vs. buy for MCP” research summary (August 2026). Full findings: https://www.cdata.com/lp/claude-mcp-report/
Every team wiring agents into production eventually hits the same question: build the MCP path yourself, or buy something purpose-built?
CData published a blunt answer from the build side.
What they measured
Across nine sessions and two test phases, they evaluated Claude Code against eight dimensions that matter for enterprise MCP reliability — including OAuth token lifecycle, pagination correctness, large-dataset behavior, and error handling. They used a real production environment and list sizes from a handful of items up to 40,000+.
The results (as published)
- Baseline Claude Code passed 1 of 8 dimensions without human intervention.
- Expert guidance and structured prompting improved outcomes, but still left 3 dimensions unresolved.
- The failures that matter most were not always loud crashes. Silent wrong data — lost rows, hallucinated pagination counts, dropped columns — can still look like a “successful” tool call.
Their conclusion is the one operators already feel: a working prototype MCP server is not the same as something you would trust in production.
The missing layer most build-vs-buy posts skip
Connector quality is real. So is managed MCP data access. Those matter.
There is a second failure mode that shows up the moment agents start acting:
- Tool returns plausible but wrong data.
- The agent treats the call as success.
- It plans, writes, emails, deploys, or opens a PR on top of garbage.
- Nobody gets a stack trace — just a quiet mess.
If production MCP reliability is hard even when an expert is watching Claude Code, it is worse when autonomous agents chain tools overnight.
That is the layer we care about at ThumbGate: pre-action governance.
- Capture human thumbs on what went wrong.
- Promote matchable prevention rules.
- Block known-bad tool patterns before the next run.
- Keep the receipt: what was allowed, warned, or denied.
We are not claiming to replace CData’s connector research. We are claiming something narrower: once tools are in the loop, you need a firewall for agent actions — not only better scaffolding.
What to do Monday morning
- Read the CData dimension grid if you are choosing build vs buy for MCP data access.
- Inventory which MCP tools your agents can call without a human in the loop.
- Add a pre-action gate on high-blast-radius tools (email, payments, production shell, write APIs).
- Log every deny and every silent “looks successful” failure you catch manually — turn those into rules.
Soft CTA
Control Hermes / coding agents from the browser:
Self-serve reliability tooling:
No enterprise pilot pitch in this post. No claim of CData partnership. Just the public numbers and the governance gap they make obvious.
Sources: CData “Build vs. buy for MCP: what the data says” announcement (Aug 17, 2026); CData public evaluation summary and report landing page.
Top comments (0)