DEV Community

Jason Lee
Jason Lee

Posted on

Strata Beats GitHub's Own MCP Server by Hiding 1,000 Tools Behind Five

Klavis AI cover image

If you've wired more than three MCP servers into Claude Code or Cursor at the same time, you've probably felt it: the agent gets slower, more indecisive, and occasionally picks the wrong tool out of a list it clearly didn't read carefully. That's not a bad prompt. It's arithmetic. Every MCP server you attach registers its full tool catalog — names, descriptions, JSON schemas, the works — directly into the model's context window before a single token of your actual request gets processed. Attach GitHub, Slack, Notion, Linear, and a couple of internal servers, and you can easily be spending thousands of tokens just describing tools the model will never call in that turn.

Klavis AI, a startup out of Y Combinator's X25 batch, built Strata to make that problem go away — not by giving agents better judgment, but by giving them fewer things to judge. Strata launched on Product Hunt as "One MCP server for AI agents to handle thousands of tools," pulled 694 upvotes and 140 comments, and the underlying open-source project has since crossed 5,800 GitHub stars under an Apache-2.0 license. The pitch is almost aggressively simple: instead of exposing 1,000 tools to your model, expose five, and let the agent ask for the rest only when it actually needs them.

That's a smaller idea than it sounds, and that's exactly why it's worth a close look — the interesting part isn't the pitch, it's whether hiding tools behind a router quietly costs you accuracy, latency, or both.

The problem, stated precisely

MCP standardized how an AI agent talks to external tools — call a server, get a manifest of its tools, feed those tool definitions to the model as part of its context, let the model decide which to invoke. That worked fine when "external tools" meant one filesystem server and maybe a database connector. It stops working cleanly once you're aggregating real infrastructure: a GitHub server alone can expose 40+ discrete tools (issues, PRs, commits, actions, releases, code search…), and a Notion or Slack server isn't much leaner. Wire up half a dozen of these and you're not giving the model options anymore — you're giving it a phone book and asking it to dial the right number on the first try, every time, while also doing the task you actually asked for.

Two failure modes show up in practice. The obvious one is token cost: tool schemas are pure context-window overhead that gets re-sent on every single turn of a conversation, whether or not any of those tools get used. The less obvious one is accuracy degradation — models measurably get worse at tool selection as the candidate list grows, because "pick the one correct function from 300 plausible-looking ones" is a genuinely harder discrimination problem than "pick the one correct function from 5." Klavis frames this as the "N × M integration problem" — N agents times M tools, all needing to be wired together and all competing for the same limited context budget — and it's the same shape of problem that's pushed a whole category of MCP gateway and registry products (Smithery, Arcade, and hosted platforms like Composio among them) to exist in the first place. Strata's specific answer is architectural rather than curatorial: it doesn't try to pick better tools for you, it changes how many tools the model ever sees at once.

What Strata actually is

Strata is a single MCP server that sits between your agent and every other MCP server you've configured. Instead of forwarding all of those servers' tools directly into the model's context, it collapses the entire catalog — regardless of how many servers or tools are behind it — into five fixed meta-tools:

  • discover_server_actions — browse available actions across your configured servers
  • get_action_details — pull the full parameter schema for one specific action, on demand
  • execute_action — actually run the action against the target server
  • search_documentation — query a server's docs when the agent is unsure what's available
  • handle_auth_failure — recover gracefully when a downstream server's OAuth token has expired

That's the entire surface area the model sees at the start of a conversation, no matter whether you've got 3 servers or 30 behind Strata. Klavis describes the flow behind those five tools as four progressive stages: intent recognition (what is the user actually trying to do), category navigation (which connected service is relevant), action selection (which specific action fits), and only then, API execution — where the full parameter schema for that one action gets loaded into context, not the schemas for the other 999 actions sitting unused.

This is the core trade Strata is making, and it's worth being explicit about it: a request that used to be a single model call ("here's the full GitHub API surface, go") becomes a short chain of calls (discover → get details → execute). You're trading a large, flat context cost paid once per turn for a smaller, staged context cost paid across a few round trips per action. Whether that's a good trade depends entirely on how many tools you'd otherwise be loading — which is precisely what the benchmark numbers below are trying to answer.

A concrete walkthrough

It's easier to see the trade with an actual request. Say you've got Strata fronting three servers — GitHub, Slack, and Notion — and the user asks: "open a GitHub issue for the bug Sarah flagged in #eng-alerts yesterday, then drop a link in the same channel."

Without Strata, a typical MCP client would front-load every tool from all three servers into the system context before the model does anything: 40-odd GitHub tools, a couple dozen Slack tools, a similar count from Notion — call it 90-120 tool definitions, each with a name, description, and full JSON Schema for its parameters. That's commonly 8,000-15,000 tokens of pure tool-catalog overhead, sent again on every turn of the conversation, for a task that will end up calling exactly two tools.

With Strata, the model sees only the five meta-tools at the start. Its first move is discover_server_actions scoped to Slack, to find the message; that returns a short list of Slack-related actions, not Slack's full schema set. Once it identifies search_messages as the right action, get_action_details pulls that one action's parameter schema — not the other 20 Slack actions it didn't need. It calls execute_action to run the search, repeats roughly the same discover → details → execute pattern against GitHub to file the issue, and again against Slack to post the reply. Total round trips: more than the single-shot version. Total tool-schema tokens loaded across the whole exchange: a small fraction of the upfront cost, because at no point did the model ever see schemas for the 100+ actions it wasn't going to call.

That's the whole bet in one example: more calls, less context per call, and — per the MCPMark numbers — a better hit rate on picking the right action in the first place, because the model is never choosing from a haystack.

Deployment: this is not a SaaS-only pitch

Strata ships two ways, and the split matters for anyone evaluating lock-in risk:

Open source, self-hosted. The open-strata package installs locally:

pipx install strata-mcp
strata add --type stdio playwright npx @playwright/mcp@latest
strata add --type http github https://api.githubcopilot.com/mcp/
strata run --port 8080
Enter fullscreen mode Exit fullscreen mode

Servers can be stdio (local subprocess), SSE, or HTTP (with optional OAuth), and configuration lives in a plain JSON file at ~/.config/strata/servers.json — nothing exotic, nothing you can't inspect or version-control yourself. A strata tool add subcommand wires it directly into Claude Code, Gemini CLI, VS Code, or Cursor's MCP config with one command, at either user or project scope.

Managed cloud, via Klavis. The hosted version at klavis.ai adds 100+ prebuilt MCP integrations with OAuth already handled (Gmail, Slack, GitHub, Notion, and others), Python and TypeScript SDKs, a REST API, and — per Klavis's own framing — "enterprise-grade performance" and "advanced AI model optimizations" that aren't part of the open-source build. There's also a separate product, MCP Sandbox, for teams that need scalable MCP environments for LLM training and reinforcement learning, which is outside the scope of what most individual developers will touch.

The practical read: you can adopt the routing pattern for free and never touch Klavis's servers, or you can pay for their managed OAuth and integration catalog if you don't want to run your own token storage. That's a meaningfully different commitment than most MCP gateway products, several of which only exist as hosted services.

The benchmark claim, and how much to trust it

The number doing most of the marketing work here comes from MCPMark, a third-party benchmark built specifically to stress-test MCP-based agent workflows against real-world tasks. On MCPMark, Klavis reports Strata achieving a +15.2% higher pass@1 rate versus the official GitHub MCP server, and +13.4% higher pass@1 versus the official Notion MCP server, when the underlying task set is identical and only the tool-exposure strategy changes. Separately, in Klavis's own human evaluation across more than 2,000 real-world queries, Strata is reported to hit 83%+ accuracy on complex, multi-app workflows.

Two things are true at once here. First, MCPMark being an independent benchmark (not something Klavis built) gives the comparison more credibility than a self-authored eval would — it's not grading its own homework on the primary number. Second, the specific comparison configuration, and the 83% human-eval figure, are Klavis's own reported results, not independently reproduced by a third party as far as this research turned up. Treat the MCPMark delta as a real, externally-anchored signal that progressive disclosure helps tool selection accuracy — and treat the 83% figure as a vendor claim worth validating against your own workload before you build a budget projection on it.

What the numbers do support, directionally, is the underlying thesis: giving a model fewer, better-scoped tool choices at each step measurably beats dumping the entire catalog into context, at least on official GitHub and Notion MCP servers under MCPMark's task suite. That's a more interesting result than "aggregation saves tokens" — it's a claim that aggregation done this specific way also improves correctness, not just cost.

Where this sits next to everything else in the MCP-tooling space

MCP tooling has split into a few distinct lanes, and it's easy to conflate them:

  • Registries and directories (Smithery, Glama, the official MCP servers list) help you find MCP servers. They don't solve the context-overload problem once you've found and connected several.
  • Hosted integration platforms (Composio being the best-known example) give you managed OAuth and a large catalog of pre-built connectors as a service, generally billed per use or per seat. They solve the "don't build your own Slack OAuth flow" problem, not the "the model is choking on 400 tool schemas" problem — though several, Composio included, have started adding their own tool-filtering layers for the same reason Strata exists.
  • Routing/aggregation layers — where Strata sits — sit between the agent and however many MCP servers you already run, and their entire job is context discipline: fewer tools visible per turn, more tools reachable overall.

Strata's differentiator inside that third lane is that it's open source with a real, inspectable local mode, not just a hosted proxy you have to trust blind. You can read src/strata/server.py and tools.py yourself; you're not taking Klavis's word for what the router does with your requests unless you choose the managed version. For a category where "an intermediary now sees every tool call your agent makes" is a legitimate concern, that's a real point in Strata's favor over gateway-only competitors.

It's also worth being precise about what Strata is not trying to be. It isn't a new protocol, a fork of MCP, or a replacement client — it's a standard MCP server that happens to proxy other MCP servers behind it, which means anything that already speaks MCP (Claude Code, Cursor, VS Code's agent mode, Gemini CLI) can adopt it without any client-side changes. That's a lower-friction integration path than tools that require a custom SDK or a non-standard transport, and it's part of why the strata tool add one-liners for four different clients work at all — Strata is playing entirely inside the existing spec, just changing what gets exposed and when.

Pricing shape

Klavis doesn't publish a public rate card in the materials this research turned up, and the open-source open-strata package itself is free — you're only paying for compute you already run. The managed cloud tier is the commercial product: 100+ prebuilt OAuth-handled integrations, the SDK/REST layer, and the unspecified "advanced AI model optimizations" Klavis mentions in its own README, positioned as the difference between "try the pattern for free" and "let us run the token custody and integration maintenance for you." Anyone evaluating this seriously for production should treat the managed tier's actual cost as an open question to confirm directly with Klavis rather than something this launch's public materials answer on their own — a gap worth flagging, since it's the kind of detail a launch page glosses over in favor of the open-source star count.

Why this matters beyond the immediate token bill

Cost. Tool schemas are sent on every turn in most MCP client implementations, not once per session. If you're running a coding agent for hours a day against 30+ tools spread across several servers, the wasted context is a recurring, compounding cost — not a one-time tax. Collapsing the visible surface to five stable meta-tools makes that overhead roughly constant regardless of how many servers you add later.

Latency, with a caveat. Smaller context per call generally means faster time-to-first-token and cheaper inference. But the progressive-discovery flow adds hops — discover, then details, then execute is up to three round trips where a flat tool list would have been one. For latency-sensitive interactive use (a chat UI where a user is watching the cursor blink), that trade needs to be measured on your own stack, not assumed. Strata's argument is that the aggregate latency, once you account for how much slower models get when reasoning over huge tool lists, nets out ahead — the MCPMark pass@1 numbers are indirect evidence for that, but they measure accuracy, not wall-clock latency.

Security surface. Centralizing OAuth handling and execution through one router is a double-edged trade. It's genuinely easier to audit one component's token handling than to trust N different MCP server implementations with varying security hygiene. It's also a single point that, if compromised, has visibility into every downstream action your agent takes. The self-hostable open-source path meaningfully changes that calculus versus a hosted-only gateway, since you can keep that chokepoint inside your own infrastructure.

Maintainability. Adding a new internal tool to your agent stack becomes "add an MCP server behind Strata" rather than "re-tune every prompt that references your growing tool list." That's a real DX win for teams whose tool catalog changes weekly, which describes most companies actively building internal agent tooling right now.

Practical use cases

The clearest fit is a coding agent (Claude Code, Cursor, or a custom harness) with 5+ MCP servers already wired in — GitHub, a project tracker, a docs search, a database inspector, maybe an internal deploy tool. That's exactly the setup where tool-list bloat is already visibly hurting tool-selection accuracy, and where strata tool add gets you to a working router in minutes.

A second fit is multi-app workflow agents — the kind that need to read a Notion doc, cross-reference a Slack thread, and file a Linear ticket in one pass. Klavis's own SDK examples show exactly this pattern: create_strata_server(servers=[GMAIL, SLACK]) gives you one MCP endpoint that internally routes across both, rather than forcing the calling agent to juggle two separate tool namespaces itself.

A third, more specialized fit is teams building internal "agent platforms" for non-engineering users, where the tool catalog is large, changes often, and needs centralized OAuth and audit logging — the managed Klavis offering is built for exactly that scenario, at the cost of depending on a third party for the token custody piece.

Limitations the launch page doesn't dwell on

The progressive-discovery flow assumes the model correctly identifies intent and category before drilling into specific actions — if it mis-navigates that first hop (picks "GitHub" when the right server was actually "GitLab," for instance), you've added round trips for a wrong answer instead of a fast wrong answer. Flat tool lists fail fast; staged discovery can fail slow.

The headline accuracy numbers are strongest specifically against GitHub's and Notion's official MCP servers under MCPMark — that's a meaningful, credible comparison, but it's not evidence that Strata beats every possible tool-exposure strategy (curated static subsets, task-specific tool filtering, or other progressive-disclosure implementations from competing routers weren't part of the published comparison this research turned up).

The open-source and managed versions are explicitly not feature-equivalent — Klavis is upfront that the hosted product includes "advanced AI model optimizations" the self-hosted build doesn't have, which is a legitimate SaaS-vendor move but means teams evaluating the free version on GitHub aren't evaluating the same thing Klavis's benchmark numbers may partly reflect, depending on which build those numbers were run against.

And as with any router sitting in the request path, you're adding a new component that needs to stay up, stay fast, and stay correctly configured — one more piece of infrastructure between your agent and the tools it needs, even if it's a piece you fully control.

Independent read

The tool-overload problem Strata is solving is real and not controversial — anyone who's configured more than a handful of MCP servers at once has watched an agent get worse, not better, as its options multiplied. What's genuinely interesting about Strata isn't the observation, it's that Klavis went and got an independent-benchmark number to back the fix instead of shipping the idea on vibes: a measured pass@1 improvement over official vendor MCP servers on a third-party benchmark is a materially stronger claim than most Product Hunt launches make, and it deserves credit for that.

Where I'd push back a little: the 83% human-eval figure is unaudited and vendor-reported, the open-source/managed feature split makes it unclear which build the headline numbers describe, and progressive disclosure's latency profile under real interactive load hasn't been published anywhere I could find this run — only the accuracy side has real evidence behind it. None of that undermines the core architectural idea, which is sound and, being Apache-2.0 licensed with 5.8k stars and an active repo, low-risk to try without committing to Klavis as a company. It does mean "15% better tool selection" shouldn't get rounded up to "15% better agent," which is a claim nobody involved actually made, but which is an easy one to accidentally repeat.

Who should try it, wait, or skip it

Try it now if you're running an agent — coding assistant or otherwise — against five or more MCP servers simultaneously and have noticed tool-selection getting flaky as you've added more. The open-source path costs you a pipx install and a config file; there's no reason not to benchmark it against your own tool list before deciding anything.

Wait if you're only running one or two MCP servers. The progressive-discovery overhead (extra round trips per action) is a real cost, and it's not clearly worth paying until your tool catalog is large enough that context bloat is your actual bottleneck rather than a hypothetical one.

Skip it if you need the managed cloud version specifically and can't accept a third party sitting in the OAuth path for tools like Gmail or Slack — that's a legitimate constraint for regulated environments, and it's not one the open-source build fully solves, since the free version still requires you to build and operate that OAuth handling yourself if you want to avoid Klavis's hosted layer entirely.

Discussion

If progressive tool disclosure genuinely trades a flatter, larger context cost for a staged, smaller one with extra round trips — at what point does an agent's tool catalog get large enough that this trade stops being obviously worth it for your stack, and have you actually measured that crossover, or are you assuming it based on how bad flat tool lists already feel?

Sources:

Top comments (0)