DEV Community

Pranay Batta
Pranay Batta

Posted on

Best MCP Gateway for 50% Token Cost Savings

#ai #mcp #programming #llm

TL;DR: Classic MCP dumps 100+ tool definitions into every LLM call. Bifrost's Code Mode generates TypeScript declarations instead, cutting token usage by 50%+ and latency by 40-50%. If you are running 3 or more MCP servers, this is the single biggest cost lever you have.

The Problem with Classic MCP

I have been testing MCP setups for a few months now. The standard approach is simple. You connect your MCP servers, and every tool definition gets sent to the LLM as part of the context window. Every single call.

With 3 MCP servers, you might have 30-40 tools. With 10 servers, easily 100+. Each tool definition includes the name, description, input schema, and parameter types. That is a lot of tokens. And you are paying for every single one of them on every request.

The math is straightforward. If your average tool definition is 200 tokens, and you have 50 tools, that is 10,000 tokens of overhead per call. At scale, this adds up fast.

How Bifrost Code Mode Changes This

Bifrost takes a different approach with its Code Mode. Instead of exposing raw tool definitions to the LLM, it generates TypeScript declaration files (.d.ts) for all connected MCP tools.

The LLM then writes TypeScript code to orchestrate multiple tools in a restricted sandbox environment. Instead of the model making 5 separate tool calls (each requiring a round trip), it writes one code block that handles all 5 operations.

Here is what this means in practice:

  • Token reduction: 50%+ compared to classic MCP. The TypeScript declarations are more compact than full JSON schemas, and the model makes fewer round trips.
  • Latency reduction: 40-50% compared to classic MCP. Fewer round trips means faster overall execution.
  • Recommended when: You are using 3 or more MCP servers.

What Code Mode Actually Does

The execution model is restricted by design. Here is what is available in the sandbox:

Available: ES5.1+ JavaScript, async/await, TypeScript, console.log/error/warn, JSON.parse/stringify, and all MCP tool bindings as globals.

Not available: ES Modules, Node.js APIs, browser APIs, DOM, timers (setTimeout/setInterval), network access.

This is not a general-purpose runtime. It is a controlled environment where the LLM can orchestrate tools safely. No arbitrary code execution, no network calls outside of the tool bindings.

You can configure tool bindings at the server level or tool level, depending on how granular you need the control to be. The docs cover the binding configuration in detail.

The Latency Numbers

Bifrost itself adds 11 microseconds of latency overhead per request. It is written in Go and handles 5,000 RPS sustained throughput. That is roughly 50x faster than Python-based alternatives.

For MCP-specific operations:

  • Sub-3ms MCP latency overall
  • InProcess connections: ~0.1ms
  • STDIO connections: ~1-10ms
  • HTTP connections: ~10-500ms (network dependent)

The MCP tool discovery is cached after the first request, so subsequent calls hit ~100-500 microseconds for discovery and ~50-200 nanoseconds for tool filtering.

Agent Mode: The Other Side

Bifrost also has an Agent Mode that turns the gateway into an autonomous agent runtime. You configure which tools are auto-approved via tools_to_auto_execute, set a max_depth to prevent infinite loops, and let the agent handle iterative execution.

This is a different use case from Code Mode. Agent Mode is for workflows where you want the LLM to act autonomously within boundaries. Code Mode is for when you want to reduce token costs on tool-heavy operations.

Deployment

Setup is zero-config. You can start with npx or Docker. The gateway supports 19+ providers out of the box (OpenAI, Anthropic, Azure, Bedrock, Gemini, Mistral, Cohere, Groq, and others), all through an OpenAI-compatible API format.

# npx
npx -y @maximhq/bifrost

# Docker
docker run -p 8080:8080 maximhq/bifrost
Enter fullscreen mode Exit fullscreen mode

Who Should Use Code Mode

If you are running fewer than 3 MCP servers, classic mode is probably fine. The overhead is manageable.

If you are running 3+, especially with 50+ tools across those servers, Code Mode is worth testing. The 50%+ token savings are significant at scale, and the 40-50% latency improvement compounds across multi-step agent workflows.

I tested this on a setup with 5 MCP servers and 80+ tools. The token savings were immediately visible in the cost dashboard. The reduced round trips also made the overall agent response noticeably faster.

GitHub: git.new/bifrost
Docs: getmax.im/bifrostdocs
Website: getmax.im/bifrost-home

Top comments (0)