Your company uses six different AI providers. OpenAI for ChatGPT, Anthropic for Claude and Groq for speed critical inference.
Each one has different API formats. Different authentication models. Different rate limits and costs. Different failure modes.
Your application code has to know about all of them. Your security team has to audit requests across all of them. Your finance team has to track costs across all of them. Your compliance team has to ensure governance across all of them.
Bifrost Gateway solves this by doing what HTTP gateways have done for decades: centralizing control. But for AI.
👀 What is an MCP gateway?
Model Context Protocol (MCP) is an open standard that lets AI models discover and execute external tools at runtime filesystems, web search, databases, ticketing systems, and custom business logic instead of being limited to text generation.
An MCP gateway sits between your applications (or external MCP clients like Claude Desktop and Cursor) and the upstream MCP servers. Instead of each client maintaining its own connections, credentials, and tool lists, the gateway:
- Aggregates tools from multiple MCP servers into one registry
- Applies governance: authentication, tool filtering, budgets, and rate limits
- Exposes a single endpoint that external MCP clients can connect to
In Bifrost, this pattern is implemented in two complementary roles:
| Role | What it does |
|---|---|
| MCP Client | Connects to external MCP servers via STDIO, HTTP, or SSE |
| MCP Server (Gateway) | Exposes aggregated tools at /mcp for Claude Desktop, Cursor, and other MCP-compatible clients |
Bifrost is both an AI gateway (routing LLM traffic to 20+ providers) and an MCP gateway (connecting to and exposing tool servers). The open-source gateway covers virtual keys, budgets, rate limits, routing, and MCP tool filtering. Bifrost Enterprise adds RBAC, SSO, audit logs, MCP Tool Groups, guardrails, clustering, and in-VPC deployment options.
⚙️ How does an MCP gateway work?
Connection layer
Each upstream MCP server is registered as an MCP client in Bifrost. Three connection protocols are supported:
| Type | Description | Best for |
|---|---|---|
| STDIO | Spawns a subprocess, communicates via stdin/stdout | Local tools, CLI utilities, scripts |
| HTTP | Sends requests to an HTTP endpoint | Remote APIs, microservices, cloud functions |
| SSE | Server-Sent Events for persistent connections | Real-time data, streaming tools |
Authentication is configured separately via a top-level auth_type on each client: none, headers, oauth, per_user_oauth, or per_user_headers. OAuth auth types apply only to HTTP and SSE connections.
Gateway endpoints
When Bifrost acts as an MCP server, external clients connect to:
| Endpoint | Method | Purpose |
|---|---|---|
/mcp |
POST | JSON-RPC 2.0 for tool discovery and execution |
/mcp |
GET | Server-Sent Events for persistent connections |
Clients configure their MCP host to point at http://your-bifrost-gateway/mcp, optionally with a virtual key in the Authorization, x-bf-vk, x-api-key, or x-goog-api-key header.
Tool discovery and execution
Tools are discovered when a client connects and refreshed on a configurable sync interval (default 10 minutes). Each tool follows the prefixed naming convention clientName-toolName (for example, filesystem-read_file).
There are two distinct execution models:
1. LLM Gateway path (inference + tools)
When your application calls /v1/chat/completions, Bifrost does not automatically execute tool calls. The default flow is stateless and explicit:
1. POST /v1/chat/completions → LLM returns tool call suggestions (NOT executed)
2. Your app reviews tool calls → Apply security rules, get user approval if needed
3. POST /v1/mcp/tool/execute → Execute approved tool calls explicitly
4. POST /v1/chat/completions → Continue the conversation with tool results
2. Pure MCP Gateway path (Claude Desktop, Cursor, etc.)
When external MCP clients connect directly to /mcp, Bifrost exposes tools over the MCP protocol. The host application (not Bifrost) runs the agent loop and decides whether each tools/call requires user confirmation. Bifrost's tools_to_auto_execute setting only applies in Agent Mode when Bifrost is also running the LLM loop, because it is ignored in pure gateway mode.
Three levels of tool filtering
A tool must pass all applicable filters to be available:
-
Client config:
tools_to_executeon each MCP client (baseline) -
Request headers:
x-bf-mcp-include-clientsandx-bf-mcp-include-toolsper request -
Virtual key config:
mcp_configsarray (takes precedence over request headers)
Deny-by-default for virtual keys: when a virtual key has no mcp_configs, no MCP tools are available. Clients not listed in mcp_configs are implicitly blocked.
curl -X POST http://localhost:8080/api/governance/virtual-keys \
-H "Content-Type: application/json" \
-d '{
"name": "staging-key",
"mcp_configs": [
{
"mcp_client_name": "staging_database",
"tools_to_execute": ["query"]
}
]
}'
Enterprise: MCP Tool Groups
In Bifrost Enterprise, Tool Groups are reusable bundles of MCP tools attachable to virtual keys, teams, customers, users, providers, or API keys. At request time, Bifrost inspects the request context and exposes only the union of tools from matching groups. This adds a policy layer above per-key configuration for organizations with many teams and use cases.
Health and resilience
Bifrost monitors connected MCP clients with configurable health checks (default: ping every 10 seconds, 5-second timeout, 5 consecutive failures before marking unstable). HTTP/SSE clients reconnect make-before-break; STDIO clients reconnect close-first. Automatic exponential backoff handles transient failures.
🔎 What to evaluate in an MCP gateway
When comparing enterprise MCP gateway solutions, these criteria map directly to what production deployments require:
1. Security posture
| Question | Bifrost answer (per docs) |
|---|---|
| Are tool calls executed automatically? | No by default. Explicit /v1/mcp/tool/execute required on the LLM path |
| Is access deny-by-default? | Yes. Virtual keys with no mcp_configs get zero tools |
| Can you limit tools per team/environment? | Yes. Via virtual key mcp_configs, request headers, and Enterprise MCP Tool Groups |
| Is per-user upstream identity supported? | Yes. per_user_oauth and per_user_headers with lazy auth |
2. Authentication and identity
Evaluate whether the gateway supports both server-level auth (one shared credential for the team) and per-user auth (each end-user connects under their own account). Bifrost supports five auth types, with OAuth limited to HTTP/SSE and implementing Authorization Code flow only (no client-credentials mode).
For enterprise SSO, Bifrost Enterprise provides User Provisioning over OIDC (Okta, Microsoft Entra, etc) with role mapping from IdP groups, app roles, or custom claims, synchronized on each session.
3. Runtime vs administrative governance
This distinction matters: RBAC in Bifrost Enterprise governs the administrative surface (who can edit MCP configs, read logs, configure guardrails) not which tools an agent executes at runtime. Runtime access is controlled by virtual keys, tool filtering, and MCP Tool Groups.
RBAC permissions are Resource × Operation pairs (for example, MCPGateway:Update, AuditLogs:View). Three system roles ship: Admin (42 permissions), Developer (27), Viewer (14).
4. Observability and compliance
Look for request logging, MCP execution logs, and audit trails for configuration changes. Bifrost Enterprise provides:
- Audit Logs: signed, filterable administrative event trails with export to JSON, JSON Lines, or Syslog
- Log Exports: automated export of request logs to S3, GCS, BigQuery, and other data lakes
- Guardrails: PII detection, secrets detection, content safety, and custom regex on LLM traffic and MCP tool executions
5. Cost control at scale
Connecting many MCP servers inflates token usage because classic MCP injects every tool definition on every model turn. Bifrost Code Mode addresses this by having the AI write Python to orchestrate tools in a sandbox rather than exposing hundreds of tool definitions directly. In Bifrost benchmarks with 508 tools across 16 servers, Code Mode reduced input tokens by 92.8% and estimated cost by 92.2% (from $377 to $29 per benchmark round) while maintaining a 100% pass rate.
Virtual keys also provide independent budgets and rate limits for cost management at the key, team, and customer level.
6. Deployment model
| Deployment | When to use |
|---|---|
Open-source gateway (npx -y @maximhq/bifrost) |
Local dev, single-node production, full MCP + LLM gateway |
| Enterprise clustered (3+ pods, PostgreSQL) | HA production with SSO, audit, guardrails |
| In-VPC | Private-network deployment with no public traffic |
| Bifrost Edge (alpha) | Endpoint governance — routes AI and MCP traffic from every laptop through your Bifrost without per-app reconfiguration |
📊 Alternatives to a dedicated MCP gateway (and their trade-offs)
As MCP adoption moves from local experiments to production, several vendors now offer gateway layers. Each with a different center of gravity. The table below summarizes how the major options compare at a glance; the sections that follow go deeper on each one.
| Capability | Bifrost | Docker MCP Gateway | Microsoft | AWS AgentCore Gateway | Kong MCP Gateway |
|---|---|---|---|---|---|
| Primary role | Unified LLM + MCP + Agents gateway | Container-native MCP orchestration | K8s MCP proxy + Azure API Management | Managed MCP gateway on Bedrock | API gateway with MCP plugins |
| MCP client + server | Yes | Client/proxy | Yes (proxy + lifecycle) | Yes | Yes (proxy + REST→MCP conversion) |
| LLM provider routing | Yes (20+ providers) | No | No | Bedrock-centric | Via separate AI Gateway plugins |
| Tool filtering / governance | Virtual keys, mcp_configs, MCP Tool Groups |
Profiles, interceptors | RBAC, APIM policies | Gateway targets, semantic search | OAuth 2.1 + per-tool ACLs |
| OAuth for MCP servers | Yes (Authorization Code, PKCE, DCR) | Yes (built-in OAuth flows) | Yes (Entra ID) | Yes (IAM + OAuth inbound) | Yes (OAuth 2.1 via AI MCP OAuth2 plugin) |
| Enterprise audit / RBAC | Yes (Enterprise tier) | Logging + call tracing | Entra RBAC + APIM governance | CloudTrail / IAM | Enterprise-only, Konnect analytics |
| Open-source option | Yes (full gateway) | Yes (MIT) | Yes (K8s gateway OSS) | Managed AWS service | Gateway OSS; MCP plugins enterprise |
| Deployment | Binary, Docker, K8s, in-VPC | Docker Desktop / Docker Engine | AKS + Azure APIM | AWS managed | Kong Gateway / Konnect |
1. Bifrost
Bifrost is the open-source, high-performance AI gateway built in Go by Maxim AI. It leads this comparison for teams that need LLM routing, MCP aggregation, and agent governance in one self-hosted platform, not an MCP-only proxy sitting beside a separate LLM gateway.
MCP connectivity. Bifrost acts as both MCP client and MCP server. It connects outward to any MCP server you register (STDIO, HTTP, or SSE) and exposes aggregated tools at /mcp for Claude Desktop, Cursor, and other MCP-compatible clients. Five auth types cover the full spectrum: none, headers, oauth, per_user_oauth, and per_user_headers.
Security by default. On the LLM path, Bifrost does not auto-execute tool calls. Your application must explicitly call POST /v1/mcp/tool/execute. Virtual keys enforce deny-by-default: a key with no mcp_configs gets zero MCP tools. Three stacked filter levels (client config, request headers, virtual key config) control which tools each caller sees.
Governance and cost control. Open-source Bifrost covers virtual keys, budgets, rate limits, routing, and MCP tool filtering. Code Mode reduces input token usage by up to 92.8% and estimated cost by 92.2% when orchestrating many MCP servers, the model writes Python to call tools in a sandbox instead of round-tripping every tool definition through the context window. Agent Mode adds opt-in autonomous execution for tools explicitly marked in tools_to_auto_execute.
Enterprise tier. Bifrost Enterprise adds RBAC (Resource × Operation permissions), SSO via OIDC (Okta, Entra, Keycloak, etc), HMAC-signed audit logs, MCP Tool Groups, guardrails (PII, secrets, content safety), clustering, adaptive load balancing, and in-VPC deployment. Enterprise is a strict superset of OSS. Same config.json schema, no re-integration.
Performance. Bifrost adds roughly 11 µs of overhead per request at 5,000 RPS in sustained benchmarks, making it suitable for latency-sensitive production workloads.
Best for: Organizations running mission-critical AI workloads that need a single gateway for LLM traffic, MCP tool access, and agent execution with enterprise-grade governance, air-gapped or in-VPC deployment, and ultra-low latency. Start free with npx -y @maximhq/bifrost; add Enterprise when SSO, audit logs, and administrative RBAC are required.
2. Docker MCP Gateway
Docker MCP Gateway is Docker's open-source (MIT) solution for orchestrating MCP servers as isolated Docker containers. It acts as a centralized proxy: AI clients connect to the Gateway once, and the Gateway manages server lifecycle, credential injection, and routing across servers grouped in profiles.
Strengths: Strong container isolation (restricted privileges, network sandboxing, resource limits), built-in OAuth flows, secrets management via Docker Desktop, dynamic tool discovery, and call-tracing interceptors (secret scanning, signature verification). Integrates natively with Docker Desktop's MCP Toolkit for a low-friction local developer experience.
Trade-offs vs Bifrost: Docker MCP Gateway is MCP-focused, it does not route LLM traffic across providers, enforce virtual-key budgets, or offer Code Mode token savings. Cross-team organizational governance (RBAC, SSO, audit logs) requires additional tooling on top. Best suited for container-native teams already standardized on Docker rather than organizations needing a unified AI + MCP control plane.
Best for: Teams invested in the Docker ecosystem who want to package, run, and secure MCP servers as containers from local development toward production.
3. Microsoft MCP Gateway (Azure)
Microsoft offers two complementary paths for MCP at enterprise scale.
Open-source MCP Gateway is a Kubernetes-native reverse proxy and management layer for MCP servers on AKS. It provides session-aware stateful routing, MCP server lifecycle management (deploy, update, delete), and integration with Azure Entra ID for bearer-token authentication. A separate Tool Gateway handles dynamic tool routing behind the proxy.
Azure API Management adds enterprise governance: expose REST APIs as MCP servers, govern existing MCP endpoints, apply rate-limiting and content-safety policies, and discover servers through Azure API Center. The AI Gateway tier supports MCP-specific features through a dedicated release channel.
Strengths: Deep Azure and Entra ID integration, K8s-native scaling, REST-to-MCP exposure without rewriting APIs, and a mature API-management policy engine.
Trade-offs vs Bifrost: Microsoft's stack is Azure-centric and split across multiple services (AKS gateway + APIM + API Center). It does not unify LLM provider routing with MCP governance in a single binary. Per-tool token-cost optimization (Code Mode) and sub-microsecond LLM gateway overhead are not part of this offering.
Best for: Enterprises already on Azure and Kubernetes who want MCP server lifecycle management and API Management-style policy enforcement within the Microsoft cloud ecosystem.
4. AWS AgentCore Gateway
Amazon Bedrock AgentCore Gateway is AWS's managed MCP gateway for connecting agents to tools hosted on AWS services and external MCP servers. It supports multiple MCP protocol versions (including 2026-07-28 stateless tool calls), semantic search across tool catalogs, and inbound authentication via IAM and OAuth.
Strengths: Fully managed. No gateway pods to size or patch. Native IAM SigV4 for AWS-hosted MCP targets. Semantic search helps agents find the right tool when catalogs grow large. Tight integration with Bedrock agent runtimes and the broader AgentCore platform.
Trade-offs vs Bifrost: AgentCore Gateway is an AWS managed service with Bedrock-centric positioning, not a self-hosted, provider-agnostic LLM gateway. Pricing follows AWS consumption models rather than a free open-source tier. Virtual-key-style budgets, Code Mode orchestration, and air-gapped on-prem deployment are outside this product's scope.
Best for: Teams building agent workloads on Amazon Bedrock who want a managed MCP aggregation layer without operating their own gateway infrastructure.
5. Kong MCP Gateway
Kong MCP Gateway extends Kong's AI Gateway (v3.12+) with MCP-specific plugins. The AI MCP Proxy plugin acts as a protocol bridge. Converting REST APIs into MCP tools (conversion-only / listener modes) or proxying upstream MCP servers in passthrough mode. The AI MCP OAuth2 plugin implements OAuth 2.1 per the MCP authorization spec, mapping token claims to per-tool ACLs.
Strengths: Leverages Kong's proven API-gateway patterns, rate limiting, observability, developer portal, service catalog. Can federate multiple team-owned MCP servers behind one aggregated endpoint with centralized OAuth. REST-to-MCP conversion lets existing Kong-managed APIs become agent tools without new MCP server code.
Trade-offs vs Bifrost: MCP Gateway capabilities are enterprise-only paid plugins on Kong Gateway or Konnect. Not available in the open-source Kong edition. Kong does not natively route LLM traffic across 20+ providers or offer Code Mode token reduction. Configuration spans Services, Routes, Consumers, and plugin modes rather than a unified MCP + LLM config surface.
Best for: Organizations already running Kong API Gateway who want to add MCP aggregation, OAuth 2.1 enforcement, and REST-to-MCP conversion to their existing API-management stack.
💻 What implementing an MCP gateway costs
Costs fall into three buckets: software licensing, infrastructure, and operational/token savings.
Software licensing
| Tier | Cost (per docs) | What you get |
|---|---|---|
| Open-source Bifrost | Free | LLM gateway, MCP client + server, virtual keys, budgets, rate limits, routing, MCP tool filtering, Code Mode, observability |
| Bifrost Enterprise | 14-day free trial (no credit card); contact sales for pricing | Everything in OSS plus RBAC, SSO, audit logs, MCP Tool Groups, guardrails, clustering, adaptive load balancing, in-VPC deployment, log exports |
Bifrost Enterprise is a strict superset of the open-source gateway. Same config.json schema, no re-integration required.
Infrastructure (Enterprise production baseline)
The Enterprise sizing guide recommends:
Gateway pods
| Setting | Recommended |
|---|---|
| Pod count | 3 minimum |
| vCPU per pod | 4 |
| RAM per pod | 16 GB |
PostgreSQL
| Configuration | vCPU | RAM |
|---|---|---|
| Default (logs in PostgreSQL) | 8 | 24 GB |
| With object storage for large logs | 8 | 16 GB |
Object storage (S3, GCS) for log payloads reduces PostgreSQL write pressure and improves dashboard log-read latency. Audit log archival to object storage is configured separately under audit_logs.object_storage.
For a single-node or development deployment, the open-source gateway runs locally with:
npx -y @maximhq/bifrost
No minimum hardware is specified for OSS. Production Enterprise sizing above is the documented baseline for HA.
Token and inference cost savings
The largest ongoing cost for MCP-heavy workloads is often LLM token usage, not gateway infrastructure. Two Bifrost features directly reduce this:
Code Mode (recommended when using 3+ MCP servers):
- Up to 92.8% fewer input tokens vs classic MCP at scale
- Up to 92.2% lower estimated inference cost
- Benchmark: 508 tools / 16 servers — $377 → $29 per round, 75.1M → 5.4M input tokens
Governance features that reduce waste:
- Virtual key budgets — dollar limits with configurable reset periods
- Rate limits — token and request throttling per virtual key
- Semantic caching — reduces cost and latency for similar queries (open-source feature)
- Tool filtering — expose only the tools a team needs, reducing context size even in classic MCP mode
Total cost framing
| Scenario | Typical cost drivers |
|---|---|
| Single team, OSS, local/dev | $0 software + existing hardware; main cost is LLM API usage |
| Production OSS, single node | Compute for one gateway instance + PostgreSQL (if used) + LLM API usage |
| Enterprise HA (3 pods + PG + object storage) | Enterprise license + ~3 × (4 vCPU / 16 GB) + PostgreSQL 8 vCPU / 16–24 GB + object storage + LLM API usage |
| Enterprise + Edge (alpha) | Above + endpoint agent rollout (contact for alpha access) |
The gateway itself adds minimal latency. Bifrost benchmarks show 11 µs overhead per request at 5,000 RPS on the LLM routing path. The economic case for an MCP gateway is primarily governance (preventing unauthorized tool access) and token efficiency (Code Mode and filtering at scale), not raw infrastructure savings.
🖋️ Conclusion
An MCP gateway turns a collection of developer tool connections into an organization-wide control plane. Without one, every agent on every laptop can connect to production systems with no central policy, no audit trail, and no cost visibility.
Bifrost provides this as both an open-source AI + MCP gateway and an Enterprise tier for organizations that need RBAC, audit-grade logging, guardrails, and high-availability deployment.
🔗 Resources:
- Bifrost GitHub: https://github.com/maximhq/bifrost
- Bifrost Docs: https://docs.getbifrost.ai
-
Bifrost CLI:
npx -y @maximhq/bifrost-cli
Thanks for reading this article! ❤️
I'd love to hear your thoughts on this mode in the comments!
Top comments (5)
For MCP gateways, I would judge the category by auditability before feature count. Once tools can touch real systems, the important questions become who approved the call, what context the model saw, which connector executed, and how a team reconstructs the decision after something goes wrong.
That’s a great point. Auditability should be a first-class feature of MCP gateways.
Which AI model do you use the most?
GPT 5.6
The gateway framing is right because the hard part is rarely the model call itself. It is auditability across providers. Once tools touch real systems, I would judge the category by who approved the call, what context was visible, which connector executed, and how a team reconstructs the decision after a bad run.