DEV Community

Cover image for Multi-Tenant MCP Gateway: Isolate Tools with Bifrost
Kamya Shah
Kamya Shah

Posted on

Multi-Tenant MCP Gateway: Isolate Tools with Bifrost

#mcp #security #ai #devops

Multi-Tenant MCP Gateway: Isolate Tools with Bifrost

TL;DR

  • A multi-tenant MCP gateway acts as a centralized policy enforcement layer between AI models and backend tools, isolating client access across organizational boundaries.
  • Direct connections between AI agents and Model Context Protocol servers fail in multi-tenant environments because MCP lacks native multi-tenant authorization boundaries.
  • Bifrost isolates tenant access through virtual keys, role-based access profiles, and dynamic MCP tool groups that filter tool schemas before models see them.
  • Centralized credential handling and downstream federated authentication prevent tenant token leakage while maintaining strict per-tenant audit trails.
  • Bifrost introduces only 11 microseconds of overhead per request at 5,000 requests per second, making policy enforcement imperceptible to end users.

Production AI applications increasingly rely on autonomous agents that connect directly to enterprise databases, internal APIs, and third-party SaaS applications. When multiple enterprise customers, internal business units, or development squads share the same AI infrastructure, deploying a multi-tenant MCP gateway becomes necessary to prevent unauthorized data exposure and credential cross-contamination. Bifrost, an open-source AI gateway written in Go, provides the control plane required to centralize Model Context Protocol (MCP) traffic, enforce tenant isolation, and govern tool execution at scale.

The Multi-Tenant MCP Challenge: Why Direct Connections Fail

Connecting AI clients directly to Model Context Protocol servers creates operational failure points as soon as more than one team or customer enters the system. The Model Context Protocol specification defines standard JSON-RPC 2.0 message exchanges for discovering tools, reading resources, and executing actions. However, MCP was initially designed around single-user, local workstation paradigms such as desktop developer environments.

In a shared SaaS platform or enterprise internal developer platform, four structural problems emerge when running unmanaged MCP connections:

  1. Tool Namespace Pollution and Context Exhaustion: If an AI agent receives the raw union of every MCP tool available across an enterprise, the model's context window fills with hundreds of schema definitions. Anthropic evaluation data shows that loading large, unfiltered tool catalogs into context degrades agent decision accuracy significantly compared to scoped discovery. Irrelevant tool definitions also consume billable input tokens on every turn.
  2. Missing Authorization Boundaries: MCP servers generally operate under a flat trust model. Once an agent establishes a connection to an MCP server, it can invoke any tool declared by that server unless custom authentication logic is added to the backend tool implementation.
  3. Credential Sprawl and Secret Leakage: Direct MCP configurations often rely on long-lived API tokens or service account keys hardcoded into environment files or client settings. If an agent prompt injection trick forces the model to inspect its environment or echo execution parameters, shared backend credentials risk exposure.
  4. No Tenant-Aware Rate Limiting or Cost Allocation: Without a mediation layer, a rogue agent or heavy tenant can exhaust shared rate limits on external APIs, starving neighboring tenants of compute capacity and tool availability. 
Without a Gateway (Flat Namespace, High Risk):
[ Tenant A Agent ] ──┐
                     ├───> [ Shared MCP Server ] ───> Backend Databases (No Isolation)
[ Tenant B Agent ] ──┘

With a Multi-Tenant MCP Gateway:
[ Tenant A Agent ] ───> [ Multi-Tenant MCP Gateway ] ───> Tenant A Tools & Scoped Credentials
[ Tenant B Agent ] ───> [ Multi-Tenant MCP Gateway ] ───> Tenant B Tools & Scoped Credentials
Enter fullscreen mode Exit fullscreen mode

Building tenant isolation into every individual MCP server forces development teams to rewrite authentication, logging, and rate-limiting code across dozen of distinct microservices. A specialized gateway centralizes these controls in the network path, decoupling tenant security from tool implementation.

Multi-Tenant Architecture: Isolation Layers in an MCP Gateway

A production-grade multi-tenant MCP gateway must establish isolation at every stage of the request lifecycle: transport authentication, tool catalog discovery, tool parameter validation, downstream execution, and observability.

An intricate geometric filtering grid floating in mid-air, allowing authorized circular tokens to pass through specific

The following matrix outlines the core requirements of multi-tenant MCP architectures and how gateway-level enforcement solves them:

Isolation Layer Risk Without Gateway Gateway Enforcement Mechanism Operational Benefit
Tenant Identity Spoofed client identities, ambient authority Cryptographic verification via JWT/OIDC or Virtual Keys Guaranteed identity attribution per request
Tool Discovery Information leakage via tools/list schema inspection Dynamic schema filtering before payload reaches LLM Lower token costs, higher model routing accuracy
Tool Execution Cross-tenant parameter injection, execution crossover Request-time entitlement checks and argument rewriting Hard boundaries on database queries and API calls
Credential Management Shared tokens across tenants, key rotation downtime Centralized vault storage and per-user/per-tenant OAuth brokering Zero secrets exposed to client applications
Noisy Neighbor Control One tenant exhausts global API quotas Hierarchical token and request rate limiters Guaranteed quality of service for all tenants
Audit Compliance Inconsistent logging across microservices Structured, immutable JSON-RPC audit trails per tenant Clean SOC 2, HIPAA, and GDPR compliance validation

In this architecture, the gateway functions as an intelligent reverse proxy for tools. Incoming requests from client applications present credentials identifying the tenant context. The gateway validates these credentials, looks up tenant entitlements, prunes unauthorized tools from the catalog, and forwards validated execution calls to upstream servers.

Enforcing Tenant Scopes with Virtual Keys and Access Profiles

In Bifrost, the foundational mechanism for establishing tenant boundaries is the Virtual Key. A virtual key (sk-bf-*) is an abstraction managed by the gateway that replaces raw provider API keys and downstream service tokens. Instead of distributing real credentials, platform teams issue distinct virtual keys to each customer, organization, or internal service.

Virtual keys act as the policy enforcement root within the gateway runtime. When an incoming JSON-RPC or HTTP request reaches Bifrost, the gateway evaluates the key against configured constraints:

  • Model and Provider Filtering: Dictates which upstream LLM engines the tenant's agents are allowed to invoke.
  • Budget Caps: Imposes hard currency or token spend ceilings over hourly, daily, weekly, monthly, or calendar-aligned billing intervals.
  • Rate Limits: Restricts requests per minute (RPM) and tokens per minute (TPM) to safeguard upstream infrastructure from abuse.
  • Attachment Hierarchy: Virtual keys link exclusively to specific teams or enterprise customers, ensuring metrics roll up accurately into financial chargeback reports. 
{
  "name": "tenant-acme-prod",
  "key_prefix": "sk-bf-acme",
  "budget": {
    "max_limit": 500.00,
    "reset_duration": "1M",
    "calendar_aligned": true
  },
  "rate_limits": {
    "requests_per_minute": 600,
    "tokens_per_minute": 200000
  },
  "metadata": {
    "tenant_id": "cust_acme_corp_8821",
    "tier": "enterprise"
  }
}
Enter fullscreen mode Exit fullscreen mode

For large deployments managing thousands of tenants, creating individual policies manually becomes inefficient. Bifrost resolves this via Access Profiles. An Access Profile serves as a reusable template bundling model access, rate limits, spending caps, and tool permissions.

When a new enterprise tenant signs up through an identity provider such as Okta, Microsoft Entra ID, or Keycloak, Bifrost automatically provisions a scoped virtual key inheriting the profile rules. If the platform team subsequently alters the Access Profile to grant access to a new secure utility tool, the policy change instantly propagates across all associated tenant keys without manual credential rotation.

Tenant boundary validation is further reinforced through Data Access Control (DAC). DAC ensures administrators managing Tenant A in the Bifrost console or REST API cannot view the prompts, logs, virtual keys, or tool execution history belonging to Tenant B. Scopes are separated into own-data, team-data, and system-wide visibility, ensuring administrative isolation mirrors runtime execution boundaries.

Dynamic Tool Filtering and MCP Tool Groups

The central security requirement of a multi-tenant MCP gateway is ensuring an agent cannot discover or execute tools outside its designated tenant boundary. Exposing administrative or cross-tenant tools in an LLM prompt invites prompt injection exploits, where an attacker tricks the model into calling unauthorized functions.

Bifrost implements a strict deny-by-default architecture through MCP Tool Filtering and MCP Tool Groups.

How Tool Filtering Operates at Runtime

Rather than exposing every tool attached to the gateway, Bifrost computes a dynamic intersection of allowed tools every time an agent queries tools/list or issues an execution request:

  1. Client-Level Restrictions: Defined globally on the upstream MCP connection, specifying the baseline tools made available by that server.
  2. Virtual Key Restrictions: Configured on the tenant's virtual key, defining an explicit allowlist of permitted tool names.
  3. Request-Level Headers: Optional runtime headers (x-bf-mcp-include-tools or x-bf-mcp-include-clients) passed by the client application to narrow its own context for a specific task. 
Incoming Request (Virtual Key: sk-bf-acme)
                    │
                    ▼
┌──────────────────────────────────────────────┐
│       Bifrost Filter Evaluation Engine       │
│                                              │
│  All Available Tools: [A, B, C, D, E, F]     │
│  Virtual Key Allowlist: [A, B, C]            │
│  Request Header Scope: [B, C, D]             │
│                                              │
│  Resulting Scope: Allowlist ∩ Header Scope   │
│  Exposed to Model: [B, C]                    │
└──────────────────────────────────────────────┘
                    │
                    ▼
Returned to Agent via JSON-RPC `tools/list`
Enter fullscreen mode Exit fullscreen mode

If an agent attempts to bypass prompt instructions and manually formats a JSON-RPC tools/call for tool D, Bifrost blocks the request at the gateway layer before the packet ever leaves the network. The upstream MCP server never receives the call, preventing backend execution entirely.

Managing Reusable Tool Bundles with Tool Groups

In enterprise environments, backend tools are rarely assigned one by one. Instead, platform administrators bundle them into logical collections using MCP Tool Groups.

A Tool Group draws tools from multiple disparate MCP servers, such as combining a PostgreSQL query tool, a Jira issue tracker, and a document retriever into a unified bundle named support-agent-tools. This group is then linked directly to a tenant's virtual key or an Access Profile.

{
  "name": "finance-tenant-tools",
  "description": "Scoped toolset for accounting agents",
  "tools": [
    {
      "mcp_client_id": "postgres-db-server",
      "tool_name": "run_read_only_query"
    },
    {
      "mcp_client_id": "stripe-api-server",
      "tool_name": "fetch_customer_invoices"
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

By mapping tool groups to virtual keys, platform operators maintain centralized control. If a security vulnerability is identified in an upstream tool, disabling that tool inside the tool group instantly revokes access across every tenant virtual key in production.

Credential Isolation and Federated Authentication

Multi-tenancy breaks down quickly if backend tools rely on a single, shared system credential. If Tenant A and Tenant B both invoke a tool that queries Google Drive or GitHub, executing those queries under a static gateway service account risks cross-tenant data leaks.

Bifrost prevents credential crossover through native integration with the Model Context Protocol authentication framework and downstream federated authentication mechanisms.

A central glowing nexus receiving transparent data streams and attaching individual cryptographic security keys before r

Per-User and Per-Tenant OAuth

Bifrost acts as a secure credential broker. When configuring an MCP client connection within the gateway, administrators select from distinct authentication modes tailored to multi-tenant isolation:

  • Static Headers: For single-tenant private MCP instances where credentials do not vary.
  • Shared OAuth 2.0: The gateway manages automated token acquisition, storage, and refresh flows using client credentials or PKCE.
  • Per-User / Per-Tenant OAuth: Bifrost maintains isolated session storage for individual callers. During runtime tool execution, Bifrost injects the specific tenant or end-user access token into the upstream request.
  • Token Exchange: Incoming OpenID Connect (OIDC) tokens presented by the AI application are exchanged via OAuth 2.0 Token Exchange (RFC 8693) for downstream resource-specific tokens.

Under this model, the large language model never sees, handles, or outputs backend authentication secrets. Downstream services receive authenticated requests carrying proper tenant-scoped identity claims, allowing database row-level security (RLS) and external API permissions to enforce data boundaries naturally.

Automated Failure Recovery and Destructive Safety

Network timeouts and expired tokens are common when orchestrating multi-step agent workflows across distributed tools. Bifrost includes automatic inline auth-failure recovery.

If an upstream MCP server rejects a tool invocation with a 401 or 403 status code, the gateway force-refreshes the caller's OAuth credential and retries the tool execution transparently. To ensure safety, Bifrost inspects MCP tool annotations: tools explicitly marked as destructive or non-idempotent are never retried automatically, preventing accidental duplicate operations such as double financial transactions or repeat database deletions.

Runtime Security: Guardrails, Prompt Injection Defense, and Edge Governance

Isolating tool access is only half the battle in multi-tenant environments. A tenant's prompt or external tool output could contain prompt injections designed to hijack the model, exfiltrate sensitive data, or poison shared context windows.

Bifrost incorporates native enterprise Guardrails directly into the gateway proxy layer. Every prompt entering the gateway and every tool result returning from an MCP server passes through content safety inspections before being rendered to the caller.

[ Tenant Request ] ──> [ Gateway Guardrails (PII & Secrets) ] ──> [ Model Routing ]
                                                                          │
                                                                          ▼
[ Client Application ] <── [ Guardrails Verification ] <── [ Tool Execution Result ]
Enter fullscreen mode Exit fullscreen mode

Defense-in-Depth Inspection Engines

Bifrost supports both native regex inspection and deep integrations with specialized security systems:

  • Native Secrets Detection: Automatically identifies API keys, private certificates, and environment tokens using Gitleaks-backed scanning, blocking secrets from leaking via agent outputs.
  • PII Redaction and Custom Regex: Strips customer phone numbers, credit card data, and healthcare identifiers before payloads reach external model providers.
  • External Safety Classifiers: Passes prompts and tool results to AWS Bedrock Guardrails, Azure Content Safety, or Patronus AI for advanced adversarial injection detection.

Beyond centralized server infrastructure, organizations must address shadow AI emerging on employee laptops. Developers and knowledge workers frequently configure desktop applications such as Claude Desktop, Cursor, or terminal-based coding agents that connect directly to unvetted local and remote MCP servers.

Beyond routing, Bifrost applies governance and security controls (virtual keys, budgets, guardrails, audit logs) centrally, and Bifrost Edge extends that same governance and security to AI traffic on employee machines, with endpoint enforcement on each device.

Bifrost Edge (currently in alpha) runs as a background endpoint service deployed fleet-wide via MDM tools such as Jamf or Microsoft Intune. It discovers local MCP server configurations inside desktop applications and reports them to the centralized dashboard. If an employee configures an unauthorized MCP server or attempts to bypass multi-tenant policies, Edge MCP governance blocks execution locally, ensuring enterprise tool policies remain consistent from internal developer laptops to cloud production servers.

End-to-End Implementation: Configuring a Multi-Tenant Setup

Deploying a multi-tenant MCP gateway involves connecting upstream tool servers, defining tenant tool groups, and binding those groups to tenant virtual keys. Bifrost can be deployed directly via Docker, Helm on Kubernetes, or within private cloud environments using In-VPC deployments.

Step 1: Register Upstream MCP Servers

First, register the external MCP servers inside Bifrost using the configuration API or management dashboard. In this example, we register a database service and an issue tracker:

curl -X POST http://localhost:8080/v1/mcp/clients \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $BIFROST_ADMIN_KEY" \
  -d '{
    "client_id": "corporate-jira-server",
    "transport": "sse",
    "url": "https://mcp.internal.example.com/jira/sse",
    "auth_type": "oauth",
    "oauth_config": {
      "client_id": "gateway-client-id",
      "client_secret": "env(JIRA_MCP_SECRET)",
      "token_url": "https://auth.example.com/oauth/token"
    }
  }'
Enter fullscreen mode Exit fullscreen mode

Step 2: Create Scoped Tool Groups

Next, define an MCP Tool Group that limits exposed operations to safe, read-only capabilities:

curl -X POST http://localhost:8080/v1/mcp/tool-groups \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $BIFROST_ADMIN_KEY" \
  -d '{
    "name": "tier-standard-support",
    "description": "Standard support toolset for tenant customer service",
    "tools": [
      {
        "mcp_client_id": "corporate-jira-server",
        "tool_name": "search_issues"
      },
      {
        "mcp_client_id": "corporate-jira-server",
        "tool_name": "get_issue_details"
      }
    ]
  }'
Enter fullscreen mode Exit fullscreen mode

Step 3: Issue Tenant Virtual Keys with Tool Restrictions

Now create a tenant-specific virtual key. Bind the virtual key to the tool group created in Step 2, apply spending limits, and enforce strict tenant metadata:

curl -X POST http://localhost:8080/v1/governance/virtual-keys \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $BIFROST_ADMIN_KEY" \
  -d '{
    "name": "tenant-globex-prod",
    "budget": {
      "max_limit": 150.00,
      "reset_duration": "1M"
    },
    "rate_limits": {
      "requests_per_minute": 120
    },
    "mcp_tool_groups": ["tier-standard-support"],
    "allowed_models": ["anthropic/claude-3-5-sonnet", "openai/gpt-4o"]
  }'
Enter fullscreen mode Exit fullscreen mode

The gateway returns a secure key (sk-bf-globex-...). Provide this key to the tenant's agentic application.

Step 4: Connecting Clients to the Gateway

The tenant application configures its AI client (such as Claude Code, Cursor, or an agent framework using the OpenAI SDK) to point at Bifrost. Bifrost exposes an aggregated MCP endpoint at /mcp handling standard JSON-RPC 2.0 communication:

{
  "mcpServers": {
    "bifrost-gateway": {
      "url": "https://gateway.example.com/mcp",
      "headers": {
        "Authorization": "Bearer sk-bf-globex-prod-9941a8e"
      }
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

When the tenant agent issues tools/list, Bifrost inspects the virtual key, validates that only search_issues and get_issue_details belong to tier-standard-support, and returns only those two schemas. Any other tools hosted on corporate-jira-server (such as delete_project or create_user) remain completely invisible and uncallable.

Step 5: Auditing and Observing Tenant Execution

All tool calls flowing through Bifrost generate structured telemetry. Bifrost exports native Prometheus metrics and OpenTelemetry (OTLP) traces to observability platforms like Datadog, Grafana, or Honeycomb.

Platform teams can inspect request latencies, track tool execution errors, and verify that tool execution overhead remains negligible. In sustained benchmarks, Bifrost adds only 11 microseconds of internal overhead per request at 5,000 requests per second, ensuring multi-tenant governance introduces no perceptible latency into production agent interactions.

Every execution event is recorded in immutable Audit Logs, documenting the exact timestamp, calling virtual key, tenant ID, tool parameters, and response status to fulfill compliance mandates.

Frequently Asked Questions

What is a multi-tenant MCP gateway?

A multi-tenant MCP gateway is an infrastructure proxy that sits between AI agents and Model Context Protocol servers to isolate tool access, authenticate requests, manage credentials, and enforce rate limits across multiple independent tenants, teams, or customers sharing the same AI platform.

How does Bifrost isolate MCP tools between different tenants?

Bifrost isolates tools by pairing virtual keys with dynamic tool filtering and MCP tool groups. When a tenant authenticates using its virtual key, Bifrost evaluates the key's permissions and prunes all unauthorized tools from the schema returned to the model, blocking unauthorized invocations at runtime.

Can an AI model bypass MCP tool restrictions if it guesses the tool name?

No. Bifrost enforces tool validation at both the schema listing stage (tools/list) and the execution stage (tools/call). If a model attempts to execute a tool not explicitly permitted by the caller's virtual key, the gateway rejects the request with a policy error and never forwards the call to the upstream tool server.

How does a multi-tenant MCP gateway prevent cross-tenant credential leaks?

Bifrost centralizes downstream credentials and supports per-user OAuth and token exchange protocols. Upstream tool tokens are injected server-side by the gateway based on validated tenant session identity. The AI model never receives, stores, or transmits backend API credentials, preventing token exfiltration via prompt injection attacks.

What is the performance impact of routing MCP requests through Bifrost?

Bifrost is written in Go and engineered for high-concurrency environments, adding only 11 microseconds of internal routing overhead per request under sustained loads of 5,000 requests per second. This ensures security policy enforcement introduces zero perceptible latency to interactive agent workflows.

Does a multi-tenant MCP gateway support rate limits and budget controls?

Yes. Bifrost provides hierarchical cost and usage governance, allowing platform teams to assign request-per-minute (RPM), token-per-minute (TPM), and periodic currency spending caps directly to tenant virtual keys, teams, and customer accounts.

Next Steps: Deploying a Multi-Tenant MCP Gateway

Operating production AI agents across multi-tenant environments requires shifting tool governance from ad-hoc application code into a dedicated, high-performance infrastructure layer. By centralizing Model Context Protocol traffic through Bifrost as an MCP gateway, engineering teams can eliminate context window pollution, protect downstream backend systems with strict access boundaries, and enforce enterprise compliance across every tenant interaction.

Platform teams evaluating solutions for agent security and tool isolation can explore the Bifrost open-source repository, review the official documentation, or request an enterprise demo to inspect multi-tenant governance capabilities in depth.

Sources

Top comments (0)