DEV Community: Kuldeep Paul

Open-Source LLM Gateways for Production: Which One Fits Your Stack in 2026?

Kuldeep Paul — Thu, 11 Jun 2026 20:49:41 +0000

Evaluating five open-source AI gateways for multi-provider LLM routing, performance, and governance in 2026. Bifrost stands out as the enterprise choice for teams that need ultra-low latency, comprehensive governance, and production-grade reliability.

Building production AI systems in 2026 means routing across multiple LLM providers, managing costs at scale, and keeping your model calls resilient when a provider stumbles. That's where open-source AI gateways come in. They sit between your application and the LLM providers you're using, unifying fragmented APIs into one clean interface while handling failover, caching, budgets, and observability. Because they're open-source, you own the code, can audit every line of the routing logic, and deploy them inside your own infrastructure (no vendor control plane required).

Bifrost, built by Maxim AI as a Go-native gateway, has emerged as the top choice for teams running mission-critical AI at scale that demand production latency, deep access controls, and self-hosted deployment without compromise. In this post, we'll walk through five production-ready open-source gateways, what makes each worth considering, and how to choose the right fit for your workload.

Understanding Open-Source AI Gateways

A self-hostable gateway is infrastructure that runs on your machines, sitting between your services and the LLM providers you call. It accepts requests through a single API endpoint, routes them to the right provider, and sends responses back to your app. Along the way, it can enforce budgets, cache responses, log access, failover across providers, and expose your tools via the Model Context Protocol (MCP) for agentic work.

What changed in 2026 is the bar for "production-ready." Today's best gateways treat MCP traffic, semantic caching, and per-team budget controls as core capabilities, not bolt-on plugins. A CNCF survey shows cloud-native organizations increasingly run their own critical infrastructure layers rather than relying on managed services, and gateways are following the same trend.

Evaluating These Gateways: What Actually Matters

When you're testing gateways, focus on these dimensions:

Overhead at scale: How much latency does the gateway add per request? Test at sustained throughput (thousands of requests per second), not toy traffic. A gateway that adds milliseconds kills your latency budget for streaming chat or multi-step agent workflows.
Provider breadth: How many LLM APIs does it support? Can you use the same gateway code with OpenAI, Anthropic, Bedrock, and self-hosted models, or do you need workarounds?
MCP and agents: Does it natively support Model Context Protocol as both client and server? Can it route agent tool calls, or does that require a separate layer?
Governance and access control: Can you set per-team or per-project budgets, rate limits, and granular permissions? For multi-tenant or regulated setups, this matters hugely.
Caching strategy: Exact-match caching is table stakes. Semantic caching (matching similar requests even when phrased differently) cuts costs dramatically for production Q&A and RAG.
Deployment flexibility: Can you run this in VPC-only, air-gapped, on Kubernetes, or in a container? Does it have external dependencies that limit where you can run it?
Open-source terms: Is the license clean (Apache 2.0, MIT)? Is there a clear boundary between free and commercial tiers?

The Bifrost buyer's guide provides a structured matrix to evaluate all of these.

The Five Best Open-Source AI Gateways

1. Bifrost

Bifrost is a high-performance gateway written in Go, engineered from day one as infrastructure-grade software. Testing shows it adds roughly 11 microseconds of per-request overhead at 5,000 requests per second, which is about 54 times less tail-latency impact than a Python proxy on the same hardware, with 68% less memory use. You can grab the full benchmark results to see how it compares. Apache 2.0 licensed, code is on GitHub.

What it does well:

Single API gateway for 20+ providers: OpenAI, Anthropic, Bedrock, Vertex AI, Azure, Mistral, Groq, Cohere, Ollama, vLLM, all accessible through one OpenAI-compatible endpoint. Flip the base URL in your existing SDK, nothing else changes.
Native MCP as client and server: Bifrost can connect to external tool servers and expose tools to clients (Claude, agents, etc.), with Agent Mode for autonomous execution and Code Mode for AI-written tool orchestration that cuts tokens and latency. The MCP resource hub has the full story.
Multi-tier budget and access control: Virtual keys are the core, where you set per-consumer budgets, rate limits, and MCP tool allow-lists across virtual key, team, and customer tiers.
Zero-downtime failover: Automatic provider routing with weighted load balancing. When a provider starts returning errors, traffic shifts seamlessly.
Dual-layer caching: Combines exact-match with semantic similarity so rephrased questions hit the cache.
Enterprise deployment: In-VPC isolation, air-gapped mode, clustering for HA, RBAC, and immutable audit trails, built to satisfy SOC 2, GDPR, and HIPAA auditors.

Deploys in under 60 seconds with npx -y @maximhq/bifrost or a single Docker image. Integrates natively with Claude Code, Codex CLI, Gemini CLI, and Cursor.

Best for: Teams operating mission-critical AI workloads that require best-in-class performance, scalability, and reliability. It serves as a centralized AI gateway to route, govern, and secure all AI traffic across models and environments with ultra low latency. Bifrost unifies LLM gateway, MCP gateway, and Agents gateway capabilities into a single platform. Designed for regulated industries and strict enterprise requirements, it supports air-gapped deployments, VPC isolation, and on-prem infrastructure. It provides full control over data, access, and execution, along with robust security, policy enforcement, and governance capabilities.

2. LiteLLM

LiteLLM is a Python-based open-source project that gives you a unified OpenAI-compatible API to 100+ providers. Available as both a Python SDK and a standalone proxy server with built-in admin UI, virtual keys, and spend dashboards. It's widely used across the ecosystem and has a healthy contributor community behind it.

What draws teams: maximum provider catalog in this roundup, straightforward Python integration, and minimal friction to get started. The downsides surface under load. Python's Global Interpreter Lock caps single-process throughput, pushing tail latency higher as concurrency climbs. Running it at meaningful scale requires standing up PostgreSQL and Redis for state management. The open-source version offers exact-match caching (no semantic smarts) and lacks native MCP support. If you're evaluating a move to Bifrost, the feature comparison breaks down the gaps.

Best for: Python teams that value breadth of provider coverage for experimentation and early prototypes, and can live with higher latency ceilings as traffic grows.

3. Kong AI Gateway

Kong AI Gateway extends Kong's widely-deployed API gateway with plugins for LLM-aware routing, prompt templates, token-rate-limiting, and request transforms. If your team already runs Kong for API management, the appeal is obvious: add AI routing without standing up another proxy.

Where it gets tricky: AI features ship as plugins layered on a general-purpose gateway, not as a purpose-built AI system. The most advanced features (fine-grained governance, MCP, rich observability) live in Kong's commercial tier. The plugin architecture and config model add operational overhead if you're new to Kong.

Best for: Teams that already have Kong in production for API management and want to extend it into LLM routing without introducing a new infrastructure component.

4. Apache APISIX

APISIX is an Apache project: a cloud-native API gateway built on high-performance NGINX and Lua, with a strong governance model and active maintainer community. Recent versions added AI plugins to handle LLM routing and token-aware rate limiting.

APISIX shines if you're already using it for general APIs and want to route AI traffic through the same platform. The catch: AI capabilities come through plugins rather than native architecture, so you don't get semantic caching, a built-in MCP gateway, or the granular budget controls (hierarchical spend limits, per-consumer rate shaping) that purpose-built AI gateways offer. The APISIX config model has a learning curve if you haven't used it before.

Best for: Teams standardized on APISIX for API management that want plugin-based LLM routing without standing up separate infrastructure.

5. Envoy AI Gateway

Envoy AI Gateway is a newer open-source effort that layers LLM awareness onto the Envoy proxy and Kubernetes Gateway API. It's built for teams already running Envoy or Istio service meshes who want AI routing as part of their existing infrastructure fabric.

The win: native Kubernetes integration and service-mesh hooks. As a newer project, it carries some rough edges: narrower provider support, lower throughput (single-digit milliseconds of overhead), no semantic caching yet, and no virtual-key-style budget hierarchy. The xDS configuration model (Envoy's config system) is powerful but steep if you're outside the Envoy ecosystem.

Best for: Teams deeply committed to Kubernetes with Envoy or Istio already in the mesh, wanting AI traffic management that plugs into their existing infrastructure.

Quick Comparison Table

Gateway	Language	License	Overhead @ scale	Native MCP	Semantic cache	Deployment fit
Bifrost	Go	Apache 2.0	~11µs at 5K RPS	Yes (both)	Yes	Enterprise, production scale
LiteLLM	Python	MIT	Hundreds of µs	No (OSS)	Exact-match only	Prototyping, broad coverage
Kong AI Gateway	Lua/Kong	Apache 2.0	Millisecond range	Plugin-based	Plugin-based	Existing Kong shops
Apache APISIX	Lua/NGINX	Apache 2.0	Millisecond range	Limited	No (OSS)	Existing APISIX shops
Envoy AI Gateway	Go/Envoy	Apache 2.0	1-3 ms	No	No	Kubernetes/Istio mesh

For deeper detail on benchmarking methodology and the governance model behind the comparison, check the benchmark hub and governance deep-dive.

Common Questions

What's the actual performance difference?

Bifrost leads on raw throughput: 11 microseconds per request at high concurrency. Its Go architecture sidesteps the Python Global Interpreter Lock that Python proxies can't escape. That translates to roughly 54 times lower tail latency (P99) compared to a Python gateway under identical load. For streaming responses or agent workflows with multiple LLM hops, that difference between microseconds and milliseconds is the difference between feeling instant and feeling slow.

Do these gateways really support MCP?

MCP support varies. Bifrost has a native MCP gateway that acts as both client and server, with tool filtering and per-consumer auth. Others either expose MCP through plugins or don't support it in their open-source version. The MCP spec defines the standard, and native support (as opposed to plugin-based) makes a real difference in simplicity and capability.

Can I use these in a regulated industry?

Yes, if the gateway supports on-prem deployment, air-gapped operation, immutable audit logs, and fine-grained access control. Self-hosting keeps all prompts, responses, and audit trails inside your infrastructure, which is mandatory for SOC 2, HIPAA, and GDPR contexts. Bifrost's enterprise deployment docs cover the compliance-grade setup: VPC isolation, clustering for HA, RBAC with SSO, and audit trails sized for auditors.

Making Your Choice

The decision tree is straightforward:

Production AI at enterprise scale with strict latency and governance requirements: Bifrost is the clear pick. Lowest overhead, native MCP, hierarchical governance, semantic caching, compliance-ready deployment.
Maximum provider breadth for experimentation: LiteLLM's 100+ provider catalog is unmatched.
Already running Kong: Kong AI Gateway saves you introducing another proxy.
Already running APISIX: APISIX AI plugins let you reuse what you have.
Kubernetes and Istio native: Envoy AI Gateway slots into your mesh.

For teams weighing multiple options, the LLM gateway buying guide provides a detailed capability matrix to score each gateway against your specific requirements.

Next Steps

If you're looking at self-hosted gateways for your production AI stack, Bifrost on GitHub is open-source and deploys in 30 seconds. The docs cover setup, configuration, and integration with coding agents. To see how it handles your actual workloads and discuss deployment, book time with the Bifrost team. They'll walk you through performance tuning and compliance setup.

Enterprise Rollout of Claude Code and Codex: Governance Without Friction

Kuldeep Paul — Thu, 11 Jun 2026 20:49:08 +0000

Bifrost is the open-source AI gateway for governing AI coding agents. Deploy virtual key governance, guardrails, and audit logs to scale Claude Code and Codex safely.

Engineering teams adopting Claude Code and Codex face a governance dilemma: how do you scale agent usage to hundreds of developers while controlling costs, preventing data leakage, and maintaining compliance? Research by Black Duck shows 85% of organizations now use AI in development, but most lack centralized control over which models engineers can access, how much they spend, or whether sensitive data makes its way into external APIs. Bifrost, an open-source Go-based gateway built by Maxim AI, lets you deploy a single control point that every Claude Code and Codex call flows through (applying cost enforcement, secret detection, and compliance logging automatically, invisibly to the developer).

What Does Governance of Coding Agents Actually Look Like?

Governance means placing a policy enforcement layer between developers and their AI tools. Instead of each engineer managing their own provider credentials, picking their own models, and sending requests directly outbound, all traffic routes through a gateway that validates access, enforces budgets, scans for unsafe content, and records audit evidence.

In ungoverned setups, developers operate in silos: each person has their own API keys, each person chooses their own model, and there's no single view of what's being called or how much it costs. When trouble comes (a runaway loop burning through the monthly budget, credentials accidentally committed into a prompt, or a regulatory audit asking for proof of what was called and when), you're left assembling evidence from scattered logs across multiple provider accounts.

Bifrost flips this: Claude Code and Codex point to one endpoint, and you apply policy once instead of per-developer. That endpoint serves as both a proxy and a policy engine, keeping control unified. The governance resource page details how virtual keys, budgets, rate limits, and content rules all work together.

The Real Risks of Ungoverned AI Coding Agent Adoption

Three compounding risks emerge when coding agents aren't centrally governed:

Runaway spend. Coding agents generate many token-heavy requests. Without per-person budget limits, a single infinite loop or a developer experimenting with expensive reasoning models can produce five-figure surprises on next month's bill, with no clear owner or explanation.
Credential exposure in prompts. Developers paste code into agents for context. If that code contains database passwords, cloud credentials, or API secrets, and no layer inspects the traffic, those secrets travel straight to an external model provider. GitGuardian reported a 40% higher rate of secret leakage in repositories using AI assistants, driven by agents reading local files and incorporating them into prompts.
Compliance becomes impossible. Auditors need an immutable trail: who called what model, when, with what budget, and what was the outcome? Without centralized logging, you're hunting across vendor dashboards, trying to correlate a user's questions with a provider's request logs. Regulated workloads in healthcare, finance, or insurance cannot move forward without that evidence.

The Coalition for Secure AI noted that today's typical developer workflow puts AI agents outside the IDE with full repository access, able to read files, execute commands, and call external systems via MCP. That capability is powerful, but without a gate in front of it, policies cannot be enforced consistently. A gateway for CLI coding agents provides exactly that gate.

How Bifrost Brings Order to Coding Agent Deployments

Bifrost positions itself as an OpenAI, Anthropic, and Google-compatible proxy that Claude Code, Codex, and other agents point to instead of calling providers directly. The proxy applies governance rules to every request and response before they traverse the wire.

Issuing per-developer virtual keys with independent budgets and model restrictions

Virtual keys form the foundation of Bifrost's access control. Each key is a credential that a developer or squad uses; each key carries its own independent configuration:

Model permissions: a principal engineer's key can access expensive research-grade models, while team members' keys are limited to production-optimized, cost-efficient alternatives by default.
Budgets: each key has a spend ceiling in dollars, resettable daily, weekly, monthly, or yearly, plus the ability to roll budgets up to team or department levels for visibility.
Token and request caps: set maximum tokens per hour or maximum requests per minute, preventing infinite loops from consuming quota or blowing a budget.

Setting a single flag, enforce_auth_on_inference, makes the virtual key header mandatory on every request. Any call without a valid key is rejected immediately. This one switch transforms Bifrost from an optional proxy into the enforced control point for every coding agent in the organization.

Real-time inspection for secrets and PII

Coding agents inherently send local file contents, shell command outputs, and code snippets into prompts (all potential sources of sensitive data). Bifrost validates both incoming requests and outgoing responses without adding latency. The guardrails module offers multiple options:

Built-in secrets scanning (powered by Gitleaks) detects API keys, database credentials, and tokens in requests before they leave your network.
Custom regex rules (including templates for PII like SSNs and credit cards) let you define patterns specific to your organization and redact or reject them.
Multi-provider integration with AWS Bedrock Guardrails, Azure Content Safety, GraySwan, and Patronus AI for industry-standard content filtering, jailbreak detection, and policy enforcement.

You can chain rules so that a sensitive prompt gets scanned by multiple guardrails in sequence, giving you defense-in-depth at the gateway rather than scattered, hard-to-maintain checks inside each application.

Immutable audit logs for compliance teams

Every agent request is logged. The enterprise edition adds immutable audit trails that align with SOC 2 Type II, GDPR, HIPAA, and ISO 27001 requirements. Logs capture the developer identity, the model called, tokens consumed, guardrail violations detected, and the timestamp. These logs export continuously to your SIEM, data lake, or compliance archive, so compliance and security teams get one consolidated record instead of having to stitch together evidence from multiple vendor consoles.

Controlling which MCP tools each agent can access

When Claude Code connects to Bifrost as an MCP gateway, it gains access to whatever MCP servers Bifrost knows about. But tool filtering means each virtual key defines which tools are available to the bearer. One developer might have access to the deployment and database query tools; another might be limited to read-only introspection. Code Mode, which lets the agent orchestrate multiple tools by writing orchestration code, cuts token usage by over half and latency by 40%, lowering both cost and the overhead of compliance logging.

Deploying Safely: Identity, Access Control, and Network Isolation

A secure rollout isn't just about governance policy. It's about making sure policy enforcement itself is protected. Bifrost Enterprise adds three critical layers:

SSO and RBAC: Connect your identity provider (Okta, Microsoft Entra) so that developer access follows your existing directory. Define roles that control who can change policy, who can view usage telemetry, and who can access sensitive audit logs. Satisfies access-control requirements in SOC 2 and ISO 27001.
In-VPC and on-prem deployment: run Bifrost inside your private network boundary using Kubernetes, ECS, or bare metal. Prompts and responses never traverse the public internet, meeting data residency requirements for regulated workloads.
Secrets management integration: offload API key and credential management to HashiCorp Vault, AWS Secrets Manager, Google Secret Manager, or Azure Key Vault. Credentials are never stored in Bifrost configuration; they're fetched on demand and never logged.

These controls exist because policy is only credible if it cannot be bypassed by deploying in a different region or routing around the gateway.

Practical Rollout: Five Steps from Day Zero to Enforcement

A phased approach lets your platform team prove the value of governance on a pilot before expanding organization-wide:

Provision the gateway. Deploy Bifrost to your VPC or a managed hosting environment, configure your LLM providers, and set up optional guardrails.
Point your agents at Bifrost. Update Claude Code and Codex to use the Bifrost endpoint (change one environment variable). The Bifrost CLI tool makes this a one-command launch for each agent.
Create and distribute virtual keys. Issue one key per developer or per squad with model permissions, spend limits, and request caps. Then enforce mandatory authentication.
Activate content safety rules. Enable secrets scanning and PII rules first (they catch the most common leakage vectors), then layer additional guardrails from content safety providers.
Wire identity and auditing. Integrate your SSO provider, assign roles, and enable continuous audit log exports to your security infrastructure.

Performance remains predictable throughout: Bifrost adds only 11 microseconds per request at 5,000 RPS in sustained load, so policy enforcement is transparent to developers using Claude Code interactively.

Addressing Common Questions

Does using a gateway change how developers work?

No. Developers use Claude Code and Codex exactly as before; governance is applied invisibly at the proxy layer. The only change is the base URL the tool points at.

Can we use different LLM providers and models for different developers?

Yes. Each virtual key specifies which providers and models it can access. One developer can run GPT-5 with Claude Code while another runs Claude Sonnet 4.5 with Codex, all governed by one central policy engine.

What stops secrets from leaking into external APIs?

Bifrost inspects every prompt and response in-line before they cross the network boundary. Gitleaks-backed secrets detection and custom regex rules identify credentials and PII, redacting or rejecting them on the spot.

Is this only for large teams?

No. The open-source Bifrost gateway handles virtual keys, budgets, rate limits, and MCP tool filtering, which covers most production teams. Larger or compliance-sensitive organizations add RBAC, SSO, immutable audit logs, and advanced content safety via the enterprise edition.

Consolidating Control: The Case for a Unified Gateway

Governing Claude Code and Codex at enterprise scale means routing every call through a single policy layer, not managing a thousand individual developer configurations. Bifrost provides that layer as an open-source AI gateway with per-developer virtual keys, inline guardrails, persistent audit logs, and in-VPC deployment. Whether you're rolling out agents to a pilot team or scaling to your entire engineering organization, the same governance engine applies consistently.

Ready to govern your coding agent rollout? Check out the Bifrost resources to explore deployment patterns, or schedule a conversation with the Bifrost team to map a secure coding agent strategy for your organization.

The Top 5 AI Governance Platforms for Running LLMs in Production

Kuldeep Paul — Thu, 11 Jun 2026 20:45:49 +0000

Comparing governance solutions for multi-team, multi-provider AI deployments in 2026. Bifrost is the top choice for engineering teams needing production-grade controls with minimal latency overhead.

When you're running LLMs across multiple teams and multiple providers, governance is no longer optional. Raw provider keys scattered across services, unbounded token budgets, and no way to audit what data is being sent where; these problems compound fast. AI governance platforms solve this by putting policy enforcement directly into the request path, before any call reaches a provider. Bifrost, the open-source AI gateway written in Go by Maxim AI, is the strongest choice for platform teams that need to route, govern, and audit mission-critical AI traffic with best-in-class performance, scalability, and reliability. This breakdown covers the top 5 platforms in the governance space and where each fits into your infrastructure.

Why Governance Matters When You're Scaling AI

Governance is the set of controls that enforce who gets access to what, how much they spend, where their data flows, and what evidence gets logged for compliance. It's moved from "nice to have" to procurement requirement. Gartner forecasts the market will hit $492M this year and exceed $1B by 2030 as regulation spreads to 75% of the world's economies.

Two things are driving the adoption curve. Cost first: a misconfigured agent or a dev using a live key without rate limits can blow through a month's budget in hours. Regulation second: the EU AI Act, NIST AI RMF, and ISO/IEC 42001 now expect enforceable, auditable controls; not just policies on a wiki. Gartner's research shows teams using dedicated governance platforms are 3.4x more likely to actually succeed at governance than those cobbling it together.

The shift is clear: you can document governance all day, but a policy doc won't stop an agent from hitting an unapproved model or exceeding budget. Only a control in the request pipeline does.

What to Look For in a Governance Platform

Strong platforms for production AI share core capabilities. Evaluate on these dimensions:

In-request enforcement: actual controls that block/allow requests on the fly, not dashboards that report after the fact.
Granular access: per-user, per-team, per-project credentials tied to specific models, providers, and permissions.
Budget + rate limits: hierarchical spend caps and token throttling that fail safely when ceilings are hit.
Audit trails: immutable logs that hold up under SOC 2, GDPR, HIPAA, and ISO audits.
Deployment flexibility: in-VPC, on-prem, air-gapped modes so regulated data never touches public infrastructure.
Low overhead: minimal added latency, since every call goes through the governance layer.

Platforms that pack all of this into a single control plane reduce your operational surface area and make compliance easier to prove.

The 5 Top Options for AI Governance

1. Bifrost

Bifrost is an open-source AI gateway that sits in the request path and enforces access, budgets, and policy for 1,000+ models across 20+ providers through one OpenAI-compatible endpoint. Instead of bolting a monitoring tool onto your stack, you route every LLM call through Bifrost, which validates permissions and budgets before forwarding to a provider. The open-source gateway adds just 11 microseconds of latency per request at 5,000 RPS. Benchmarked in production, so governance doesn't come at a performance cost.

The core abstraction is the virtual key. Each key represents a consumer, a developer, a service, a team, or a customer, and encodes a specific policy: allowed providers, allowed models, spend limit, and request quota. Raw provider API keys stay locked in Bifrost; they never touch your services. This alone eliminates one of the biggest sources of key sprawl and cost leakage.

Spend control is hierarchical. Budgets and rate limits cascade through Customer → Team → Virtual Key → Provider Config layers. Every applicable budget must have balance for a request to pass. Once a limit is hit, requests fail gracefully with a policy error instead of continuing to charge. Budgets reset on calendar boundaries, daily, weekly, monthly, yearly, so your reconciliation matches your finance team's calendar.

For stricter environments, Bifrost Enterprise extends the open-source gateway. It adds role-based access control, OIDC/SSO integration with Okta and Entra, immutable audit logs for compliance frameworks, and content-safety guardrails. For regulated workloads, VPC-isolated deployment keeps all traffic inside private infrastructure with zero public egress. Governance extends to agentic use cases too: running as an MCP gateway, Bifrost applies the same access rules and policy enforcement to Model Context Protocol tool invocations.

Best for: Bifrost is built for enterprises running mission-critical AI workloads that require best-in-class performance, scalability, and reliability. It serves as a centralized AI gateway to route, govern, and secure all AI traffic across models and environments with ultra low latency. Bifrost unifies LLM gateway, MCP gateway, and Agents gateway capabilities into a single platform. Designed for regulated industries and strict enterprise requirements, it supports air-gapped deployments, VPC isolation, and on-prem infrastructure. It provides full control over data, access, and execution, along with robust security, policy enforcement, and governance capabilities.

2. Native Cloud Governance

The hyperscalers, AWS, Google Cloud, Azure, bundle governance with their managed AI services. You get identity integration, usage budgets, and content moderation for their models, all in your existing cloud console.

The catch: breadth. These suites work well for single-cloud, single-provider teams. The moment you add external providers or self-hosted models, you're managing governance across multiple dashboards with incompatible policy models and separate audit logs. Enterprises spanning multiple providers end up stitching things together manually.

Best for: single-cloud teams fully committed to one provider's model family who want governance bundled with their existing IAM and billing.

3. Risk and Compliance Platforms

A distinct category focuses on governance from the compliance angle: model inventories, risk registers, bias audits, and mapping controls to frameworks like NIST AI RMF and ISO/IEC 42001. These platforms produce the documentation and evidence that auditors and regulators expect.

But they don't enforce policy on live requests. They describe governance, they don't execute it. Most teams pair these with an infrastructure-layer gateway that does the actual enforcement.

Best for: compliance, legal, and risk teams needing lifecycle documentation, framework attestation, and audit evidence for a portfolio of AI systems.

4. Red-Teaming and Security Testing

Red-teaming platforms govern AI by discovering weaknesses through automated adversarial testing, jailbreaks, prompt injection, data extraction, unsafe completions. Baked into CI/CD, they shift adversarial testing left, making it a pre-deployment gate instead of a quarterly review.

They address a real threat class, but they govern model behavior under attack rather than day-to-day access and budgets. They complement, but don't replace, a gateway managing traffic on live requests.

Best for: security teams needing continuous adversarial testing of models and agents before each deployment.

5. Evaluation and Observability

These platforms govern through quality measurement: grading outputs, tracing production behavior, alerting on regression. Maxim AI, the company behind Bifrost, offers agent simulation and evaluation and production observability with distributed tracing and live alerts.

Quality scoring is essential, but it lives alongside, not inside, the access-and-budget enforcement layer. Teams typically run an evaluation platform for quality assurance and a gateway for runtime access control.

Best for: teams measuring agent quality, running pre-release evaluations, and observing production behavior across your AI stack's lifecycle.

How Bifrost Enforces Governance In-Path

Bifrost enforces governance where it matters: directly in the request flow. Every LLM call through Bifrost is validated against the policies attached to its virtual key before any data reaches a provider. Because it's a drop-in replacement for standard SDKs, you add governance by changing one line: the base URL.

The enforcement pipeline combines multiple checks:

Identification: the virtual key reveals which consumer this is and what permissions they have.
Model allowlist: checks if the requested model is permitted by the key's policy.
Budget validation: verifies every applicable budget (consumer, team, key, provider) still has balance.
Rate-limit validation: checks token and request limits at the key and provider levels.
Routing: excludes providers that hit their limits; directs traffic to a permissioned alternative.

The result: a runaway job can't silently overspend, a contractor's key can't reach production models, and a provider outage doesn't cascade. For deeper context on how these pieces fit, the governance resource hub and LLM Gateway Buyer's Guide provide capability matrices and architectural details.

Picking the Right Governance Platform

The choice depends on where your enforcement bottleneck is.

Multi-provider, mission-critical traffic: you need an infrastructure-layer gateway enforcing access and budgets on every request.
Compliance and audit evidence: a risk-and-compliance platform handles documentation and framework mapping.
Model security posture: a red-teaming tool shifts adversarial testing into your release pipeline.
Output quality and monitoring: an evaluation platform measures behavior and traces production.

Most mature deployments use more than one. But the foundation is always a control plane that enforces access, spend, and policy at request time. that's where regulators and finance teams focus. Bifrost provides that layer as an open-source gateway, and Bifrost Enterprise adds RBAC, SSO, audit logging, and VPC isolation for regulated environments, all while keeping the 11-microsecond overhead.

Getting Started with Runtime Governance

Governance works best when it's enforced in the request path, not bolted on afterward. Bifrost delivers that enforcement as an open-source AI gateway with virtual keys, hierarchical budgets, rate limits, audit logs, and private-cloud deployment in one platform, with just 11 microseconds of added latency at production scale. To see how Bifrost can centralize governance and access control across your infrastructure, book a demo with the Bifrost team.

How to Control MCP Server Access at Scale: Top 5 Gateway Tools

Kuldeep Paul — Thu, 11 Jun 2026 20:45:20 +0000

Choosing the right MCP gateway matters. Bifrost stands out as the best option for teams running production AI agents that need governance, security, and speed without compromise.

When you're connecting AI agents to multiple MCP servers, things get messy fast. Your first agent uses two servers. Your second adds three more. By the time you have five agents across your org, you're staring at dozens of connections, no visibility into which agent can call which tool, and token costs that grow with every new server you add. An MCP gateway sits in the middle, creating a single checkpoint where you control access, authenticate, audit, and manage costs across everything.

Bifrost is an open-source AI gateway built in Go that solves this by acting as both an MCP client and server at once. But it's not the only option out there. This post walks through five gateways you can use to govern MCP access in production, comparing them on ease of setup, governance depth, audit capabilities, and how they handle costs at scale.

What Does an MCP Gateway Actually Need to Do?

MCP server access governance means controlling who can do what with your tools, logging everything, and keeping costs predictable. Here's what a serious gateway needs to handle:

Tool-level access control: lock down which teams, customers, or services can touch which tools. When a key has no permission for a tool, it shouldn't see it at all (deny-by-default).
Centralized identity: manage credentials to your MCP servers in one place, not scattered across agent configs.
An audit trail: record every tool call with who made it, what happened, and why. Non-negotiable for compliance.
Cost boundaries: set budgets and rate limits so one runaway agent doesn't tank your bill.
Deployment options: the ability to run this in your own VPC, on-prem, or air-gapped if your industry or data residency requires it.

The MCP gateway resource goes deeper on what these controls look like in practice. The five tools below are compared against those criteria.

1. Bifrost: The All-in-One Approach

Bifrost combines MCP governance with LLM routing in one system. It routes requests to your models and governs access to your tools, so you're not stitching together separate infrastructure.

The architecture is straightforward: Bifrost sits between your agents and your MCP servers. It aggregates connections from multiple servers and exposes them through a single /mcp endpoint, with access enforced per request. At 5,000 requests per second, it adds only 11 microseconds of overhead, so governance doesn't become a bottleneck.

Governance happens through virtual keys. Each key is a bundle of permissions: which providers you can talk to, your budget, your rate limit, and which tools you can call. Crucially, the default is deny (a key with no MCP configuration sees zero tools until you explicitly add them).

Authentication to protected servers uses OAuth 2.0 with automatic token refresh and PKCE. On the cost side, Bifrost's Code Mode lets models orchestrate tools through code rather than loading massive tool definitions into the prompt, cutting tokens by up to 92% at scale.

For enterprise teams, Bifrost adds MCP tool groups so you define tool sets once and attach them to keys, teams, or customers. Federated auth turns your existing APIs into MCP tools from OpenAPI specs or Postman collections (no code required). Audit logs export to any SIEM for SOC 2, HIPAA, GDPR, and ISO 27001 audits. It runs anywhere: VPC, on-prem, air-gapped, with SSO via Okta or Entra.

2. Docker MCP Gateway: Container-Native Isolation

If your team lives in Docker, Docker MCP Gateway is worth a look. It runs each MCP server in its own container with signed images and built-in secrets handling.

The appeal is isolation at the container boundary. Each server runs independently, image signatures verify provenance, and secrets come through Docker's native systems, not hardcoded configs. For teams already shipping everything via containers, it's a natural fit.

The catch: Docker MCP Gateway is more of a toolkit than a turnkey solution. You get the building blocks (containers and signed images), but you layer on identity management, per-consumer access lists, and audit logging yourself. It also governs tools at the container level, not per-request, and doesn't handle model routing, so you'd run LLM traffic separately.

Best for: Container teams with Docker expertise who want full control over their setup and are willing to build governance layers on top.

3. Kong AI Gateway: For API-First Teams

Kong added MCP support through an AI MCP Proxy plugin. It translates between MCP and HTTP, letting MCP clients talk to REST APIs without rewriting them as MCP servers.

If you already use Kong for API management, extending it to MCP traffic keeps everything under one roof. Your existing rate-limiting, authentication, and observability policies apply to agent tool access too. It's familiar territory for platform teams that already standardize on Kong.

But MCP is bolted onto Kong rather than native to it. Tool-level access control, deny-by-default filtering, and token-optimization features like Code Mode aren't Kong's focus. LLM routing stays separate. For a deeper comparison, check the LLM gateway buyer's guide.

Best for: Teams with existing Kong deployments who want to manage agent tools through the same gateway and policies they use for REST APIs.

4. Azure API Management for MCP: Microsoft Ecosystem

Microsoft's approach: extend existing Azure services rather than shipping a new product. Azure API Management provides policy enforcement and OAuth 2.0, Entra ID handles identity and RBAC, and Azure Container Apps hosts MCP servers with Kubernetes-native scaling.

For Azure-centric orgs, it's appealing. You're using identity and policy infrastructure you already operate, reducing the number of new systems your platform team has to maintain and secure. Microsoft maintains an open-source MCP gateway for Kubernetes alongside the managed services.

The downside: Azure-first architecture locks you in. If you're multi-cloud, management complexity and vendor concerns become real. MCP capabilities are spread across multiple Azure services rather than unified in one layer, and portable deployments across AWS, GCP, and on-prem get harder.

Best for: Large enterprises committed to the Microsoft stack that want identity and policy controls integrated with Azure services.

5. Cloudflare AI Gateway and MCP Server Portals: Edge-First

Cloudflare's model: use Cloudflare AI Gateway, MCP Server Portals, and Cloudflare Gateway to create a security layer at the network edge. It includes shadow MCP detection to catch unauthorized server connections.

If you're already on Cloudflare's edge network and use Cloudflare One and Workers, this integrates naturally. MCP control happens at the network edge alongside your existing Zero Trust policies.

The dependency: you need Cloudflare infrastructure to make this work. If you're not on that platform, it's harder to justify. Model routing across LLM providers still happens separately from MCP access governance.

Best for: Orgs already invested in Cloudflare One and Workers that want edge-based MCP control with built-in shadow server detection.

Making the Choice

The right gateway depends on two things: do you need tool governance and model routing in one system (you probably do), and what infrastructure does your team already run?

Should governance and routing be unified?

In production? Almost always yes. Having tool access, model routing, audit logs, and identity in one control plane means simpler audits, smaller attack surface, and fewer integration points to debug. Bifrost is the only gateway here that unifies both by design.

What matters most for compliance?

Immutable audit logs, deny-by-default filtering, and the ability to run in isolated environments. Regulated industries (healthcare, finance, government) need proof of every tool call and the option to run in a VPC or offline. Bifrost's audit and access controls map directly to SOC 2, HIPAA, GDPR, and ISO 27001.

How do gateways cut token costs?

Most use budgets and rate limits. But there's a structural problem: if every request loads full tool definitions into the model context, costs climb with each new server. Bifrost's Code Mode flips this (the model writes orchestration code instead of carrying tool schemas, cutting tokens significantly).

Start Governing MCP Access

Controlling MCP server access at scale requires per-consumer tool filtering, centralized credentials, immutable audit trails, and cost controls that hold up as you add servers. Bifrost delivers all of these in one high-performance control plane, with deny-by-default access, tool groups for teams and customers, federated auth for enterprise APIs, and audit logs ready for SIEM export.

Ready to see it in action? Book a demo with the Bifrost team, or check out the Bifrost resources hub for step-by-step guides.

What Happens When MCP Server Access Goes Ungoverned: 5 Critical Security Risks

Kuldeep Paul — Thu, 11 Jun 2026 20:44:54 +0000

Without central governance, direct MCP connections expose agents to tool poisoning, prompt injection, privilege escalation, credential theft, and missing audit trails. Bifrost centralizes MCP access control, validation, and logging.

When an AI agent connects directly to MCP servers without a governance checkpoint, you've essentially handed the model keys to your infrastructure and hoped nothing goes wrong. Bifrost, a Go-based open-source MCP gateway from Maxim AI, intercepts every tool call with access controls, input/output validation, and audit logging. This post walks through the five most dangerous gaps in ungoverned MCP setups and how a centralized gateway fills each one.

Defining Ungoverned MCP Server Access

An ungoverned MCP server configuration is one where agents talk directly to external Model Context Protocol servers with no intermediary layer to check tools, filter what models can access, validate incoming and outgoing data, identify callers, or keep records. The Model Context Protocol, an open standard released by Anthropic in late 2024, lets AI models dynamically discover and invoke external tools across filesystems, databases, APIs, and internal services at runtime.

MCP's strength (instant tool availability) is also its vulnerability surface. When agents contact tool servers directly, three risk factors align dangerously: privileged system access, potentially hostile input (from documents, web pages, tool responses), and a communications channel to exfiltrate data. All five risks below follow from that convergence.

Tool poisoning via malicious metadata embedded in tool definitions
Indirect prompt injection through tool response content
Excessive privilege with no enforcement of least-privilege access
Secret exposure across prompts, arguments, and tool output
Missing forensics with no record of who ran what tool and when

Risk 1: Tool Poisoning in Tool Metadata and Descriptions

Tool poisoning occurs when a rogue or compromised MCP server stores malicious instructions inside tool properties like names, descriptions, or JSON schema. The model reads these properties as trusted input, visible to the LLM but invisible to the user watching the terminal. Invariant Labs first documented this attack as a subset of indirect prompt injection, and OWASP now classifies MCP tool poisoning as its own threat category, with a structural gap between initial connection and runtime behavior.

Ungoverned setups broadcast every available tool to the model with zero filtering, making it trivial for a poisoned definition to steer the agent toward data theft or unauthorized operations. Bifrost's tool filtering flips the default to deny: unless a virtual key explicitly permits access to a specific client and its tools, nothing is available. Instead of exposing your entire tool library, teams whitelist only what each agent legitimately needs, which curtails how far one poisoned tool can propagate. Bifrost also consolidates your tool registry in one place, letting your security team review a single governed surface instead of hunting through every agent's independent MCP connections.

Risk 2: Indirect Prompt Injection Via Tool Responses

Indirect prompt injection is when hostile instructions slip into content an agent processes: a malicious support ticket, a booby-trapped web page, or the result returned from a tool. Because the model treats all context equally, it can't separate a real user instruction from a planted one. Microsoft's security researchers detailed this attack pattern for MCP, and known incidents follow the same playbook: in mid-2025, a Supabase agent with database-level privileges got tricked by injected SQL in user-submitted tickets, leaking OAuth integration tokens to a public forum.

The solution requires a layer that watches every prompt and response in flight, not distributed filters inside each application. Bifrost enforces guardrails at the gateway, scanning both LLM inputs and MCP tool inputs/outputs in real time. The same safety rules that screen user prompts also inspect what gets passed to and returned from your tools, with options for prompt-injection defense, PII masking, and content filtering via native or third-party providers. If a rule is violated, Bifrost can stop the request, erase sensitive fields, or write it to an audit log, giving you one control point across your entire agent fleet rather than fractured per-service policies that drift over months.

Risk 3: Privilege Creep: When Every Agent Can Call Every Tool

Privilege creep is the default misconfiguration: agents get broad API credentials and the full tool menu because building tight per-consumer scoping without a control plane is tedious. The outcome: your internal prototype and your customer-facing agent invoke the same destructive operations. Bifrost inverts this by making least privilege the starting point.

Virtual keys are Bifrost's primary governance unit; each holds its own allow-list of which MCP clients and tools a consumer can access, plus model and cost limits. Bifrost enforces the allow-list twice: once when planning which tools the model sees, and again when the model actually tries to invoke a tool, so a key meant for read-only diagnostics can't silently pivot to writing data. For larger deployments, governance extends to RBAC and tool groups, named collections of tools you attach to keys, teams, or users and resolve per request. Every model call only ever touches the tools its consumer is authorized for.

Risk 4: Secrets in Flight: When Credentials Leak Into Logs and Responses

Secrets leak when API keys, OAuth tokens, or passwords flow through prompts, tool parameters, or results and end up in logs, vector databases, or attackers' hands. Ungoverned MCP deployments make this worse in two ways: you tend to embed provider credentials in every agent separately, and there's zero inspection as a secret transits a tool invocation.

Bifrost solves both. Provider credentials stay encrypted inside the gateway and never ship to client code; they rotate independently of the virtual keys that reference them, so secrets never live in app config or .env files. On detection, Bifrost's secrets guardrail, powered by Gitleaks under the hood, catches leaked keys, tokens, and certificates in prompts and completions before they escape. For teams in regulated industries, in-VPC and self-hosted options keep request payloads, detection events, and all credentials within your network perimeter, and vault integrations (HashiCorp Vault, AWS Secrets Manager, Google Secret Manager, Azure Key Vault) handle rotation and access.

Risk 5: No Audit Trail: When You Can't Answer "What Happened?"

Ungoverned MCP setups scatter tool-call evidence across application logs, if it's captured at all, which makes incident postmortems and compliance audits painful. The MCP spec itself says best practice is a human in the loop with ability to veto tool invocations, and that's only credible if every action is recorded.

Once Bifrost becomes the single checkpoint between agents and tools, each tool call becomes a first-class log entry with full metadata: tool name, source MCP server, arguments sent, response received, latency, the virtual key that triggered it, and the originating LLM request. You can slice by virtual key to see what a specific consumer executed, or by tool to track which server gets hammered. For compliance, immutable audit logs meet SOC 2, GDPR, HIPAA, and ISO 27001 standards and can ship to external SIEM and data-lake systems for archival and investigation.

How Bifrost Governs MCP From Request to Tool Return

Bifrost sits as both an MCP client (connecting to your external tool servers) and an MCP server (exposing one controlled endpoint to agents and desktop clients like Claude Desktop). Routing every tool call through a single control plane is what makes all five protections consistent rather than ad hoc per-application. By default, Bifrost blocks auto-execution of tool calls; the model returns suggestions, and your code must explicitly approve and invoke them, preserving human judgment for risky operations. For teams that want automation, Agent Mode lets you set an auto-approval list while keeping the boundary explicit.

Bifrost also tackles cost and context bloat. When agents connect to many servers, Code Mode has the model generate compact Python code to orchestrate tools in a sandbox instead of stuffing every tool definition into the prompt, cutting input tokens by up to 92.8% and shrinking the payload that poisoned definitions can corrupt. The overhead is tiny: 11 microseconds per request at 5,000 RPS, so governance never becomes your latency bottleneck. The cost-and-access deep dive shows how filtering, logging, and Code Mode combine at production scale.

Common Questions About MCP Security

What's the #1 MCP attack vector?

Prompt injection (including tool poisoning) tops the list. It's scary because it combines three factors: an attacker controls tool metadata, the model reads it as gospel truth, and the agent has real system access. This pattern shows up in almost every documented MCP incident.

Does a gateway actually stop prompt injection?

A gateway alone doesn't make your model immune, but it adds layers that direct connections lack: request and response guardrails on every tool call, deny-by-default filtering, and tamper-evident logs. Together they shrink both the odds of a successful attack and how much damage one can do.

How do you actually implement least privilege for MCP agents?

Give each agent its own credentials and tool permission set instead of sharing universal access. Bifrost virtual keys apply a per-consumer allow-list to both what the model sees and what it can actually invoke, enforced at planning time and execution time so privilege doesn't escalate mid-call.

Starting With Bifrost

Ungoverned MCP server setups concentrate privileged tool access, untrusted input vectors, and exfiltration paths into one blind spot. A governed MCP gateway decouples those concerns: deny-by-default tool filtering, runtime guardrails on prompts and responses, credential isolation, and immutable logs, all running between your agents and external tools. Bifrost provides this control plane as an open-source AI gateway with enterprise governance, and the full feature set is documented in the resources hub. Ready to see how Bifrost locks down MCP access across your agent infrastructure? Book a demo with the team.

Running Bifrost in Cluster Mode: The Path to Enterprise AI Deployments

Kuldeep Paul — Thu, 11 Jun 2026 20:44:10 +0000

Bifrost cluster mode distributes gateway traffic across multiple nodes with synchronized state, automatic node failover, and zero-downtime rolling updates.

If you're running an AI gateway in production, you've probably wondered what happens when a single instance fails. Cluster mode is Bifrost's answer: multiple gateway nodes that act as a single system, sharing rate limits, budgets, and governance rules in real time. Bifrost, the open-source AI gateway built in Go by Maxim AI, supports cluster mode to give teams the high-availability infrastructure that production AI workloads demand. Let's walk through what it is, why you'd use it, how it actually works under the hood, and how to deploy it on Kubernetes with Helm.

Understanding Cluster Mode

Cluster mode runs multiple gateway instances as an interconnected system. In the default setup (cluster mode disabled), each instance is isolated. It talks to a shared database, but handles its own rate limits and governance state. With cluster mode on, every node synchronizes those counters and policies with every other node, creating a single logical gateway spread across many machines.

Here's the practical difference: without clustering, each node enforces a rate limit independently. So if you set a limit of 1,000 requests per minute, each node gets its own 1,000 rps quota. With clustering, the fleet shares a single 1,000 rps quota. One node uses 600, another uses 300, and the limit holds cluster-wide.

The tech here is production-grade. Gossip-based state synchronization makes sure every node converges on the same view of the cluster in seconds. Cluster mode requires PostgreSQL, because SQLite doesn't support multi-node coordination. The feature set is part of Bifrost Enterprise, though the open-source image accepts cluster config values.

Why Teams Deploy Clustering

Single nodes fail. Cloud environments are noisy. Rate limits need to be accurate across the whole system. That's why enterprises move to clustered gateways.

Here's what clustering solves:

No single point of failure: When a node goes down (hardware fault, pod eviction, zone disruption), traffic automatically shifts to healthy instances. The gateway keeps serving requests.
Shared rate limits and budgets: A 1,000 requests-per-minute limit holds across the whole fleet, not per node. Same for budget tracking. Budget counters stay consistent everywhere.
Unified governance: Virtual keys, routing rules, providers, and access policies sync to every node instantly. Configuration drift disappears.
Handle spikes gracefully: Horizontal scaling adds replicas, distributing load instead of crushing a single instance.
Zero-downtime updates: Roll out new code one pod at a time while the rest keep serving traffic.
Multi-region awareness: Tag nodes with region labels so requests route to the closest instance.

For teams building capacity plans and high-availability strategies, the enterprise deployment guide covers cluster sizing and HA topologies in detail.

How Clustering Actually Works

Bifrost cluster mode is fully peer-to-peer. There's no primary node, no external coordinator, no special "cluster leader node" in the request path. Every node is equal. Each one discovers peers automatically, monitors whether every other node is alive, and gets state updates as they happen.

Clustering uses two separate communication channels with different jobs. Membership signals (which nodes are in the cluster, are they alive?) run over a memberlist gossip layer on port 10101 (TCP and UDP). This uses the SWIM protocol, which is designed for scalable, eventually consistent group membership. Everything else, including usage counters, config changes, routing rules, virtual keys, RBAC, and MCP tool definitions, travels over a dedicated gRPC channel on port 10102 (TCP). Splitting these two keeps the membership system isolated from application traffic, and each can be tuned independently.

Bifrost replicates 30+ entity types across the cluster: model catalog, providers, governance counters, routing rules, RBAC, MCP tools, pricing, and prompt deployments. All of it gets synced. Each message has a unique ID and timestamp. Nodes keep a short-lived deduplication cache, so if a message arrives twice, it's only processed once. All nodes converge to the same state within seconds (eventual consistency).

Leader election is automatic. One cluster-wide leader, one leader per region. The rule is simple: lexicographically first healthy member wins. The cluster re-evaluates every 30 seconds, so leadership transfers automatically when nodes join, leave, or fail. The leader handles singleton tasks (like fetching upstream pricing once instead of every node doing it), then broadcasts the result to the rest.

Deploying on Kubernetes with Helm

The easiest deployment path is Kubernetes + Helm with native pod discovery. The default mesh clustering model means nodes reach each other directly over gossip and gRPC. Minimum recommendation: three nodes (survives one failure), five nodes (survives two failures), seven or more for even higher redundancy.

A minimal cluster Helm config looks like:

# cluster-values.yaml
replicaCount: 3

storage:
  mode: postgres

postgresql:
  external:
    enabled: true
    host: "your-postgres-host.example.com"
    port: 5432
    user: bifrost
    database: bifrost
    sslMode: require
    existingSecret: "postgres-credentials"
    passwordKey: "password"

bifrost:
  encryptionKeySecret:
    name: "bifrost-encryption"
    key: "encryption-key"
  cluster:
    enabled: true
    region: "us-east-1"
    discovery:
      enabled: true
      type: kubernetes
      k8sNamespace: "default"
      k8sLabelSelector: "app.kubernetes.io/name=bifrost"
    gossip:
      port: 10101

Kubernetes discovery works by querying the API for pods matching your label selector. The gossip layer finds peers that way. You'll need service account RBAC to list pods. After creating your PostgreSQL and encryption secrets and configuring RBAC, just run:

helm install bifrost bifrost/bifrost -f cluster-values.yaml

Both ports 10101 (TCP+UDP) and 10102 (TCP) need to be open between pods, so check your NetworkPolicies and security groups.

Other discovery methods

Bifrost supports six ways to discover cluster members, so it works anywhere:

Kubernetes: Pod discovery via the API, best for cloud-native setups.
DNS: Headless services or A-record resolution, great with StatefulSets.
Consul and etcd: For teams already running those systems.
UDP broadcast and mDNS: Local-network discovery for on-prem and dev.

On serverless platforms without peer-to-peer networking (like Google Cloud Run), Bifrost offers broker mode. Each node makes one outbound connection to a central relay that fans out messages to the rest. Same reliability, different topology. See the in-VPC deployment docs for air-gapped and private-cloud setups.

Production HA Patterns

A solid Bifrost cluster combines the multi-node foundation with scheduling, autoscaling, and shutdown policies that keep things running through real-world disruptions. Goal: a fleet that survives node faults, scales for demand, and deploys new versions without dropping requests.

Spread replicas across hosts: Use pod anti-affinity so instances don't land on the same node. One machine can't take down the whole gateway.
Guard availability during maintenance: Set a PodDisruptionBudget with minAvailable: 2. Kubernetes won't let voluntary disruptions (drains, autoscaling) drop you below that threshold.
Graceful shutdown: Termination grace period + preStop hook ensures streaming responses finish cleanly instead of getting cut off.
Smart autoscaling: Enable HPA with conservative scale-down. Let the cluster grow for spikes, but don't aggressively kill pods that are still busy.
Layer in provider resilience: Mix cluster-level HA with automatic provider failover and adaptive load balancing. Survive node failures AND provider outages.
Monitor health: Export Prometheus metrics. Use the built-in cluster topology view to confirm every node is alive and receiving messages.

Rolling upgrades work because nodes negotiate capabilities with each other. New and old versions can run side by side without losing quorum. Deploy one pod at a time using standard Kubernetes rolling strategies.

Common Questions

Does cluster mode need a specific database?

Yes, PostgreSQL. Bifrost cluster mode coordinates shared state through the database, so it's a hard requirement. SQLite only works for single instances.

How many nodes should I run?

Three is the minimum and handles one failure. Five tolerates two. Seven+ gives you three fault tolerance. Pick based on how much disruption you can absorb.

Can I use cluster mode on serverless?

Yes, with broker mode. Instead of direct peer-to-peer connections, every node connects outbound to a central relay. Works on Cloud Run and similar platforms.

Do cluster upgrades require downtime?

Nope. Nodes support mixed versions during rolling upgrades. Replace one pod at a time, keep the rest serving traffic.

Getting Started

Bifrost cluster mode is the deployment pattern for teams that need a highly available AI gateway. It delivers gossip-based state sync, automatic failover, shared rate limits, and zero-downtime deploys across a peer-to-peer cluster. The enterprise deployment guide and the full Bifrost resources hub have reference topologies, sizing guidance, and HA walkthroughs.

Ready to see how Bifrost cluster mode fits your production setup? Book a demo with the Bifrost team.

Stop Runaway LLM Bills: Cost Overrun Prevention with Bifrost

Kuldeep Paul — Thu, 11 Jun 2026 20:42:59 +0000

Bifrost tackles LLM cost overruns by enforcing budgets at the gateway, deduplicating expensive requests, and cutting token consumption before charges hit.

Token bills blow up in production when spending spins beyond what teams predicted during development. It only takes an exposed API key, a recursive loop that hammers multiple providers, or an agent reloading tool schemas on every single call to turn a tidy monthly bill into a nightmare. According to Gartner's latest forecast, $2.59 trillion in worldwide AI spending is coming in 2026, a 47% jump year-over-year, and much of that flows through production LLM systems where few teams have real spend controls in place. Bifrost, the open-source AI gateway written in Go by Maxim AI, puts cost limits front-and-center by catching budget overages, eliminating duplicate work, and trimming token usage at the request level. Here's how to prevent cost overruns at the infrastructure layer.

Understanding What Drives Production Cost Overruns

When production LLM costs spike unexpectedly, it's not random. Unplanned token spend happens when live traffic patterns clash with the budget numbers a team built into prototypes. The culprits are usually the same few things: unrestricted keys floating around services with no per-team limits, the same query hitting the provider multiple times instead of using a cached result, giant context windows stuffed with unused tool listings, and spending dashboards that only tell you about the problem after it's already expensive.

The top cost-overrun sources in production:

Unguarded API keys spread across the stack , raw provider credentials without per-consumer spend caps let any service or customer rack up charges unchecked.
Repeated equivalent queries , identical or nearly-identical requests that should return a stored answer instead hit the provider every time, wasting spend on work the system already solved.
Oversized contexts , agents dumping complete tool catalogs or full conversation histories into the prompt, padding input token costs on each request.
Reactive cost monitoring , alerting dashboards that notify after overspending happened rather than blocking the call that triggered it.

Bifrost handles all four pain points from one place. Since it's a drop-in replacement that sits between services and providers, enforcement happens on every request without touching application code.

Why the Gateway Is the Best Place to Control LLM Costs

Cost controls in individual services don't scale. Application teams each set different limits, spend accounting stays fragmented, and you end up with no unified spending ceiling anywhere. A gateway layer sees everything: it's where all requests converge before they hit a provider and incur charges. That's the chokepoint for cost governance.

Bifrost manages traffic to 1000+ models via one OpenAI-compatible endpoint and layers in just 11 microseconds of latency overhead per call at 5,000 RPS in real-world benchmarks. Enforcement can't become a performance hit or teams will bypass it. The gateway already intercepts every request, so adding budget checks, response caching, and token reduction there is cheap in CPU and fast. The LLM Gateway Buyer's Guide helps teams weigh performance, feature depth, and cost-control options across platforms.

Setting Hard Spend Limits Before Money Gets Spent

The critical difference: blocking a request that would bust your budget, not alerting you after the damage is done. Gateway-level budget and rate limits prevent overspending by rejecting requests that exceed your ceilings, not by warning you afterward.

The key governance tool is the virtual key. Instead of handing raw provider keys to every service, Bifrost issues virtual keys tied to a specific budget, rate limit, model whitelist, and routing policy. Raw provider credentials never leak across your infrastructure, which closes off one of the biggest cost-leak vectors.

Bifrost's budget hierarchy is nested:

Customer-level , isolate spending by account (critical for multi-tenant SaaS).
Team-level , set spending ceilings per department or product area.
Virtual-key-level , cap spend by specific service or application.
Provider-level , budget limits per model provider inside a single key.

A request has to pass every applicable budget check before it goes through. When any budget is exhausted, Bifrost sends a 402 response and stops LLM traffic for that key until the budget window resets; the key stays active for non-LLM operations. Token and request rate limits trigger a 429 when thresholds are crossed within a configured period. Reset windows can be rolling or calendar-aligned (monthly resets hit the 1st, not a rolling 30-day window).

For SaaS, per-customer keys mean one customer can't drive up costs for everyone else. Budget isolation is built in. This governance and access control approach was designed for enterprise teams and regulated industries where uncontrolled spend isn't just a finance problem, it's a compliance violation.

Cutting Costs by Caching Semantically Similar Queries

A chunk of production traffic is functionally redundant: support chatbots answer the same questions, search-powered pipelines re-embed identical queries, automation tools replay similar instructions. Every time a repeat request hits the provider, you pay for work the system already completed.

Bifrost includes semantic caching as a native feature with a two-part strategy. Exact hash matches get served instantly, and prompts that don't match exactly still hit the cache through vector similarity with a tunable threshold (default: 0.8). Cached hits land in roughly 5ms, call it near-instant vs. the seconds a live provider call takes, so you save time and tokens on the same request.

Bifrost's semantic cache includes:

Dual-layer architecture , combine exact-match speed with fuzzy vector similarity to catch near-duplicates.
Adjustable similarity threshold , decide how close a match has to be; different rules for different endpoints.
Multiple vector backends , plug in Weaviate, Redis/Valkey, Qdrant, or Pinecone.
Streaming-compatible , cached responses stream back with chunks in the right order.
Per-request toggles , endpoints that need fresh responses can skip the cache.

The cache lives in the gateway, so every service behind Bifrost gets caching without separate integration per service. Governance can tune when caching helps without sacrificing freshness where it matters.

Code Mode: Slashing Token Costs in Multi-Tool Agents

Multi-tool agent workflows are a special cost problem: every single request includes the full schema catalog for every available tool. Connect an agent to five MCP servers with 20 tools each, and the LLM gets 100 tool definitions before it even sees the user's question. The model burns tokens parsing schemas instead of working, and this repeats every call.

Bifrost solves this with Code Mode, part of the MCP gateway. Code Mode flips the script: instead of listing all tools directly, it hands the model four meta-tools that orchestrate everything else. The LLM writes Python executing in an isolated sandbox, and only the final result bubbles back up to the context. Intermediate tool outputs stay sandboxed rather than cycling through the model's context.

The savings compound as you add tools. Bifrost's own testing shows Code Mode slashes input token usage to as low as 92.8%, cuts costs by ~92.2%, and executes ~40% quicker in deployments with many tools, with input-token gains ranging from 58.2% to 92.8% depending on tool count. The reason: classic MCP spend grows with each tool, but Code Mode is bounded by what the model reads. Coding agents and CLI tools running through the gateway get the same spend controls and multi-provider governance. The detailed technical breakdown on token costs and access control walks through multi-server setups.

Making Spend Attribution Clear and Actionable

Controlling costs starts with knowing where it's coming from. Figuring out which team, application, and model is eating your budget requires attribution: tags for users, projects, and API keys. Without that traceability, a budget is just a number, not a control.

Bifrost includes native request monitoring and built-in observability (Prometheus, OpenTelemetry/OTLP, compatible with Grafana, New Relic, Honeycomb). Request activity ties back to the virtual key, team, and customer responsible for it, giving finance and infrastructure teams spend visibility at the same detail they need to set budgets. This cost governance setup is what makes per-consumer budgets actually enforceable. Organizations operating under compliance rules get more: Bifrost Enterprise layers on audit logging, role-based access, and private-network deployment, extending cost governance into restricted and regulated settings.

Enforcement and monitoring: what's the difference?

Monitoring tells you after overspend happens. Enforcement stops the request that causes it. Bifrost does both: budget and rate limits block in real time, while Prometheus/OpenTelemetry telemetry show where spend concentrates so you can tune future limits.

Will adding cost controls add latency?

No. Bifrost runs 11 microseconds overhead per request at 5,000 calls per second, so checks, deduplication, and token reduction layer in without becoming a performance tax teams work around.

How to get started?

First step: replace raw provider keys with virtual keys enforcing budgets and rate limits. Next, enable semantic caching on high-traffic endpoints. Last, flip on Code Mode for any agent talking to three or more tool servers.

Getting Spend Under Control

Controlling production LLM spend boils down to three things: enforcing limits, ditching duplicate work, and cutting token bloat at the infrastructure layer, not scattered through every app. Bifrost combines real-time budget walls, smart dual-layer caching, and Code Mode token cuts in a single open-source gateway that's a drop-in swap for current SDKs. Every control lives at the gateway, so you get multi-layer cost governance, caching, and observability without rolling custom metering. Ready to lock down production spending? Book a demo with the Bifrost team to see it in action.

Running a High-Performance AI Gateway on Kubernetes

Kuldeep Paul — Thu, 11 Jun 2026 20:38:16 +0000

Bifrost, the open-source AI gateway, handles thousands of concurrent LLM requests on Kubernetes with near-zero overhead, autoscaling, and centralized governance, everything you need for enterprise-grade production traffic.

When AI requests arrive at scale (hundreds or thousands per second), even milliseconds of added latency compound into user-visible slowdowns and unnecessary token costs. A high-performance AI gateway on Kubernetes lets you absorb that load with a declarative, horizontally scalable deployment while maintaining full control over data, policy, and request routing. Bifrost, an open-source AI gateway written in Go, is purpose-built for enterprise teams handling mission-critical AI workloads at high concurrency. This guide covers deploying Bifrost on Kubernetes at production scale, from initial Helm installation through multi-replica cluster mode, autoscaling, and enterprise-grade governance.

Core Requirements for a Production AI Gateway on Kubernetes

More than just a proxy is needed to handle enterprise AI traffic. A gateway that can sustain thousands of concurrent requests requires:

Horizontal scaling: pods that scale in and out automatically, driven by CPU and memory metrics.
Consistent state across all replicas: shared rate limits, budgets, and policy counters that don't drift as the cluster scales.
Clean shutdown under load: in-flight streams (especially SSE responses) that finish gracefully during pod termination or node drain.
Minimal request overhead: every microsecond of latency matters at this scale.
Observability integrated from the start: metrics, distributed traces, and readiness checks baked into the workload.

Bifrost ships as a first-class Kubernetes resource. The official Helm chart maps all configuration values directly to the runtime, so your cluster always matches what's in your values file. No configuration drift, no surprises.

Concurrency Architecture: Why Implementation Matters at Scale

Below 100 requests per second, gateway overhead is imperceptible. At 1,000 RPS and beyond, the architecture of the gateway itself decides whether service quality holds steady or collapses.

Bifrost is compiled to a single Go binary with goroutines handling concurrent work. This contrasts with Python-based proxies, which face the Global Interpreter Lock and asyncio overhead, both of which constrain parallelism. Internally, Bifrost uses a worker-pool concurrency model: requests are distributed to workers in a round-robin pattern, queue buffers are sized for traffic bursts, and when the system saturates, backpressure policies either queue excess work or drop it cleanly.

Performance at high concurrency is measurable. When stress-tested at 5,000 requests per second, Bifrost adds only 11 microseconds of overhead per request. Comparative benchmark data shows 54 times lower P99 latency and roughly 68% lower memory consumption versus a Python gateway under identical load. At enterprise scales handling sustained high-concurrency traffic, this gap between implementations is what separates predictable tail latency from service degradation.

Deploying Bifrost on Kubernetes: The Helm Path

The quickest way to get a gateway running is the official Helm chart. First, register the repository, then provision an encryption key and deploy:

helm repo add bifrost https://maximhq.github.io/bifrost/helm-charts
helm repo update

kubectl create secret generic bifrost-encryption-key \
  --from-literal=encryption-key="$(openssl rand -base64 32)"

helm install bifrost bifrost/bifrost \
  --set image.tag=v1.4.11 \
  --set bifrost.encryptionKeySecret.name="bifrost-encryption-key" \
  --set bifrost.encryptionKeySecret.key="encryption-key"

In production deployments, the Bifrost gateway relies on PostgreSQL for the backing store instead of SQLite, and runs three or more replicas for high availability. The switch to Postgres is what enables state sharing across pods. Within the chart, the Helm deployment guide exposes a client-facing config section that directly controls concurrency:

bifrost:
  client:
    initialPoolSize: 1000        # preallocate this many request workers
    dropExcessRequests: true     # shed overload instead of buffering infinitely
    enableLogging: true
    enforceGovernanceHeader: true

A high initialPoolSize pre-reserves worker capacity to handle expected load spikes. Setting dropExcessRequests to true means the gateway will reject requests gracefully when overwhelmed, rather than letting request queues grow unbounded. Both settings are critical to keeping a high-concurrency AI gateway predictable at the traffic ceiling.

Multi-Replica Deployments: Sharing State via Cluster Mode

Just running multiple pod replicas is not enough. If each pod enforces rate limits independently, you end up with the limit multiplied across replicas. That's where cluster mode comes in: it synchronizes in-memory state (rate limit counters, budget spent, policy rules) across all pods using a gossip protocol.

On Kubernetes, the recommended approach queries the API server to discover peer pods by label, so new replicas are auto-discovered without manual peer lists:

bifrost:
  cluster:
    enabled: true
    discovery:
      enabled: true
      type: kubernetes
      k8sNamespace: "default"
      k8sLabelSelector: "app.kubernetes.io/name=bifrost"
    gossip:
      port: 7946

The pod's service account needs read permissions on pods in that namespace, set up via Role and RoleBinding. Other discovery options (DNS, static peer lists, Consul, etcd) work too for environments where Kubernetes API access isn't available. More advanced HA patterns, including region-aware routing and broker mode for Cloud Run, are covered in the full clustering guide. Note: cluster mode is an enterprise feature and requires PostgreSQL.

Intelligent Scaling: Autoscaling Without Dropping Requests

A gateway must scale out when load spikes and scale back in afterward, all without terminating active requests. The Bifrost Helm chart wires three pieces together: the Horizontal Pod Autoscaler, pod anti-affinity rules, and graceful termination:

replicaCount: 3

autoscaling:
  enabled: true
  minReplicas: 3
  maxReplicas: 15
  targetCPUUtilizationPercentage: 70
  targetMemoryUtilizationPercentage: 75
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300   # wait before shrinking
      policies:
        - type: Pods
          value: 1
          periodSeconds: 120

terminationGracePeriodSeconds: 90       # allow streams to finish
lifecycle:
  preStop:
    exec:
      command: ["sh", "-c", "sleep 20"]

affinity:
  podAntiAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchLabels:
            app.kubernetes.io/name: bifrost
        topologyKey: kubernetes.io/hostname

The scale-down window prevents unnecessary churn during brief traffic dips. The extended grace period and preStop hook give streaming responses time to finish before a pod is removed. Spread replicas across different nodes with pod anti-affinity to ensure a single node failure doesn't bring the gateway offline. Combined with provider failover, this setup keeps the gateway online through infrastructure events and upstream provider issues alike.

Governance, Observability, Compliance at Enterprise Scale

High throughput alone means little without governance, visibility, and compliance. Bifrost centralizes all three.

Governance. Virtual keys are your primary control lever: each carries access permissions, spending limits, and request rate caps. Turning on is_vk_mandatory forces every request through a governed key. Budgets and rate limits can be set at the key, team, or customer level, and in cluster mode those counters stay synchronized across the entire replica set. For teams building fine-grained control at scale, the governance resource hub lays out the full model.

Observability. Bifrost exposes Prometheus metrics at /metrics and ships a ServiceMonitor for automatic scraping. It also supports OpenTelemetry for end-to-end distributed tracing. Health probes hook directly into Kubernetes liveness and readiness checks. Worker and queue metrics feed capacity planning decisions.

Compliance and security. Bifrost Enterprise supplies guardrails for request filtering and secrets detection, plus RBAC for access control. Audit logs are immutable and support SOC 2, GDPR, HIPAA, and ISO 27001. Strict data residency is possible through in-VPC deployment.

Combining throughput with policy and compliance is the hallmark of a gateway that works in production. The same benchmark data that informs scaling decisions also guides replica sizing and resource requests for your traffic profile.

Next Steps: Running Bifrost in Your Cluster

Deploying a high-performance AI gateway on Kubernetes distills to: Helm-based declarative deployment, PostgreSQL cluster mode for shared state, autoscaling tuned for graceful shutdown, and built-in governance plus observability. Bifrost packages these together as a single Kubernetes workload designed for high-concurrency production AI traffic, with a nearly transparent overhead profile under sustained load.

Ready to see Bifrost handling your enterprise AI workloads? Book a demo with the team.

Connect Claude Code to Groq With Bifrost

Kuldeep Paul — Thu, 11 Jun 2026 20:37:44 +0000

Bifrost is an open-source LLM gateway that redirects Claude Code to any inference provider, including Groq, without modifying Claude Code itself. This walkthrough covers the entire configuration.

By default, Claude Code connects exclusively to Anthropic's API and runs only Claude models. For individual developers, this constraint is fine. Teams seeking fast inference on open-source models for cost-sensitive applications, or those wanting to direct specific workloads to specialized hardware, find the default limiting. Bifrost, the open-source AI gateway written in Go by Maxim AI, stands between Claude Code and any LLM provider, translating at the protocol level. By configuring Claude Code to use Bifrost's Anthropic-compatible endpoint, you can direct all inference to Groq or any other supported provider without modifying Claude Code's binary or internals.

The Case for Groq

Groq's inference service relies on Language Processing Units (LPUs), custom silicon optimized for neural network computation. This LPU-based approach guarantees consistent latency: each token generation takes the same time, removing the variance present on traditional GPU platforms. Independent performance tests show Groq delivering 4-7x greater token throughput than leading GPU systems, with initial-token latency 3-4x lower at equivalent scale.

For Claude Code users, this speed advantage applies in specific contexts:

Experimental and exploratory work using openly-licensed models (Llama 4 Scout, llama-3.3-70b-versatile, DeepSeek R1 Distill) for faster prototyping at lower per-token cost than paid frontier models
Chained multi-step reasoning where each tool invocation requires a new LLM call; per-call speed improvements compound across the chain
Selective model usage where routine tasks run on quick, low-cost open models, while complex reasoning stays on Anthropic's paid tiers

An important limitation: Groq's API lacks support for image input, vector embeddings, voice synthesis, or transcription. It processes text-based chat and function calling exclusively. Features like Claude's computer_use tool are unsupported. Deploy Groq for text and function-call work; keep Anthropic available for multimodal or proprietary-feature requirements.

Bifrost's Role in Connecting Claude Code to Groq

Bifrost presents an Anthropic-compatible interface at the /anthropic path. When Claude Code points here instead of Anthropic's servers, Bifrost acts as the intermediary. It accepts Claude Code's request in Anthropic format, routes it to whichever provider you've configured, and returns the response in the shape Claude Code expects.

Since Groq implements OpenAI's API, Bifrost maps through its OpenAI layer with Groq-specific tuning: fields like store, service_tier, and prompt_cache_key are ignored, and server-sent events use Groq's schema. Full support for function calling is included. The Bifrost provider configuration layer and the request routing layer are orthogonal; you configure Groq independently from configuring which Claude Code requests hit Groq.

Step 1: Launch Bifrost

The simplest path is via npx with Node.js 18+.

npx -y @maximhq/bifrost -app-dir ./bifrost-data

The gateway starts at http://localhost:8080 and initializes a bifrost-data directory for configs and data. Visit http://localhost:8080 in a browser to reach the control panel.

Docker is also an option:

docker run -p 8080:8080 maximhq/bifrost:latest

Step 2: Add Groq as a Provider

The Bifrost control panel and config.json both work for provider setup. In the dashboard, go to Models > Model Providers, locate Groq, add it if missing, and enter your Groq API key (raw or as env.GROQ_API_KEY). Under Allowed Models, choose All Models or restrict to a list.

For config.json in your bifrost-data folder:

{
  "providers": {
    "groq": {
      "keys": [
        {
          "name": "groq-key-1",
          "value": "env.GROQ_API_KEY",
          "models": ["*"],
          "weight": 1.0
        }
      ]
    }
  }
}

Before starting Bifrost, export your Groq key:

export GROQ_API_KEY="your-groq-api-key"

Verify activation in the dashboard: Models > Model Providers > Groq will list the available Groq models.

Step 3: Generate a Virtual Key

Virtual keys let Claude Code authenticate to Bifrost without exposing your real provider keys. Each key can carry routing policies, spending caps, and rate-limit rules.

In the dashboard, go to Governance > Virtual Keys, create a new entry, name it, select Groq as an allowed provider, and copy the generated value. Pass this to Claude Code in the configuration step.

Step 4: Connect Claude Code to Bifrost

Now that Bifrost is running with Groq configured, point Claude Code at Bifrost. Two options: directly edit settings.json, or run Bifrost CLI, a guided terminal tool that sets up Claude Code without requiring you to manage environment variables or files.

Option A: Direct settings.json Edit

Claude Code loads environment settings from settings.json. Location varies by OS and scope:

macOS / Linux / WSL system-wide: ~/.claude/settings.json
Windows system-wide: %USERPROFILE%\.claude\settings.json
Project-level: .claude/settings.json in the project folder

Merge the following into your settings.json's env block (shown here as the key alone; combine with your other settings):

"env": {
  "ANTHROPIC_BASE_URL": "http://localhost:8080/anthropic",
  "ANTHROPIC_AUTH_TOKEN": "your-bifrost-virtual-key",
  "ANTHROPIC_DEFAULT_HAIKU_MODEL": "groq/llama-3.3-70b-versatile",
  "ANTHROPIC_DEFAULT_SONNET_MODEL": "groq/llama-3.3-70b-versatile"
}

ANTHROPIC_BASE_URL sends Claude Code's calls to Bifrost's local Anthropic layer. ANTHROPIC_AUTH_TOKEN provides the virtual key in the Authorization: Bearer header for Bifrost's auth and routing. Model names prefixed with groq/ pin Haiku and Sonnet slots to Groq.

If Claude Code is active, exit and relaunch. Prevent caching issues by running /logout first.

Option B: Bifrost CLI (no Manual Configuration)

Bifrost CLI is a guided terminal launcher. It connects Claude Code to an active Bifrost instance interactively, handling base URL setup, key storage, model selection, and MCP server hookup—all without editing files or setting env variables.

From a second terminal (gateway already running):

npx -y @maximhq/bifrost-cli

The interactive flow asks five questions:

Bifrost gateway location (typically http://localhost:8080)
Your virtual key (optional; press Enter to skip)
Pick Claude Code as your agent
Filter by groq/ and select your model (e.g., groq/llama-3.3-70b-versatile)
Confirm and launch

The CLI writes environment variables on the fly, auto-registers Bifrost's MCP service for your tools, and saves settings to ~/.bifrost/config.json between runs. Keys go to the system keyring—never saved as text.

On model selection: Claude Code depends on function calling for file operations, terminal execution, and code changes. Verify your chosen Groq model supports tool use. The GroqCloud documentation lists each model's capabilities.

Step 5: Test the Setup

Once Claude Code is pointed at Bifrost (via either method), run:

claude --model groq/llama-3.3-70b-versatile

Inside the session, type /model to confirm the active model. Check Bifrost's dashboard Logs tab to see the request routed to Groq; a 200 status indicates success.

To change models mid-session, use:

/model groq/llama-3.3-70b-versatile

Automatic Failover and Routing Rules

Bifrost opens up failover on demand: if Groq returns errors or hits rate limits, Bifrost can automatically reroute to Anthropic or another provider without Claude Code knowing. Build a fallback chain in the dashboard (Features > Fallbacks) or in config.json:

{
  "fallbacks": [
    {
      "from": "groq/llama-3.3-70b-versatile",
      "to": ["anthropic/claude-haiku-4-5"]
    }
  ]
}

Routing rules give you condition-based switching too. Example: send Claude Code requests (identifiable by claude-cli user-agent) to Groq by default, but route others to Anthropic. This creates per-client model behavior on a single instance.

Governance and Observability Built In

Routing Claude Code via Bifrost unlocks the full governance stack at no cost. Per-key spending caps and request limits prevent unexpected bills. Logs capture every request's provider, model, token count, and timing. For teams with many Claude Code users, the LLM Gateway Buyer's Guide details how to layer virtual keys by team, project, or owner.

Observability is ready immediately via the Bifrost dashboard. Prometheus metrics live at /metrics for your monitoring systems. OTLP export is available for Datadog, Grafana, or any standards-compatible backend.

For teams needing to centralize MCP tool management alongside model routing, Bifrost can also work as an MCP gateway: link upstream MCP servers, expose them through /mcp, and register them once with Claude Code via a single command. This cleanly separates inference routing from tool control.

Wrapping Up

Getting Claude Code to Groq through Bifrost boils down to three shared steps (start gateway, enable Groq, make a virtual key) and then connecting Claude Code, either via direct settings.json editing or Bifrost CLI for a guided, no-variables approach. From there, Claude Code defaults to Groq, with failover, governance, and dashboards ready to go.

The same flow works for any Bifrost-supported provider: flip the model name to anthropic/, openai/, bedrock/, vertex/, or any of 20+ platforms, and Claude Code adapts instantly.

For multi-team scaling with load balancing, clustering, vault integration, OIDC, or private-cloud deployment, schedule a demo with the Bifrost team.

AI Gateways for Production: A Technical Overview

Kuldeep Paul — Thu, 11 Jun 2026 20:37:17 +0000

Route, govern, and observe LLM traffic from a single control plane. Bifrost unifies 20+ providers through one OpenAI-compatible API.

Imagine your production AI system calling OpenAI, Anthropic, AWS Bedrock, and Google Vertex AI, each with its own SDK, authentication scheme, and rate-limit surface. Duplicated integration code sprawls across your codebase, access controls are inconsistent, and no single vantage point shows cost or latency. This is the problem an AI gateway solves. It acts as a unified entry point between applications and LLM providers, centralizing routing, security, and observability through one API. Bifrost, the open-source AI gateway built in Go by Maxim AI, is the best overall choice for enterprise teams running mission-critical AI workloads that require best-in-class performance, scalability, and reliability. This guide covers what an AI gateway does, where it differs from traditional API gateways, which production-grade capabilities matter, and how to evaluate one for your stack.

Defining an AI Gateway

Between your application and your LLM providers sits a middleware layer called an AI gateway. It exposes a single API to all models, then applies routing rules, fallback logic, authentication, caching, rate limiting, and monitoring to every request passing through.

Why does this layer exist? Direct integrations do not scale. Every provider has its own SDK, its own quota semantics, and its own response format; adding a new model multiplies integration and maintenance burden. A 2025 overview from IBM frames the problem as a need for unified layer connecting applications to models while applying governance and security policies consistently.

In operation, the gateway becomes the single source of truth for AI traffic: which application made which request, at what cost, with what latency, under which policy constraints. Both Bifrost's documentation and the LLM Gateway Buyer's Guide detail the concrete capabilities that matter, which structure the rest of this discussion.

Why Gateways Differ from Traditional API Gateways

A conventional API gateway manages microservices: it enforces authentication, limits requests by count, and routes by path. LLM workloads break these assumptions, which is why a specialized layer has emerged.

The key differences:

Pricing is token-based, not request-based. Two identical requests can differ wildly in cost depending on model selection and output length. Rate limits and budgets must therefore track tokens and spending, not just request count.
Providers enforce per-key quotas. When OpenAI's rate limits are hit on one key, the gateway must distribute load across keys or fall back to a different provider entirely.
Sensitive data lives in payloads. Customer information, source code, and credentials flow through prompts regularly, making prompt screening and redaction a gateway-level responsibility.
Responses are format-heterogeneous. Streaming semantics, error types, and provider-specific parameters differ across vendors; the gateway must normalize all of this into one consistent shape.

A traditional API gateway can sit in front of an LLM endpoint, but it lacks the ability to reason about tokens, model failover decisions, or content safety. An AI gateway is purpose-built for these exact problems.

Capabilities Every Production Gateway Needs

Bifrost's feature set covers everything teams should expect from a production gateway:

Unified API, drop-in ready. A single OpenAI-compatible endpoint works for every provider. The drop-in SDK pattern means OpenAI, Anthropic, LangChain, and LiteLLM code needs only a base URL change.
Multi-provider, multi-model routing. Support for 20+ providers and 1,000+ models, including OpenAI, Anthropic, Bedrock, Vertex, Azure, and more, with consistent responses regardless of where a request routes.
Failover and load balancing. When a provider fails or hits quota, automatic fallbacks reroute to alternates. Weighted key distribution handles load balancing.
Semantic caching. Similar queries pull responses from cache instead of making new API calls, cutting cost and latency through semantic matching.
Access control. Virtual keys bundle per-consumer permissions, budgets, and rate limits into one governance entity.
Monitoring and tracing. Every request is logged with full metadata. Prometheus metrics emit natively, and OpenTelemetry spans integrate with any observability backend.
Content safety. Guardrail plugins screen prompts and completions against AWS Bedrock Guardrails, Azure Content Safety, and Patronus AI services in-line.
MCP tool management. Acting as an MCP gateway, Bifrost centralizes Model Context Protocol tool discovery and execution, so agents access tools through the same governed gateway.

These capabilities reinforce one another: failover only works with multi-provider support, governance depends on a single control point, and observability is incomplete unless all traffic flows through the same layer.

How Bifrost Operates at Scale

Built in Go, the Bifrost gateway achieves minimal overhead. Production benchmarks at 5,000 requests per second show just 11 microseconds of added latency, with zero request failures.

Getting started takes minutes. One Docker command or NPX invocation brings the gateway online with zero required configuration. Via the web UI or API, teams configure providers, API keys, routing rules, and access policies. Existing application code points at Bifrost by changing the SDK base URL. All requests then flow through the gateway, automatically gaining failover, caching, governance, and tracing with no additional code.

The observability pipeline operates continuously. The gateway records each request including provider, model, token volumes, cost, response time, and cache hit status. Prometheus scraping works natively, and OpenTelemetry traces feed into any compatible backend, so AI traffic lands in the same monitoring dashboards platform teams already run.

For agent workloads, the same infrastructure handles tools. Bifrost serves as both an MCP client and server, connecting external tool providers and exposing them to clients (Claude Desktop, Cursor, etc.), with per-key tool filtering that restricts each consumer to permitted tools only.

Evaluation Framework for Production Deployments

Selecting a gateway is an infrastructure decision, so apply the same rigor as any critical-path system. Four evaluation areas distinguish production-ready deployments from development experiments:

Latency impact. The gateway lives on the request path, so validate overhead under your actual traffic profile, not in a single request benchmark.
Availability guarantees. Single points of failure are unacceptable. Bifrost Enterprise offers clustering with automatic discovery and zero-downtime rolling updates.
Deployment options. Regulated industries demand the gateway run within their own network boundary. Bifrost Enterprise provides in-VPC, on-prem, and isolated deployments, along with audit logs that satisfy SOC 2, GDPR, HIPAA, and ISO 27001 requirements.
Governance breadth. Ensure the platform supports nested budgets, RBAC, and identity provider federation, not merely API key management.

For detailed capability comparisons across multiple gateways, the LLM gateway evaluation guide provides a structured matrix.

Timing plays a role as well. Early gateway adoption, when your system runs one provider, carries minimal switching friction (just a base URL change) while delivering failover and cost tracking before an outage forces the decision. Late adoption, after multiple providers are entrenched, becomes a migration project.

Adopting Bifrost

An AI gateway collapses fragmented provider integrations into one governed, observable, resilient layer, and starting early avoids integration debt. Bifrost deploys in seconds as open-source software and scales from a developer laptop to enterprise clusters processing thousands of requests per second. Explore how Bifrost fits your infrastructure by booking a demo with the Bifrost team.

MCP at Scale: When to Use Code Mode Instead of Classic Tool Calling

Kuldeep Paul — Thu, 11 Jun 2026 20:36:43 +0000

Classic MCP loads every tool schema on every request, driving up costs. Code Mode in Bifrost cuts input tokens by 92.8% by letting AI orchestrate through Python.

Building AI agents that connect to dozens of external tools creates a hidden cost: each request loaded with 150+ tool schemas, consuming most of the model's token budget just parsing definitions instead of solving the task. This is the central tradeoff between classic MCP and Code Mode. Bifrost, the open-source AI gateway built in Go by Maxim AI, gives teams both options, letting them pick based on deployment scale and workflow complexity. Understanding when each approach makes sense can cut costs by orders of magnitude.

The Default Approach: Direct Tool Calling

Classic MCP tool calling is how most MCP implementations work out of the box. The gateway pushes all connected tool definitions into the model's context before each request arrives, the model generates tool-call suggestions, and the application handles execution separately through an explicit approval step.

Tool execution in Bifrost follows a deliberate, stateless flow: each chat request asks the model what to do, returns the suggestions without running them, lets the application review for safety, and then executes the approved actions via a separate call before resuming conversation. Bifrost simultaneously acts as an MCP client connecting to external servers via STDIO, HTTP, or SSE, and as an MCP server exposing those tools to clients such as Claude Desktop. See the MCP overview for the full mechanics.

This design delivers tight control. Tool execution never happens without a reviewer in the loop, all operations are logged, and the calling code maintains conversation state. The downside emerges at scale: as servers multiply, the model rereads the entire tool list on every turn, and every intermediate step in a multi-tool task has to flow back through the context before the next move.

The Alternative: Code Mode

Code Mode flips the model: instead of injecting all tool definitions, the system shows the AI four small meta-tools and lets it write a Python orchestration script that runs in a controlled sandbox. The model sees compact schemas rather than a catalog of 150 definitions.

When Code Mode is enabled in Bifrost, four lightweight tools appear in every request:

listToolFiles: reveal available MCP servers and their tools
readToolFile: fetch Python function signatures for a server as needed
getToolDocs: retrieve full details for a single tool on demand
executeToolCode: submit Python code for sandbox execution with full tool bindings

The model discovers what's available, pulls the signatures it needs, and writes a short Python script that Bifrost runs via a Starlark interpreter. Any intermediate outputs from tool calls stay sandboxed and never return to the model; only the final result comes back. This mirrors the pattern described by Anthropic's engineering team, which found that a Google Drive to Salesforce integration shrank from 150,000 tokens down to 2,000 when tool calls gave way to code execution. Bifrost embeds this as a first-class gateway feature, picking Python over JavaScript (models see more Python in training data) and including a dedicated doc-lookup tool to compress context even further.

Comparing the Two Patterns

Both approaches operate through the same MCP gateway. The difference lies in what the model sees, how many interaction cycles happen, and where orchestration logic runs. Switching from one to the other is just a per-client configuration toggle.

Aspect	Classic MCP	Code Mode
Tool visibility	Complete list in context, repeated per turn	Four meta-tools; definitions fetched on demand
Model interaction cycles	One step per tool invocation	3 to 4 total across whole workflow
Where intermediate outputs go	Returned to model	Processed inside the sandbox
Where workflow logic lives	Conversation loop	Python script
Cost scaling	Grows linearly with tool count	Capped by actually-used files
Right for	Few tools, straightforward calls	Many servers, intricate multi-step work

Classic MCP's simplicity shines for lean tool sets. Code Mode trades that immediacy for resource efficiency: tools stay behind the meta-tool interface, and the model only reads the stubs and docs it actually uses.

The Token Math as Tool Count Climbs

The efficiency gains from Code Mode multiply as deployments scale. At large tool counts, classic MCP cost balloons because the entire catalog reloads every turn, whereas Code Mode cost plateaus based on which stub files the model opens.

Bifrost ran benchmarks across increasing MCP footprints, comparing Code Mode to classic MCP with the same test queries each time. The benchmark writeup shows the separation widening:

96 tools across 6 servers: input token drop of 58.2%, cost drop of 55.7%
251 tools across 11 servers: input token drop of 84.5%, cost drop of 83.4%
508 tools across 16 servers: input token drop of 92.8% (75.1M falling to 5.4M), cost drop from $377 to $29

The largest test maintained a 100% pass rate (65 of 65 succeeded), showing the token reduction came without sacrificing task completion. Per-query math at 500 tools shows roughly a 14x saving, dropping from 1.15M to 83K tokens. These runs also cut LLM round trips from many calls to 3 to 4 total, and accelerated execution by ~40%. Using published benchmarks, teams can test their own MCP configurations.

How Code Mode Executes, Step by Step

Code Mode replaces the back-and-forth loop with a single discover-then-run sequence. Once activated for an MCP client, that client's tools disappear from direct availability and only become accessible through the four meta-tools.

A typical interaction unfolds like this:

Discovery: the model calls listToolFiles to examine available servers
Schema loading: readToolFile retrieves the Python signatures for a relevant server
Documentation lookup: if needed, getToolDocs pulls deeper docs for one tool
Execution: the model writes Python code that executeToolCode runs in the sandbox, returning a compact outcome

The Starlark sandbox intentionally limits what code can do: no import statements, no filesystem or network access, default 30-second timeout. Tools from Code Mode clients appear as global objects in the sandbox; calls run synchronously, so scripts can chain multiple tool operations and return a single aggregated result. Bifrost offers two ways to organize the stub files: server-level puts all tools from one server in one file, while tool-level creates one file per tool for servers with large schemas.

Code Mode is toggled per MCP client. A single Bifrost instance can run some clients in classic mode and others in Code Mode simultaneously, and adding new servers follows the standard connection flow either way. Complete setup instructions and the full meta-tool API are in the Code Mode docs.

Deciding Between Classic MCP and Code Mode

Code Mode wins for deployments with 3+ MCP servers, workflows that require multiple orchestrated steps, or scenarios where token cost and latency are top concerns. Stick with classic calling for single or dual-server setups with straightforward, one-step tool use. The patterns need not be all-or-nothing: advanced teams use Code Mode for heavy-duty servers like web search or data access, and keep direct calling for quick utilities.

What if I only have one or two MCP servers?

The token savings from Code Mode rarely justify the added conceptual overhead with minimal server counts. Classic calling has lower mental complexity and debugging is simpler. Code Mode's payoff accelerates as the tool count climbs.

Will Code Mode slow my agents down?

No. By consolidating multiple tool steps into one sandboxed script, Code Mode actually cuts the count of LLM interactions by 3 to 4 times and typically improves execution speed by 40% in large deployments. Classic MCP can still be preferable for single, immediate tool calls where latency is critical.

Can I run both at once?

Absolutely. Code Mode is configured individually per MCP client, so a Bifrost setup can have some clients in classic mode and others in Code Mode. This approach lets teams migrate the largest, most expensive servers first while keeping lightweight tools on the simpler path.

How secure is Code Mode auto-execution?

Agent Mode lets Code Mode scripts run without human approval, but with safeguards. Bifrost extracts every tool reference from the code and executes only if all of them are on the allowlist; any off-list calls bounce back for approval. Paired with granular tool filtering per virtual key, this keeps autonomous runs inside configured guardrails. For compliance-heavy teams and regulated industries, Bifrost operates as a full-featured MCP gateway with access logs, cost attribution, and tool-level permissions.

Next Steps

Picking between classic MCP and Code Mode fundamentally depends on scale. For small tool inventories, classic calling is clear and direct. For agents managing many connected servers, Code Mode constrains token consumption and latency even as the catalog expands, while sustaining reliability. Both ride the same infrastructure; toggling Code Mode happens once servers connect via the Bifrost quickstart. The MCP specification provides full protocol details for teams digging deeper into the standard itself.

Ready to optimize MCP orchestration and governance at scale? Schedule a demo with the Bifrost team, or browse the Bifrost resources hub for benchmarks and deployment patterns.

Claude Code Governance: How an AI Gateway Secures Agent Access

Kuldeep Paul — Thu, 11 Jun 2026 20:35:56 +0000

Effective Claude Code governance requires a gateway-layer control point. Bifrost delivers that control through virtual keys, credential isolation, MCP tool filtering, and immutable audit trails for every agent session.

From a single terminal session, Claude Code reads full repositories, runs shell commands, and invokes external MCP tools, all authenticated with a raw provider API key. When developers share the same key, the same model permissions, and an unrestricted tool surface, a single leaked credential or an over-privileged session crosses from a billing problem into a security incident. Claude Code governance is how platform teams insert identity, access control, and audit between the agent and the provider, so every request is scoped, attributable, and recoverable after the fact. Bifrost, the Go-based open-source AI gateway built by Maxim AI, enforces those controls at the gateway layer through virtual keys, credential isolation, MCP tool filtering, guardrails, and audit logs. This guide explains what governance actually requires and how Bifrost applies it without touching the developer workflow.

The Five Security Controls That Claude Code Governance Requires

At its core, Claude Code governance means running every agent request through a control point that authenticates the caller, keeps real provider credentials off developer machines, limits which models and tools each session can reach, and captures each action in a tamper-proof log. An AI gateway is that control point; the Claude Code client itself does not need to change.

The risks that make this necessary are well established. The OWASP Top 10 for LLM Applications places excessive agency and sensitive information disclosure in its list of the most critical LLM security risks, and an autonomous coding agent that holds file access, shell access, and tool-calling privileges exercises that agency on every run. Five controls address those risks comprehensively:

Credential isolation: provider keys are held centrally in the gateway, not distributed to individual developer machines
Identity and attribution: each request is tied to a developer, team, or project rather than a shared API key
Access scoping: per-key policies define which models, providers, and MCP tools any given session is allowed to reach
Containment: rate limits and guardrails bound the damage a compromised or runaway session can cause
Audit and evidence: request-level records covering session, model, token, and user data for compliance purposes

All five controls are applied through gateway-level governance, which means they take effect on every Claude Code session uniformly, regardless of developer, provider, or deployment environment.

Why Anthropic's Native Controls Leave Gaps

Out of the box, Claude Code authenticates directly with a single provider key per developer. Anthropic's Team and Enterprise plans include SSO and admin controls, but they stop short of fine-grained, cross-provider access control and request-level audit logs that platform teams need. Direct-to-provider usage produces four security gaps:

Broadly scoped credentials: a raw provider key stored on a laptop can be copied, committed to version control, or reused in ways the platform team cannot track
Uniform access across roles: every developer, from intern to staff engineer, reaches the same provider catalog, model set, and tool surface
No MCP tool boundary: a connected MCP server exposes all of its tools to every session
No per-request identity or audit record: the provider console shows aggregate usage, not who ran which session, against which model, at what cost

Anthropic's cost documentation reports that 90 percent of users stay under $30 per active day, which still exposes organizations to a long tail of sessions costing hundreds of dollars. A flat, identical-access model cannot contain that tail. The gateway layer is where these gaps close, specifically through virtual keys that attach identity and policy to every request before any provider receives it.

Routing Claude Code Through Bifrost

Bifrost intercepts Claude Code's requests at the transport layer before they reach any upstream provider. Connecting Claude Code to the gateway takes two environment variable changes; no modifications to the Claude Code binary, extensions, or workflow are needed.

# Route Claude Code through the Bifrost gateway
export ANTHROPIC_BASE_URL="http://localhost:8080/anthropic"

# Authenticate using a Bifrost virtual key (no Anthropic account required)
export ANTHROPIC_AUTH_TOKEN="your-virtual-key"

# Start Claude Code
claude

Bifrost exposes an Anthropic-compatible API surface, so streaming responses, tool calls, and extended thinking all continue to function without modification. The full setup, including settings.json configuration and provider passthrough patterns for Bedrock, Vertex, and Azure, is covered in the Claude Code integration guide.

Does the gateway put provider credentials on developer machines?

No. With the recommended ANTHROPIC_AUTH_TOKEN method, Claude Code transmits only the Bifrost virtual key. Real provider credentials are stored centrally in the gateway and injected per-request. Revoking or rotating a key takes effect immediately on the next request, with no key rotation ceremony and no environment variable changes required across developer machines.

What does this add to the developer's daily workflow?

In practice, nothing. Only the base URL and the virtual key differ from a direct-to-Anthropic setup. Developers run the same claude command and keep all existing session features, including /model switching. Bifrost adds 11 microseconds of per-request overhead at 5,000 requests per second in sustained performance benchmarks.

Virtual Keys: Where the Security Boundary Is Defined

Virtual keys are the core governance primitive in the Bifrost AI gateway. Each key represents one authenticated consumer, whether that is an individual developer, a squad, a CI pipeline, or an external tenant, and carries a policy defining exactly what that consumer can do:

Provider and model scoping: a key configured for Sonnet and Haiku cannot call Opus or reach an unlisted provider; attempting to switch out-of-scope via /model returns a clear error rather than executing silently
MCP tool filtering: tool filtering on a per-virtual-key basis determines which tools each agent session can call, so a developer building a support agent sees CRM and ticketing tools, while an SRE sees Kubernetes and observability tools
Tenant isolation: virtual keys scoped per team or customer keep access, budgets, and usage data cleanly separated across organizational boundaries

MCP tool filtering is the most direct answer to the excessive agency risk: the full set of connected tools remains available in the gateway, but each session's visible surface is constrained to what that role is authorized to use. A single GitHub or database MCP server no longer functions as an all-or-nothing grant to every developer.

Rate Limits and Guardrails as Containment Controls

Even with access scoped correctly, a compromised key or a misbehaving automated session needs to be bounded. Bifrost provides two containment mechanisms at the gateway layer: rate limits and guardrails.

Rate limits operate at the virtual key level, capping requests per minute for each consumer. This covers subagent loops, automation scripts that run unbounded, and leaked keys being driven programmatically. Setting a tighter per-minute ceiling on CI automation keys than on interactive developer keys means one failed pipeline cannot consume the team's entire throughput allocation.

Guardrails evaluate prompts and completions as they transit the gateway. Secrets detection scans for API keys, tokens, and credentials that a developer might paste into context or that a model might reproduce in its output. Custom regex patterns allow teams to redact or block organization-specific sensitive strings. Content-safety enforcement through AWS Bedrock Guardrails, Azure Content Safety, and Patronus AI applies consistently across all Claude Code sessions, configured once in the gateway rather than per developer.

Building an Audit and Compliance Record

Governance that cannot be demonstrated to auditors has limited value. Audit logs in Bifrost generate an immutable, request-level record of every Claude Code session, covering model, token counts, tool call activity, and the virtual key or user behind each request. This record meets the evidence requirements for SOC 2, HIPAA, GDPR, and ISO 27001.

For workloads in regulated industries, observability adds Prometheus metrics and OpenTelemetry distributed traces, each tagged with virtual key, team, model, and tool identifiers. Security and platform teams can query exactly which model a developer or agent used, when, and how many tokens it consumed. Bifrost can be deployed in-VPC or in air-gapped environments so no inference traffic or credential exits the organization's own infrastructure; SSO through OIDC maps virtual keys to directory identities in Okta or Microsoft Entra, closing the loop between organizational identity management and per-request attribution.

Start Governing Claude Code Agent Access

Claude Code governance is, first, a security problem. The provider console provides a spending record; it does not keep credentials off laptops, restrict tool access by role, stop a runaway session, or prove who ran what. Placing Bifrost between Claude Code and the provider adds virtual keys, tool filtering, guardrails, and audit logs behind the same Anthropic-compatible endpoint the agent already targets, with no change to how developers work. The Bifrost governance resource page documents the full control set, and the LLM Gateway Buyer's Guide covers how gateway options compare on governance and security depth.

To apply Claude Code governance to your engineering organization's actual usage, book a Bifrost demo with the team.