DEV Community: James Whitfield

Why Governing AI at the Device Beats Governing It at the Firewall

James Whitfield — Wed, 24 Jun 2026 18:06:52 +0000

AI governance at the device provides the visibility and control that network-level security is not designed to offer. For teams managing shadow AI, endpoint governance with a tool like Bifrost Edge is the only way to apply policy where the activity actually happens.

The rapid adoption of generative AI has created a significant governance blind spot for most organizations. Employees use desktop AI applications, browser-based chat tools, and coding agents that communicate directly with model providers, bypassing traditional network security controls. This "shadow AI" usage means that sensitive data can leave the company without any oversight, creating substantial compliance and security risks. While many security teams look to firewalls as a control point, governing modern AI traffic requires moving enforcement from the network perimeter to the device itself.

This article examines the limitations of firewall-based AI governance and explains why a device-centric approach is more effective for managing the security and compliance risks of enterprise AI.

The Limits of the Firewall in the AI Era

For decades, firewalls have been a cornerstone of enterprise security, filtering network traffic based on ports, protocols, and IP addresses. However, they were not designed to understand the nuances of AI interactions.

Blindness to Content and Context

Traditional firewalls and even Next-Generation Firewalls (NGFWs) are fundamentally blind to the semantic content of AI traffic. They can block or allow a connection to an AI service, but they cannot inspect the prompt a user submits or the response a model generates. A firewall rule cannot distinguish between a developer asking an AI to write a benign unit test and one pasting proprietary source code into a public model. This lack of content awareness makes it impossible to enforce data governance policies at the network level.

The Encryption Problem

Most AI traffic is encrypted using TLS, which means that without deep packet inspection (DPI), firewall appliances cannot see the contents of the requests. While some solutions offer TLS interception, it introduces significant performance overhead, can break applications, and raises privacy concerns. Attackers and unapproved applications often use standard encrypted channels like HTTPS, which firewalls are configured to trust and allow, rendering them ineffective at stopping data exfiltration through sanctioned ports.

Inability to See Local and Agentic AI

A growing category of AI tools, including local models and endpoint agents, operates without generating easily observable network signals. An open-weight model running directly on a developer's machine, or an AI agent accessing local files and system APIs, is completely invisible to a network firewall. These tools can process sensitive information, interact with other applications, and execute commands without ever sending a packet across the perimeter that a firewall could inspect.

The Case for AI Governance at the Device

Governing AI on the endpoint closes the visibility gaps left by network-level controls. By placing an agent on the device, security and platform teams can see and control AI activity at its source.

Complete Visibility into All AI Usage

Endpoint governance provides a complete inventory of all AI tools in use across a fleet, including desktop apps, browser extensions, coding agents, and even locally running MCP servers. This visibility is the first and most critical step toward effective governance; you cannot govern what you cannot see. An endpoint agent can identify every AI application, who is using it, and what data it accesses, creating the foundation for policy enforcement.

Context-Aware Policy Enforcement

Unlike a firewall, a device-level agent understands the full context of an AI interaction. It can see the user, the application generating the request, the content of the prompt, and the local files or data being accessed. This allows for highly granular, context-aware policies. For example, a policy could allow employees to use a sanctioned AI tool for general queries but block prompts containing personally identifiable information (PII) or source code. This level of control is impossible at the firewall.

Real-Time Intervention Before Data Leaves

Governing at the device means policy is enforced before sensitive data ever leaves the machine. An endpoint agent can inspect a prompt or a file upload in real time, block it based on content, and prevent it from being sent to an external AI model. This preemptive control is a significant advantage over network-based data loss prevention (DLP), which can only react after the data has already traversed the internal network.

A Unified Approach: AI Gateway and Endpoint Governance

The most effective AI governance strategy combines a centralized control plane with distributed endpoint enforcement. This is the model offered by Bifrost, an open-source AI gateway, and its endpoint component, Bifrost Edge.

The Bifrost AI gateway acts as the central policy engine. It is where administrators configure virtual keys, set budgets and rate limits, define routing rules, and apply security guardrails. For traffic configured to route through it, the gateway provides comprehensive control and observability.

However, to address shadow AI, that governance must be extended to the endpoint. Bifrost Edge is an agent that runs on macOS, Windows, and Linux devices and transparently routes all AI traffic through the organization's Bifrost gateway.

This combined approach delivers several key benefits:

Centralized Policy, Distributed Enforcement: Policies are defined once in the Bifrost gateway and enforced consistently everywhere, from backend services to every employee's laptop.
Zero-Configuration for Users: Employees continue using their preferred AI tools like Claude Desktop, ChatGPT, and Cursor without changing any settings. Edge automatically handles the routing in the background.
Fleet-Wide Deployment: Bifrost Edge is designed for enterprise scale and can be rolled out silently across thousands of machines using existing MDM platforms like Jamf, Intune, or Kandji.
Complete Audit Trail: Every AI prompt and response from every device is logged, providing a complete audit trail for compliance and security investigations.

Moving Beyond the Perimeter

Relying on firewalls for AI governance is like trying to secure a modern office building with only a front-door security guard. While essential, it fails to address the activity happening inside. As AI becomes more integrated into daily workflows and more autonomous, security controls must move closer to the data and the user.

Endpoint governance is not a replacement for firewalls but a necessary evolution of the security stack. By shifting enforcement to the device, organizations can gain the visibility and contextual control required to manage the risks of shadow AI, enabling employees to innovate responsibly without exposing the business. Teams evaluating AI governance solutions can request a demo of Bifrost to see how its combined gateway and endpoint approach works.

What to Look For in an Endpoint AI Governance Agent

James Whitfield — Wed, 24 Jun 2026 18:06:51 +0000

As employees adopt desktop AI clients, coding agents, and browser-based tools, organizations need a way to enforce security and compliance policies directly on the device. An endpoint AI governance agent is the solution, but not all agents are created equal. An effective agent must extend a centralized policy engine, govern all forms of AI traffic, and operate transparently without disrupting workflows. Bifrost and its endpoint agent, Bifrost Edge, provide a complete model for this.

The use of unsanctioned AI tools by employees, often called "shadow AI," has become a primary security and governance challenge for enterprises. When employees use personal accounts for ChatGPT, wire up local Model Context Protocol (MCP) servers, or use unapproved coding assistants, they operate outside of the organization's control plane. This creates significant risks, including data leakage, compliance violations, and intellectual property loss. An open-source AI gateway like Bifrost can centralize policy for known traffic, but it cannot govern what it cannot see.

This is the problem an endpoint AI governance agent is designed to solve. By running directly on employee machines, the agent brings all AI activity—from desktop apps to browser sessions—under a unified policy, closing the visibility gap that shadow AI exploits.

The Core Problem: Shadow AI and Ungoverned MCP Servers

Shadow AI is the practice of using AI tools without approval or oversight from IT and security teams. It happens because employees seek to be more productive, adopting tools that fit their workflow, often unaware of the associated risks. A recent Gartner report noted that a significant percentage of employees use personal AI accounts for work, and many admit to uploading sensitive data to these unapproved tools.

A critical and often invisible component of shadow AI is the proliferation of "Shadow MCP." The Model Context Protocol (MCP) is the standard that allows AI agents to interact with external tools, databases, and APIs. Employees, particularly developers, often run local MCP servers to connect AI agents to internal resources. These unmanaged servers create unaudited pathways to sensitive data, operating completely outside of established security frameworks.

An endpoint agent's first job is to make this invisible activity visible and bring it under a consistent governance model.

Key Capabilities of an Effective Endpoint AI Agent

When evaluating an endpoint AI governance solution, teams should look for specific capabilities that address the realities of modern AI usage. A policy document is not enough; effective governance requires technical enforcement on the device itself.

1. Unified Policy Engine: The Gateway + Edge Model

An endpoint agent should not be a standalone tool with its own set of rules. It must be an extension of a central policy engine. The most effective architecture is a combined AI gateway and endpoint agent model.

The AI Gateway as a Control Plane: The gateway is where administrators define all governance policies: virtual keys, access controls, budgets, rate limits, audit logging, and security guardrails. It serves as the single source of truth for the entire organization.
The Endpoint Agent as the Enforcer: The agent runs on each machine (macOS, Windows, Linux) and ensures all AI traffic from that device routes through the central gateway. This means the same policies apply everywhere, without needing to configure each app or tool separately.

The Bifrost AI gateway and Bifrost Edge exemplify this model. Policies configured in the gateway are automatically enforced by the Edge agent on every employee's machine, providing consistent governance for all AI activity.

2. Comprehensive Application and MCP Governance

An agent must be able to see and control the full spectrum of AI tools used by employees. This includes four key surfaces:

Desktop AI Apps: Standalone clients like Claude Desktop and the ChatGPT app.
AI in the Browser: Web-based interfaces such as claude.ai and chatgpt.com.
Coding Agents: Tools used in the terminal and IDE, like Claude Code and Codex CLI.
MCP Servers: Local and remote tool servers that agents connect to.

A robust endpoint solution provides a fleet-wide inventory of all discovered apps and MCP servers. From this central dashboard, administrators should be able to approve or deny tools with policies that propagate to every device. With Bifrost Edge, administrators can manage application access and govern MCP servers from a single interface, ensuring that only sanctioned tools can operate.

3. Transparent Operation and Zero-Configuration Experience

For an endpoint agent to be successful, it must not create friction for users. The ideal agent is invisible after a one-time setup.

No Per-App Setup: Users should not need to change base URLs, swap SDKs, or reconfigure their tools. The agent should intercept and route traffic transparently at the machine level.
SSO Integration: Initial setup should be seamless, using the organization's existing single sign-on (SSO) provider (Okta, Microsoft Entra, Google Workspace) to link the device to a user and sync their policies.
User Control: While the agent enforces company policy, it should provide a simple interface for users to see their status, active policies, and switch between different virtual keys if their role requires it.

Bifrost Edge is designed for this transparency. It lives in the menu bar or system tray, requires a single browser-based sign-in, and automatically governs all supported AI applications without any user intervention.

4. Fleet-Wide Deployment and Management

Endpoint governance is only effective if it covers every device. An enterprise-grade agent must support silent, large-scale deployment through standard Mobile Device Management (MDM) platforms.

Look for support for tools like:

Jamf
Microsoft Intune
Kandji
Workspace ONE
JumpCloud

MDM solutions allow administrators to push the agent to every corporate device with a managed configuration, ensuring consistent rollout and policy enforcement from day one. Bifrost Edge offers full MDM deployment support, enabling platform teams to secure their entire fleet without manual setup on each machine.

5. Extension of Existing Security and Compliance Controls

Finally, an endpoint AI governance agent must integrate with and extend the security controls an organization already relies on. Because all traffic is routed through the central AI gateway, every request from the endpoint inherits the same protections.

Guardrails: PII detection, secrets scanning, and custom content policies configured in the gateway are applied to prompts and responses from desktop and browser apps. Bifrost's guardrail capabilities are extended to the endpoint via Edge.
Audit Logs: All endpoint AI activity is captured in the gateway's immutable audit logs, providing the documentation needed for compliance with frameworks like SOC 2, HIPAA, and the EU AI Act.
Cost and Access Controls: Budgets and rate limits defined by virtual keys apply to endpoint usage, preventing cost overruns from unsanctioned tools.

Conclusion: The Endpoint is the New Control Plane

As AI becomes more decentralized, endpoint governance moves from a "nice-to-have" to a core component of any enterprise AI strategy. Written policies are insufficient to manage the risks of shadow AI. Organizations need a technical solution that provides visibility and enforces policy directly on the device.

An effective endpoint agent works as part of a unified system, extending a central AI gateway's policies to every machine. It governs all types of AI tools, operates transparently for users, and deploys at scale. By adopting a solution like the Bifrost AI gateway with the Bifrost Edge agent, organizations can turn their endpoint blind spots into a governed, secure, and compliant part of their AI ecosystem. Teams evaluating solutions can request a Bifrost demo to see how the gateway and endpoint agent work together.

Sources

Governing AI on Every Company Computer: A Reference Architecture

James Whitfield — Wed, 24 Jun 2026 18:06:06 +0000

To effectively govern AI, organizations need a reference architecture that extends from a central gateway to every endpoint. This post outlines a three-plane model for the control plane (gateway), the enforcement plane (endpoint agent), and the management plane (MDM) that closes governance gaps left by shadow AI.

The adoption of generative AI has created a significant governance gap in most organizations. While security and platform teams focus on controlling AI through centrally managed gateways, employees are using unapproved desktop apps, browser-based tools, and coding agents on their company computers. This "shadow AI" operates outside of established controls, creating unmonitored pathways for data exfiltration, compliance violations, and other security risks.

An effective AI governance strategy cannot stop at the network edge; it must extend to the endpoint where AI tools are actually used. A policy document is not a technical control. Real governance requires an architecture that ensures the rules defined centrally are enforced on every device. This article presents a vendor-agnostic reference architecture for achieving this, composed of three distinct but interconnected planes: a control plane, an enforcement plane, and a management plane.

The Challenge: Shadow AI and the Endpoint Blind Spot

Shadow AI refers to the use of AI tools within an organization without IT approval or oversight. It's a modern variant of shadow IT, but with higher stakes due to the ability of AI models to process and retain sensitive data. When an employee pastes proprietary code or confidential customer data into a public AI chatbot, that information can become part of the model's training data, effectively leaking it.

The core problem is that a gateway-only approach to AI governance is incomplete. An AI gateway can only enforce policies on traffic that is explicitly configured to pass through it. The tools employees download and use on their own—from the ChatGPT desktop app to coding agents like Claude Code—bypass these gateways by default, leaving a massive blind spot. This is where a comprehensive architecture becomes necessary.

A Three-Plane Reference Architecture for Endpoint Governance

A robust solution for governing AI across a fleet of computers relies on the clear separation of duties across three architectural layers. This model draws inspiration from established frameworks like the NIST AI Risk Management Framework and Zero Trust principles, applying them to the specific problem of endpoint AI control.

The Control Plane (AI Gateway): Where policy is defined.
The Enforcement Plane (Endpoint Agent): Where policy is applied.
The Management Plane (MDM): How the enforcement agent is deployed and managed.

This separation ensures that policy decisions are centralized and consistent, while enforcement is distributed and universally applied.

Plane 1: The Control Plane (The AI Gateway)

The control plane is the centralized policy engine for all AI traffic. This is typically an AI gateway, a specialized proxy that sits between AI applications and the various model providers (like OpenAI, Anthropic, or Google).

The gateway serves as the single point for defining and managing the rules of AI engagement for the entire organization. Its key responsibilities in this architecture include:

Unified Policy Management: Central definition of all AI policies, such as which models are approved, which users or teams have access, and what data is allowed.
Virtual Keys and Budgets: Instead of scattering provider API keys across applications, the gateway uses virtual keys to abstract them. These keys are tied to specific projects, teams, or users and can have granular budgets and rate limits attached.
Guardrails: The gateway inspects prompts and responses in real-time to block sensitive data (like PII or API keys) from leaving the organization and to prevent harmful or non-compliant content from being returned by the model.
Audit Logging: It creates an immutable, centralized log of all AI interactions, which is essential for security audits and compliance with regulations like SOC 2, HIPAA, or GDPR.

Platforms like Bifrost, which combine a gateway with an endpoint component, exemplify this model where the gateway acts as the "brain" of the operation.

Plane 2: The Enforcement Plane (The Endpoint Agent)

The enforcement plane's job is to ensure the policies defined in the control plane are applied to every AI tool running on a user's machine. This is accomplished by a lightweight, always-on agent that is installed on each company computer.

This agent is not a second policy engine. Its sole purpose is to intercept AI-related traffic on the device and route it through the central AI gateway (the control plane). This transparent routing is the mechanism that closes the shadow AI gap.

Key responsibilities of the enforcement agent include:

Application and Tool Discovery: The agent identifies all known AI desktop applications, browser-based AI tools, and CLI coding agents running on the machine. This provides administrators with a real-time inventory of AI usage across the fleet.
Transparent Traffic Routing: It automatically and transparently routes traffic from these applications to the organization's AI gateway. The user does not need to configure anything; the tools they already use are simply brought under governance.
Policy Enforcement: By forcing traffic through the gateway, the agent ensures all policies—budgets, guardrails, audit requirements—are applied, even to previously ungoverned tools.
Block/Allow Capabilities: Based on policy from the control plane, the agent can either route an application's traffic or block it entirely, preventing the use of unapproved AI tools on company devices.

This architecture ensures that governance follows the user and the application, regardless of how or where the AI tool is run.

Plane 3: The Management Plane (MDM Platform)

The management plane is the mechanism for deploying, configuring, and maintaining the enforcement agent across the entire fleet of company devices. This is the role of Mobile Device Management (MDM) or Unified Endpoint Management (UEM) platforms like Jamf, Microsoft Intune, or Kandji.

MDM platforms are critical for operationalizing endpoint governance at scale. Their responsibilities include:

Silent, Fleet-Wide Deployment: Pushing the enforcement agent to all managed devices without requiring any user interaction.
Managed Configuration: Securely delivering the initial configuration to the agent, such as the address of the organization's AI gateway. This avoids hardcoding sensitive information and allows for centralized updates.
Health and Status Monitoring: Reporting on the installation status and health of the enforcement agent across all devices.
Lifecycle Management: Handling updates and uninstallation of the agent as needed.

Using an MDM platform turns the endpoint agent from a tool that must be manually installed into a standard, non-bypassable component of the corporate software stack.

Putting It All Together: The Governed AI Request Flow

When these three planes work in concert, the flow for governing a request from an employee's laptop looks like this:

Deployment: The MDM platform (Management Plane) deploys the endpoint agent (Enforcement Plane) to a new company laptop with a managed configuration pointing it to the corporate AI gateway (Control Plane).
User Action: An employee opens an AI tool on their laptop—for example, the Claude Desktop app.
Interception & Routing: The endpoint agent detects the network request from the app, intercepts it, and securely routes it to the AI gateway.
Policy Adjudication: The AI gateway receives the request. It checks the user's identity, attaches the correct virtual key, verifies the request against assigned budgets and guardrails, and logs the entire transaction.
Execution: If the request is compliant, the gateway forwards it to the upstream LLM provider. The response is routed back through the gateway, inspected by guardrails again, and then sent to the user's application.

This architecture creates a system where policy is managed centrally, but enforcement is everywhere. It makes AI usage both safe and visible, eliminating the risks of shadow AI without having to block tools that make employees more productive.

Sources

The Endpoint Is the New Perimeter for AI Governance

James Whitfield — Wed, 24 Jun 2026 18:05:58 +0000

With AI applications running on every device, traditional network security has become insufficient for governance. Effective AI risk management requires extending policies to the endpoint, which tools like Bifrost achieve by combining a central gateway with an endpoint agent.

The adoption of generative AI tools like ChatGPT, Claude Desktop, and various coding assistants is happening faster than most IT and security teams can track. Employees, seeking to be more productive, are connecting to these tools from their work laptops, often using unapproved applications and personal accounts. This trend, known as "shadow AI," creates a significant governance blind spot. When sensitive company data is entered into these unsanctioned tools, it bypasses all traditional security controls, exposing organizations to data leakage, compliance violations, and intellectual property loss.

For decades, cybersecurity has been centered on the network perimeter. This model is breaking down. With remote work, cloud services, and now AI applications, the perimeter has dissolved and reformed around each individual device. To effectively govern AI, security and platform teams must adopt a new approach that treats the endpoint as the true perimeter. One of the tools built to address this is Bifrost, an open-source AI gateway that can be extended with an endpoint agent to enforce central policies on every machine.

Why the Network Perimeter Fails for Modern AI Workflows

Traditional security tools like firewalls and network-based Data Loss Prevention (DLP) systems were designed for a world where corporate data stayed within a defined corporate network. Modern AI usage patterns make this model obsolete:

Direct-to-Cloud Connections: Desktop applications and browser-based AI tools connect directly to cloud providers, bypassing on-premise network monitoring.
Encrypted Traffic: Nearly all AI traffic is encrypted with TLS, making it opaque to passive network inspection without complex and often brittle "man-in-the-middle" decryption.
Dynamic Endpoints: Employees work from various locations and networks, meaning their devices are frequently outside the corporate network perimeter where policies could be applied.

This mismatch leaves security teams unable to answer basic questions: Which AI tools are employees using? What data is being sent to them? Are we complying with regulations like GDPR or SOC 2? This lack of visibility is not just a policy gap; it's a critical, unmanaged risk.

What is Endpoint AI Governance?

Endpoint AI governance shifts the point of control from the network to the device itself. Instead of trying to inspect traffic as it crosses a central point, policies are enforced directly on the laptops and workstations employees use every day. This approach aligns with the principles of Zero Trust architecture, which operates on the philosophy of "never trust, always verify" for every user and device, regardless of its location.

By managing AI usage at its source, organizations can gain visibility and enforce rules consistently, whether an employee is in the office, at home, or connected to public Wi-Fi.

How Endpoint Governance Works: The Gateway and the Agent

A complete endpoint governance solution consists of two integrated components: a central policy engine and a distributed enforcement agent.

The AI Gateway as the Central Control Plane

The foundation of this model is an AI gateway like Bifrost. The gateway serves as the single control plane where all governance policies are defined. This is where administrators configure the rules of AI engagement for the entire organization, including:

Virtual Keys: Creating distinct access credentials for different users, teams, or projects.
Budgets and Rate Limits: Controlling costs and preventing abuse.
Routing Rules: Directing traffic to approved models and providers.
Guardrails: Implementing security policies, such as detecting and redacting secrets or PII.

This centralized management ensures policies are consistent and easy to update. However, the gateway can only enforce these policies on traffic that is explicitly directed to it.

The Endpoint Agent Extends Governance Everywhere

The second component is an endpoint agent, such as Bifrost Edge, which is installed on each employee's machine. This agent works transparently in the background to intercept all AI-related traffic from any application and route it through the organization's central AI gateway.

This combination is powerful. It means the same robust governance and security controls defined in the Bifrost gateway are applied to all AI traffic automatically. Beyond routing, Bifrost applies security controls (virtual keys, budgets, guardrails, audit logs) centrally, and Bifrost Edge extends that same governance and security to AI traffic on employee machines, with endpoint enforcement on each device. This closes the shadow AI gap without requiring users to change their behavior or reconfigure their favorite tools.

Key Capabilities of an Endpoint AI Governance Solution

When evaluating solutions for endpoint AI governance, organizations should look for a platform like the Bifrost AI gateway with its Edge component that provides comprehensive visibility and control.

Visibility and Control over AI Applications

The first step in governing shadow AI is seeing it. An endpoint agent can inventory every AI application installed on the company's fleet of devices. This data feeds into a central dashboard where administrators can see which tools are being used, by whom, and how often. Based on this visibility, they can create and enforce policies to allow approved applications and block unsanctioned ones directly on the device.

MCP Server Discovery and Governance

Modern AI agents and coding tools often connect to Model Context Protocol (MCP) servers to interact with external tools and data sources. These servers represent another vector for ungoverned data exchange. An endpoint solution should provide MCP server discovery and governance, giving security teams a full inventory of these connections and the ability to allow or deny them based on corporate policy.

Fleet-Wide Deployment and Management

Manually installing and configuring agents across hundreds or thousands of devices is not scalable. A true enterprise solution must support silent, large-scale deployment through Mobile Device Management (MDM) platforms. Look for support for tools like Jamf, Microsoft Intune, Kandji, and Workspace ONE, which allow for automated, policy-driven rollouts across an entire fleet of macOS, Windows, and Linux machines.

The Business Impact of Endpoint-First AI Governance

Adopting an endpoint-first approach to AI governance allows organizations to move from a reactive to a proactive security posture. It enables teams to embrace the productivity benefits of AI tools while mitigating the associated risks. The result is a framework that supports secure innovation, provides a clear audit trail for compliance, and protects the organization's most sensitive data, no matter where its employees work.

Next Steps for Securing AI

As AI becomes more integrated into daily workflows, the risks associated with ungoverned use will only grow. The traditional network perimeter is no longer a reliable line of defense. By shifting focus to the endpoint, organizations can build a more resilient and comprehensive governance strategy. Teams evaluating how to secure their AI usage can request a Bifrost demo to see how a combined gateway and endpoint solution works, or review the open-source repository to explore the core technology.

Sources

Endpoint AI Governance vs. Network Controls: What Each Can and Can't Do

James Whitfield — Wed, 24 Jun 2026 18:04:39 +0000

A comparison of network-level and device-level controls for governing AI. Traditional network security is blind to encrypted, semantic threats, making endpoint governance a required layer for visibility into how AI is actually used.

The rapid adoption of generative AI has created a significant blind spot for enterprise security and IT teams: shadow AI. When employees use unapproved AI tools—from desktop clients like Claude and ChatGPT to coding agents in an IDE—they operate outside of established corporate controls. This unsanctioned usage creates risks of data leakage, compliance violations, and uncontrolled costs. Traditional network controls, like firewalls and proxies, were not designed to manage this new traffic, prompting a shift toward endpoint-first governance.

This article compares endpoint AI governance with traditional network controls, examining what each can and cannot do. It explains why network-level visibility is insufficient for modern AI applications and how an AI gateway combined with an endpoint agent provides a more complete solution. An open-source AI gateway like Bifrost acts as a central policy engine, while an endpoint agent extends those policies to the device where AI is actually used.

What are Traditional Network Controls?

Network controls are security measures deployed at the boundaries of a corporate network. They are designed to monitor and filter traffic moving between an organization's internal network and the public internet. For decades, these have been the foundation of enterprise security.

Key types of network controls include:

Firewalls: These inspect network traffic based on predefined rules, blocking or allowing packets based on IP addresses, ports, and protocols. While modern next-generation firewalls (NGFWs) have more advanced capabilities, their focus remains on network-level data.
Web Proxies/Secure Web Gateways (SWGs): These act as intermediaries for internet-bound traffic, enforcing acceptable use policies and filtering for malicious content. They can block access to entire domains, such as known AI service websites.
Data Loss Prevention (DLP): Network DLP solutions inspect outbound traffic for sensitive data patterns, such as credit card numbers or social security numbers, and can block transmissions that violate policy.
Cloud Access Security Brokers (CASBs): These tools provide visibility and control over an organization's use of cloud services, enforcing security policies as users access cloud-based applications.

Where Network Controls Fall Short with AI Traffic

The architecture of AI applications fundamentally breaks the assumptions that underpin traditional network security. AI traffic is encrypted, contextual, and increasingly initiated by applications that run entirely on the endpoint, bypassing network inspection points.

1. Blindness to Encrypted Traffic

Nearly all communication with AI services uses TLS 1.3 or other modern encryption protocols. This means that network-level controls like firewalls and legacy DLP systems cannot inspect the content of the prompts or the responses. They can see that a connection was made to api.openai.com, but have no visibility into what was sent. While some organizations use TLS inspection (SSL decryption), this practice is becoming less effective due to techniques like certificate pinning and can introduce performance and privacy issues.

2. Lack of Semantic Understanding

Traditional security tools are built to analyze structured data and network packets, not the semantic meaning of natural language. A network firewall cannot understand the intent behind a prompt. For example, these two prompts are identical from a network perspective, but represent entirely different levels of risk:

"Summarize the key ideas in the latest public earnings report." (Benign)
"Summarize this internal M&A document and draft a public press release." (High Risk)

A network DLP solution might catch a specific keyword or pattern, but it cannot interpret the context that makes the second prompt a serious data leak risk.

3. The Endpoint Blind Spot: Desktop, CLI, and Local Models

The biggest gap for network controls is the rise of AI tools that run directly on employee machines. This includes:

Desktop Apps: The official ChatGPT and Claude applications.
IDE and CLI Tools: GitHub Copilot, Codex CLI, and other coding assistants.
Local Models: Open-source models running via tools like Ollama.

Traffic from these applications may never traverse the corporate network perimeter in a way that allows for inspection. An employee on a home network using a desktop AI app is completely invisible to a corporate firewall. This "shadow AI" usage accounts for a growing volume of enterprise AI interactions.

4. Inability to Govern Agentic Workflows

Modern AI is moving beyond simple request-response interactions to autonomous agents that can execute multi-step tasks. These agents can access local files, call external APIs, and interact with other applications on the endpoint. Network controls have no visibility into these on-device actions and cannot distinguish between a sanctioned workflow and a compromised agent exfiltrating data.

What is Endpoint AI Governance?

Endpoint AI governance shifts the control plane from the network perimeter to the device itself. It uses a lightweight agent installed on each machine (laptop, desktop) to monitor and manage AI usage at the source. This approach provides visibility into all forms of AI activity, regardless of the application, network, or user location.

How Endpoint Agents Work

An endpoint agent for AI governance typically performs several key functions:

Application and Process Discovery: It identifies every AI application running on the device, including desktop clients, CLI tools, and even custom scripts making calls to AI models.
Traffic Interception: The agent transparently routes all AI-related traffic from the device to a central control plane, like an AI gateway, before it goes to the external AI provider. This works for all apps without requiring individual configuration.
Policy Enforcement: It enforces centralized policies on the device. For instance, it can block the use of unapproved applications or prevent sensitive data from being sent in a prompt.
Contextual Visibility: It captures the full context of an interaction, including the user, the application used, the content of the prompt, and the model's response.

The Combined Solution: AI Gateway + Bifrost Edge

Neither network controls nor endpoint governance alone provides a complete solution. A truly effective strategy combines a centralized policy engine with endpoint enforcement.

This is the model used by platforms like Bifrost.

The AI Gateway as the Control Plane: The Bifrost AI gateway sits in an organization's infrastructure and serves as the central point for defining all AI policies. This is where administrators configure virtual keys, set budgets and rate limits, enable guardrails for content filtering, and manage routing across different AI providers. The gateway ensures all known traffic is governed.
Bifrost Edge for Endpoint Enforcement: Bifrost Edge is the lightweight agent deployed to every employee machine. It automatically and transparently intercepts all AI traffic—from desktop apps, browsers, and CLIs—and routes it through the organization's Bifrost gateway. This closes the "shadow AI" gap by ensuring that the policies defined in the gateway are applied to the AI tools employees actually use, with no per-app setup needed.

This two-layer approach offers several advantages:

Complete Visibility: See all AI usage, whether from sanctioned server-side applications or unsanctioned desktop tools.
Consistent Policy: A single set of rules for access, security, and cost management applies everywhere.
User Transparency: Developers and other employees can continue using their preferred tools without changing workflows, while the organization maintains governance.

Conclusion: Control Must Follow the Data

Traditional network controls are essential for general cybersecurity but are ill-equipped to govern modern AI. Their inability to inspect encrypted traffic, understand semantic context, and see activity on the endpoint renders them ineffective against the primary risks of shadow AI.

Endpoint AI governance provides the necessary visibility and control where AI usage actually occurs: on the device. By pairing an endpoint agent like Bifrost Edge with a central policy engine like the Bifrost AI gateway, organizations can enforce consistent security, compliance, and cost policies across every application and every user. For teams looking to get a handle on AI risk, the endpoint is the new perimeter.

Sources

What a Device-Level AI Control Agent Actually Does

James Whitfield — Wed, 24 Jun 2026 18:04:34 +0000

As AI tools become standard in the workplace, organizations face a critical governance gap. A device-level AI control agent closes this gap by extending security and management policies directly to the endpoint where work happens.

The proliferation of AI tools like ChatGPT, Claude, and GitHub Copilot has created a significant challenge for IT and security teams: employees are using powerful, cloud-based AI on company devices with little to no oversight. This phenomenon, often called "Shadow AI," introduces risks ranging from data leakage of sensitive intellectual property to non-compliance with regulations like GDPR and HIPAA. While some organizations attempt to block these tools outright, a more effective long-term strategy involves governing their use. This is the role of a device-level AI control agent.

The Problem: Shadow AI and the Governance Gap

An AI gateway is a common piece of infrastructure used to manage access to large language models (LLMs). It can enforce budgets, apply security guardrails, and audit requests. However, a gateway can only control traffic that is explicitly configured to pass through it. It has a fundamental blind spot: the endpoint.

Employees often use AI in ways that bypass centralized gateways:

Web-based AI: Using services like chatgpt.com or claude.ai directly in the browser.
Desktop Applications: Installing native clients like Claude Desktop or the ChatGPT app.
Developer Tools: Integrating AI agents directly into an IDE or command-line interface (CLI).

This ungoverned usage means that prompts containing proprietary code, customer data, or strategic plans can be sent to third-party models without any record or control, creating a serious security risk. A 2023 report by Cisco highlighted that company data is being entered into generative AI applications, and many organizations lack adequate controls to monitor this.

What Is a Device-Level AI Agent?

A device-level AI control agent is a piece of software installed directly on an employee's computer (macOS, Windows, or Linux) that monitors and controls AI-related activity at the source. It acts as a local enforcement point for the organization's centralized AI policies.

Instead of being blind to endpoint activity, the agent gives administrators two core capabilities:

Visibility: It discovers and inventories all AI applications and services being used across the fleet of managed devices.
Control: It enforces policies on that usage, such as blocking unauthorized applications or routing traffic from approved tools through the company's secure AI gateway.

This approach shifts the control plane from a purely network-based solution to one that is endpoint-aware, closing the governance gap left by shadow AI.

How It Works: The Technical Mechanics

Device-level agents typically work by intercepting network traffic locally on the machine. This is often accomplished by installing a trusted root certificate and establishing a local proxy or network filter that can inspect and reroute traffic before it leaves the device.

The workflow generally follows these steps:

Deployment: The agent is deployed silently to all company devices using a Mobile Device Management (MDM) platform like Jamf, Microsoft Intune, or Kandji.
Policy Sync: Once installed, the agent authenticates the user and machine and syncs the latest AI governance policies from a central control plane (often the organization's AI gateway).
Traffic Interception: The agent monitors network connections originating from the device. When it identifies traffic destined for a known AI service, it intercepts the request.
Policy Enforcement: The agent checks the request against its local policy cache.
- If the application is disallowed, the connection is blocked on the device.
- If the application is allowed, the agent transparently reroutes the traffic to the company's secure AI gateway. The gateway then applies its own fine-grained policies (e.g., budget checks, rate limits, PII redaction) before forwarding the request to the LLM provider.
Auditing: All actions—both allowed and blocked—are logged for security and compliance auditing.

This ensures that all AI usage, whether from a browser or a desktop app, is subject to the same set of corporate rules.

Key Capabilities of Modern Agents

When evaluating device-level AI control solutions, organizations typically look for a few key features:

Cross-Platform Support: The agent must run natively on all operating systems used by the company (macOS, Windows, Linux).
Broad Application Coverage: It should be able to identify and control a wide range of AI tools, including web, desktop, and CLI agents.
Centralized Management: The ability to define policies once and have them enforced across the entire fleet of devices.
MDM Integration: Seamless deployment and configuration through existing enterprise device management systems is essential for large-scale rollouts.
Low Performance Overhead: The agent must be lightweight and not noticeably impact the user's device performance or network speed.
User Transparency: The agent should provide clear feedback to the user when an action is blocked due to policy, avoiding confusion and helpdesk tickets.

As enterprise AI adoption matures, managing usage at the endpoint is no longer optional. A device-level control agent provides the necessary visibility and enforcement to allow employees to use AI tools productively while protecting the organization's sensitive data.

Endpoint AI Governance: How to Bring Every Laptop App Under Company Control

James Whitfield — Wed, 24 Jun 2026 18:03:50 +0000

Bringing AI applications on employee laptops under corporate governance requires a new approach that combines a central policy engine with an endpoint agent. Tools like Bifrost and its Edge component provide a blueprint for enforcing security, compliance, and cost controls on the AI people actually use.

The rapid adoption of desktop AI applications and coding assistants has created a significant governance gap in most organizations. Employees use tools like ChatGPT, Claude Desktop, and Cursor to improve productivity, but this activity often happens outside of established security and compliance frameworks. This phenomenon, known as "shadow AI," introduces substantial risk, including the potential for data leaks, compliance violations, and intellectual property loss. To solve this, engineering and security leaders need a mechanism to extend corporate governance to the endpoint. An emerging and effective model combines a centralized AI gateway like Bifrost, an open-source AI gateway from Maxim AI, with an endpoint agent that enforces the gateway's policies directly on each laptop.

The Challenge of Shadow AI

Shadow AI refers to the use of artificial intelligence tools by employees without the knowledge or approval of their IT and security departments. It's a modern variant of shadow IT, but with unique risks tied to how AI models process data. When an employee pastes proprietary source code into a public web-based chatbot or uses a desktop AI app to summarize a confidential document, that data leaves the organization's control. According to Gartner, this trend is expanding the corporate attack surface in ways that traditional security tools were not designed to handle.

The core challenges include:

Lack of Visibility: Security teams cannot see which AI tools are in use, what data is being shared, or how frequently. This makes it impossible to assess risk or apply consistent policy.
Data Exfiltration: Sensitive information, from customer data to unreleased financial reports and source code, can be inadvertently uploaded to third-party AI models, where it may be stored or used for training.
Compliance Violations: Ungoverned AI usage can violate regulations like GDPR, HIPAA, or the EU AI Act, which hold the organization responsible for how personal and sensitive data is processed.
Inconsistent Security: Without a central point of control, there is no way to enforce security measures like data redaction, access controls, or audit logging for AI traffic originating from employee laptops.

Common Approaches to Endpoint Control (and Their Limits)

Organizations traditionally rely on a few methods to control application usage on corporate devices, but these often fall short when applied to the dynamic nature of AI tools.

Acceptable Use Policies (AUPs): Policies are a necessary foundation, but they are not a technical control. They rely on employee awareness and compliance, and they provide no visibility or enforcement mechanism.
Network-Level Blocking: Using firewalls or proxies to block domains for known AI services can be a blunt instrument. It can hinder productivity and is easily bypassed with VPNs or by using desktop applications that communicate over standard encrypted channels.
Mobile Device Management (MDM): MDM platforms like Microsoft Intune and Jamf are powerful tools for managing device fleets. They can be used to deploy software and enforce configurations, including blocking specific applications. However, MDM on its own is often focused on application inventory and blocklists, not on inspecting and governing the content of the API traffic flowing from those applications.

While these methods have a role, they don't solve the core problem: how to allow productive use of approved AI tools while ensuring all traffic adheres to company-wide security and compliance rules.

A Modern Approach: AI Gateway + Endpoint Agent

A more effective strategy for endpoint AI governance pairs a centralized AI gateway with a lightweight agent deployed on each machine. This model separates policy management from enforcement, creating a robust and scalable solution.

The Bifrost AI gateway serves as the central control plane. Here, administrators configure all governance policies:

Virtual Keys: Instead of managing raw API keys, teams create virtual keys that control access, assign budgets, and set rate limits per user, team, or project.
Guardrails: The gateway inspects prompts and responses, using tools like secrets detection or custom rules to prevent sensitive data from reaching an external model.
Routing and Fallbacks: Policies can route requests to specific models or providers and configure automatic fallbacks to maintain availability during an outage.
Audit Logs: All AI requests are centrally logged, creating an immutable record for compliance and security reviews.

Bifrost Edge, the endpoint agent, extends these gateway policies to every corporate laptop. It runs on macOS, Windows, and Linux, routing all detected AI traffic from desktop apps, browsers, and coding agents through the Bifrost gateway automatically.

How It Works: Centralized Policy, Local Enforcement

The "AI Gateway + Bifrost Edge" model provides a clear workflow for bringing endpoint AI under governance.

Policy Configuration: Admins define all security, access, and cost-control policies in the central Bifrost gateway. This includes setting up guardrail profiles and virtual keys for different teams.
Endpoint Deployment: The Bifrost Edge agent is deployed to all laptops, typically via an MDM solution. This rollout is silent to the end-user.
Automatic Routing: Once installed, Edge identifies AI traffic from supported applications and transparently routes it through the company's Bifrost instance. No manual configuration is needed in the AI apps themselves.
Policy Enforcement: Because all traffic now flows through the gateway, every request is subject to the centrally defined policies. A developer using Claude Code in their terminal is governed by the same rules as a production service using the gateway's API. This includes Bifrost's governance and security controls, with Bifrost Edge extending that same endpoint enforcement to each device.

Governing AI Apps and MCP Servers

Endpoint AI governance goes beyond just LLM API calls. Modern coding assistants increasingly use the Model Context Protocol (MCP) to interact with local and remote tools. An effective governance solution must see and control this layer as well.

Bifrost Edge provides deep visibility into the AI ecosystem on each device. The platform inventories all installed AI applications and configured MCP servers across the fleet. From a central dashboard, administrators can then make allow/deny decisions.

App Governance: Admins can approve specific applications for use and block others. This application policy is enforced directly on the device by Edge.
MCP Governance: Edge discovers which MCP servers are configured in tools like Cursor or Claude Code. Admins can then create an allowlist of approved servers, blocking potentially risky or unknown tool providers. This capability is critical for preventing data exfiltration through agentic workflows.

Fleet-Wide Deployment with MDM

Rolling out an endpoint agent across hundreds or thousands of devices requires integration with existing IT infrastructure. Bifrost Edge is designed for managed deployment through standard MDM platforms like Jamf, Microsoft Intune, Kandji, and others.

The MDM platform pushes the Edge agent and a managed configuration file to each device. This ensures that every agent is pre-configured to connect to the organization's Bifrost gateway. The user experience is seamless: after a one-time single sign-on (SSO) authentication, the agent runs in the background, providing continuous governance without interrupting workflows.

Getting Started with Endpoint AI Governance

Bringing all AI activity under a unified governance framework is becoming a critical function for security and engineering teams. The combination of a central AI gateway and an endpoint agent provides a scalable and effective solution to the challenge of shadow AI. By enforcing policies at the point of use, organizations can enable employees to leverage powerful AI tools while protecting sensitive data and maintaining compliance.

Teams evaluating solutions for endpoint AI governance can request a Bifrost demo to see how the gateway and Edge agent work together, or review the open-source repository to explore the gateway's core capabilities.

Sources

Device-Level AI Policy Enforcement Explained for Security Teams

James Whitfield — Wed, 24 Jun 2026 18:03:39 +0000

Device-level AI policy enforcement extends centralized security and governance controls to the endpoint, closing the visibility gap created by shadow AI. For security teams, tools like Bifrost Edge provide the necessary mechanisms to discover, govern, and secure AI usage on every company machine.

The adoption of generative AI has introduced a significant security blind spot for most organizations: the endpoint. While security teams focus on governing AI traffic from production applications, employees are independently using desktop AI clients, browser-based AI tools, and coding agents that bypass centralized controls. This ungoverned usage, often called "shadow AI," creates unmonitored pathways for data exfiltration and introduces serious compliance risks. To address this, organizations are turning to device-level AI policy enforcement, a strategy that extends security policies directly to every employee's computer. One of the leading tools in this space is Bifrost, an open-source AI gateway from Maxim AI, which uses its Bifrost Edge component to apply gateway-level policies on the endpoint.

The Security Risk of Ungoverned Endpoint AI

When employees use tools like ChatGPT, Claude Desktop, or integrated coding agents without oversight, they operate outside the organization's security perimeter. This creates several critical risks:

Data Leakage: Sensitive information, such as intellectual property, customer data, or PII, can be inadvertently pasted into prompts, with no mechanism to prevent or audit its transmission to third-party AI providers.
Compliance Violations: For organizations subject to regulations like SOC 2, HIPAA, or GDPR, the absence of audit logs and access controls for AI usage on endpoints can lead to non-compliance and significant penalties.
Lack of Visibility: Security teams cannot answer fundamental questions about AI usage across the company: Which AI tools are in use? Who is using them? What kind of data is being submitted?
Inconsistent Security: Policies applied to official, sanctioned AI applications are rendered ineffective if employees can simply use an ungoverned alternative on their local machine.

What is Device-Level AI Policy Enforcement?

Device-level AI policy enforcement is the practice of installing a lightweight agent on endpoint devices (laptops, desktops) to intercept and manage all outbound AI traffic. Instead of relying on application-level SDKs or network-level proxies, this approach enforces policy directly on the machine where the AI interaction originates.

The core principle is to extend the same robust governance framework used for production AI traffic to the endpoint. This is achieved through a combined-platform approach. A central AI gateway like Bifrost serves as the control plane for defining all security and governance policies. An endpoint agent, Bifrost Edge, is then deployed to the fleet of devices, where it enforces those central policies transparently.

How Endpoint AI Governance Works

Implementing AI governance at the device level involves a few key components working in concert. The process is designed to be largely invisible to the end-user while providing comprehensive control for security administrators.

The Endpoint Agent

The foundation of this model is a lightweight agent that runs natively on macOS, Windows, and Linux. This agent is installed on every company-managed machine, typically as part of a standard device build. Its primary job is to monitor for and intercept traffic directed at known AI services and models. The Bifrost Edge agent, for example, detects traffic from a wide range of supported applications, including desktop clients, browser-based AI, and developer-focused coding agents.

Centralized Policy Engine

The endpoint agent does not make policy decisions on its own. It connects to a central AI gateway, which acts as the policy engine. Administrators use the gateway's interface to configure all governance rules:

Virtual Keys: Access is managed through virtual keys, which associate usage with specific users, teams, or projects.
Access Control: Administrators can create allow and deny lists for specific AI applications or the powerful external tools they connect to via the Model Context Protocol (MCP). Bifrost provides dedicated controls for both app governance and MCP governance.
Guardrails: Data loss prevention is handled by guardrails that can detect and redact secrets, PII, or other sensitive patterns before a prompt leaves the device.
Budgets and Rate Limits: Cost controls and usage limits are defined centrally and enforced for every request, regardless of its origin.

Transparent Traffic Routing

Once a policy is synced from the gateway, the agent begins enforcing it. When a user attempts to use an AI tool, the agent intercepts the outbound request. If the application and the request content comply with policy, the agent securely routes the traffic through the organization's central AI gateway. If the application is denied or the prompt violates a guardrail, the request is blocked on the device before any data is transmitted. This entire process happens transparently, requiring no changes to the user's workflow or applications.

Key Security Benefits of Endpoint Enforcement

Adopting a device-level enforcement strategy provides immediate and tangible security benefits. It moves AI governance from a reactive, partial solution to a proactive, comprehensive one.

Complete Visibility: The system creates a real-time inventory of all AI applications and MCP servers being used across the organization. Security teams finally have a definitive dataset to understand the company's AI footprint.
Mitigation of Shadow AI: By enforcing allow/deny lists, organizations can standardize on approved AI tools and block the use of unsanctioned or high-risk applications, effectively eliminating shadow AI.
Consistent Data Protection: The same security guardrails protecting production AI traffic are extended to every endpoint. This ensures that policies for redacting sensitive data are applied universally.
Comprehensive Audit Trails: Every AI prompt and response from an endpoint is logged centrally. This provides the immutable audit trail required for security investigations and compliance with standards like SOC 2.

Integration with Existing Security Infrastructure

A critical feature of any enterprise-grade security solution is its ability to integrate with existing operational workflows. Modern endpoint AI governance platforms are designed for this. Agents like Bifrost Edge are deployed and managed using standard Mobile Device Management (MDM) tools.

This allows for seamless, silent rollout across an entire fleet of devices using platforms that IT and security teams already manage, such as:

Jamf
Microsoft Intune
Kandji
Omnissa Workspace ONE
JumpCloud

This MDM-native deployment model means that device-level AI governance becomes a scalable and manageable component of an organization's broader endpoint security strategy, not a siloed tool requiring separate processes.

Conclusion: Securing the Last Mile of AI

As AI becomes more integrated into daily workflows through a diverse set of endpoint tools, securing only the infrastructure layer is no longer sufficient. Device-level AI policy enforcement closes the "last mile" security gap, bringing the ungoverned world of shadow AI under centralized control. By combining a powerful AI gateway as a policy engine with a transparent endpoint agent, security teams can gain full visibility, enforce consistent data protection, and ensure compliance across every application on every device.

Security teams evaluating solutions in this space can request a Bifrost demo to see its endpoint governance capabilities or review the open-source repository to understand the underlying gateway technology.

Why Network-Level AI Filtering Is Not Real AI Control

James Whitfield — Wed, 24 Jun 2026 18:02:49 +0000

Many organizations attempt to govern AI usage with network-level filtering, but modern encryption and AI-specific protocols render this approach ineffective. A dedicated AI gateway like Bifrost combined with endpoint visibility provides the deep control required for genuine security and governance.

As organizations grapple with the rapid adoption of AI tools, a common first instinct is to control their use at the network perimeter. The thinking is straightforward: if all traffic passes through a central firewall or proxy, it should be possible to inspect that traffic, identify requests to AI services, and block them. However, this approach, which relies on traditional network-level filtering, consistently falls short of providing meaningful AI control. Modern application protocols and the specific nature of AI traffic make it nearly impossible for network devices to see, understand, or act on AI requests with any reliability.

This analysis examines the technical limitations of network-level AI filtering and contrasts it with a purpose-built approach. Solutions like Bifrost, an open-source AI gateway, combined with endpoint agents, offer a more robust method for governing AI by operating at the correct layer of the technology stack.

The Limits of Network-Level Inspection

Network security appliances like firewalls and secure web gateways have historically relied on techniques such as deep packet inspection (DPI) to analyze and filter traffic. These tools inspect the content of network packets to identify signatures, keywords, or destinations associated with prohibited applications. While effective for simpler protocols of the past, this method is fundamentally breaking down against modern, encrypted web traffic.

Challenge 1: Pervasive Encryption (TLS 1.3)

The vast majority of web traffic, including virtually all API calls to AI providers like OpenAI and Anthropic, is encrypted with Transport Layer Security (TLS). The current standard, TLS 1.3, significantly enhances privacy and security by encrypting more of the handshake process than its predecessors.

A key change in TLS 1.3 is the encryption of the server certificate, which means that even the Server Name Indication (SNI) extension, previously a reliable indicator of the destination hostname, can be hidden through Encrypted Client Hello (ECH). As ECH adoption grows, network devices lose the ability to reliably determine which service a user is connecting to without decrypting the traffic.

Decrypting TLS traffic, often called "TLS inspection" or "man-in-the-middle" interception, is a resource-intensive process with significant drawbacks:

Performance Degradation: Decrypting and re-encrypting every connection adds substantial latency and requires powerful, expensive hardware.
Security Risks: It weakens end-to-end security by creating a central point of failure and can break applications that use certificate pinning for security.
Maintenance Overhead: Managing the trusted certificates required for interception across an entire organization is a complex operational burden.

Without full TLS decryption, a network filter is mostly blind. It can see an encrypted connection to an IP address owned by a major cloud provider like AWS or Google Cloud, but it cannot know if that connection is for an AI service or any of the thousands of other services hosted on that same infrastructure.

Challenge 2: The Rise of HTTP/3 and QUIC

The web is also shifting from TCP to the QUIC protocol, which underpins HTTP/3. Unlike TCP, which can be inspected at the transport layer, QUIC is encrypted by default. Its headers, which contain critical session information, are part of the encrypted payload. This design choice was made explicitly to prevent network middleboxes from interfering with the protocol's evolution and to enhance user privacy.

As a result, network appliances that rely on analyzing TCP headers are completely unable to interpret QUIC traffic. They cannot distinguish between different streams within a connection or identify the application-layer protocol being used. For a network filter, a QUIC connection is an opaque, encrypted UDP stream. According to Google, over 60% of their client traffic is already over QUIC, and its adoption is accelerating across the web. This shift renders signature-based network filtering increasingly obsolete.

Challenge 3: Lack of AI-Specific Context

Even if a network device could decrypt and inspect the traffic, it lacks the context to make intelligent governance decisions. An AI gateway operates at the application layer and understands the structure of an AI request. A network firewall does not.

Consider these critical governance questions that are impossible to answer at the network level:

Model Usage: Is this request for gpt-4o or a much cheaper, less capable model?
Prompt Content: Does the prompt contain personally identifiable information (PII), credentials, or intellectual property?
Budget Control: Has this specific user or project team exceeded its allocated budget for the month?
Tool Usage: Is the AI agent attempting to use a high-risk external tool via Model Context Protocol (MCP)?
Failover Logic: If a request to Anthropic's Claude 3.5 Sonnet fails, should it be retried on Google's Gemini 1.5 Pro?

A network filter sees a stream of text going to an endpoint. It cannot parse the JSON payload to identify the model, analyze the prompt for sensitive data with guardrails, or associate the request with a specific virtual key to enforce a budget. This lack of context makes granular control impossible.

A Better Approach: AI-Native Governance

Effective AI governance requires moving control from the network perimeter to a dedicated layer that understands AI traffic. This is the role of an AI gateway like Bifrost, which acts as a central control plane for all AI requests.

An AI gateway is a specialized service that sits between applications and AI providers. Because it operates at the application layer, it can terminate the TLS connection cleanly and analyze the full request payload. This enables a level of control that network filtering cannot achieve.

Key Capabilities of an AI Gateway

Granular Routing: Direct requests to specific models or providers based on payload content, user identity, or other metadata using advanced routing rules.
Budget and Rate Limit Enforcement: Apply fine-grained budgets and rate limits to users, teams, or projects.
Content Guardrails: Inspect prompts and responses for sensitive data, secrets, or policy violations before they reach the model or the user.
Automatic Failover: Ensure application reliability by automatically rerouting requests when a primary provider experiences an outage or performance degradation.
Unified Observability: Maintain detailed audit logs of every request, including prompt content, model used, cost, and latency, for compliance and analysis.
Semantic Caching: Reduce costs and improve latency by caching responses to semantically similar queries, a concept completely foreign to network-level tools.

Closing the Loop with Endpoint Control

An AI gateway governs traffic that is configured to pass through it. However, a significant amount of AI usage happens directly on employee machines through desktop apps like Claude Desktop, browser-based tools like ChatGPT, and coding agents in the terminal. This "shadow AI" bypasses any central gateway.

This is where an endpoint agent like Bifrost Edge becomes essential. Edge is a lightweight agent deployed on employee machines that automatically routes all AI traffic through the organization's central Bifrost AI gateway. This combined approach ensures that the same governance, security, and observability policies apply everywhere, whether the AI request originates from a production server or a developer's laptop.

With Bifrost Edge, organizations can discover which AI apps and MCP servers are in use across the fleet, and enforce centralized allow/deny policies on the device itself. This extends the gateway's control to the last mile, providing comprehensive visibility and enforcement that network-level filtering cannot match.

Conclusion: Use the Right Tool for the Job

Attempting to control modern AI applications with traditional network filters is an exercise in futility. The convergence of strong encryption, modern protocols like QUIC, and the need for application-specific context renders network-level inspection ineffective. It provides a false sense of security while failing to address the real risks of ungoverned AI usage.

A purpose-built AI gateway offers a superior solution by operating at the correct layer of abstraction. It understands the nuances of AI requests and provides the granular control over routing, security, and costs that organizations require. When combined with an endpoint agent to govern shadow AI, this approach delivers a complete, reliable, and future-proof platform for AI governance. Teams evaluating AI control solutions can request a Bifrost demo or explore the open-source repository to learn more.

Sources

How Encrypted Client Hello (ECH) Works - Cloudflare
QUIC Protocol Specification (RFC 9000) - Internet Engineering Task Force (IETF)
Google's Adoption of QUIC - The Chromium Projects
An Overview of TLS 1.3 and its Advantages - SSL.com

What Endpoint AI Governance Is and Why It Matters in 2026

James Whitfield — Wed, 24 Jun 2026 18:02:44 +0000

Endpoint AI governance is the practice of monitoring and controlling the AI tools employees use directly on their devices. With the rise of "shadow AI," where unapproved tools access sensitive data, governing the endpoint has become a critical security and compliance function for enterprises in 2026.

The use of AI in the enterprise is no longer centralized. Employees across every department use a mix of sanctioned and unsanctioned AI tools on their company laptops to analyze data, write code, and draft documents. While these tools increase productivity, they also create a significant blind spot for security and compliance teams known as "shadow AI." This is the use of AI systems outside of established organizational controls, and it’s why endpoint AI governance has shifted from a niche concept to an operational necessity.

Recent industry data highlights the scale of the problem. According to one 2026 report, 91% of AI tools used in enterprises are unmanaged by IT. Another notes that 80% of organizations report moderate to pervasive shadow AI use. This ungoverned activity creates substantial risk, including data leakage, compliance violations, and intellectual property loss. Endpoint AI governance addresses this risk directly by applying policy and visibility where the activity occurs: on the user's device.

What Is Endpoint AI Governance?

Endpoint AI governance is the set of policies, tools, and controls used to manage AI applications on employee devices. It brings the laptop, desktop, and mobile device into the scope of an organization's AI governance strategy, recognizing that this is where many AI interactions now happen.

A complete approach covers the primary ways employees interact with AI today:

Desktop AI Apps: Standalone applications like Claude Desktop or the ChatGPT app.
AI in the Browser: Web-based interfaces such as claude.ai and chatgpt.com.
Coding Agents: Tools used in a developer's terminal and IDE, like Claude Code and other copilots.

Unlike traditional network-based security, which can be bypassed, endpoint governance places the control directly on the machine. This allows organizations to see and manage AI usage regardless of network location, whether an employee is in the office, at home, or connected to a public hotspot.

The Rise of Shadow AI and Its Inherent Risks

Shadow AI is the AI-era evolution of shadow IT. It describes employees using AI tools for work-related tasks without the approval or oversight of their organization. This behavior isn't malicious; it's driven by a desire for efficiency. However, it introduces significant risks:

Data Exposure: Employees frequently paste sensitive information—including customer PII, financial data, and proprietary code—into public AI tools. These platforms may use inputs for model training, creating an uncontrolled data outflow that most data loss prevention (DLP) tools are not configured to catch.
Compliance Violations: Industries with strict data handling regulations (like healthcare and finance) face enormous compliance risks from ungoverned AI. Using unapproved tools can violate standards like GDPR, HIPAA, and CCPA before the organization is even aware.
Intellectual Property Loss: The terms of service for many AI tools grant the provider rights over user-generated content, potentially compromising a company's ownership of its own intellectual property.
Expanded Attack Surface: Unregulated AI tools can introduce new security vulnerabilities, including prompt injection attacks and malicious code generation, expanding the enterprise attack surface in ways traditional security tools don't address.

The scale of this problem is significant. A 2025 IBM report found that breaches involving shadow AI cost companies an average of $670,000 more than standard breaches. Gartner predicts that by 2030, 40% of enterprises will suffer an AI-related security breach directly caused by shadow AI.

How Endpoint AI Governance Works

Effective endpoint AI governance is not about blocking AI tools outright. Attempts to ban popular tools often fail, pushing usage further into the shadows as employees find workarounds like personal accounts and unmanaged devices. The more sustainable approach is to govern usage by routing all AI traffic—from both approved and unapproved applications—through a central control plane.

This is typically achieved with a lightweight agent installed on each endpoint. The agent works transparently to:

Discover and Inventory: It first identifies all AI applications and services being used on the device, providing a clear picture of the organization's actual AI footprint.
Route Traffic: It transparently redirects all AI-related traffic from the device to a central AI gateway. This happens in the background without requiring users to change their workflows.
Enforce Policy: The AI gateway, acting as the control plane, applies the organization's policies. This can include blocking specific applications, enforcing budget and rate limits, and applying security guardrails like PII redaction or secrets detection to prompts before they reach a third-party model.
Audit and Log: All activity is logged centrally, creating an immutable audit trail for compliance and security reviews.

This model allows employees to continue using the tools that make them productive while ensuring all interactions adhere to company policy.

Key Benefits of an Endpoint-First Approach

Implementing a robust endpoint AI governance strategy provides several critical advantages for the modern enterprise.

Complete Visibility: You cannot protect what you cannot see. Endpoint governance provides a comprehensive, real-time inventory of all AI tools in use across the organization, closing the visibility gap left by network-only solutions.
Consistent Policy Enforcement: Centralized control means the same rules apply everywhere. Whether an employee is using a sanctioned enterprise tool or a free web-based app, their prompts are subject to the same guardrails and audit requirements.
Reduced Data Leakage Risk: By inspecting prompts and uploads before they leave the device, endpoint solutions can detect and block sensitive data from being sent to external models, directly mitigating a primary risk of shadow AI.
Improved Compliance Posture: With detailed audit logs of all AI interactions, organizations can demonstrate compliance with regulatory frameworks like the EU AI Act and standards like the NIST AI Risk Management Framework.
Cost Management: By routing all traffic through a central gateway, organizations can monitor usage, enforce budgets per user or team, and optimize spending across different AI models and providers. A recent Gartner survey showed that less than half of organizations actively manage their AI-related costs.

The AI Gateway and Endpoint Agent Model

The most effective architecture for endpoint AI governance combines a powerful AI gateway as the central control plane with an endpoint agent that extends its reach to every device. This is the model used by solutions like Bifrost, an open-source AI gateway from Maxim AI.

The Bifrost AI gateway serves as the policy engine where administrators configure virtual keys, budgets, security guardrails, and audit logging. Bifrost Edge, its corresponding endpoint agent, is deployed to company devices via MDM. Edge ensures that all AI traffic from desktop apps, browsers, and coding agents routes through the gateway, inheriting its security and governance policies automatically.

This combined approach allows organizations to embrace the productivity gains of AI without sacrificing control. It governs AI usage where it actually happens, providing the visibility and enforcement needed to manage risk in an era of decentralized AI adoption.

Integrating LLMs into a Go service without losing your mind (or adding 550ms latency)

James Whitfield — Tue, 07 Apr 2026 13:08:25 +0000

Right, so. This is a post I wish existed six months ago when we were first wiring LLMs into our Go backend at Huma.

Most of the tutorials out there for LLM integration assume you're in Python. Which is fine — a lot of ML infrastructure is Python, and libraries like LangChain, LiteLLM, and friends are well-documented. But if you're running a Go service stack and you want to add LLM calls without bolting on a whole Python sidecar, the path is less obvious.

Here's what we actually learned, including where we went wrong.

The problem we were solving

We build remote patient monitoring software. Clinicians use dashboards to track patients with chronic conditions — vitals, medication adherence, care notes. We added an LLM-powered summarization layer: given a week's worth of patient data, produce a brief natural-language summary for the clinician at the start of a shift.

Simple enough use case. The constraints were:

Go service stack (everything is Go at Huma, has been for years)
Latency-sensitive — this is in a hot path used during ward rounds, so we had a soft target of sub-1s end-to-end
HIPAA-relevant, so we needed to be clear about data routing
Needed provider failover — if OpenAI has an outage during peak hours, we can't just have it be broken

Attempt 1: Python sidecar

The first thing we did was the obvious thing: stood up a small FastAPI service in a sidecar container running LiteLLM. Our Go services would call the sidecar over HTTP, sidecar would call the provider.

It worked. And then we measured it.

p50 latency (model only):     ~400ms
p50 latency (sidecar added):  ~950ms
p99 latency (model only):     ~800ms
p99 latency (sidecar added):  ~1,400ms

The sidecar overhead was 500-600ms. Not at the HTTP level — at the total stack level, including the Python runtime, the LiteLLM library initialisation overhead on cold requests, and the round trip.

We spent a week trying to optimise it. Got it down to about 300ms overhead. Still not good enough.

Why Python proxies are painful for Go services

This isn't a Python criticism post. Python is fine for what it's for. But if you're integrating a Python proxy into a Go service, you're dealing with:

Different deployment lifecycle — your Go binary is a single static artifact. Python has a requirements.txt, a virtualenv, a cold start time, and dependencies that drift. Keeping these in sync across environments is friction.
Overhead from the library layer — libraries like LiteLLM do a lot. Request transformation, response normalisation, retry logic, fallback handling. That's all valuable, but it has a cost. When it runs in Python on every request path, you feel it.
Debugging across the boundary — when something goes wrong, your traces span two different runtimes with different logging formats. This sounds minor and isn't.

What we tried instead

I found Bifrost — an open-source LLM gateway written in Go. Apache 2.0, runs as a proper binary. The headline claim is 11µs overhead, which I was skeptical about.

It turned out to be accurate, or close enough that it doesn't matter. In practice we measured sub-1ms gateway overhead even under load. The difference from the Python sidecar was stark.

To get started locally:

npx -y @maximhq/bifrost

That spins up the gateway on localhost:8080. For production we containerise it and run it as a sidecar in the same pod as our Go service — tiny image, proper Go binary, low resource footprint.

The Go integration

Bifrost exposes a standard OpenAI-compatible HTTP API, so integrating it from Go is just HTTP client work. Here's our client struct:

package llmclient

import (
    "bytes"
    "context"
    "encoding/json"
    "fmt"
    "net/http"
    "time"
)

type Client struct {
    baseURL    string
    apiKey     string
    httpClient *http.Client
}

func New(baseURL, apiKey string) *Client {
    return &Client{
        baseURL: baseURL,
        apiKey:  apiKey,
        httpClient: &http.Client{
            Timeout: 30 * time.Second,
        },
    }
}

type Message struct {
    Role    string `json:"role"`
    Content string `json:"content"`
}

type CompletionRequest struct {
    Model       string    `json:"model"`
    Messages    []Message `json:"messages"`
    Temperature float32   `json:"temperature,omitempty"`
}

type CompletionResponse struct {
    ID      string `json:"id"`
    Choices []struct {
        Message      Message `json:"message"`
        FinishReason string  `json:"finish_reason"`
    } `json:"choices"`
    Usage struct {
        PromptTokens     int `json:"prompt_tokens"`
        CompletionTokens int `json:"completion_tokens"`
    } `json:"usage"`
}

func (c *Client) Complete(ctx context.Context, model, userPrompt string) (*CompletionResponse, error) {
    reqBody := CompletionRequest{
        Model: model,
        Messages: []Message{
            {Role: "user", Content: userPrompt},
        },
    }

    body, err := json.Marshal(reqBody)
    if err != nil {
        return nil, fmt.Errorf("marshalling request: %w", err)
    }

    req, err := http.NewRequestWithContext(
        ctx,
        http.MethodPost,
        c.baseURL+"/v1/chat/completions",
        bytes.NewReader(body),
    )
    if err != nil {
        return nil, fmt.Errorf("creating http request: %w", err)
    }

    req.Header.Set("Content-Type", "application/json")
    req.Header.Set("Authorization", "Bearer "+c.apiKey)

    resp, err := c.httpClient.Do(req)
    if err != nil {
        return nil, fmt.Errorf("sending request: %w", err)
    }
    defer resp.Body.Close()

    if resp.StatusCode != http.StatusOK {
        return nil, fmt.Errorf("upstream returned status %d", resp.StatusCode)
    }

    var result CompletionResponse
    if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
        return nil, fmt.Errorf("decoding response: %w", err)
    }

    return &result, nil
}

Nothing exotic. It's just Go HTTP. The gateway handles all the provider-specific bits.

Handling provider failover

The piece that mattered most for our use case was failover. Bifrost handles this through its routing config — you set up primary and fallback providers, and it handles the retry and fallback logic transparently. From our Go service's perspective, it's still just one HTTP call.

Here's a simplified version of how we define the fallback in the Bifrost config:

providers:
  - name: openai
    api_key: ${OPENAI_API_KEY}
    weight: 100
  - name: anthropic
    api_key: ${ANTHROPIC_API_KEY}
    weight: 0  # fallback only

routing:
  strategy: failover
  fallback_on: [5xx, timeout]

When OpenAI returns a 5xx or times out, requests automatically fall over to Anthropic. The clinician's dashboard keeps working. Nobody gets paged.

For HIPAA purposes, we audited the data routing — Bifrost passes requests through to the configured provider, it doesn't store or log the content by default. We added our own audit logging at the Go service layer before the request goes out.

The latency after switching

p50 latency (gateway overhead):  <1ms
p50 latency (total, model + gw): ~420ms
p99 latency (total):              ~870ms

Down from 1,400ms p99 to ~870ms. Most of what's left is the model.

That difference is real in the context we're in. Clinicians using a tool during ward rounds notice 1.4 seconds. They don't notice 870ms.

What I'd do differently from the start

If I were starting this integration again:

Don't reach for a Python sidecar reflexively. If your stack is Go, there are Go-native options now.
Measure the proxy overhead early, not after you've invested. We assumed sidecar overhead would be negligible. It wasn't.
Design failover in from the start. Bolting it on later meant a sprint of refactoring. The gateway approach made this much simpler.
Keep the LLM client behind an interface. We have a Summarizer interface in our domain layer that the LLM client implements. That means we can swap implementations in tests without touching anything else.

type Summarizer interface {
    Summarize(ctx context.Context, patientData PatientWeeklySummary) (string, error)
}

// In tests:
type mockSummarizer struct{}

func (m *mockSummarizer) Summarize(_ context.Context, _ PatientWeeklySummary) (string, error) {
    return "Patient stable. No significant changes.", nil
}

Standard Go stuff, but worth saying explicitly: don't let the LLM client leak into your domain logic.

The tl;dr is: if you're building Go services and need LLMs, the Python proxy path has a real latency cost that's easy to miss until you measure it. There are proper Go-native options now. Worth knowing they exist before you commit to an architecture that'll hurt you later.

Happy to share more specifics on the HIPAA logging setup or the failover config if useful — drop it in the comments.