Enterprise AI deployments expose sensitive data to external APIs, create compliance risks (GDPR/HIPAA), and lack centralized security controls. Without proper gateways and guardrails, organizations cannot enforce data governance, audit LLM interactions, or prevent unauthorized access.
This guide covers enterprise AI security through gateways and guardrails. This guide will use Bifrost as a reference for a gateway that provides comprehensive enterprise security through self-hosted deployment, virtual keys with granular permissions, Vault integration, and complete audit trails—enabling GDPR/HIPAA/SOC2 compliance.
Security Challenges
Data Exposure: Prompts contain PII, financial data, trade secrets
Compliance: GDPR, HIPAA, SOC2 requirements
Access Control: No per-team/user authorization
Audit Trails: Cannot track who accessed what
Rate Limits: No protection against abuse
Budget Controls: Uncontrolled costs
Solution: AI Gateway with Enterprise Security
maximhq
/
bifrost
Fastest enterprise AI gateway (50x faster than LiteLLM) with adaptive load balancer, cluster mode, guardrails, 1000+ models support & <100 µs overhead at 5k RPS.
Bifrost AI Gateway
The fastest way to build AI applications that never go down
Bifrost is a high-performance AI gateway that unifies access to 15+ providers (OpenAI, Anthropic, AWS Bedrock, Google Vertex, and more) through a single OpenAI-compatible API. Deploy in seconds with zero configuration and get automatic failover, load balancing, semantic caching, and enterprise-grade features.
Quick Start
Go from zero to production-ready AI gateway in under a minute.
Step 1: Start Bifrost Gateway
# Install and run locally
npx -y @maximhq/bifrost
# Or use Docker
docker run -p 8080:8080 maximhq/bifrost
Step 2: Configure via Web UI
# Open the built-in web interface
open http://localhost:8080
Step 3: Make your first API call
curl -X POST http://localhost:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "openai/gpt-4o-mini",
"messages": [{"role": "user", "content": "Hello, Bifrost!"}]
}'
That's it! Your AI gateway is running with a web interface for visual configuration…
1. Authentication and Authorization
Virtual Keys (per-team/user access control):
curl -X PUT http://localhost:8080/api/governance/virtual-keys/vk-eng-team \
-H "Content-Type: application/json" \
-d '{
"team_id": "team-engineering",
"provider_configs": [
{"provider": "openai", "allowed_models": ["gpt-4o-mini"]}
]
}'
SSO Integration (Google, GitHub):
- Single sign-on enforcement
- Team-based access
- Centralized user management
SAML/OIDC Support (Enterprise):
- Corporate identity provider integration
- Role-based access control
- Multi-factor authentication
2. Data Sovereignty
Self-Hosted Deployment:
# Deploy in-VPC
npx -y @maximhq/bifrost
# All data stays in your infrastructure
Benefits:
- Prompts/responses never leave your network
- GDPR/HIPAA compliance
- Complete data control
- Zero vendor lock-in
3. Budget and Rate Limiting
Hierarchical Budget Enforcement:
curl -X POST http://localhost:8080/api/governance/customers \
-H "Content-Type: application/json" \
-d '{
"name": "Acme Corp",
"budget": {"max_limit": 10000, "reset_duration": "1M"}
}'
Rate Limits:
curl -X PUT http://localhost:8080/api/governance/virtual-keys/vk-dev \
-H "Content-Type: application/json" \
-d '{
"rate_limit": {
"request_max_limit": 100,
"request_reset_duration": "1h"
}
}'
4. Comprehensive Audit Logging
Every Request Logged:
- Timestamp
- User ID (via x-bf-user-id header)
- Virtual key used
- Model requested
- Token usage
- Cost
- Latency
Prometheus Metrics:
# Requests by user
sum(bifrost_requests_total) by (user_id)
# Audit trail query
bifrost_requests{user_id="alice"}
5. Secrets Management
HashiCorp Vault Integration:
{
"vault": {
"type": "hashicorp",
"address": "https://vault.company.com:8200",
"token": "${VAULT_TOKEN}",
"mount": "secret",
"sync_interval": "300s"
}
}
Benefits:
- Centralized secret management
- Automatic key rotation
- Audit logging
- Access policies
6. Model and Provider Filtering
Restrict Access:
curl -X PUT http://localhost:8080/api/governance/virtual-keys/vk-dev \
-H "Content-Type: application/json" \
-d '{
"provider_configs": [
{"provider": "openai", "allowed_models": ["gpt-4o-mini"]}
]
}'
Result: Developers limited to approved models/providers
7. MCP Tool Filtering
Control Agent Tool Access:
curl -X PUT http://localhost:8080/api/governance/virtual-keys/vk-agent \
-H "Content-Type: application/json" \
-d '{
"mcp_tool_filter": ["filesystem/read_file", "web_search/search"]
}'
Behavior: Agent can only use approved tools
Complete Enterprise Setup
# 1. Deploy self-hosted (in-VPC)
npx -y @maximhq/bifrost
# 2. Configure SSO
curl -X PUT http://localhost:8080/api/config \
-H "Content-Type: application/json" \
-d '{
"auth": {
"google_oauth": {
"client_id": "...",
"client_secret": "env.GOOGLE_CLIENT_SECRET"
}
}
}'
# 3. Create customer with budget
curl -X POST http://localhost:8080/api/governance/customers \
-H "Content-Type: application/json" \
-d '{
"name": "Acme Corp",
"budget": {"max_limit": 10000, "reset_duration": "1M"}
}'
# 4. Create team
curl -X POST http://localhost:8080/api/governance/teams \
-H "Content-Type: application/json" \
-d '{
"name": "Engineering",
"customer_id": "customer-acme",
"budget": {"max_limit": 5000, "reset_duration": "1M"}
}'
# 5. Create virtual key with restrictions
curl -X PUT http://localhost:8080/api/governance/virtual-keys/vk-eng \
-H "Content-Type: application/json" \
-d '{
"team_id": "team-engineering",
"budget": {"max_limit": 1000, "reset_duration": "1M"},
"rate_limit": {
"request_max_limit": 1000,
"request_reset_duration": "1h"
},
"provider_configs": [
{"provider": "openai", "allowed_models": ["gpt-4o-mini"]}
]
}'
# 6. Configure Vault
curl -X PUT http://localhost:8080/api/config \
-H "Content-Type: application/json" \
-d '{
"vault": {
"type": "hashicorp",
"address": "https://vault.company.com:8200"
}
}'
Result:
- Self-hosted (data sovereignty)
- SSO authentication
- Hierarchical budgets
- Rate limits
- Model restrictions
- Vault secrets management
- Complete audit trails
Compliance
GDPR: Self-hosted deployment, data deletion capabilities, audit logs
HIPAA: In-VPC deployment, encryption at rest/transit, access controls
SOC2: Audit logging, access controls, secrets management
Get Started:
npx -y @maximhq/bifrost
Docs: https://getmax.im/bifrostdocs
GitHub: https://git.new/bifrost
Key Takeaway: Enterprise AI security requires authentication (SSO/SAML), data sovereignty (self-hosted), budget controls (hierarchical), audit logging (complete trails), secrets management (Vault), and access restrictions (model/provider/tool filtering). Everthing can be
Top comments (0)