API keys authenticate software. Policy objects decide what that software is allowed to do.
The previous post in this series ended on a question asked in a meeting room: who actually decided we're allowed to do this? The argument was that the answer has to exist before a gateway can enforce anything, and that the gateway's job is to make the answer repeatable.
This post is about what that answer looks like when it stops being a decision and becomes an object.
For decades, credentials have answered one question: who are you? Passwords identified people. API keys identified software. OAuth scopes added delegation because identity alone could not express that an application should act on a user's behalf within limits. Each step added context, because the decisions around access kept getting more nuanced.
Enterprise AI adds a question those credentials were never built to answer. Two applications can both authenticate successfully, and still one should be limited to inexpensive models while the other is cleared for frontier reasoning. One has a generous experimentation budget; the other has hard cost controls. One may invoke sensitive MCP tools, the other should never see that they exist.
Those are not authentication decisions. They are governance decisions arriving at request time, and identity cannot carry them.
What Is AI Access Control, and Why Don't API Keys Provide It?
The pattern is familiar to anyone who has watched authorization evolve. A credential starts as an identifier and slowly accumulates context, because the systems around it need to know more than who is calling.
Enterprise AI pushes several questions into the same moment. Which business unit owns this workload and absorbs its cost? Which providers are approved? Which specific models, given that "approved for OpenAI" and "approved for GPT-4o" are very different statements? Which budget applies, and what happens when it runs out? Which tools may be called? When does this access expire?
A credential that only proves identity forces every application to answer those questions for itself. That is how you end up with twenty codebases containing twenty slightly different interpretations of the same policy, which is exactly the drift the first post described.
What Does a Virtual Key Contain?
Bifrost's virtual keys are one implementation of the pattern. Rather than treating a credential as an identifier, the key becomes the runtime representation of a governance decision. Authentication still establishes identity. The virtual key determines how the request is handled once it arrives.
The Marketing budget decision from the previous post, the one Finance made in a meeting, ends up looking like this:
{
"name": "Marketing Experimentation",
"team_id": "team-marketing",
"provider_configs": [
{ "provider": "openai", "allowed_models": ["gpt-4o-mini"] }
],
"budget": { "max_limit": 2000.00, "reset_duration": "1M" },
"rate_limit": { "token_max_limit": 10000, "token_reset_duration": "1h" },
"expires_at": "2026-12-31T00:00:00Z",
"is_active": true
}
Everything in that object came out of a conversation rather than a codebase. The allow list came from a security review. The budget came from Finance. The expiry came from whoever decided this was an experiment rather than a permanent capability. The application presenting this key implements none of it.
Two details worth noticing. The allow list shapes discovery, not just enforcement: calling the models endpoint with this key returns only the providers it may reach, so an application never sees an inventory it cannot use. And because provider entries carry weights, the same object that expresses permission also expresses routing preference, which makes failover a policy decision rather than application logic.
How Virtual Key Hierarchy Maps AI Budgets to Teams and Customers
Policy objects would be unmanageable if every key stood alone. Bifrost stacks them: a customer contains teams, teams contain virtual keys, and budgets can attach at any level. A key belongs to one team or one customer, never both, which keeps cost attribution unambiguous. When a request arrives, the budgets in that chain are evaluated together, so a key can be under its own limit and still be refused because the team above it is exhausted. That sounds like an inconvenience until you have watched one enthusiastic team burn a quarter's allocation in nine days.
What makes the modeling worth it is that the structure mirrors the organization. Cost attribution stops being a monthly reconciliation exercise, because ownership was answered when the key was created rather than when the invoice arrived. Budgets and rate limits are the mechanism; the hierarchy is what makes them mean something.
What Happens When an AI Access Policy Blocks a Request?
This is the part most governance writing skips, and it is the part that determines whether any of it works in production.
Every constraint on the key has a defined failure. The request does not silently degrade or fall back to something cheaper without telling you.
That last note is worth more than it looks. Policy is evaluated before anything leaves the building, so a refused request costs nothing. Governance that saves money on the calls it prevents is a different proposition from governance that only records the calls it allowed.
The full set of refusals:
| What the policy says | What the caller gets |
|---|---|
| Budget exhausted |
402, budget_exceeded, with the overage in the message |
| Model not on the allow list |
403, model_blocked
|
| Provider not approved |
403, provider_blocked
|
| Key past its expiry |
403, with the expiry named as the reason |
| Token or request rate exceeded |
429, rate_limited, with the window stated |
| Key deactivated |
403, virtual_key_blocked
|
Look at the first row for a moment. A governance decision made by Finance surfaces to the application as 402 Payment Required, a status code that has existed since HTTP/1.1 and almost never had a legitimate use. Organizational policy is being expressed in ordinary protocol semantics, which means the retry logic, alerting, and error handling your team already has can respond to it without learning anything new.
The behavior is also deterministic in the boring ways that matter during an incident. Expiry fails closed and blocks both inference and tool execution. An expired key is not deleted or deactivated, so it remains visible for auditing and can be restored by extending the timestamp. And a key that is both inactive and expired reports as inactive, because when two conditions are true you want one predictable answer rather than a race.
That last detail connects back to the previous post's argument about evidence. A key that disappears when it expires destroys the record of what was permitted and when. A key that expires and stays visible preserves it.
Is AI Gateway Governance Enforced by Default?
Here is the thing to check on day one.
Governance is optional by default. A request arriving without a virtual key header is allowed through, ungoverned. That is a reasonable default for adoption, because it lets you introduce a gateway without breaking every existing integration on the first afternoon. It is a poor default to still be running six months later, when leadership believes policy is enforced and a meaningful share of traffic is routing around it.
Enforcement is a single setting, enforce_auth_on_inference. Turn it on and a valid key becomes mandatory; requests without one are rejected outright. Put that setting in your governance checklist rather than trusting that installing the gateway did it for you.
Do Virtual Keys Just Replace API Key Sprawl?
It is the fair objection. Replace scattered provider keys with hundreds of policy objects and you have arguably traded one management problem for another.
Two things make it a real trade rather than a lateral move. The sprawl already exists, distributed across twenty codebases where nobody can see it and no auditor can enumerate it; consolidating it into objects makes existing complexity visible rather than creating new complexity. And these objects are configuration, so they diff, they version, and they answer "who was approved for which models last quarter" by inspection rather than archaeology.
Adoption cost is also lower than it looks. Virtual keys ride the header conventions applications already use, whether that is the OpenAI style bearer token, the Anthropic style key header, or the Google and Azure equivalents. Most applications adopt governance by changing a base URL and a key value, not by taking on a new SDK.
Identity Answers Who. Policy Answers How.
Virtual keys do not replace role-based access control, and framing them as a competitor gets the relationship backwards. RBAC is good at what a person is permitted to do. Policy objects express what a workload is permitted to do, which is a separate question that persists whether a human is in the loop or not. An overnight batch job has no user, and still needs a budget, an allow list, and an owner.
That distinction is only going to matter more. Agentic systems make requests nobody explicitly initiated, invoke tools nobody selected in the moment, and spend money nobody watched being spent. The credential accompanying those requests has to carry organizational intent, because there is no human present to supply it.
The first post argued that governance has to exist before infrastructure can enforce it. This is the shape that enforcement takes: a decision made once in a room, encoded once in an object, and evaluated on every request without anyone having to remember it.
If you want to see how it is implemented rather than described, Bifrost is open source and the governance documentation and repository are both worth an hour.
This article was written in collaboration with the Bifrost team. The architectural perspective and conclusions expressed here are my own.
Top comments (2)
Really enjoyed this series. 👏
The part I find genuinely clever is turning an organizational decision into a policy object. Moving from “Finance decided Marketing gets $2k/month and these models” to something that can actually be evaluated at runtime is a huge step forward. Governance stops being a document and becomes something the system can enforce.
I also strongly agree with the point about enforce_auth_on_inference. I think this should eventually be a MUST in production. A permissive mode makes sense during adoption, but once the gateway is the enforcement boundary, requests without a valid policy should fail closed. 🔐
The hierarchical budget model also makes a lot of sense to me. I would probably take that even further and build policy hierarchies similar to AWS: organization → team → workload/agent → request, where each level can further restrict the permissions inherited from above.
And I really liked the attention to failure modes and auditability. 402, 403, 429, explicit reasons, expiry, policy versions... These are exactly the details I think about early when building production systems, because they are painful to add after the fact.
What makes the article particularly interesting to me is that the same model can extend naturally to agent capabilities and MCP tools, not only model access.
Really good work. You can feel the years of enterprise architecture experience behind these decisions. 🚀
Runtime enforcement is the important phrase here. Policy documents are useful for alignment, but the system becomes real only when the tool boundary can refuse an action at the moment of execution. I would also log the policy rule that authorized or denied each call.