DEV Community: ModelPlane

Embedding Groups: Dimension-Aware Routing for /v1/embeddings

ModelPlane — Sun, 09 Aug 2026 02:22:59 +0000

Embedding Groups: Dimension-Aware Routing for /v1/embeddings

If you're building on embeddings, you already know the pain: you pick a provider, bake model="text-embedding-3-large" into your code, and pray the pricing and latency stay acceptable. The moment you want to switch or add a fallback, you discover the dirty secret of embedding vectors — they're meaningless unless every vector in your store has the same dimension. A 3072-dimension vector from one model and a 1536-dimension vector from another can't be compared, clustered, or searched. Your vector database will happily store both, and then your similarity queries return garbage.

ModelPlane's embedding groups solve this by treating dimension as a first-class routing constraint. The gateway validates that every backend in an embedding model group produces vectors with the same dimension — at configuration time, before you route a single request, and at runtime, so a provider change can't silently corrupt your index.

The abstraction: model groups, not model IDs

The core ModelPlane abstraction is the model group: request.model is a name you control, not a provider model ID. You define a group like prod-embeddings, point it at one or more backends, and your application code never changes when you swap providers.

For chat and completion models, a model group can contain wildly different backends — gpt-4o and claude-3-5-sonnet have different tokenizers, different pricing, different latency profiles, but they all produce text. The routing engine doesn't care about the output shape; it just needs to get a string back.

Embeddings break that assumption. The output of an embedding call is a vector, and vectors are only useful when they share a dimensional space. You can't fall back from a 3072-dimension model to a 1536-dimension model without silently breaking every downstream consumer of those vectors.

This is why embedding groups are dimension-aware. The gateway enforces a contract that your code can't — that every backend in a group produces vectors your vector store can actually use.

Why dimension mismatch is a silent production killer

Let's be concrete about the failure mode. You have a RAG pipeline. You chunk documents, embed them with text-embedding-3-large (3072 dimensions), and store them in Pinecone or pgvector. Your app queries with the same model, gets a 3072-dimension query vector, and everything works.

Now your embedding provider has an outage. You have a fallback configured — good engineering instinct. The fallback is text-embedding-3-small (1536 dimensions). The gateway routes your query to the fallback, returns a 1536-dimension vector, and your vector store... accepts it. Pinecone doesn't reject vectors with the wrong dimension. pgvector doesn't either. Your similarity search now compares 3072-dimension stored vectors against a 1536-dimension query vector.

The results are nonsense. Not obviously broken — just subtly wrong rankings, missing relevant documents, and a production incident that takes hours to diagnose because the error messages are unhelpful or nonexistent.

This is the kind of bug that doesn't show up in staging. Staging has one provider. Production has fallbacks. And fallbacks without dimension validation are landmines.

How embedding groups enforce the contract

ModelPlane's routing engine, tryTargetsRecursively, walks a target tree according to a routing strategy: single, fallback, loadbalance, or conditional. For embedding groups, the gateway adds a dimension check on top of that tree.

At configuration time, when you create or update an embedding model group, the gateway validates that all targets resolve to models with the same embedding dimension. It uses the provider catalog's model metadata to check this before the group is saved. If you try to mix a 3072-dimension model with a 1536-dimension model, the gateway rejects the configuration.

At runtime, the gateway verifies the dimension of each embedding response against the group's expected dimension. If a provider returns a vector with the wrong dimension — whether due to a model change, a provider-side error, or a misconfigured override — the gateway treats it as a failed target and advances to the next backend in the fallback chain.

This means your fallback chain is safe by construction. You can configure prod-embeddings with text-embedding-3-large as primary and text-embedding-3-small as fallback — but the gateway will reject that group at creation time because the dimensions don't match. You'll need to pick backends that actually produce compatible vectors.

What you can build with dimension-aware groups

The constraint isn't a limitation — it's what makes the abstraction useful. Here's what becomes possible:

True embedding HA. Pick two or more providers that offer models with the same dimension. For example, OpenAI's text-embedding-3-large (3072) and a compatible model from another provider. Configure them in a fallback group. When one provider has an outage or rate-limits you, the gateway routes to the next, and every vector that comes back is guaranteed to be queryable.

Load-balanced embedding pipelines. If you have a high-volume ingestion pipeline, spread embedding requests across multiple backends with weighted load balancing. The gateway distributes traffic, and you never have to worry about a fraction of your vectors being incompatible with the rest.

Conditional routing by request metadata. Route embedding requests based on customer tier, region, or feature flag — but only among backends that produce the same dimension. The dimension check applies regardless of which strategy mode you use.

Provider migration without re-embedding. Want to move from one embedding provider to another? If they offer same-dimension models, you can run both in a load-balanced group, migrate traffic gradually, and re-embed your corpus at your own pace. The gateway ensures every new vector is compatible with your existing store.

A concrete example

Here's what an embedding group looks like in practice. First, create the group via the ModelPlane API:

curl -X POST https://modelplane.dev/api/model-groups \
  -H "Authorization: Bearer $MODELPLANE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "prod-embeddings",
    "routing_config": {
      "strategy": { "mode": "fallback" },
      "targets": [
        { "kv_ref": "cred:user:openai", "override_params": { "model": "text-embedding-3-large" } },
        { "kv_ref": "cred:user:voyage", "override_params": { "model": "voyage-3-large" } }
      ]
    }
  }'

If text-embedding-3-large and voyage-3-large have the same dimension, the group is created. If not, the gateway returns a validation error telling you the dimensions don't match.

Now your application code is trivial. Point your OpenAI client at ModelPlane and use the group name as the model:

from openai import OpenAI

client = OpenAI(
    base_url="https://modelplane.dev/v1",
    api_key="your-modelplane-api-key",
)

# This routes to prod-embeddings, which handles fallback and dimension validation
response = client.embeddings.create(
    model="prod-embeddings",
    input="The vector database is the new hotness, but only if your vectors are compatible.",
)

vector = response.data[0].embedding
print(f"Dimension: {len(vector)}")

That's it. Your code doesn't know which provider answered, doesn't know about fallbacks, and doesn't need to check dimensions. The gateway handles all of it.

The dimension check is the feature

Most routing gateways treat embeddings as an afterthought — just another endpoint to proxy. ModelPlane treats them as a distinct workload with a distinct contract. The dimension check isn't a validation nicety; it's the difference between a fallback that saves you from an outage and a fallback that silently corrupts your production data.

Think about what your vector store actually needs. It needs every vector to have the same dimension. It needs that invariant to hold across provider outages, model deprecations, and traffic spikes. Your application code can't enforce that invariant — it's too far from the routing decision. Your vector store can't enforce it — it accepts whatever you give it. The only place to enforce it is at the routing layer, where the gateway sees every request and every response.

That's what embedding groups do. They make dimension compatibility a property of the route, not a hope about the provider.

Beyond embeddings: the same principle applies

The dimension check is a specific instance of a general principle: the routing layer should enforce the contracts your application depends on. For chat models, that means thinking normalization — one thinking parameter, encoded per provider, so reasoning output comes back in a consistent shape. For system prompts, it means injection at the router, so safety and formatting instructions live with the route, not copy-pasted into every client. For embeddings, it means dimension validation, so your vector store never receives a vector it can't use.

ModelPlane's provider catalog covers the major embedding providers, and the same routing engine that handles your chat traffic handles your embedding traffic. One endpoint, one API key, one dashboard for usage and billing across all of it.

Start with a free account

If you're building on embeddings, you know the pain of provider lock-in and the fear of silent corruption. Embedding groups give you the safety of a validated contract and the flexibility of routing — without changing your application code.

Create your first embedding group — free $5 credits, no card required. Point your OpenAI client at https://modelplane.dev/v1, define a model group, and let the gateway handle the rest.

This post is part of a series on building production-grade AI infrastructure with ModelPlane. Check out the rest:

Multimodal-aware routing: vision requests find vision-capable backends

ModelPlane — Sun, 09 Aug 2026 02:22:58 +0000

Your app sends a text-only request. It also sends an image-with-a-question request. Right now, both go to the same model — usually the most expensive one that can handle both. That's the wrong unit of integration.

The provider is the wrong unit of integration. The model group is the right one. And once you think in model groups, a question you couldn't answer before becomes obvious: why does a text-only request need to hit a vision-capable backend at all?

Multimodal-aware routing means your model group filters its targets by the request's modalities. Text requests go to a cheap text tier. Image requests skip incompatible backends and land on a vision-capable one. You get lower cost, fewer failed requests, and no code changes in your app.

Here's how to build it with ModelPlane.

The problem: one model for every modality is expensive and fragile

Most LLM apps are multimodal in practice even when they don't think of themselves that way. A support bot sends text. A document analyzer sends images. A coding assistant sends screenshots of error messages. A data pipeline sends charts.

If you hard-code model="gpt-4o" in your client, every one of those requests — text and image alike — goes to the same backend. That's the default, and it's wrong for two reasons.

First, cost. Vision-capable flagship models are priced at a premium over text-only tiers. A text-only request that gets routed to a vision model is paying for capability it doesn't use. When 80% of your traffic is text, that's a tax on the majority of your requests.

Second, fragility. Not every model accepts images. If you route an image request to a text-only model, you get a 400 error — or worse, the model silently ignores the image and answers from text alone. You've shipped a bug you can't see.

The fix isn't to write modality-checking logic in every service. It's to make the routing layer aware of what the request contains.

Model groups: the abstraction that makes this possible

A model group is a name you control — prod-chat, vision-pipeline, whatever — that maps to a routing strategy and a list of backend targets. Your client sends model="prod-chat" to https://modelplane.dev/v1, and the gateway expands that name into a routing config and runs the request through the routing engine.

The key insight: request.model is a name you control, not a provider model id. You can change what that name resolves to without touching your code. That's what makes multimodal-aware routing possible — the routing config, not the client, decides which backend answers.

The routing engine supports four strategy modes: single, fallback, loadbalance, and conditional. For multimodal-aware routing, you combine fallback (for reliability) with target-level filtering based on request content.

How multimodal-aware routing works

ModelPlane's routing engine walks a target tree for each request. The fallback strategy tries targets in order and advances to the next on specified status codes — that's your high-availability backbone. What multimodal-aware routing adds is a filter at the target level: if the request contains an image, skip targets that can't handle images.

Concretely, your model group's routing config looks like this:

{
  "strategy": { "mode": "fallback" },
  "targets": [
    {
      "kv_ref": "cred:<userId>:<credId>",
      "override_params": { "model": "deepseek-v4-flash" },
      "modalities": ["text"]
    },
    {
      "kv_ref": "cred:<userId>:<credId2>",
      "override_params": { "model": "gpt-5.4-mini" },
      "modalities": ["text", "image"]
    },
    {
      "kv_ref": "cred:<userId>:<credId3>",
      "override_params": { "model": "gpt-5.6-sol" },
      "modalities": ["text", "image"]
    }
  ]
}

The gateway inspects the incoming request. If it's text-only, the first target (cheap, fast, text-only) handles it. If it contains an image, the router skips the first target and goes straight to a vision-capable backend. If that backend fails, it falls through to the next vision-capable one.

No client-side logic. No modality checks in your app code. The routing layer handles it.

A concrete example: DeepSeek-first, vision-capable fallback

Let's build a real model group. The CTA for this post is "Build a DeepSeek-first, vision-capable-fallback group." Here's why that's the right shape.

DeepSeek's V4 family is cost-efficient and has a 1M-token context window with a 384K max output — excellent for long-generation text workloads. But it's not a vision model. If you route an image request there, you get an error.

So you build a group that's DeepSeek-first for text, with vision-capable fallbacks for image requests:

from openai import OpenAI

client = OpenAI(
    base_url="https://modelplane.dev/v1",
    api_key="gw-...",  # your ModelPlane gateway key
)

# Text-only request → routes to deepseek-v4-flash
text_response = client.chat.completions.create(
    model="prod-chat",
    messages=[
        {"role": "user", "content": "Summarize this changelog in three bullets."}
    ],
)

# Image request → skips text-only targets, routes to a vision-capable backend
image_response = client.chat.completions.create(
    model="prod-chat",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "What's the error in this screenshot?"},
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "https://example.com/error-screenshot.png"
                    },
                },
            ],
        }
    ],
)

Same client. Same model group name. Different backends, chosen automatically by the router based on what the request contains.

The first request hits DeepSeek V4 Flash — cheap, fast, text-only. The second request skips it and lands on a vision-capable model. If that model is rate-limited or returns a 5xx, the fallback strategy advances to the next vision-capable target. Your app never sees a failure.

Why routing beats client-side modality checks

You could implement modality detection in your app. Check if the message contains an image, then pick a model. It's not hard — a few lines of code.

But it's the wrong place for that logic, for the same reason system prompts belong at the router, not in your app: behavior should live with the route, not copy-pasted into every client.

Client-side modality checks have three problems:

They drift. You have five services calling models. Each one implements its own modality check. One service forgets. Another uses a different image format. Now you have inconsistent behavior across your stack.

They can't react to provider changes. Your text-only model gets a vision update. Your vision model gets deprecated. With client-side checks, you're updating code and redeploying. With router-side filtering, you edit the model group config and you're done.

They can't enforce policy. A client can always send an image to a text-only model. The router can't stop it. With multimodal-aware routing, the gateway enforces the filter — an image request physically cannot land on a text-only target.

The router is the single enforcement point. That's where modality awareness belongs.

Combining multimodal filtering with other routing strategies

Multimodal-aware routing isn't a replacement for the other strategies — it composes with them.

Fallback + multimodal. Your vision-capable primary target fails. The router advances to the next vision-capable target. Text requests never trigger the vision fallback chain, so you're not paying vision prices for text traffic.

Loadbalance + multimodal. You want to split traffic across two vision-capable models by weight — 70% to one, 30% to another — while text-only requests go to a third, cheaper tier. The modality filter narrows the candidate pool; the load-balance strategy picks among the survivors.

Conditional + multimodal. You route by customer tier: free-tier users get the cheap vision model, paid users get the flagship. The modality filter runs first, then the conditional router picks among the vision-capable targets.

The routing engine's target tree handles all of this. Multimodal filtering is just another constraint on which targets are eligible for a given request.

What this means for your LLM stack

Multimodal-aware routing changes how you think about your model inventory. Instead of one model per use case, you have a pool of backends with different capabilities, and the router matches requests to the cheapest capable target.

The practical wins:

Lower cost. Text requests stop paying vision premiums. If 80% of your traffic is text, that's a direct reduction in your bill.
Fewer failures. Image requests never hit text-only backends. No more silent image-ignoring bugs.
One client. Your app sends model="prod-chat" and never thinks about modalities again.
Provider flexibility. Swap DeepSeek for another text tier, add a new vision model, rebalance weights — all in the model group config, no deploys.

The provider is the wrong unit of integration. The model group is the right one. Multimodal-aware routing is what happens when you take that seriously — your routing layer knows what your requests contain, and it routes accordingly.

Build a DeepSeek-first, vision-capable-fallback group and watch your text traffic stop paying vision prices.

Ready to route smarter? Start free — $5 credits, no card. Create your first model group in minutes and see multimodal-aware routing in action.

This post is part of a series on building production-grade LLM infrastructure with ModelPlane. Stay tuned for deep dives into routing strategies, model groups, unified thinking, system-prompt injection, high availability, coding-plan routing, price-aware routing, billing, the provider catalog, teams, and the routing engine itself.

Inside the ModelPlane routing engine

ModelPlane — Sat, 08 Aug 2026 02:18:35 +0000

Inside the ModelPlane routing engine

Every LLM request your app sends is a bet: that the provider you hard-coded will be up, fast, and cheap enough. That bet is why routing layers exist. But a routing layer only helps if it's fast, reliable, and transparent enough to sit on every single call you make. This post walks through exactly what happens inside the ModelPlane routing engine from the moment your request hits the gateway to the moment tokens come back — and why the design keeps accounting off the hot path.

The request lifecycle: auth, resolve, gate, route, account

A single request through ModelPlane passes through five distinct stages. Understanding them matters because each one is a place where a naive gateway either slows you down, leaks credentials, or drops usage data.

Authentication — the gateway resolves your Bearer gw-* key into a tenant context.
Config resolution — it figures out which routing config applies to this request.
Billing gate — a pre-request check confirms you have credits.
Routing — it walks the target tree, picks a backend, and makes the upstream call.
Accounting — it records usage and decrements credits asynchronously.

Let's go through each stage with the concrete mechanisms, because the details are where the reliability lives.

Stage 1: Authentication — the `gw-*` key is auth only, never forwarded

The first thing the gateway does is authenticate the request. It resolves the Bearer gw-* token (or a Supabase JWT for management calls) into a TenantContext. That context carries your workspace identity and the credentials needed to call upstream providers.

The critical security property here: the Authorization header containing your gw-* key is stripped from the request before it's ever forwarded upstream. Your gateway key is tenant authentication only. It is never the credential an upstream provider sees. That separation is what makes BYOK safe — your provider keys are encrypted per-tenant and only hydrated at routing time, never exposed to your clients.

Stage 2: Config resolution — `request.model` is a name you control

Once authenticated, the gateway needs to decide how to route. This is where the core abstraction kicks in: request.model is not a provider model ID. It's a model group — a name you control that maps to a set of targets and a routing strategy.

The resolution order is:

Explicit per-request config (x-portkey-config header, legacy) — overrides everything.
request.model as a model group — looked up in KV cache first, then Supabase. Credentials for the group's backends are decrypted from their kv_refs at this point.
Direct provider call (x-portkey-provider header) — a single-provider bypass.

For most users, path #2 is the one that matters. You define a model group called prod-chat in the dashboard, point it at three backends with a fallback strategy, and then in your code you just send model="prod-chat". The gateway resolves that name to the full routing config.

from openai import OpenAI

client = OpenAI(
    base_url="https://modelplane.dev/v1",
    api_key="gw-...",  # your ModelPlane gateway key
)

# "prod-chat" is a model group, not a provider model ID.
# The gateway resolves it to targets + strategy server-side.
response = client.chat.completions.create(
    model="prod-chat",
    messages=[{"role": "user", "content": "Explain routing engines."}],
)

Stage 3: The billing gate — a snapshot, not a lock

Before routing, the preRequestValidator checks your available credits. If you're out, you get a 402. If you're on the unlimited plan, the gate is skipped entirely.

This is a deliberate design choice: a pre-request balance snapshot gate plus post-request async deduction. It's fast — no distributed transaction on the hot path — but it means there's a theoretical window where concurrent requests could overspend. The team has documented this as a known gap (PRD/002 covers reliability fixes like an outbox pattern and atomic balance updates). For the vast majority of usage, the snapshot gate is the right tradeoff: it catches the "I'm out of credits" case instantly without adding a database round-trip to every inference call.

Stage 4: The routing engine — walking the target tree

This is the heart of the system. The routing engine's tryTargetsRecursively walks the target tree according to the strategy mode you configured on the model group. There are four modes:

Mode	Behavior	Use case
`single`	One target, no alternatives	Dev/test, or when you want zero ambiguity
`fallback`	Try in order; advance on specified status codes	High availability — the minimum viable LLM stack
`loadbalance`	Pick by weight (`selectProviderByWeight`)	A/B tests, cost-weighted splits
`conditional`	Evaluate a query DSL against request metadata to pick a named target	Customer-tier routing, region-based routing, feature flags

For each target, tryPost builds the upstream call, runs any configured hooks, checks the cache, and calls the retry handler. If a target fails with a status code that the strategy says to retry on, the engine advances to the next target in the tree.

The fallback mode is where the reliability value lives. A production-grade setup looks like this:

{
  "strategy": { "mode": "fallback" },
  "targets": [
    { "provider": "openai", "model": "gpt-4o" },
    { "provider": "anthropic", "model": "claude-3-5-sonnet" },
    { "provider": "deepseek", "model": "deepseek-v3" }
  ]
}

When OpenAI returns a 5xx or times out, the engine automatically tries Anthropic, then DeepSeek. Your app sees one endpoint and one model name; the gateway handles the chaos.

Stage 5: Accounting — off the hot path, but not off the record

After the upstream response is returned, recordUsage runs asynchronously. It writes to usage_logs and decrements credits, using a reference ID to keep the operation idempotent. This is what keeps the inference hot path fast — the gateway doesn't block your response on a database write.

The honest caveat: because accounting is fire-and-forget, there's a small risk of dropped usage records under extreme load. This is a documented tradeoff, not a hidden bug. The team's roadmap includes moving to a more durable outbox pattern. For now, the async model buys you latency at the cost of a tiny accounting tail-risk.

Why the provider abstraction matters

The routing engine doesn't care which provider it's calling. The provider system abstracts every upstream behind a uniform, OpenAI-compatible interface. Requests are transformed from the OpenAI parameter shape into each provider's native format, and responses are normalized back.

This is what makes the provider catalog so powerful. You can route to OpenAI, Anthropic, Google Gemini, DeepSeek, MiniMax, Zhipu, OpenRouter, or Novita AI — all through the same base_url and the same client library. The gateway handles the protocol differences, the parameter mapping, and the response normalization.

The abstraction also extends to reasoning models. A single thinking parameter on the request is encoded per provider — effort scalars for some, thinking objects for others, passthrough for the rest. Your code doesn't branch on provider; the gateway's ReasoningMapping handles it.

Multi-tenancy without slowing down inference

The routing engine runs in a multi-tenant environment, which means isolation has to be baked in, not bolted on. The design keeps it off the hot path:

Credentials are encrypted per-tenant (AES-GCM) and stored in Workers KV. They're decrypted only when a model group's config is hydrated.
Routing config is cached in KV, so the gateway doesn't hit the database on every request.
Tenancy is workspace-scoped — all resources (API keys, backends, model groups, usage) belong to a workspace, and the billing account owns the plan and credits.

The result: tenant isolation and encrypted credential handling don't add a round-trip to your inference call. The security model is a property of the infrastructure, not a per-request cost.

The target tree is the unit of resilience

Here's the mental model that makes all of this click: your model group is a target tree, and the routing strategy is how the tree is traversed. A single model ID in your code is a leaf. A fallback chain is a branch. A weighted load-balance is a branch with probabilities. A conditional router is a branch with a decision function.

Once you think in target trees, the engineering choices become obvious:

Fallback isn't a feature — for anything production-bound, it's the minimum viable LLM stack. Provider outages are when-not-if.
Load-balancing by weight lets you express cost policy in the routing config, not in application code.
Conditional routing moves business logic (customer tier, region) into the gateway, where it's auditable and changeable without a deploy.

The hot path stays hot

The whole design philosophy can be summarized in one sentence: the inference hot path does only what it must, and everything else happens around it. Auth is a fast token lookup. Config resolution is a KV cache hit. The billing gate is a snapshot check. Routing is an in-memory tree walk. Accounting is async.

That's why ModelPlane can sit on every call your app makes without becoming the bottleneck. The gateway isn't doing anything clever per request — it's doing the right things per request and deferring everything else.

Build your first target tree

The fastest way to understand the routing engine is to build a model group with a fallback chain. Create a group called prod-chat, add two or three backends, set the strategy to fallback, and point your OpenAI client at https://modelplane.dev/v1. Then kill one provider and watch the gateway route around it.

Start free — $5 credits, no card. Your first model group is 60 seconds away.

ModelPlane for teams: orgs, workspaces & shared keys

ModelPlane — Sat, 08 Aug 2026 02:18:34 +0000

ModelPlane for teams: orgs, workspaces & shared keys

The moment a second engineer joins your project, the "just use my API key" approach to LLM infrastructure breaks. Someone commits a key to a repo, another person rotates it, and suddenly production is down because the key that was working yesterday is gone. You need multi-tenancy, but you don't need the complexity of building it yourself.

ModelPlane treats team access as a first-class routing problem, not an afterthought. The gateway separates identity, resources, and billing into distinct entities, gives you scoped keys that can't leak upstream, and lets an organization share credits across every workspace. This is the difference between a gateway that works for one developer and a gateway that works for a team.

The multi-tenant mess you're actually in

Most teams start with a single shared key. It's pragmatic until it isn't. You can't tell which teammate burned through the budget, you can't revoke access for one person without breaking everyone, and you definitely can't enforce different routing policies for different services.

Hard-coding a provider key in your backend is worse. That key is a direct line to your provider account, and it gets forwarded with every request. If it leaks, someone else can spend your money. If you need to rotate it, you're redeploying services.

The abstraction that fixes this is the same one that fixes provider lock-in: the model group. Your application sends requests to a model group like prod-chat, and the gateway decides which backend answers. Team access works the same way — you manage who can call that group, not which provider key they hold.

Three entities, one clean boundary

ModelPlane's multi-tenancy model separates three concerns that most systems conflate:

Entity	Responsibility
User	Login identity, audit actor
Workspace	Resource owner — API keys, backends, model groups, usage
Billing Account	Plan, credits, transactions

A user is who you are. A workspace is what you own. A billing account is how you pay. When these are the same thing, you get the single-developer experience. When they're separate, you get teams.

The workspace is the unit of isolation. Every resource — every API key, backend, model group, and usage log — belongs to a workspace. This means you can have a staging workspace and a production workspace with completely different routing configurations, and the keys for one can't touch the other's resources.

Organizations take this further. An organization requires a paid plan and can have multiple workspaces. Crucially, an organization's workspaces share the organization's credits. Your team gets one billing surface, not a dozen individual accounts to reconcile.

Key scopes: private vs. shared

The gateway API key is the credential your code actually uses. It's a gw-* bearer token that authenticates the request to ModelPlane. Two properties make it fundamentally safer than a raw provider key:

It's never forwarded upstream. The gateway strips the Authorization header carrying your gw-* key before making the upstream call. Your provider credentials are resolved server-side from encrypted storage. A leaked gateway key can't be used to call OpenAI directly.
Only the hash is stored. The plaintext key is returned exactly once at creation. If you lose it, you create a new one. There's no database of recoverable secrets to steal.

Keys come in two scopes, mirroring how your team actually works:

Private keys — personal and member-scoped. Every user, including org members, can create these. They're yours, and they're tied to your identity for audit purposes.
Shared keys — available to organization owners and admins. These are workspace-scoped credentials that multiple services can use without being tied to one person's account.

This is the key insight: shared keys are for machines, private keys are for humans. A CI pipeline gets a shared key scoped to the workspace it deploys to. A developer debugging locally uses their private key, and their usage shows up in audit logs under their identity.

Setting up an organization

The portal keeps tenant context visible before every resource action — you always know which org and workspace you're operating in. The flow is straightforward:

Create an organization (requires a paid plan).
Invite members with owner, admin, or member roles.
Create workspaces for different environments or product lines.
Add backends and model groups per workspace.
Issue shared keys for services, private keys for people.

A free user gets one auto-created default workspace. Paid users can have multiple personal workspaces. Organizations are where the real team structure lives.

Routing that respects team boundaries

The point of all this structure is that routing policies become team-manageable. Consider a typical setup:

A prod-chat model group in the production workspace with a fallback chain across three providers.
A staging-chat group in the staging workspace pointing at cheaper models.
A data-extraction group with a system prompt injected for structured output.

Each group has its own backends, its own routing strategy, and its own usage accounting. The team that owns the data pipeline can change its routing without touching the chat application's configuration. The billing account sees one consolidated view of credits consumed across all workspaces.

This is what "LLM gateway for teams" actually means: not just a shared endpoint, but shared infrastructure with per-team control planes.

Code: one client, scoped by key

Here's how a service uses a shared key to talk to a model group. The code is identical to calling OpenAI directly — only the base_url and model name change.

from openai import OpenAI

client = OpenAI(
    base_url="https://modelplane.dev/v1",
    api_key="gw-your-shared-key-here",  # workspace-scoped, never forwarded upstream
)

response = client.chat.completions.create(
    model="prod-chat",  # a model group, not a provider model id
    messages=[
        {"role": "user", "content": "Summarize the incident report."}
    ],
)

print(response.choices[0].message.content)

The same code works for a developer using their private key. The gateway resolves the model group prod-chat to the appropriate backend, applies the routing strategy, and accounts the usage to the workspace that owns the key.

If the production workspace's primary provider has an outage, the fallback chain kicks in — the application doesn't need to know, and the developer doesn't need to change anything. The model group abstracts away both the provider and the team's access policy.

Billing that doesn't require a spreadsheet

The shared-credits model is the operational win. When an organization's workspaces share credits, you eliminate the monthly ritual of collecting individual usage reports and trying to figure out who owes what.

The gateway records usage per API key, per model group, and per model. You can see at a glance:

Which workspace consumed the most tokens this month.
Which model group is driving cost.
Whether the staging group is accidentally handling production traffic.

This visibility turns billing from an argument into a report. And because the gateway handles the routing, you can also make cost a routing criterion — prefer cheaper models for internal tools, escalate to premium models only for customer-facing features.

The honest limitations

Multi-tenancy is a design goal, and the rollout is a process. The full org/workspace model is a v1 target, and some edge cases — like per-workspace concurrency limits surfacing billing atomicity gaps — are still being hardened. If you're evaluating ModelPlane for a large enterprise, it's worth confirming the specific org features you need are fully live.

For most teams, though, the core value is immediate: separate workspaces, scoped keys, shared billing, and routing policies that don't require a shared credential or a shared fate.

Stop sharing keys, start sharing infrastructure

The single most important step for any team building on LLMs is to stop treating provider keys as team infrastructure. They're not — they're a liability. ModelPlane replaces them with a routing layer that gives you the isolation of separate accounts and the simplicity of a shared one.

Create an organization, set up your workspaces, and issue scoped keys. Your future self, debugging a production incident at 2 AM, will thank you.

CTA: Create an organization.

This post is part of a series on building production-grade LLM infrastructure. Here's what else we've covered and what's coming:

One endpoint, every model — why we built ModelPlane
Routing strategies, explained — fallback, load-balance, conditional
Model groups — the one abstraction that decouples your app from providers
Bring your own keys, safely — how BYOK encryption works
High availability for LLM apps — a fallback playbook
Stop paying twice — route your coding-plan quota into production
Price-aware routing — cut your LLM bill without changing models
Credits, usage & billing, explained — one transparent bill
1600+ models, one API — the provider catalog
One gateway, two regions — routing to global and China models
ModelPlane for teams — orgs, workspaces & shared keys (you are here)
Meet the assistant — an AI helper in your gateway dashboard
Inside the routing engine — how a request flows from auth to upstream
One thinking parameter, every model — unified reasoning
The system prompt belongs at the router — not in your app

Bring Your Own Keys, Safely: The BYOK Security Model Behind ModelPlane

ModelPlane — Fri, 07 Aug 2026 03:43:30 +0000

Bring Your Own Keys, Safely: The BYOK Security Model Behind ModelPlane

If you're building on top of LLMs, you've already made the security decision that matters most: you're not training your own models. You're renting intelligence from someone else's API. The question is how you manage the credentials that unlock that intelligence.

Most teams hardcode a provider key into a service, or worse, share one key across an entire organization. When that key leaks—and it will—you're not just out a few dollars. You're exposed to prompt injection attacks, data exfiltration, and a billing nightmare that takes weeks to untangle.

The solution isn't to stop using LLMs. It's to route your traffic through a gateway that treats credentials as first-class, tenant-isolated secrets. That's the BYOK (Bring Your Own Key) model at the heart of ModelPlane. It's not just about convenience—it's about building a security boundary between your application and the providers you depend on.

The Wrong Unit of Integration

Before we talk about keys, let's talk about what you're actually integrating with. Most teams think in terms of providers: "We use OpenAI," or "We're on Anthropic now." That's the wrong mental model.

The provider is the wrong unit of integration. Your application shouldn't care whether a request is answered by gpt-4o, claude-3-5-sonnet, or deepseek-v3. It should care about the capability: a fast chat model, a reasoning model, a cheap batch model. That capability is what we call a model group.

A model group is a name you control—like prod-chat—that maps to one or more provider backends with a routing strategy. When your app sends a request to model="prod-chat", ModelPlane decides which backend answers, based on fallback rules, load-balancing weights, or conditional logic.

This abstraction changes the security conversation. Instead of managing N provider keys across M services, you manage one gateway key per environment, and the gateway manages the provider keys for you. The keys become infrastructure, not application logic.

Two Keys, Two Jobs

The first thing to understand about ModelPlane's security model is that there are two entirely different types of credentials in play. Confusing them is the root of most LLM gateway security failures.

Gateway API Key (gw-*): This is what your application sends to ModelPlane. It starts with gw- and is a bearer token that authenticates your tenant. It's how we know which workspace is making the request, which model groups it can access, and which billing account to charge.

Backend Credential: This is the actual provider API key—your OpenAI key, your Anthropic key, your DeepSeek key. You upload these to ModelPlane via the Backends page or API, and we store them encrypted.

The critical rule: the gateway key is for authentication only. It is never forwarded upstream. When your request hits our edge, the Authorization header carrying gw-* is stripped before the request is routed to any provider. Forwarding it would be a bug—and it's a bug we've designed against at the middleware level.

This separation means a leaked gateway key gives an attacker access to your ModelPlane usage, not to your underlying provider accounts. They could burn your credits, but they can't exfiltrate your OpenAI or Anthropic keys. That's a meaningful reduction in blast radius.

How Backend Credentials Are Stored

When you upload a provider key to ModelPlane, it doesn't go into a database table where a SQL injection or a rogue admin could read it. It goes into Cloudflare Workers KV, encrypted.

The encryption model is worth understanding because it's the difference between "we encrypt your data" and "we encrypt your data in a way that actually protects you."

Each credential is stored under a key that includes your user ID: cred:{userId}:{credId}. The payload is encrypted with AES-GCM, using a root key from a Worker secret. But here's the important part: that root key isn't used directly. It's fed through HKDF with your user ID as the salt, deriving a per-user encryption key.

This means two things. First, your credentials are encrypted at rest with a key that's unique to you. Second, when the gateway needs to use a credential, it decrypts only the keys belonging to the current tenant. Cross-tenant access—even by accident—is rejected at the hydration layer.

We also never cache decrypted credentials. The routing cache may hold model-group metadata, but the API keys themselves are only decrypted in memory for the duration of a request, then discarded.

What This Means for Your Threat Model

Let's be concrete about what this architecture protects against.

Scenario 1: A gateway key leaks. An attacker gets a gw-* token from a compromised service. They can make requests through your model groups, spending your credits. But they cannot extract your provider keys, and they cannot access your other tenants' data. You revoke the key, and the attack surface closes.

Scenario 2: A database backup leaks. Supabase, Workers KV, S3—any of these could theoretically be compromised. But the provider keys in KV are AES-GCM encrypted with per-user keys. The gateway key hashes in Supabase are SHA-256, which means the plaintext is unrecoverable. A backup leak is a nuisance, not a catastrophe.

Scenario 3: An insider threat. A ModelPlane employee with database access still can't read your provider keys. The encryption keys are in Worker secrets, not in the database. This is the BYOK promise: you're not trusting us with your keys, you're trusting us with encrypted keys.

The Practical Setup

Here's what this looks like in practice. You sign up for ModelPlane, add your first backend, and point your OpenAI-compatible client at our endpoint.

First, add a backend credential. You can do this through the portal or via the API:

curl -X POST https://modelplane.dev/api/backends \
  -H "Authorization: Bearer $GATEWAY_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "provider": "openai",
    "apiKey": "sk-your-openai-key",
    "options": { "models": ["gpt-4o", "gpt-4o-mini"] }
  }'

The response gives you a backendId. You reference that ID in a model group, which defines the routing strategy. Then your application only ever sees the gateway key.

Here's a complete Python example using the standard openai client:

from openai import OpenAI

client = OpenAI(
    base_url="https://modelplane.dev/v1",
    api_key="gw-your-gateway-key",  # never a provider key
)

# "prod-chat" is a model group, not a provider model ID
response = client.chat.completions.create(
    model="prod-chat",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain BYOK security in one paragraph."},
    ],
)

print(response.choices[0].message.content)

Notice what's missing: there's no provider key in this code. There's no sk- or anthropic- token. If this file leaks to GitHub, the attacker gets a gateway key that can be revoked in seconds, not a provider key that gives them direct access to your OpenAI account.

The Gateway Key Lifecycle

Gateway keys are designed to be ephemeral and revocable. When you create one via POST /api/keys, the plaintext is returned exactly once. After that, only the SHA-256 hash is stored in Supabase. There's no "forgot my key" recovery—you create a new one and rotate.

This is a feature, not a bug. It means a key that's been exposed in logs, in a leaked .env file, or in a commit history is worthless to anyone who finds it later. You can't look up the plaintext, and neither can we.

ModelPlane also distinguishes between Private and Shared keys. Private keys belong to an individual member and are scoped to their permissions. Shared keys are created by org owners or admins and can be used across a workspace. This lets you give a CI/CD pipeline its own key, then revoke it without affecting your developers' keys.

Why This Matters for Teams

If you're a solo developer, the security model matters because it's one less thing to worry about. If you're an engineering lead at a company with 50 engineers, it's a compliance requirement.

The multi-tenant design means each workspace gets its own isolated credential store. Your team's keys are encrypted with your workspace's derived keys. Another team on the same ModelPlane instance—even another workspace in your own organization—cannot access them.

This is the difference between a gateway that's bolted onto your stack and one that's designed for multi-tenancy from the ground up. The tenancy model (User/Workspace/Billing Account) is baked into every resource: API keys, backends, model groups, usage logs. When you audit who has access to what, the answer is clear.

The Honest Tradeoffs

No security model is perfect, and we're not going to pretend otherwise.

First, BYOK means you're responsible for your provider keys. If you upload a key to ModelPlane and then leak it elsewhere, that's on you. We encrypt it at rest, but we can't protect you from your own .env file hygiene.

Second, there's a trust boundary you're accepting: you're trusting ModelPlane to handle your keys correctly. We've designed the system so that a compromise of our infrastructure doesn't expose your plaintext keys, but you're still relying on our encryption implementation being correct.

Third, the latency/accuracy tradeoff in billing means usage accounting is eventually consistent. That's a billing concern, not a security one, but it's worth knowing that the balance gate is a pre-request snapshot, and deductions happen asynchronously.

The Bottom Line

The provider key is the crown jewel of your LLM infrastructure. It's the credential that can read your prompts, generate your responses, and spend your money. Treating it like a regular API key—hardcoded in services, shared across teams, stored in plaintext—is a risk that will eventually materialize.

ModelPlane's BYOK model gives you a clean separation: gateway keys for your applications, encrypted backend credentials for your providers, and a routing layer that never confuses the two. The gateway key is auth; the backend key is access. Keeping those separate is the foundation of a secure LLM stack.

Start free with $5 in credits—no card required. Add your first backend, create a model group, and see how the security model holds up under real traffic. Your provider keys will thank you.

This post is part of our series on building production-grade LLM infrastructure. Here's what we've covered and what's coming:

One endpoint, every model — why we built ModelPlane and the core abstractions of model routing
Routing strategies, explained — fallback, load-balance, and conditional routing patterns
Model groups — the one abstraction that decouples your app from providers
Bring your own keys, safely — the BYOK security model (you are here)
High availability for LLM apps — a fallback playbook for provider outages
Stop paying twice — route your coding-plan quota into production
Price-aware routing — cut your LLM bill without changing models
Credits, usage & billing, explained — one transparent bill across all providers
1600+ models, one API — the ModelPlane provider catalog
One gateway, two regions — routing to global and China models
ModelPlane for teams — orgs, workspaces & shared keys
Meet the assistant — an AI helper that lives in your gateway dashboard
Inside the ModelPlane routing engine — how a single request flows from auth to upstream
One thinking parameter, every model — unified reasoning across providers
The system prompt belongs at the router — per-model-group system-prompt injection

Credits, usage & billing, explained

ModelPlane — Fri, 07 Aug 2026 03:43:29 +0000

Credits, usage & billing, explained

The problem with LLM usage tracking isn't that it's hard to measure tokens — it's that every provider gives you a different dashboard, and none of them tell you what you actually spent through your application. When you route through a gateway, usage becomes a property of your architecture, not a per-provider afterthought. ModelPlane gives you one surface for usage across all your model groups and providers, so you can answer "what did this feature cost?" without stitching together five vendor consoles.

The multi-dashboard problem

If you've built an AI feature in production, you know the ritual. You have a script, a spreadsheet, or a half-remembered URL for each provider's usage page. OpenAI shows you one number. Anthropic shows you another. DeepSeek shows you a third. And none of them map to the requests your application actually made, because your application doesn't call providers directly — it calls a model group.

The mismatch is the core issue. Provider dashboards are organized by provider account. Your costs are organized by feature, customer, or environment. When you hard-code model="gpt-4o" in your code, you've coupled your application's cost structure to a single vendor's accounting. You can't see which of your features is burning credits, which customer tier is expensive to serve, or whether your fallback routing is quietly sending traffic to a premium model you didn't intend.

This is why the abstraction matters. When request.model is a name you control — a model group like prod-chat — usage tracking becomes something you can actually act on.

One surface for usage across model groups

ModelPlane's billing model is built around a single concept: credits. One credit is equivalent to one US dollar. Every request through the gateway is accounted for in credits, regardless of which provider answered it. The cost formula is straightforward:

cost = (inputTokens × inputUnitPrice + outputTokens × outputUnitPrice) / 1_000_000

The pricing for each model lives in the provider catalog, so the gateway can compute the cost of every request without you configuring per-model rates. This means your usage logs are not just token counts — they're actual spend, normalized across every provider.

The usage_logs table is the heart of this. Each entry records the request's dimensions: which API key made the call, which model group it went through, and which underlying model actually answered. That last point is critical. With fallback routing, the model that answers isn't always the one you asked for. Your logs show you the ground truth.

This gives you three views of your spend that provider dashboards can't:

By model group. What does prod-chat cost this month? What about customer-support or internal-coding? Each group is a line item.
By API key. Which application or service is consuming the most? Private keys per developer, shared keys per environment.
By model. Even within a group, you can see the split — how much traffic went to the fast tier versus the premium fallback.

The balance gate: what happens before a request runs

ModelPlane's billing is designed to be low-latency and simple. Before a request runs, the gateway checks your credit balance. If you're out of credits, you get a 402 insufficient_credits response — fast, clear, and before any upstream call is made. This is the pre-request balance gate.

This design is a deliberate tradeoff. The gateway doesn't do a strong-consistency reservation of funds before every request. That would add latency to every single call. Instead, it takes a snapshot of your balance, gates the request, and then accounts for the cost after the request completes, asynchronously.

The result is that billing is fast enough to sit on the hot path of every inference call. The tradeoff is that the balance you see can be slightly stale — up to 300 seconds, in some cases, because the balance may be served from a cache. For almost all use cases, this is the right trade. You don't want your LLM gateway to be the slow part of your stack, and you don't want it to reject requests because of a race condition in accounting.

What happens after: async accounting

Once the request completes, the gateway calculates the cost and deducts it from your balance. This is a fire-and-forget operation — it doesn't block the response to your application. The response you get is the model's output; the accounting happens in the background.

This design is simple and fast, but it's worth being honest about the edge cases. The async deduction has no retry mechanism in its current implementation. If the background task fails, or the Worker running it is evicted, the usage row and the deduction could be silently lost. This is a known reliability gap, documented in the project's internal PRD. For most teams, this is acceptable — the cost of a few lost usage rows is far less than the cost of adding strong consistency to every request. But if you're building billing infrastructure on top of usage logs, you should know the data is eventually consistent, not exactly-once.

The system also handles the accounting atomically at the database level. The consume_credits RPC is atomic and allows a brief negative balance, which prevents a burst of concurrent requests from being rejected due to a race between the balance check and the deduction.

Plans and pricing: what you actually pay

ModelPlane's pricing is designed to be transparent. There are three tiers:

Free: You get $5 in credits on signup. No card required. This is enough to build and test a real feature.
Basic: $10 per month, which includes $15 in credits. You can top up with additional credits as needed.
Unlimited: A special plan for high-volume internal use cases, where the credit gate is bypassed entirely.

On top of the plan, there's a flat gateway fee of approximately $0.02 per 1 million tokens. This is the cost of routing — the abstraction, the fallback, the unified billing. It's a rounding error compared to the cost of the tokens themselves, but it's what pays for the gateway to exist.

The key number to understand is the included credit. On the Basic plan, you pay $10 and get $15 in credits. That means the effective cost of the gateway is negative — you're getting more credit than you pay for, and the gateway fee comes out of the included credit. The math works because the gateway fee is so small relative to model costs.

Seeing it in action

Here's what it looks like to point your OpenAI client at ModelPlane and see usage flow into one dashboard:

from openai import OpenAI

client = OpenAI(
    base_url="https://modelplane.dev/v1",
    api_key="gw-your-gateway-key"
)

# "prod-chat" is a model group, not a provider model id.
# It routes across your configured backends with your chosen strategy.
response = client.chat.completions.create(
    model="prod-chat",
    messages=[
        {"role": "user", "content": "Summarize this week's support tickets."}
    ]
)

print(response.choices[0].message.content)

That's it. Your application code doesn't know or care which provider answered. It doesn't know whether the request went to a coding-plan quota or a pay-per-token endpoint. It just knows it called prod-chat.

After this request runs, you can query the usage API to see exactly what happened:

curl -H "Authorization: Bearer gw-your-gateway-key" \
  https://modelplane.dev/api/usage

The response shows you the request's token count, the cost in credits, the model group, the API key, and the actual model that served it. This is the data you need to answer the questions that matter: "What does this feature cost per user?" "Is my fallback routing sending too much traffic to the expensive model?" "Which customer tier is burning through credits?"

The honest tradeoff: latency vs. accuracy

Every billing system makes a tradeoff between accuracy and latency. ModelPlane's choice is explicit: low latency wins. The pre-request balance check is a snapshot, not a reservation. The post-request deduction is asynchronous, not synchronous. This means your requests are never slowed down by billing logic, and your gateway can handle high concurrency without becoming a bottleneck.

The cost of this choice is that your balance and usage data are eventually consistent. You might see a request in your logs a few seconds after it completes. Your balance might be slightly stale for a few minutes. And in rare failure cases, a usage row might be lost entirely.

For most teams, this is the right trade. If you're building a customer-facing billing system on top of your LLM usage, you should build your own reconciliation layer. But if you want to see what your AI features cost, and you want to catch a runaway model group before it burns through your credits, this is more than sufficient.

Why this beats provider dashboards

The multi-dashboard problem isn't just inconvenient — it's a blind spot. When you can't see usage by model group, you can't see which features are profitable. When you can't see usage by API key, you can't see which developer or service is consuming the most. When you can't see the actual model that answered, you can't tell if your fallback strategy is working as intended.

ModelPlane gives you the dimensions that matter for running an AI product. The abstraction of the model group — the same abstraction that decouples your code from providers — also decouples your accounting from providers. Your costs are organized the same way your architecture is organized.

This is the point. The provider is the wrong unit of integration for your code, and it's the wrong unit of integration for your billing. ModelPlane makes the model group the unit of both. One endpoint, one dashboard, one bill.

The bottom line

LLM usage tracking doesn't have to be a nightmare of spreadsheets and vendor consoles. When you route through a gateway, usage becomes a first-class property of your application. ModelPlane gives you one surface to see it all: credits, spend, and usage across every model group and provider.

The system is fast because it's simple. The balance gate is a snapshot, the deduction is async, and the accounting is eventually consistent. That's the right trade for a gateway that sits on every inference call.

If you're tired of stitching together provider dashboards, point your OpenAI client at ModelPlane and see your usage in one place.

See your usage in one dashboard — start free with $5 in credits, no card required.

This post is part of a series on building production-grade LLM applications with ModelPlane.

The ModelPlane series:

DEV Community: ModelPlane

Embedding Groups: Dimension-Aware Routing for /v1/embeddings

Embedding Groups: Dimension-Aware Routing for /v1/embeddings

The abstraction: model groups, not model IDs

Why dimension mismatch is a silent production killer

How embedding groups enforce the contract

What you can build with dimension-aware groups

A concrete example

The dimension check is the feature

Beyond embeddings: the same principle applies

Start with a free account

Multimodal-aware routing: vision requests find vision-capable backends

The problem: one model for every modality is expensive and fragile

Model groups: the abstraction that makes this possible

How multimodal-aware routing works

A concrete example: DeepSeek-first, vision-capable fallback

Why routing beats client-side modality checks

Combining multimodal filtering with other routing strategies

What this means for your LLM stack

Inside the ModelPlane routing engine

Inside the ModelPlane routing engine

The request lifecycle: auth, resolve, gate, route, account

Stage 1: Authentication — the gw-* key is auth only, never forwarded

Stage 2: Config resolution — request.model is a name you control

Stage 3: The billing gate — a snapshot, not a lock

Stage 4: The routing engine — walking the target tree

Stage 5: Accounting — off the hot path, but not off the record

Why the provider abstraction matters

Multi-tenancy without slowing down inference

The target tree is the unit of resilience

The hot path stays hot

Build your first target tree

ModelPlane for teams: orgs, workspaces & shared keys

ModelPlane for teams: orgs, workspaces & shared keys

The multi-tenant mess you're actually in

Three entities, one clean boundary

Key scopes: private vs. shared

Setting up an organization

Routing that respects team boundaries

Code: one client, scoped by key

Billing that doesn't require a spreadsheet

The honest limitations

Stop sharing keys, start sharing infrastructure

Bring Your Own Keys, Safely: The BYOK Security Model Behind ModelPlane

Bring Your Own Keys, Safely: The BYOK Security Model Behind ModelPlane

The Wrong Unit of Integration

Two Keys, Two Jobs

How Backend Credentials Are Stored

What This Means for Your Threat Model

The Practical Setup

The Gateway Key Lifecycle

Why This Matters for Teams

The Honest Tradeoffs

The Bottom Line

Credits, usage & billing, explained

Credits, usage & billing, explained

The multi-dashboard problem

One surface for usage across model groups

The balance gate: what happens before a request runs

What happens after: async accounting

Plans and pricing: what you actually pay

Seeing it in action

The honest tradeoff: latency vs. accuracy

Why this beats provider dashboards

The bottom line

Stage 1: Authentication — the `gw-*` key is auth only, never forwarded

Stage 2: Config resolution — `request.model` is a name you control