Most of an adapter layer is renaming. The part that decides whether the migration lands is the handful of capabilities the target provider does not have, where there is nothing to rename because there is nothing on the other side.
A gap is not a rename
It is worth being precise about the difference, because the two failure modes are opposite. A rename is a field that exists on both sides under different names: OpenAI’s Chat Completions API takes the standing instruction as a message with role system in the messages array, Anthropic’s Messages API takes it as a top-level system parameter alongside messages, and Google’s Gemini API takes it as systemInstruction alongside contents. Three names, one concept, and a table settles it. If you get a rename wrong you find out immediately, because the request is rejected or the instruction visibly stops working.
A gap is different in kind. The source provider offers a guarantee the target does not offer at any price: decoding constrained to a JSON Schema, more than one tool invocation in a single assistant turn, token log probabilities, a reproducible seed. There is no correct value to put in a field. You either give the guarantee up, reconstruct something that resembles it in your own code, or refuse to run on that provider.
Gaps are dangerous mainly because of how they present. The request does not fail. Many providers, and almost every OpenAI-compatible endpoint in front of an open-weights model, accept unknown parameters and ignore them. You send response_format with a JSON Schema; the endpoint returns 200; the output is prose. You send seed; you get 200 and non-reproducible output. Nothing in the response says the parameter was discarded, so the failure surfaces later as a parse error in production rather than as a 400 during the port. Assume silent acceptance is the default and verify each capability with a request whose response proves the parameter took effect.
Four responses to a gap, one of them wrong
When your adapter meets a request it cannot express on the target provider, it has exactly four things it can do, and choosing between them per capability — deliberately, once, in one place — is most of what a good adapter layer is.
- Emulate. Rebuild the behaviour client-side. Structured output becomes prompt instruction plus schema validation plus a repair turn. Parallel tool calls become a serialised loop. The shape the caller sees is preserved; the guarantee usually is not, and the difference is what you must document.
- Degrade, loudly. Drop the capability and say so in a place a human sees. A request that asked for
logprobsand ran on a provider without them should come back with a field on your own response object recording that the capability was unavailable, so downstream code can branch instead of readingundefined. - Reject at configuration time. If a workload genuinely requires a capability — an evaluation harness that needs reproducible sampling, a classifier that reads token probabilities — the right behaviour is for the route to fail to start, not for the ten-thousandth request to behave oddly. This is the cheapest failure available and the one most often skipped.
- Drop it silently. This is the wrong one, and it is the default that happens when nobody chooses. Passing the caller’s parameter bag straight through to a provider that ignores unknown keys is the same as choosing this.
The gaps you will actually hit
Sorted roughly by how often they stop a port, with the honest verdict on each. The details of who supports what change; the shape of the problem does not.
Schema-constrained output
The strongest form of this is not a prompt instruction but a decoding constraint: the sampler is restricted at each step to tokens that keep the output a valid instance of a supplied JSON Schema, so an invalid document is not merely unlikely, it is unreachable. OpenAI documents this as Structured Outputs under response_format with a json_schema type and a strict flag, and documents a subset of JSON Schema that the constraint supports (OpenAI, Structured Outputs guide). A weaker form, often called JSON mode, guarantees only that the output parses as JSON, not that it matches your schema. A third tier is nothing at all. Shimmable in shape, not in guarantee — the shim and its failure modes are a page of their own.
Several tool calls in one turn
One assistant turn containing two or more independent tool invocations, which you execute concurrently and return together. Where the provider emits at most one, the shim is a serialised loop, and it is not transparent: the model sees each result before choosing the next call, which changes what it asks for. Serialising parallel tool calls covers the loop and the semantics it changes.
Token log probabilities
Per-token scores for the chosen token and the top alternatives, used for confidence gating, classification by scoring candidate labels, and detecting hedging. Not shimmable at all — see below.
Deterministic sampling
A seed parameter that makes repeated identical requests likely to return identical output. Best effort even where it exists, and absent on many providers. This library already covers what to do when the provider has no seed; the short version is that temperature 0 narrows the variance without removing it and is not a substitute.
Sampling parameters that do not exist on both sides
top_k is available on some chat APIs and not others; frequency and presence penalties are an OpenAI-shaped idea with no direct counterpart in every other API; logit_bias is narrower still. Ranges differ even where names match, which is the subject of mapping sampling parameters between APIs.
Assistant prefill
Ending the request with a partial assistant turn so the model continues it rather than starting fresh — the cheapest way to force an output to begin with a specific character. Where it exists it is extremely useful and where it does not there is no equivalent, because the chat template on the server decides what the model is handed.
Stop sequences, counts and semantics
Every chat API takes stop strings, under stop or stop_sequences or inside a generation-config object, but the maximum count differs and so does whether the matched string is included in the returned text. A prompt that relied on four stop strings and moves to an API that accepts fewer needs its framing changed, not its parameters clamped.
Multiple completions per request
Asking for several independent samples in one call. Where the target has no such parameter, the shim is trivially n requests, and the only thing you lose is any prompt-processing saving the provider might have made once instead of n times.
Make the capability set data
The failure this section prevents is capability knowledge scattered across the codebase as conditionals on a provider name. Once four call sites test if provider === "x", adding a fifth provider means finding all four, and the one you miss is the outage. Put the capabilities in one table and branch on the capability, never on the vendor.
// capabilities.ts — one row per route, checked in one place.
export type Capabilities = {
schemaConstrainedOutput: "strict" | "json-only" | "none";
parallelToolCalls: boolean;
logprobs: boolean;
seed: boolean;
topK: boolean;
assistantPrefill: boolean;
maxStopSequences: number;
imageInput: "url-or-base64" | "base64-only" | "none";
};
// A route asks the question; it never asks who the provider is.
function planStructuredOutput(caps: Capabilities, schema: JSONSchema) {
switch (caps.schemaConstrainedOutput) {
case "strict": return { mode: "native", schema };
case "json-only": return { mode: "json-mode-plus-validate", schema };
case "none": return { mode: "prompt-plus-validate", schema };
}
}
Two rules make this hold up. First, the table is a claim about a route, not about a vendor: the same model family behind two different OpenAI-compatible gateways can have different real capabilities, because the capability lives in the serving stack. Second, every entry needs a probe — a small request whose response distinguishes support from silent acceptance — that you can run against a route on demand. For schema-constrained output, ask for an object with one required integer field and a description that invites prose; if you get prose, the flag is a lie. For a seed, send the same request twice at a temperature above zero and compare. Without probes the table is a document, and documents go stale in the direction that hurts.
Three things you cannot shim
Being clear about these saves the week somebody would otherwise spend trying.
- Log probabilities. They are a byproduct of the forward pass. If the provider does not return them, the numbers do not exist anywhere on your side of the wire, and nothing you can compute from the text approximates them. Asking the model how confident it is produces a number that reads like a probability and is not one. If a feature depends on
logprobs, that feature constrains your provider list. - A true decoding constraint. A validation shim tells you afterwards that the output was wrong and lets you ask again. It cannot make invalid output impossible, because the sampling already happened. The difference is invisible when the model complies at a high rate and becomes the whole story in the tail — long outputs, deep nesting, enums with many members. Plan for a residual failure rate rather than assuming zero.
- Bit-exact reproducibility. Even with a seed, providers document this as best-effort, and batching, hardware and silent server updates all perturb it. Build evaluations that tolerate variance instead of building on an assumption of determinism — temperature zero is not determinism is the longer argument.
A capability table has to live somewhere, and the awkward part is that it is per-route rather than per-vendor, so every service that calls models ends up with its own copy drifting from the others. Multigrid keeps one behind a single API, which mainly means the answer to “does this route support schema-constrained output” is the same answer in every service. Building it yourself, the equivalent discipline is one module that owns the table and a probe suite that runs against it on a schedule.
Top comments (0)