A couple of years ago, "OpenAI compatible" was mostly a marketing phrase — a way for a smaller provider to say "we're kind of like OpenAI, but cheaper." Today it means something more specific and more useful: a growing number of providers, both open-source and commercial, expose the exact same request/response shape that OpenAI's Chat Completions API uses. Same JSON fields, same streaming format, same SDK compatibility.
I think that's worth paying attention to for a reason that has nothing to do with any single vendor: it's quietly turning into a de facto API standard for the LLM space, the same way SQL became a de facto standard for relational databases even though every database has its own extensions and quirks underneath.
Why a de facto standard matters more than a "better" API
Nobody designed the OpenAI Chat Completions format as an industry standard on purpose. It became one for a boring but powerful reason: it got adopted first, tooling was built around it (LangChain, LlamaIndex, countless internal SDKs), and switching costs for anyone not supporting it kept rising. Once enough of the ecosystem assumes a given request shape, matching it becomes the path of least resistance for new entrants — not because it's the best possible design, but because compatibility is worth more than a marginally cleaner API.
This is a familiar pattern outside AI too. SQL isn't universally loved as a language design, but "just support SQL" became the rational choice for new databases because the alternative was asking every user to rewrite their queries. REST won over more "correct" API styles for similar reasons. Standards compound: the more things support them, the more costly it becomes not to.
What this actually changes for developers
If you build on top of LLM APIs, the practical effect of this convergence is portability. In principle, code written against the OpenAI SDK format should be able to point at a different provider's endpoint by changing a base URL and an API key, without rewriting request-building or response-parsing logic. In practice, this holds reasonably well for the core chat completion flow, and less well for provider-specific features (function calling variants, extended context handling, multimodal inputs) that haven't fully converged yet.
That partial convergence is, I think, the more interesting story than "everyone supports OpenAI's format now." A handful of things I've noticed working across providers that expose an OpenAI-compatible endpoint:
Basic chat completion and streaming — generally solid
Standard parameter names (temperature, max_tokens, etc.) — generally solid
Function/tool calling — inconsistent enough that I still test it per-provider before relying on it
Vendor-specific extras (like some providers' native reasoning-trace fields) — not portable, and probably shouldn't be treated as if they will be
This matters for how teams should think about vendor lock-in. A lot of the "avoid lock-in" advice in this space treats OpenAI-compatible endpoints as a full guarantee of portability. A more accurate framing is: the compatibility layer meaningfully lowers switching costs for the common path, but doesn't eliminate them for anything provider-specific you've come to depend on.
Where this shows up in practice
A number of gateway-style services have emerged specifically to sit on top of this convergence — offering one OpenAI-compatible endpoint that routes to multiple underlying models. RouteAI is one example I've used for this; the useful part isn't the gateway itself so much as what it implies: enough providers now agree on a shared request format that building a router on top of it is even feasible. That wouldn't have made sense a couple of years ago when every provider's API looked meaningfully different.
I'd expect this convergence to keep going for the common cases (chat, streaming, basic tool calls) while staying genuinely fragmented for anything closer to the frontier of what a given model can do — long-context handling, native multimodal input, or model-specific reasoning features. Standards tend to solidify around the stable, well-understood parts of a technology and stay contested around the parts that are still evolving.
The takeaway
"OpenAI compatible" is worth treating as an emerging API convention rather than a vendor claim to be skeptical of. It doesn't make every LLM interchangeable, and it doesn't remove the need to actually test a model for your use case — but for the core request/response shape, it's increasingly safe to build against it, the way you'd build against SQL knowing full well that vendor-specific extensions exist underneath.
Curious whether others building on multiple LLM providers have hit the same split — solid on the basics, fragmented on the advanced features — or whether your experience has been different.
TL;DR: "OpenAI compatible" has evolved from a marketing phrase into a de facto API convention for LLMs, similar to how SQL became a standard for databases. Basic chat/streaming is portable across providers; advanced features (tool calling variants, provider-specific extras) still aren't — so treat compatibility as reducing switching costs, not eliminating them.
Here's the tool I referenced in this post: www.fastrouteai.com
Top comments (0)