DEV Community

shashank ms
shashank ms

Posted on

Integrating OpenAI SDK with LLMs

#aiinfrastructure #oxlo #ai

The OpenAI SDK has become the default client for LLM applications not because it is the only option, but because it established a clean, predictable pattern for chat completions, embeddings, and multimodal tasks. For developers, this means less boilerplate and faster iteration. For providers, compatibility with this SDK is now a baseline requirement. Oxlo.ai supports the full OpenAI SDK specification out of the box, using a single base URL swap to route requests to more than 45 open-source and proprietary models across seven categories.

The SDK as a Universal Interface

When you build with the OpenAI SDK, you are not just targeting one provider. You are using a chat.completions.create abstraction that has been adopted across the industry. This method handles HTTP negotiation, SSE streaming, token counting, and tool schemas so you do not have to. Oxlo.ai exposes the exact same surface area, including streaming responses, function calling, JSON mode, vision inputs, and multi-turn conversations. If your application already uses the SDK for prototyping, moving to Oxlo.ai requires no new dependencies.

Configuring the Client for Oxlo.ai

The only change is configuration. Point the client at https://api.oxlo.ai/v1, pass your Oxlo.ai API key, and select any model from the catalog.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_OXLO_API_KEY"
)

response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[{"role": "user", "content": "Explain request-based pricing."}],
    stream=False
)

print(response.choices[0].message.content)

This pattern is identical whether you are calling general-purpose models, deep reasoning models such as DeepSeek R1 671B MoE, or agentic coding models like Kimi K2.6. The SDK does not care who hosts the weights as long as the response schema matches, and Oxlo.ai returns fully compatible payloads.

Core Features That Travel with You

Most production workloads need more than a single string response. They need streaming, structured output, and tool use. Oxlo.ai supports all of these through the standard SDK methods.

Streaming reduces time-to-first-token and improves perceived latency:

stream = client.chat.completions.create(
    model="deepseek-r1-671b",
    messages=[{"role": "user", "content": "Write a Python function to validate an email address."}],
    stream=True
)

for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

Function calling and JSON mode follow the same schemas you already use. You can attach tool definitions to tools and set response_format={"type": "json_object"} without client-side branching. Vision inputs using image URLs or base64 strings also work through the standard messages payload, enabling models like Gemma 3 27B and Kimi VL A3B.

Beyond Chat: Embeddings, Audio, and Images

The SDK is not limited to chat. Oxlo.ai exposes the same endpoint conventions for auxiliary tasks.

Top comments (0)