Using Oxlo with OpenAI SDK for LLM

#aiinfrastructure #oxlo #ai

Most inference providers force you to rewrite client code or manage fragmented SDKs. Oxlo.ai removes that friction. Because the platform is fully OpenAI SDK compatible, you can point your existing Python or Node.js client to Oxlo.ai and immediately access 45+ models across seven categories, from reasoning LLMs to vision and audio, without changing your application logic. The base URL is https://api.oxlo.ai/v1, and authentication uses a standard API key.

Why OpenAI SDK Compatibility Matters

Switching inference backends usually means refactoring HTTP clients, rewriting error handling, or maintaining multiple SDK versions. Oxlo.ai avoids all of that. The platform exposes chat/completions, embeddings, images/generations, audio/transcriptions, and audio/speech endpoints that follow the OpenAI schema exactly. That means streaming responses, function calling, JSON mode, vision inputs, and multi-turn conversations work out of the box with the official openai Python package or Node.js SDK.

For teams running agentic workflows or long-context pipelines, this drop-in behavior is not just convenient. It protects engineering time and eliminates migration risk. You keep the same retry logic, the same token counting helpers, and the same streaming parsers. The only difference is the base URL and the API key.

Getting Started

Sign up for an account and grab an API key. Oxlo.ai offers a free tier with 60 requests per day and access to 16+ models, including a 7-day full-access trial. If you need higher throughput, the Pro plan provides 1,000 requests per day across all models, while Premium adds priority queue access at 5,000 requests per day. Enterprise customers can get dedicated GPUs and unlimited requests. For exact plan details, see https://oxlo.ai/pricing.

Install the official OpenAI SDK.

Python

pip install openai

Node.js

npm install openai

Set your environment variable.

export OXLO_API_KEY="your-api-key"

Instantiate the client with the Oxlo.ai base URL.

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key=os.environ.get("OXLO_API_KEY")
)

Chat Completions Example

The following example calls Llama 3.3 70B, the general-purpose flagship. You can swap the model string for Qwen 3 32B, DeepSeek R1 671B MoE, GPT-Oss 120B, DeepSeek V4 Flash, Kimi K2.6, or GLM 5 without changing any other code.

response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[
        {"role": "system", "content": "You are a precise technical assistant."},
        {"role": "user", "content": "Explain request-based inference pricing."}
    ],
    temperature=0.2
)

print(response.choices[0].message.content)

Handling Streaming Responses

Streaming uses the same interface you already know. Because Oxlo.ai keeps popular models warm, there are no cold starts, so the first chunk arrives with predictable latency.

stream = client.chat.completions.create(
    model="deepseek-r1-671b-moe",
    messages=[{"role": "user", "content": "Write a Python function that validates email addresses."}],
    stream=True
)

for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

Tool Use and Function Calling

Oxlo.ai supports function calling and tool use on compatible models. This is critical for agentic workloads. The snippet below registers a weather tool with the model and lets the LLM decide when to invoke it.

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Get the current weather in a given location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string"},
                    "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
                },
                "required": ["location"]
            }
        }
    }
]

response = client.chat.completions.create(
    model="qwen3-32b",
    messages=[{"role": "user", "content": "What is the weather in Berlin?"}],
    tools=tools,
    tool_choice="auto"
)

print(response.choices[0].message.tool_calls)

Vision and Multimodal Inputs

Vision models such as Gemma 3 27B and Kimi VL A3B accept image inputs through the standard chat/completions format. Pass image URLs or base64 strings inside the message content array, exactly as you would with the OpenAI API. JSON mode is also available for structured output.

When to Choose Oxlo.ai

Oxlo.ai uses request-based pricing: one flat cost per API request regardless of prompt length. Unlike token-based providers

DEV Community