Integrating OpenAI SDK with LLM: A Step-by-Step Guide

#aiinfrastructure #oxlo #ai

The OpenAI SDK has become the de facto standard for interacting with large language models. Its clean interface and broad language support mean developers can prototype quickly and ship faster. What many teams overlook is that the SDK is not limited to a single provider. Any inference platform that exposes an OpenAI-compatible API can serve as a drop-in backend, which lets you keep your existing client code while switching models, optimizing latency, or controlling costs.

Why Standard Interfaces Matter

Standardized client libraries reduce friction. When your application speaks the same API shape regardless of the underlying model, you avoid rewriting request logic, parsing custom response formats, or maintaining multiple HTTP clients. The OpenAI SDK handles retries, streaming parsers, and error normalization for you. For engineering teams, this means less vendor lock-in and faster iteration when evaluating new models.

Project Setup

You only need two things to get started: the official openai package and an API key from your chosen provider.

pip install openai

Store your credentials securely. Most teams use environment variables so that switching providers later is a one-line change.

Basic Chat Completion

Here is a minimal example in Python. The structure is identical whether you are calling OpenAI, Oxlo.ai, or any other compatible platform.

import os
from openai import OpenAI

client = OpenAI(
    base_url=os.getenv("LLM_BASE_URL"),
    api_key=os.getenv("LLM_API_KEY")
)

response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[{"role": "user", "content": "Explain request-based pricing in one sentence."}]
)

print(response.choices[0].message.content)

Switching to Oxlo.ai

Oxlo.ai is a developer-first AI inference platform that is fully OpenAI SDK compatible. Changing your backend is as simple as updating the base_url and api_key.

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key=os.getenv("OXLO_API_KEY")
)

From this point, every method you already use, streaming, function calling, JSON mode, and vision input, works without modification. Oxlo.ai offers more than 45 open-source and proprietary models across seven categories, including general-purpose LLMs such as Llama 3.3 70B and Qwen 3 32B, reasoning models such as DeepSeek R1 671B MoE and Kimi K2.6, code specialists such as Qwen 3 Coder 30B, and vision models such as Gemma 3 27B. There are no cold starts on popular models, so latency stays predictable.

Streaming and Tool Use

The OpenAI SDK abstracts streaming into an iterator. Enabling it on Oxlo.ai requires no syntax changes.

stream = client.chat.completions.create(
    model="deepseek-v4-flash",
    messages=[{"role": "user", "content": "Write a Python function that validates email addresses."}],
    stream=True
)

for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")

Function calling follows the same pattern. Define your tools in the tools parameter and handle the model's response exactly as you would with any other OpenAI-compatible endpoint.

Vision and JSON Mode

Multimodal requests and structured outputs are also supported. You can pass image URLs or base64-encoded data alongside text prompts, and you can constrain the model to emit valid JSON by setting response_format={"type": "json_object"}. Oxlo.ai supports vision through models such as Kimi VL A3B and Gemma 3 27B, and JSON mode is available across the chat completions endpoint.

When to Consider Request-Based Pricing

Most inference providers, including Together AI, Fireworks AI, OpenRouter, Replicate, and Anyscale, bill by the token. For short prompts this is straightforward, but costs scale linearly with input length. If you are building agentic workflows, RAG pipelines, or any system that sends long context windows repeatedly, token-based pricing can become expensive quickly.

Oxlo.ai uses request-based pricing: one flat cost per API request regardless of prompt length. For long-context and agentic workloads, this can be significantly cheaper than token-based alternatives. Oxlo.ai also provides a free tier with 60 requests per day across more than 16 models, plus a 7-day full-access trial, so you can validate the cost structure against your own traffic patterns before committing. See https://oxlo.ai/pricing for current plan details.

Conclusion

The OpenAI SDK is not tied to a single provider. By standardizing on base_url and api_key configuration, you can treat inference as a portable layer. Oxlo.ai fits naturally into this architecture. It preserves the SDK semantics you already know, adds a broad model catalog with no cold starts, and replaces token-based billing with flat per-request pricing. If your workloads are growing in context length or complexity, it is worth benchmarking Oxlo.ai alongside your current backend.