Leveraging Oxlo for Voice Assistant LLMs

Building a voice assistant that feels natural requires more than a fast language model. You need streaming speech-to-text, low-latency text-to-speech, and an LLM layer that handles multi-turn context, tool calls, and interruptions without ballooning costs as conversations grow longer. Oxlo.ai provides the full stack for this workflow, with request-based pricing, no cold starts, and native OpenAI SDK compatibility.

Architecture of a Voice LLM Stack

A production voice assistant typically runs in three stages: audio ingestion, inference, and audio playback. The ingestion layer transcribes user speech. The inference layer processes the transcript against conversation history, system prompts, and available tools. The playback layer streams synthesized speech back to the user. Latency at every stage compounds, so the inference layer must support streaming responses and function calling without unnecessary overhead.

Oxlo.ai offers streaming responses, function calling, JSON mode, and multi-turn conversations out of the box. Because the platform uses request-based pricing, long transcripts and extended conversational context do not linearly increase your inference cost in the same way token-based providers do. This makes Oxlo.ai particularly suitable for voice agents that maintain state across dozens of turns.

Audio Ingestion and Playback

Oxlo.ai hosts Whisper Large v3, Whisper Turbo, and Whisper Medium for audio transcriptions, plus Kokoro 82M for text-to-speech. These endpoints are fully OpenAI API compatible, so you can point your existing transcription and speech clients at https://api.oxlo.ai/v1 without rewriting your audio pipeline.

For voice assistants, Whisper Turbo is often the right balance of speed and accuracy for real-time use. Kokoro 82M delivers natural sounding speech with minimal latency. Both are available under the same request-based model, which simplifies cost forecasting compared to token-based or per-second audio pricing schemes. See the Oxlo.ai pricing page for current plan details.

The LLM Layer for Voice Agents

The core of the assistant is the chat model. Voice conversations demand streaming tool use, vision support for multimodal inputs, and large context windows for session memory. Oxlo.ai offers several strong candidates:

• Qwen 3 32B: Optimized for multilingual reasoning and agent workflows.
• Llama 3.3 70B: General-purpose flagship with broad tool compatibility.
• Kimi K2.6: Advanced reasoning, agentic coding, and vision with a 131K context window.
• DeepSeek V4 Flash: Efficient MoE architecture with a 1M context window, ideal for long-running sessions where full conversation history must remain in context.
• DeepSeek V3.2: Strong coding and reasoning model, also available on the free tier.

All of these models support streaming, function calling, and vision where applicable. Because Oxlo.ai charges per request rather than per token, you can keep a large rolling context window or include verbose system prompts without the cost scaling that typically discourages long-context voice agents on token-based platforms.

Voice Pipeline Example

Below is a minimal Python pattern showing how to wire transcription, inference, and speech synthesis through Oxlo.ai using the OpenAI SDK. This example assumes you have captured audio bytes and want to stream a spoken response.

from openai import OpenAI

client = OpenAI(
    api_key="your-api-key",
    base_url="https://api.oxlo.ai/v1"
)

# 1. Transcribe user audio
with open("user_prompt.wav", "rb") as audio_file:
    transcript = client.audio.transcriptions.create(
        model="whisper-large-v3-turbo",
        file=audio_file
    )

# 2. Stream LLM response with tool use enabled
stream = client.chat.completions.create(
    model="kimi-k2-6",
    messages=[
        {"role": "system", "content": "You are a concise voice assistant."},
        {"role": "user", "content": transcript.text}
    ],
    tools=[{
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string"}
                },
                "required": ["location"]
            }
        }
    }],
    stream=True
)

# Collect text for TTS
response_text = ""
for chunk in stream:
    delta = chunk.choices[0].delta
    if delta.content:
        response_text += delta.content
        # You can also stream to a WebSocket or audio player here

# 3. Synthesize speech
speech = client.audio.speech.create(
    model="kokoro-82m",
    voice="af_bella",
    input=response_text
)

speech.stream_to_file("assistant_response.wav")

This pattern works with any model on Oxlo.ai. Switching from Kimi K2.6 to Llama 3.3 70B or DeepSeek V4 Flash is a single parameter change, with no cold starts to interrupt the user experience.

Cost and Latency Advantages

Voice assistants are inherently stateful. A single session can accumulate thousands of tokens of context across system instructions, tool schemas, and prior turns. On token-based providers, this means every new user utterance becomes more expensive as the conversation progresses.

Oxlo.ai uses flat per-request pricing. A request costs the same whether you send a short prompt or a long transcript with full conversation history. For voice agents, this removes the penalty for maintaining rich context and allows you to design more helpful, verbose system prompts without cost anxiety. In many long-context and agentic workloads, this model can be significantly cheaper than token-based alternatives. Details are available on the Oxlo.ai pricing page.

Additionally, Oxlo.ai offers no cold starts on popular models. That matters for voice, where a multi-second delay before the first streamed token makes the assistant feel broken.

Choosing the Right Model

Match the model to the assistant's job:

• General knowledge and tool use: Llama 3.3 70B or Qwen 3 32B.
• Coding or technical reasoning: Kimi K2.6, DeepSeek V3.2, or Minimax M2.5.
• Long sessions with full history: DeepSeek V4 Flash, with its 1M context window and efficient MoE architecture.
• Multilingual deployments: Qwen 3 32B or GLM 5.
• Vision-enabled assistants: Kimi K2.6 or Gemma 3 27B for image inputs during voice sessions.

All models are accessible through the same endpoint and SDK, so A/B testing and fallback routing require no infrastructure changes.

Conclusion

Voice assistants demand a platform that handles audio, streaming inference, and stateful context without cost surprises or cold-start delays. Oxlo.ai provides the necessary audio models, LLMs, and request-based pricing structure to build these systems efficiently. If you are evaluating infrastructure for a voice agent, run your next prototype on Oxlo.ai and compare latency and cost against your current token-based provider.