Voice-controlled interfaces have moved from novelty to infrastructure. Whether you are building a hands-free coding agent, a customer support bot, or an embedded device assistant, the architecture is usually the same: capture audio, transcribe speech to text, reason with a language model, synthesize a response, and stream it back to the user. The challenge is not just latency or accuracy. It is cost structure. Audio transcripts and multi-turn agentic sessions generate long contexts, and conventional token-based billing penalizes the exact conversational workloads that voice applications require. Oxlo.ai addresses this with a request-based pricing model that charges one flat cost per API call regardless of prompt length, making it a natural fit for voice pipelines that would otherwise accumulate expensive input tokens on every turn.
The Voice Agent Pipeline
A production voice interface typically runs four stages in a loop. First, audio capture records the user. Second, a speech recognition model converts the raw audio into text. Third, a large language model processes the text, maintains conversation history, and decides whether to respond directly or invoke a tool. Fourth, a text-to-speech model turns the reply into audio for the user. Oxlo.ai provides fully OpenAI-compatible endpoints for each of these stages: audio/transcriptions for speech-to-text, chat/completions for reasoning, and audio/speech for synthesis. Because every stage is reachable through the same base URL and SDK, you can prototype an entire voice agent without stitching together multiple providers.
Transcription with Whisper
Oxlo.ai hosts several Whisper variants, including Whisper Large v3, Whisper Turbo, and Whisper Medium. These cover the typical accuracy versus latency trade-off. Large v3 is ideal for offline or high-accuracy scenarios, while Turbo works well for real-time assistants that need fast turnaround. All are accessible through the standard OpenAI audio transcription interface.
from openai import OpenAI
import os
client = OpenAI(
base_url="https://api.oxlo.ai/v1",
api_key=os.environ["OXLO_API_KEY"]
)
with open("user_command.wav", "rb") as audio_file:
transcript = client.audio.transcriptions.create(
model="whisper-large-v3",
file=audio_file,
response_format="text"
)
print(transcript)Because Oxlo.ai does not charge by the token, a long meeting transcript costs the same flat per-request price as a short command. For applications that process lengthy audio, that predictability removes the surprise bills that often come with token-based transcription services.
Reasoning and Tool Use
Once you have text, you need an LLM that can reason, remember context, and optionally trigger external actions. Oxlo.ai offers more than 45 models across seven categories, including general-purpose flagships such as Llama 3.3 70B and agentic models such as Qwen 3 32B. If your voice assistant needs to generate code, DeepSeek R1 671B MoE or DeepSeek V4 Flash are available. For vision-enabled devices, Kimi K2.6 supports image input alongside its 131K context window.
A voice assistant is most useful when it can do things, not just say things. Oxlo.ai supports function calling and JSON mode, so you can define tools for home automation, calendar lookups, or database queries and let the model decide when to invoke them. Streaming responses are also supported, so you can begin synthesizing speech as soon as the first tokens arrive rather than waiting for the full completion.
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": "You are a concise voice assistant."},
{"role": "user", "content": transcript}
],
stream=True,
tools=[{
"type": "function",
"function": {
"name": "set_reminder",
"description": "Set a user reminder",
"parameters": {
"type": "object",
"properties": {
"time": {"type": "string"},
"message": {"type": "string"}
},
"required": ["time", "message"]
}
}
}]
)
# Stream the text reply for downstream TTS
for chunk in response:
if chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="")Synthesis with Kokoro
The final stage converts the LLM output into natural speech. Oxlo.ai provides Kokoro 82M, a lightweight text-to-speech model that delivers fast synthesis without the overhead of larger proprietary systems. You can call it through the same OpenAI SDK using the audio/speech endpoint.
speech = client.audio.speech.create(
model="kokoro-82m",
input="Your reminder has been set for 3 PM."
)
with open("response.wav", "wb") as f:
f.write(speech.content)Because there are no cold starts on popular models, the first request after idle time returns immediately. That behavior is critical for voice UIs, where users expect sub-second feedback and will abandon interactions that stall.
Why Flat Pricing Fits Voice Workloads
Voice interactions are inherently long-context. A five-minute audio clip can produce thousands of tokens of transcript. In a multi-turn agentic session, every previous exchange is typically appended to the prompt, so input length grows with each loop. On token-based providers such as Together AI, Fireworks AI, OpenRouter, Replicate, or Anyscale, this means cost scales with every word of history.
Oxlo.ai uses request-based pricing: one flat cost per API request regardless of prompt length. For voice-controlled interfaces that shuttle large transcripts between Whisper, an LLM, and Kokoro, this model can be 10-100x cheaper than token-based alternatives for long-context workloads, especially as sessions deepen. You can explore the exact tiers on the Oxlo.ai pricing page. The Free plan offers 60 requests per day across more than 16 models, which is enough to prototype a voice assistant before committing to a paid tier. Pro, Premium, and Enterprise plans scale from 1,000 to 5,000 requests per day and beyond, with Enterprise offering dedicated GPUs and a guaranteed 30% savings versus your current provider.
Drop-In Integration
Oxlo.ai is a fully OpenAI-compatible drop-in replacement. You point the OpenAI SDK to https://api.oxlo.ai/v1, swap the model names, and keep the rest of your logic unchanged. This compatibility extends to Python, Node.js, and cURL. If you already have a voice prototype running against another backend, migration is usually a matter of changing two lines of configuration.
The platform also supports vision input, embeddings, and image generation through the same unified API. If your voice assistant later needs to describe a scene from a camera feed or generate a visual response, you can call Gemma 3 27B or Oxlo.ai Image Pro without introducing new client libraries.
From Prototype to Production
When moving from demo to deployed product, predictability matters. Request-based pricing lets you forecast costs from user session counts rather than estimating average token lengths. Combined with Oxlo.ai's streaming responses, function calling, and absence of cold starts, you get a backend that behaves like a real-time service rather than a batch inference job.
If you are evaluating infrastructure for your next voice project, consider starting on the Oxlo.ai Free tier. Build the full pipeline with Whisper, Llama 3.3 70B or Qwen 3 32B, and Kokoro 82M, measure the latency under real audio loads, and compare your projected bill against token-based alternatives. For long-context and agentic voice workloads, the difference is often substantial.
Top comments (0)