Integrating LLM with Voice Assistant

#aiinfrastructure #oxlo #ai

Voice assistants have moved beyond rigid command-and-control systems. Modern implementations use large language models to handle open-ended dialogue, but building the full pipeline requires reliable speech-to-text, low-latency inference, and natural text-to-speech. Oxlo.ai provides all three in a single developer platform with flat per-request pricing, making it a strong foundation for voice agents that need predictable costs and fast response times.

Architecture of an LLM-Powered Voice Assistant

A production voice assistant typically follows a three-stage pipeline: speech-to-text (STT), language model reasoning, and text-to-speech (TTS). Latency at every stage directly impacts user experience, so each component must be fast and consistently available. On Oxlo.ai, the audio and chat endpoints share the same base URL and authentication, which simplifies integration and avoids the operational overhead of stitching together separate providers.

Speech-to-Text with Oxlo.ai Whisper

The first step is converting user audio into text. Oxlo.ai hosts Whisper Large v3, Whisper Turbo, and Whisper Medium behind an OpenAI-compatible audio/transcriptions endpoint. Because Oxlo.ai does not charge by the token, a long user monologue costs the same as a short command: one flat request.

import openai

client = openai.OpenAI(
    api_key="YOUR_OXLO_API_KEY",
    base_url="https://api.oxlo.ai/v1"
)

with open("user_prompt.wav", "rb") as audio_file:
    transcript = client.audio.transcriptions.create(
        model="whisper-large-v3",
        file=audio_file,
        response_format="text"
    )

print(transcript)

Orchestrating Conversation with an LLM

Once you have the transcript, you pass it to a chat model. Voice assistants often maintain multi-turn context and carry long system prompts for persona or tool instructions. On token-based platforms, that context length directly inflates cost. Oxlo.ai uses request-based pricing, so the price stays flat no matter how much conversation history you include in the payload.

For general dialogue, Llama 3.3 70B works well as a flagship option. If you need agentic tool use or multilingual reasoning, Qwen 3 32B is available on the same endpoint. The example below uses JSON mode to return a structured response, which is useful when you want to separate the assistant's spoken text from any internal tool calls.

response = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[
        {"role": "system", "content": "You are a concise voice assistant. Respond in JSON with keys: 'reply' and 'action'."},
        {"role": "user", "content": transcript}
    ],
    response_format={"type": "json_object"}
)

output = response.choices[0].message.content
print(output)

Text-to-Speech with Oxlo.ai Kokoro

The final stage converts the LLM output into audio. Oxlo.ai offers Kokoro 82M text-to-speech through the audio/speech endpoint. Like the other stages, this is a single API request with no hidden token fees.

import json

parsed = json.loads(output)
reply_text = parsed["reply"]

speech = client.audio.speech.create(
    model="kokoro-82m",
    voice="af",
    input=reply_text
)

speech.stream_to_file("assistant_response.wav")

End-to-End Implementation

Putting the stages together, a minimal Python loop might look like this. The example assumes you already have a recorded WAV file, but the core logic works with any audio source.

import json
import openai

client = openai.OpenAI(api_key="YOUR_OXLO_API_KEY", base_url="https://api.oxlo.ai/v1")

def process_turn(audio_path: str) -> str:
    # 1. Transcribe
    with open(audio_path, "rb") as f:
        stt = client.audio.transcriptions.create(
            model="whisper-large-v3", file=f, response_format="text"
        )

    # 2. Generate response
    chat = client.chat.completions.create(
        model="qwen3-32b",
        messages=[
            {"role": "system", "content": "You are a helpful voice assistant. Keep answers brief."},
            {"role": "user", "content": stt}
        ]
    )
    text = chat.choices[0].message.content

    # 3. Synthesize speech
    tts = client.audio.speech.create(
        model="kokoro-82m",
        voice="af",
        input=text
    )
    tts.stream_to_file("response.wav")
    return "response.wav"

Why Oxlo.ai for Voice Workloads

Voice applications are inherently unpredictable: a user might ask a one-word question or dictate a five-minute story. Token-based billing makes cost forecasting difficult for these workloads. Oxlo.ai's flat per-request pricing removes that variance entirely, and the platform's request-based model can be significantly cheaper for long-context sessions.

Beyond pricing, Oxlo.ai eliminates cold starts on popular models, which keeps assistant response times consistent. The full stack of STT, LLM, and TTS is available through a single OpenAI-compatible base URL, so you can prototype with the Python SDK and move to production without retooling.

If you are building a voice agent, start with the Oxlo.ai pricing page to compare plans, then point your existing OpenAI client to https://api.oxlo.ai/v1 and run the pipeline above.

DEV Community