Integrating LLM with Speech Recognition

Building voice-enabled applications usually means stitching together a transcription service, an LLM provider, and sometimes a text-to-speech engine. Each hop adds latency, cost accounting complexity, and another set of credentials to rotate. Oxlo.ai simplifies this stack by exposing both audio transcription and chat inference behind a single OpenAI-compatible endpoint, with request-based pricing that removes the token-cost penalty on long audio transcripts.

The Standard Pipeline

Most voice-to-insight systems follow a three-stage pattern. First, an automatic speech recognition (ASR) model converts raw audio into text. Second, an LLM processes that text to summarize, extract entities, or reason about next actions. Third, an optional text-to-speech (TTS) model turns the response back into audio for conversational interfaces.

Oxlo.ai hosts models for every stage. Whisper Large v3 handles transcription with strong accuracy across accents and noisy environments. Llama 3.3 70B, Qwen 3 32B, DeepSeek R1 671B MoE, and Kimi K2.6 provide reasoning, coding, and multilingual understanding over the resulting text. Kokoro 82M offers lightweight TTS when you need voice output. Because every model sits behind the same base URL and SDK, you can move from audio file to spoken response without leaving the Oxlo.ai stack.

End-to-End Code

The OpenAI SDK drops in directly. Set base_url to https://api.oxlo.ai/v1 and point your existing transcription and chat code at Oxlo.ai models.

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key=os.environ["OXLO_API_KEY"]
)

# 1. Transcribe audio with Whisper Large v3
with open("recording.wav", "rb") as audio_file:
    transcription = client.audio.transcriptions.create(
        model="Whisper Large v3",
        file=audio_file
    )

# 2. Summarize and extract action items with Llama 3.3 70B
chat = client.chat.completions.create(
    model="Llama 3.3 70B",
    messages=[
        {
            "role": "system",
            "content": "You are a meeting assistant. Summarize the transcript and list action items."
        },
        {
            "role": "user",
            "content": transcription.text
        }
    ],
    response_format={"type": "json_object"}
)

print(chat.choices[0].message.content)

This single-client pattern keeps credential management simple and eliminates cross-provider networking overhead. If you need voice output, you can call Kokoro 82M through the same client.audio.speech.create interface.

Long Context and Cost

Transcripts from hour-long meetings, podcasts, or call center recordings can easily exceed tens of thousands of tokens. On token-based providers, that input length drives cost up linearly. Oxlo.ai uses request-based pricing: one flat cost per API call regardless of how long the transcript is. For transcription-to-LLM workflows, this means you can pass the full raw text into a model like DeepSeek V4 Flash with its 1 million token context window, or Kimi K2.6 with 131K context, without watching the meter run on every additional paragraph.

If your workload involves agentic loops, where the LLM iteratively reasons over the transcript and calls tools, the savings compound. You pay per request, not per token consumed across multiple reasoning steps.

Structured Output and Tool Use

Raw transcripts are rarely the final product. You usually need structured data: speaker labels, sentiment scores, or extracted calendar events. Oxlo.ai supports JSON mode and function calling on its chat models, so you can constrain the LLM to emit valid JSON or invoke external tools directly from the transcript content.

For example, after transc