Building applications that listen and respond requires stitching together two distinct inference workloads: automatic speech recognition (ASR) to convert audio into text, and a large language model (LLM) to reason over that text. Most developers default to separate providers for each stage, which adds latency, vendor fragmentation, and unpredictable billing. Oxlo.ai offers a unified alternative. With fully OpenAI-compatible endpoints for both audio transcriptions and chat completions, you can run Whisper and a production LLM on the same platform using a single API key and consistent request-based pricing. This guide walks through a complete integration, from raw audio to structured LLM output.
Architecture Overview
A typical voice-to-insight pipeline has three layers. First, an audio file is encoded and sent to an ASR endpoint. Second, the resulting transcript is formatted and injected into an LLM prompt. Third, the LLM emits a completion, which might be a summary, a structured JSON object, or a follow-up question.
Oxlo.ai exposes both stages through the same base URL, https://api.oxlo.ai/v1. You can call audio/transcriptions with Whisper Large v3, Whisper Turbo, or Whisper Medium, then immediately call chat/completions with models such as Llama 3.3 70B, DeepSeek R1 671B MoE, or Kimi K2.6. Because both endpoints share the same SDK semantics, you reuse the same client constructor, authentication header, and error-handling logic.
Prerequisites
- Python 3.9 or higher
- An Oxlo.ai API key from your dashboard
- An audio file (WAV or MP3, mono or stereo)
- The OpenAI Python SDK:
pip install openai
Step 1: Transcribe Audio with Whisper
Oxlo.ai hosts several Whisper variants. Whisper Large v3 offers the highest accuracy for noisy or multi-speaker audio. Whisper Turbo trades a small amount of accuracy for lower latency. Whisper Medium is useful for prototyping on the Free tier.
The transcription endpoint accepts the same parameters as OpenAI’s. Set response_format to verbose_json if you need per-segment timestamps for speaker diarization or audio alignment downstream.
import openai
client = openai.OpenAI(
api_key="YOUR_OXLO_API_KEY",
base_url="https://api.oxlo.ai/v1"
)
audio_file = open("meeting.wav", "rb")
transcription = client.audio.transcriptions.create(
model="whisper-large-v3",
file=audio_file,
response_format="text"
)
transcript = transcription.text
print(transcript)Step 2: Structure the Transcript for the LLM
Raw transcripts often contain filler words, false starts, and formatting artifacts. Before sending the text to an LLM, it helps to define a system prompt that normalizes the input. For example, if you are building a meeting assistant, instruct the model to treat the transcript as unedited speech-to-text output.
Keep the context window in mind. If the transcript exceeds the target model’s context limit, chunk it by time or sentence boundaries, then send each chunk through the LLM in parallel. Oxlo.ai’s request-based pricing means the cost of each chunk is flat, so you can scale parallelism without scaling per-token cost.
Step 3: Generate Insights with an LLM
Once the transcript is clean, route it to chat/completions. For general summarization, Llama 3.3 70B provides a strong balance of speed and accuracy. If the audio is a technical code review and you need deep reasoning, DeepSeek R1 671B MoE or Kimi K2.6 are better fits. For agentic workflows where the model must decide which tools to call, Qwen 3 32B and GLM 5 support robust function calling.
The following example extracts action items and deadlines from a meeting transcript, returning strict JSON.
import json
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{
"role": "system",
"content": (
"You are an assistant that extracts action items from meeting transcripts. "
"Return a JSON object with a key 'action_items' containing a list of objects. "
"Each object must have 'owner', 'task', and 'deadline'."
)
},
{
"role": "user",
"content": f"Transcript:\n{transcript}"
}
],
response_format={"type": "json_object"}
)
result = json.loads(response.choices[0].message.content)
print(json.dumps(result, indent=2))Because Oxlo.ai supports JSON mode and function calling across its chat models, you can replace the freeform extraction above with deterministic tool schemas when building production agents.
Optimizing Latency and Cost
Audio workloads are inherently long-context. A ten-minute recording can easily produce several thousand tokens of transcript, and if you prepend system instructions or few-shot examples, the prompt grows even larger. On token-based platforms, this input length directly inflates cost. Oxlo.ai uses request-based pricing: one flat cost per API call regardless of prompt length. For transcription-heavy pipelines, this can make long-form audio analysis far more predictable.
Additionally, Oxlo.ai does not impose cold starts on popular models. After the initial Whisper call, your LLM request is served immediately, which keeps end-to-end latency low for interactive voice applications.
See https://oxlo.ai/pricing for current plan details.
Handling Long-Form Audio
When audio exceeds the length limits of a single ASR request, split it into overlapping windows (for example, sixty-second chunks with a five-second overlap). Transcribe each chunk through Oxlo.ai’s audio/transcriptions endpoint, then deduplicate the overlapping text with fuzzy string matching or semantic similarity.
Because Oxlo.ai charges per request rather than per token, chunking does not introduce hidden costs from duplicated prompt text. You can safely re-send overlapping context to preserve coherence without worrying about token duplication.
Complete Example: Voice-to-Structured-Data
Below is a single script that transcribes an audio file and produces a structured summary. It uses the Oxlo.ai client for both stages.
import openai
import json
client = openai.OpenAI(
api_key="YOUR_OXLO_API_KEY",
base_url="https://api.oxlo.ai/v1"
)
def audio_to_summary(audio_path: str) -> dict:
# Stage 1: Transcription
with open(audio_path, "rb") as f:
tx = client.audio.transcriptions.create(
model="whisper-large-v3",
file=f,
response_format="text"
)
transcript = tx.text
# Stage 2: Structured extraction
completion = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{
"role": "system",
"content": (
"Summarize the following transcript into JSON with keys: "
"'summary', 'key_decisions', and 'participants'."
)
},
{"role": "user", "content": transcript}
],
response_format={"type": "json_object"}
)
return json.loads(completion.choices[0].message.content)
if __name__ == "__main__":
output = audio_to_summary("podcast_episode.mp3")
print(json.dumps(output, indent=2))This pattern generalizes to any voice interface. Swap in DeepSeek V4 Flash for near-state-of-the-art reasoning over long transcripts, or switch to Whisper Turbo if you are optimizing for streaming transcription.
Conclusion
Integrating speech recognition with LLMs does not require managing multiple SDKs, billing dashboards, or token calculators. Oxlo.ai provides both ASR and LLM inference behind one OpenAI-compatible API, with request-based pricing that removes the penalty for long audio transcripts. Start with Whisper for transcription, route the output to Llama 3.3 70B or DeepSeek R1 for reasoning, and deploy the pipeline without cold starts. For pricing and model availability, visit https://oxlo.ai/pricing.
Top comments (0)