Integrating LLM with Speech Recognition Tasks: A Step-by-Step Guide

Speech recognition pipelines rarely stop at transcription. Most production workflows feed the resulting text into a large language model for summarization, entity extraction, or agentic reasoning. Building this two-stage pipeline usually means stitching together separate providers for audio and text, which adds latency, compatibility friction, and unpredictable billing. Oxlo.ai simplifies the stack by offering both speech-to-text and LLM inference behind a single OpenAI-compatible API, with request-based pricing that keeps costs flat even when transcripts grow long.

Architecture Overview

A typical integration follows three stages: ingestion, transcription, and language understanding. Raw audio enters a transcription model, the resulting text is passed to an LLM, and the LLM returns structured output or a conversational response. Because Oxlo.ai hosts both audio/transcriptions and chat/completions endpoints, you can run the entire pipeline against one base URL. The platform supports Whisper Large v3, Whisper Turbo, and Whisper Medium for transcription, alongside more than 45 chat and reasoning models such as Qwen 3 32B, Llama 3.3 70B, and DeepSeek R1 671B MoE. There are no cold starts on popular models, so both stages respond immediately.

Prerequisites

You need an Oxlo.ai API key, Python 3.9 or later, and the OpenAI Python SDK. Install the SDK with pip:

pip install openai

Configure the client to point to Oxlo.ai:

import openai

client = openai.OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_OXLO_API_KEY"
)

Step 1: Transcribe Audio with Whisper

Send an audio file to the audio/transcriptions endpoint. Oxlo.ai runs Whisper Large v3 and Whisper Turbo, so you can balance accuracy against speed depending on your workload.

audio_file = open("meeting.wav", "rb")

transcription = client.audio.transcriptions.create(
    model="whisper-large-v3",  # or whisper-turbo for faster processing
    file=audio_file,
    response_format="text"
)

transcript = transcription.text
print(transcript)

For longer recordings, such as interviews or podcasts, transcripts can quickly reach tens of thousands of tokens. Because Oxlo.ai charges per API request rather than by token volume, the transcription step is a single request regardless of audio duration.

Step 2: Structure and Reason with an LLM

Once you have the transcript, pass it to a chat model. You can use JSON mode or function calling to enforce structured output. The example below uses Qwen 3 32B to extract action items and decisions.

response = client.chat.completions.create(
    model="qwen3-32b",
    messages=[
        {
            "role": "system",
            "content": "You are an assistant that extracts action items from meeting transcripts. Respond with valid JSON only."
        },
        {
            "role": "user",
            "content": f"Extract action items and decisions from the following transcript:\n\n{transcript}"
        }
    ],
    response_format={"type": "json_object"}
)

structured_output = response.choices[0].message.content
print(structured_output)

Because Oxlo.ai uses request-based pricing, sending a long transcript to a large context model does not trigger the steep cost escalation common with token-based providers. This makes the platform particularly cost effective for podcast summarization, legal deposition analysis, and other long-form audio workflows.

Step 3: Streaming and Chunking for Real-Time Workflows

If you are processing live audio streams or lengthy recordings, you may want to chunk the audio and consume LLM responses as they are generated. Oxlo.ai supports streaming responses on the chat/completions endpoint, and popular models start instantly with no cold starts.

stream = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[
        {"role": "system", "content": "Summarize each chunk of a live transcript in one sentence."},
        {"role": "user", "content": transcript_chunk}
    ],
    stream=True
)

for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

Step 4: Add Text-to-Speech for Voice Agents

A complete voice agent pipeline requires speech output as well. Oxlo.ai offers text-to-speech through the audio/speech endpoint, including models such as Kokoro 82M. After the LLM generates a response, you can synthesize it back to speech without leaving the platform.

speech = client.audio.speech.create(
    model="kokoro-82m",
    voice="default",  # use a voice supported by the model
    input="Your appointment has been rescheduled to 3 PM."
)

speech.stream_to_file("response.mp3")

Putting It All Together

The script below ties transcription, structured extraction, and speech synthesis into one flow. All requests hit the same Oxlo.ai base URL, so you manage one API key and one client configuration.

import openai

client = openai.OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_OXLO_API_KEY"
)

# 1. Transcribe
with open("input.wav", "rb") as f:
    tx = client.audio.transcriptions.create(
        model="whisper-large-v3",
        file=f,
        response_format="text"
    )

# 2. Extract structured data
completion = client.chat.completions.create(
    model="qwen3-32b",
    messages=[
        {"role": "system", "content": "Extract speaker names, dates, and action items as JSON."},
        {"role": "user", "content": tx.text}
    ],
    response_format={"type": "json_object"}
)

result = completion.choices[0].message.content
print("Structured result:", result)

# 3. Synthesize response
speech = client.audio.speech.create(
    model="kokoro-82m",
    voice="default",
    input=f"Processing complete. I found the following action items: {result}"
)

speech.stream_to_file("output.mp3")

Why Request-Based Pricing Matters for Speech Workloads

Audio transcription and follow-up LLM reasoning create a double billing risk on token-based platforms. The transcription itself is often priced by audio minute or output token, and the LLM pass is priced by input and output tokens. When a one-hour interview produces a long transcript, the LLM stage can become expensive. Oxlo.ai flattens both stages into simple per-request costs. You pay one request for the transcription and one request for the LLM call, regardless of how verbose the transcript becomes. For teams running high-volume voice agents, compliance scanning, or media pipelines, that predictability removes the need to truncate or compress transcripts to save money. See the exact tiers at https://oxlo.ai/pricing.

Conclusion

Integrating speech recognition with LLMs does not require multiple vendors or complex orchestration. Oxlo.ai hosts the audio and language models you need behind a single OpenAI-compatible endpoint, supports streaming and structured output, and removes token-length billing from the equation. Whether you are building a voice assistant, a meeting summarizer, or an audio compliance scanner, you can run the entire pipeline on Oxlo.ai with the standard OpenAI SDK and a single API key.