We are going to build a two-stage speech recognition pipeline that transcribes raw audio with Whisper and then structures the output using a large language model. This approach turns messy meeting recordings into cleaned transcripts with speaker labels and extracted action items. Oxlo.ai hosts both the audio transcription and chat endpoints under one request-based pricing model, so running this on long recordings does not scale in cost with input length. See https://oxlo.ai/pricing for plan details.
What you'll need
- Python 3.10 or newer
- An Oxlo.ai API key from https://portal.oxlo.ai
- The OpenAI SDK:
pip install openai - A sample audio file named
meeting_sample.wav
Step 1: Configure the Oxlo.ai client
The OpenAI SDK connects directly to Oxlo.ai by swapping the base URL and API key. This single client handles both audio transcription and chat completion calls.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")Step 2: Transcribe audio with Whisper
Oxlo.ai hosts Whisper Large v3 with no cold starts. We open the audio file and send it to the transcriptions endpoint.
audio_path = "meeting_sample.wav"
with open(audio_path, "rb") as f:
transcription = client.audio.transcriptions.create(
model="whisper-large-v3",
file=f
)
raw_transcript = transcription.text
print("Raw transcript:")
print(raw_transcript[:800])Step 3: Design the enrichment prompt
This system prompt instructs the LLM to correct errors, label speakers, and extract action items as structured JSON.
SYSTEM_PROMPT = """You are a meeting transcript analyst. Your job is to take a raw audio transcript and return a structured JSON object.
Tasks:
1. Correct obvious transcription errors and add proper punctuation.
2. Identify speakers. If names are unknown, use Speaker 1, Speaker 2.
3. Extract action items with an assignee and due date if mentioned.
4. Return only valid JSON with no markdown formatting.
The JSON schema must be:
{
"cleaned_transcript": "string with speaker labels and full text",
"speakers": ["list of speaker labels"],
"action_items": [
{"task": "string", "assignee": "string or null", "due": "string or null"}
]
}
Raw transcript:"""Step 4: Enrich the transcript with Llama 3.3 70B
We pass the raw transcript to Llama 3.3 70B, Oxlo.ai's general-purpose flagship. A low temperature keeps the output deterministic and close to the source audio.
import json
def enrich_transcript(raw_text):
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": raw_text},
],
temperature=0.2,
max_tokens=4096,
)
content = response.choices[0].message.content.strip()
return json.loads(content)
structured = enrich_transcript(raw_transcript)
print(json.dumps(structured, indent=2))Run it
Save the complete script as transcribe.py, set your API key, and run python transcribe.py. Below is realistic output for a short engineering standup recording.
$ python transcribe.py
Raw transcript:
okay so um the api migration is almost done i just need to update the docs by wednesday sarah can you review the endpoints before friday yeah i can do that great and we should also schedule a load test for next monday
{
"cleaned_transcript": "Speaker 1: The API migration is almost done. I just need to update the docs by Wednesday.\nSpeaker 1: Sarah, can you review the endpoints before Friday?\nSpeaker 2: Yeah, I can do that.\nSpeaker 1: Great. We should also schedule a load test for next Monday.",
"speakers": ["Speaker 1", "Speaker 2"],
"action_items": [
{"task": "Update the docs", "assignee": "Speaker 1", "due": "Wednesday"},
{"task": "Review the endpoints", "assignee": "Speaker 2", "due": "Friday"},
{"task": "Schedule a load test", "assignee": null, "due": "next Monday"}
]
}Next steps
Swap in qwen-3-32b or kimi-k2.6 to test multilingual transcripts, or stream the LLM response for real-time meeting assistance. You can also extend the prompt to output speaker diarization timestamps if you chunk the audio before Whisper.
Top comments (0)