LLM vs Other AI Models: Understanding the Differences

Developers often default to LLMs for every AI task, but that wastes money and accuracy. In this tutorial, we will build a Research Archive Agent that transcribes audio, retrieves relevant passages with embeddings, and synthesizes answers with an LLM. You will see exactly where specialized models outperform generalist LLMs, and why Oxlo.ai's request-based pricing makes mixing model types affordable.

What you'll need

• Python 3.10 or newer
• An Oxlo.ai API key from https://portal.oxlo.ai
• The OpenAI SDK: pip install openai

Step 1: Configure the Oxlo.ai client

We instantiate the OpenAI client pointing at Oxlo.ai's base URL. This same client handles chat, embeddings, and audio because Oxlo.ai exposes fully compatible endpoints.

from openai import OpenAI
import os

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key=os.environ.get("OXLO_API_KEY", "YOUR_OXLO_API_KEY")
)

Step 2: Transcribe audio with Whisper

LLMs cannot ingest raw audio. We use Whisper Large v3, an encoder-decoder model built specifically for speech recognition. It converts an audio file into structured text that our pipeline can process downstream.

def transcribe_audio(file_path: str) -> str:
    with open(file_path, "rb") as audio_file:
        response = client.audio.transcriptions.create(
            model="whisper-large-v3",
            file=audio_file
        )
    return response.text

Step 3: Chunk and embed the transcript

LLMs struggle to perform exact semantic search across thousands of documents. Embedding models like BGE-Large map text into dense vectors so we can retrieve the top-k most relevant chunks later without loading an entire archive into an LLM context window.

import math

def chunk_text(text: str, chunk_size: int = 300, overlap: int = 50):
    words = text.split()
    chunks = []
    for i in range(0, len(words), chunk_size - overlap):
        chunk = " ".join(words[i:i + chunk_size])
        chunks.append(chunk)
    return chunks

def embed_chunks(chunks: list[str]) -> list[list[float]]:
    response = client.embeddings.create(
        model="bge-large",
        input=chunks
    )
    return [item.embedding for item in response.data]

Step 4: Retrieve relevant context

We compute cosine similarity between the query embedding and chunk embeddings. This is fast, deterministic, and far cheaper than asking an LLM to scan a full transcript every time.

def cosine_similarity(a, b):
    dot = sum(x * y for x, y in zip(a, b))
    norm_a = math.sqrt(sum(x * x for x in a))
    norm_b = math.sqrt(sum(x * x for x in b))
    return dot / (norm_a * norm_b)

def retrieve(query: str, chunks: list[str], embeddings: list[list[float]], top_k: int = 3):
    query_resp = client.embeddings.create(model="bge-large", input=[query])
    query_vec = query_resp.data[0].embedding
    scored = [(cosine_similarity(query_vec, emb), chunk) for chunk, emb in zip(chunks, embeddings)]
    scored.sort(reverse=True)
    return [chunk for _, chunk in scored[:top_k]]

Step 5: Synthesize answers with an LLM

Now we bring in the LLM. Its job is not transcription or retrieval. It is reasoning over the retrieved context to produce a coherent answer.

SYSTEM_PROMPT = """You are a research archive assistant. Answer the user's question using ONLY the provided context chunks. If the answer is not in the context, say "I don't have enough information." Do not make up facts."""

The function below sends the retrieved chunks and the user question to Llama 3.3 70B.

def ask_llm(question: str, context_chunks: list[str]) -> str:
    context = "\n\n".join(f"Chunk {i+1}:\n{chunk}" for i, chunk in enumerate(context_chunks))
    user_message = f"Context:\n{context}\n\nQuestion: {question}"
    
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": user_message},
        ],
        temperature=0.2,
    )
    return response.choices[0].message.content

Step 6: Wire everything into an agent

We combine the pieces so the agent delegates each subtask to the right model type. This architecture is what makes multi-model systems reliable and cost-effective on Oxlo.ai.

class ResearchArchiveAgent:
    def __init__(self):
        self.chunks = []
        self.embeddings = []
    
    def ingest_audio(self, file_path: str):
        transcript = transcribe_audio(file_path)
        self.chunks = chunk_text(transcript)
        self.embeddings = embed_chunks(self.chunks)
        print(f"Ingested {len(self.chunks)} chunks.")
    
    def query(self, question: str) -> str:
        if not self.chunks:
            return "No archive loaded."
        relevant = retrieve(question, self.chunks, self.embeddings)
        return ask_llm(question, relevant)

Run it

Here is how we call the finished agent. The example assumes you have an audio file named interview.mp3.

if __name__ == "__main__":
    agent = ResearchArchiveAgent()
    agent.ingest_audio("interview.mp3")
    
    print(agent.query("What was the main technical challenge discussed?"))
    print(agent.query("Which tools did the team decide to use?"))

Example output:

Ingested 42 chunks.

The main technical challenge discussed was maintaining sub-100ms latency while processing multilingual audio streams in real time.

The team decided to use Oxlo.ai for inference, BGE-Large for semantic retrieval, and Whisper for transcription.

Wrap-up

Try swapping the LLM to qwen-3-32b for multilingual transcripts, or switch to deepseek-v3.2 if you want the agent to generate Python analysis scripts from the retrieved text. Because Oxlo.ai charges per request rather than per token, running embeddings and LLM calls in the same workflow stays predictable even when your archive grows.