DEV Community

shashank ms
shashank ms

Posted on

Choosing the Best LLM Model for Your Needs

#learnai #oxlo #ai

Building a model router that picks the best Oxlo.ai LLM for any user request saves money and improves output quality. I built a small Python agent that classifies tasks and routes them to the right model, from fast code generation to deep reasoning. It runs entirely on Oxlo.ai and uses request-based pricing so long prompts do not inflate costs.

What you'll need

Step 1: Map the Oxlo.ai model catalog

Create a dictionary of available models with tags so the router can filter by capability. I keep this in a separate module so non-technical teammates can edit it without touching the logic.

MODEL_CATALOG = {
    "deepseek-v3.2": {
        "tags": ["coding", "fast", "free-tier"],
        "context": 128000,
        "description": "Fast coding and reasoning, free tier available"
    },
    "qwen-3-32b": {
        "tags": ["multilingual", "agent", "reasoning"],
        "context": 128000,
        "description": "Multilingual reasoning and agent workflows"
    },
    "llama-3.3-70b": {
        "tags": ["general", "chat", "balanced"],
        "context": 128000,
        "description": "General-purpose flagship"
    },
    "deepseek-r1-671b": {
        "tags": ["deep-reasoning", "math", "coding"],
        "context": 64000,
        "description": "Deep reasoning and complex coding"
    },
    "kimi-k2.6": {
        "tags": ["reasoning", "coding", "vision", "long-context"],
        "context": 131072,
        "description": "Advanced reasoning, agentic coding, vision, 131K context"
    },
    "qwen-3-coder-30b": {
        "tags": ["code", "specialist"],
        "context": 128000,
        "description": "Dedicated code generation"
    }
}

Step 2: Define the router system prompt

The router is itself an LLM call. Its job is to read the user request, consult the catalog, and return a JSON object with the selected model ID and a short reason.

ROUTER_PROMPT = """
You are a model router. Your job is to select the single best Oxlo.ai model for the user's request.

Available models:
- deepseek-v3.2: fast coding/reasoning, free tier
- qwen-3-32b: multilingual reasoning, agent workflows
- llama-3.3-70b: general-purpose flagship, balanced
- deepseek-r1-671b: deep reasoning, complex coding, math
- kimi-k2.6: advanced reasoning, coding, vision, 131K context
- qwen-3-coder-30b: dedicated code specialist

Respond with valid JSON only:
{
  "model": "<model_id>",
  "reason": "<one sentence>"
}

Rules:
- Choose qwen-3-coder-30b only for pure code generation.
- Choose deepseek-r1-671b for math, logic puzzles, or multi-step reasoning.
- Choose kimi-k2.6 if the user mentions images, vision, or needs very long context.
- Choose deepseek-v3.2 for quick coding tasks or if cost sensitivity is high.
- Default to llama-3.3-70b for general chat and mixed tasks.
"""

Step 3: Build the classifier function

This calls Oxlo.ai with the router prompt. I use deepseek-v3.2 because it is fast, has a free tier, and handles classification well. The request-based pricing on Oxlo.ai means a long router prompt does not spike the cost.

from openai import OpenAI
import json

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

def select_model(user_message):
    response = client.chat.completions.create(
        model="deepseek-v3.2",
        messages=[
            {"role": "system", "content": ROUTER_PROMPT},
            {"role": "user", "content": user_message},
        ],
        response_format={"type": "json_object"},
    )
    decision = json.loads(response.choices[0].message.content)
    return decision["model"], decision["reason"]

Step 4: Add the execution layer

Once the router picks a model, this function forwards the original user message to that model. The call is identical to the OpenAI SDK, just pointed at Oxlo.ai.

def run_task(model_id, user_message):
    response = client.chat.completions.create(
        model=model_id,
        messages=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": user_message},
        ],
    )
    return response.choices[0].message.content

Step 5: Wire up the CLI

This main block ties the classifier and runner together. It prints the routing decision so you can verify the logic before you fully trust it.

if __name__ == "__main__":
    import sys

    user_message = sys.argv[1] if len(sys.argv) > 1 else "Explain quantum computing in simple terms."

    model_id, reason = select_model(user_message)
    print(f"Router selected: {model_id}")
    print(f"Reason: {reason}\n")

    output = run_task(model_id, user_message)
    print(output)

Run it

Save the script as router.py and test it with two different workloads. The first is a coding task, the second is deep reasoning.

python router.py "Write a Python function that validates an email address with regex."

# Example output:
# Router selected: qwen-3-coder-30b
# Reason: Pure code generation task.
#
# import re
# def validate_email(email):
#     pattern = r"^[\w\.-]+@[\w\.-]+\.\w+$"
#     return re.match(pattern, email) is not None

python router.py "Solve the tower of Hanoi for 4 disks step by step."

# Example output:
# Router selected: deepseek-r1-671b
# Reason: Multi-step logical reasoning puzzle.
#
# To solve Tower of Hanoi with 4 disks...

Next steps

Add a vision pipeline by checking if the input contains image URLs and forcing the router to kimi-k2.6. You could also cache the routing decision in Redis so identical queries skip the classifier entirely, cutting latency and saving requests on the Oxlo.ai pricing tiers.

Top comments (0)