I built a command-line Model Selector that reads a workload description and recommends the best Oxlo.ai model from a structured rubric. It turns tribal knowledge about latency, context windows, and flat per-request pricing into an automated advisor. This helps teams stop guessing and start matching capability to cost.
What you'll need
- Python 3.10 or newer
- The OpenAI SDK:
pip install openai - An Oxlo.ai API key from https://portal.oxlo.ai
Step 1: Scaffold the Oxlo.ai client
I start by creating a new file, model_selector.py, and wiring up the OpenAI SDK to point at Oxlo.ai. This client handles every request we make.
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.oxlo.ai/v1",
api_key=os.environ.get("OXLO_API_KEY", "YOUR_OXLO_API_KEY")
)
class ModelSelector:
def __init__(self, client):
self.client = client
Step 2: Encode the selection rubric
The agent is only as good as its system prompt. I codify our internal model-selection criteria into a constant so the LLM has the full Oxlo.ai catalog and tradeoffs in context.
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.oxlo.ai/v1",
api_key=os.environ.get("OXLO_API_KEY", "YOUR_OXLO_API_KEY")
)
SYSTEM_PROMPT = """You are an expert ML engineer who helps teams pick the right model from the Oxlo.ai catalog.
Your decision rubric:
1. General-purpose chat or fast iteration: llama-3.3-70b (flagship, low latency).
2. Deep reasoning, math, or complex coding: deepseek-r1-671b (MoE, highest quality, slower) or kimi-k2.6 (advanced reasoning + vision + 131K context).
3. Agentic workflows with tool use: qwen-3-32b (multilingual reasoning, agent workflows) or glm-5 (744B MoE, long-horizon agentic tasks) or minimax-m2.5 (coding, agentic tool use).
4. Long-context summarization over 100K tokens: deepseek-v4-flash (1M context, efficient MoE) or kimi-k2.6 (131K context).
5. Vision tasks (image input): kimi-k2.6 (vision + reasoning) or gemma-3-27b. For vision-language specifically, kimi-vl-a3b is also an option.
6. Pure coding or cost-sensitive prototyping: deepseek-v3.2 (coding and reasoning, free tier available) or qwen-3-coder-30b.
7. Large open-source GPT-class model: gpt-oss-120b.
Rules:
- Recommend exactly one primary model and one fallback.
- Explain the tradeoff in one sentence (latency, capability, or context).
- If the user needs vision, only recommend vision-capable models.
- Note that Oxlo.ai uses flat per-request pricing, so long inputs do not increase cost the way token-based providers charge."""
class ModelSelector:
def __init__(self, client):
self.client = client
Step 3: Add the recommendation method
Now I add the method that calls Oxlo.ai. I use llama-3.3-70b because it is fast and follows structured instructions reliably. The low temperature keeps the rubric from drifting.
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.oxlo.ai/v1",
api_key=os.environ.get("OXLO_API_KEY", "YOUR_OXLO_API_KEY")
)
SYSTEM_PROMPT = """You are an expert ML engineer who helps teams pick the right model from the Oxlo.ai catalog.
Your decision rubric:
1. General-purpose chat or fast iteration: llama-3.3-70b (flagship, low latency).
2. Deep reasoning, math, or complex coding: deepseek-r1-671b (MoE, highest quality, slower) or kimi-k2.6 (advanced reasoning + vision + 131K context).
3. Agentic workflows with tool use: qwen-3-32b (multilingual reasoning, agent workflows) or glm-5 (744B MoE, long-horizon agentic tasks) or minimax-m2.5 (coding, agentic tool use).
4. Long-context summarization over 100K tokens: deepseek-v4-flash (1M context, efficient MoE) or kimi-k2.6 (131K context).
5. Vision tasks (image input): kimi-k2.6 (vision + reasoning) or gemma-3-27b. For vision-language specifically, kimi-vl-a3b is also an option.
6. Pure coding or cost-sensitive prototyping: deepseek-v3.2 (coding and reasoning, free tier available) or qwen-3-coder-30b.
7. Large open-source GPT-class model: gpt-oss-120b.
Rules:
- Recommend exactly one primary model and one fallback.
- Explain the tradeoff in one sentence (latency, capability, or context).
- If the user needs vision, only recommend vision-capable models.
- Note that Oxlo.ai uses flat per-request pricing, so long inputs do not increase cost the way token-based providers charge."""
class ModelSelector:
def __init__(self, client):
self.client = client
def recommend(self, user_requirements: str) -> str:
response = self.client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": user_requirements},
],
temperature=0.2,
max_tokens=500,
)
return response.choices[0].message.content
Step 4: Wrap it in a CLI loop
Finally, I add a small interactive loop so we can test descriptions without editing the file each time. I also catch Ctrl+C so the script exits cleanly.
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.oxlo.ai/v1",
api_key=os.environ.get("OXLO_API_KEY", "YOUR_OXLO_API_KEY")
)
SYSTEM_PROMPT = """You are an expert ML engineer who helps teams pick the right model from the Oxlo.ai catalog.
Your decision rubric:
1. General-purpose chat or fast iteration: llama-3.3-70b (flagship, low latency).
2. Deep reasoning, math, or complex coding: deepseek-r1-671b (MoE, highest quality, slower) or kimi-k2.6 (advanced reasoning + vision + 131K context).
3. Agentic workflows with tool use: qwen-3-32b (multilingual reasoning, agent workflows) or glm-5 (744B MoE, long-horizon agentic tasks) or minimax-m2.5 (coding, agentic tool use).
4. Long-context summarization over 100K tokens: deepseek-v4-flash (1M context, efficient MoE) or kimi-k2.6 (131K context).
5. Vision tasks (image input): kimi-k2.6 (vision + reasoning) or gemma-3-27b. For vision-language specifically, kimi-vl-a3b is also an option.
6. Pure coding or cost-sensitive prototyping: deepseek-v3.2 (coding and reasoning, free tier available) or qwen-3-coder-30b.
7. Large open-source GPT-class model: gpt-oss-120b.
Rules:
- Recommend exactly one primary model and one fallback.
- Explain the tradeoff in one sentence (latency, capability, or context).
- If the user needs vision, only recommend vision-capable models.
- Note that Oxlo.ai uses flat per-request pricing, so long inputs do not increase cost the way token-based providers charge."""
class ModelSelector:
def __init__(self, client):
self.client = client
def recommend(self, user_requirements: str) -> str:
response = self.client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": user_requirements},
],
temperature=0.2,
max_tokens=500,
)
return response.choices[0].message.content
if __name__ == "__main__":
selector = ModelSelector(client)
print("Oxlo.ai Model Selector")
print("Describe your workload (e.g., 'I need a coding model with vision for a React app'):\n")
try:
while True:
user_input = input("> ")
if user_input.lower() in ("exit", "quit"):
break
answer = selector.recommend(user_input)
print(f"\n{answer}\n")
except (KeyboardInterrupt, EOFError):
print("\nShutting down.")
Run it
Save the file as model_selector.py, export your key, and run it. Here is a sample session where I ask for a long-context coding agent.
$ export OXLO_API_KEY="sk-oxlo.ai-..."
$ python model_selector.py
Oxlo.ai Model Selector
Describe your workload (e.g., 'I need a coding model with vision for a React app'):
> I need an agent that can read a 200K token codebase and propose architectural refactors.
Primary: deepseek-v4-flash
Fallback: kimi-k2.6
Tradeoff: deepseek-v4-flash gives you a 1M context window in an efficient MoE architecture, while kimi-k2.6 caps at 131K but adds stronger vision and agentic coding capabilities. Because Oxlo.ai charges per request rather than per token, either choice stays predictable even with massive inputs.
Next steps
To make this production-ready, wire the selector into a Slack bot or CI pipeline so teams can query it before spinning up new workloads. You could also swap in qwen-3-32b if you want the advisor itself to run agentic tool calls, such as fetching real-time model status from Oxlo.ai before it answers.
Top comments (0)