I Sent 500 Requests to Auto-Routing. It Saved Me 47% and I Never Picked a Model.

I Sent 500 Requests to Auto-Routing. It Saved Me 47% and I Never Picked a Model.

Last month I looked at my API bill and noticed something. Almost 60% of my requests were simple. Summarization, translation, factual Q&A. But I was sending all of them to the same expensive model, because I had set it as the default months ago and never thought about it again.

I was paying sports-car prices for grocery-store trips.

So I ran an experiment. 500 real prompts. Manual model selection vs auto-routing. The results changed how I think about AI API design.

My Old Workflow: "Set It and Forget It"

I picked DeepSeek-V4 Pro because it's strong, and it is. The problem is that I used it for tasks a 20x cheaper model could handle identically. When a user asks "What's the capital of France?", both models give the same answer. But one costs $8 per million output tokens and the other costs $0.90.

The Experiment

I collected 500 prompts and categorized them:

Category	Count	Example
Simple (summarization, translation, facts)	300	"Summarize this article in 3 bullet points"
Medium (content writing, explanation)	150	"Write a product announcement email"
Complex (code review, reasoning, logic)	50	"Find the bug in this 200-line function"

Manual: All 500 to deepseek-v4-pro
Auto-routing: model="auto", the API decides

The Results

	Manual	Auto-routing	Difference
Total input tokens	300,000	300,000	Same
Total output tokens	200,000	180,000	Shorter
Total cost	$2.80	$1.48	-47.1%
Avg latency	~1.2s	~0.8s	Faster

The auto-router sent:

• 300 simple to qwen3.8-flash ($0.12)
• 150 medium to qwen3.8-max ($0.96)
• 50 complex to deepseek-v4-pro ($0.40)

Complex tasks still got the powerful model. Everything else got the cheapest model that could handle it. I changed 12 characters in my code.

What Surprised Me

1. Quality didn't drop on simple tasks.. Output was indistinguishable.

2. Output got shorter in a good way. The cheaper models gave more direct answers with less fluff. I was paying for verbosity I didn't need.

3. The biggest win wasn't money. It was time. I stopped thinking "which model?" for every feature. I write model="auto" and move on.

The Code

from openai import OpenAI

client = OpenAI(
    base_url="https://trelayai.com/v1",
    api_key="sk-your-key"
)

def ask(prompt, force_complex=False):
    model = "deepseek-v4-pro" if force_complex else "auto"
    response = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
        temperature=0.7
    )
    return response.choices[0].message.content

force_complex=True only when you know the task needs top-tier reasoning. Everything else gets auto. No model registry. No routing logic. No maintenance.

When Auto-Routing Works:

Mixed workloads, user-facing chatbots, batch processing, internal tools.

When It Doesn't:

Predictable specialized workloads, latency-critical paths, model-specific features.

Getting Started

1. Sign up at trelayai.com (email + password, under a minute)
2. Get your API key from the dashboard
3. Point your OpenAI client to https://trelayai.com/v1
4. Set model="auto" where you normally hardcode a model name
5. Check your usage dashboard after a week

No credit card required. Models: Qwen, DeepSeek, Kimi, GLM.

I've been running this in production for three weeks. If you've tried something similar or think this is a terrible idea, drop a comment.
Sign up at trelayai.com — no credit card required.