What DigitalOcean's new inference router picks for simple prompts

On 12 October DigitalOcean changed how routing works on its Inference platform. There used to be four preset routers, router:general, router:software-engineering, router:writing and router:knowledge-base-document. Now there is one model name, inference-router, and an optional routing_preference of balanced, budget or quality. You send the request, the router picks the model, and the response tells you which one it picked and so which one you pay for.

I wanted to know what it picks for the ordinary requests a small app sends all day: a one-line code question, some arithmetic, a polite email, "hi". So I sent the same 20 prompts through all four old presets, through the new router in all three preferences, and straight to four fixed models, and logged the model, the tokens and the time for every call.

The setup

Twenty short prompts, each with a short answer:

• code: a Python one-liner to reverse a string, a bash command for the five largest files, a KeyError fix, a second-highest-salary SQL query, a ZIP code regex
• reasoning: 17 × 23, a train arrival time, a syllogism, Kubernetes or a single VM for a small Django app
• writing: a product blurb, a politer rewrite, a four-line poem, a meeting decline, a one-sentence summary, a German translation
• trivia: the capital of Australia, the author of Middlemarch
• a question about an "attached policy document" with nothing attached
• hi
• three interview tips

Every call went to https://inference.do-ai.run/v1/chat/completions with max_tokens: 600 and nothing else set. I priced each call with the model's input and output rates from DigitalOcean's pricing page as of 15 October. Six of the prompts have answers a script can check (391, 17:15, Canberra, George Eliot, "no", [::-1]). Every configuration got every one of those right, so this post is about cost and behaviour, not about which model is smarter.

What the new router picked

All three preferences picked Kimi K3 for most of the prompts.

Preference	Kimi K3	GPT-6 Astra	Claude Opus 5.5	Other
balanced	14	3	2	GLM-5.3
budget	15	1	2	Sonnet 5.5, GLM-5.3 Flash
quality	14	3	2	GLM-5.3 Flash

Kimi K3 costs $3 per million input tokens and $15 per million output. GPT-6 Astra is $10 and $50, the most expensive model in this test. Astra got "What is the capital of Australia? One word." in all three preferences. It answered "Canberra" in five tokens.

budget hardly differed from the other two. It picked Sonnet 5.5 once and GLM-5.3 Flash once, and sent everything else to Kimi K3, Opus or Astra. The docs do say budget "still weighs quality, so the cheapest model does not automatically win". On these prompts it never picked a cheap model except once.

What that costs

Same 20 prompts, one pass each, priced per thousand requests:

Configuration	Cost per 1,000 requests	Median latency	Empty answers
inference-router, budget	$5.55	9.0 s	3
inference-router, quality	$5.04	8.3 s	3
inference-router, balanced	$4.94	6.7 s	2
Kimi K3, direct	$5.92	7.7 s	4
Claude Opus 5.5, direct	$3.48	2.9 s	0
old router:general	$0.31	2.9 s	3
Claude Haiku 5.5, direct	$0.07	2.0 s	0
DeepSeek V4 Flash, direct	$0.05	2.5 s	0

On this set of prompts the new router cost more than calling Claude Opus 5.5 directly for every request, took more than twice as long, and returned some empty answers where Opus returned none. budget was the most expensive of the three preferences.

The smallest example is hi. Every model answered "Hi there! How can I help you today?" Through the router it went to Kimi K3, which thought about it for 110 to 183 reasoning tokens first:

Route	Model	Output tokens	Cost
balanced	Kimi K3	206	$0.0034
budget	Kimi K3	148	$0.0025
Haiku 5.5, direct	Haiku 5.5	14	$0.000008

That is 300 to 420 times the price for the same greeting. On one request that's nothing. A chat app's opening messages add up.

Where the empty answers come from

Kimi K3 is a reasoning model, and on some prompts it spent all 600 tokens thinking and never wrote an answer. The response came back with finish_reason: "length" and empty content. The bash prompt did this in all three preferences: 600 reasoning tokens, 13 to 15 seconds, $0.009 and nothing to show for it. Haiku 5.5 answered the same prompt in 2.8 seconds for $0.0002.

That's partly down to my setup. A 600-token cap is fine for a model that answers directly, but not for one that reasons first. With the router you don't know in advance which kind you'll get. If you set max_tokens for the cheap direct model you'd expect, a reasoning model can use the whole budget up and hand you nothing. If you raise it, the expensive cases get more expensive. Either way, check finish_reason.

The old presets behaved very differently

The four preset routers still work, as the docs promise. They sent most of the same prompts to DeepSeek V4 Flash, and a response header says why:

x-model-router-selected-route: fallback

Across the four presets, 94 of the 114 calls that succeeded came back as fallback. The rest named a route: "Summarization" went to GLM-5.3 Flash, "Translation" to GPT-5.6 Luna, and "Opinion Advice & Recommendation" to GLM-5.3. router:general sent 18 of 20 prompts to the same model on both passes. The new router leaves that header empty, so the model field is all you get.

The old presets cost $0.06 to $0.34 per thousand requests on these prompts, roughly 15 to 90 times cheaper than the new router. They're no longer shown in the UI, and the docs recommend moving to inference-router.

What I got wrong

I ran out of money halfway through. I had planned two passes over everything, about 440 calls. After 271 successful calls, every request started returning this:

402 {"id": "Payment Required", "message": "This account has been blocked"}

The prepaid inference credit had run out. The new router alone cost about $0.31 for its 60 first-pass calls, and I hadn't checked the balance against that before starting. So every number in the tables comes from one complete pass. Only the old router:general got a full second pass, and it picked the same model for 18 of the 20 prompts. The new router got a second pass on four prompts in balanced and picked the same model each time.

At first I thought the old routers were adding a hidden system prompt. "What is 17 * 23?" cost 97 input tokens. Then the direct DeepSeek V4 Flash calls showed the same 84 to 99 tokens with no router involved. That's the model's own chat template, not the router. Opus counts the same prompt as 25.

Two of the direct DeepSeek calls failed with 529 Platform overloaded, so DeepSeek's direct row covers 18 prompts, not 20.

Caveats

These are 20 short, easy prompts, one pass each. DigitalOcean says the router is trained for software engineering, agentic workflows and reasoning, and its own docs say a single prompt "is not a measurement of routing quality". Long multi-step agent tasks are where a router choosing Kimi K3 or Opus might pay for itself, and I didn't test those. The router is in public preview and DigitalOcean says it re-evaluates its model pool as models ship, so the picks will change. I didn't test the Responses or Messages APIs, cached prompts, or the "personalize against your own rubric" option.

What I would take from it

• Routing is free during the preview, and the model field tells you exactly what you were billed for. Log it. That's how I found out what my requests actually cost.
• For short, simple requests, a fixed small model was 70 to 110 times cheaper here and just as correct on every checkable answer.
• budget did not mean cheap on these prompts. Measure before relying on it.
• If you route to a pool that includes reasoning models, check finish_reason and size max_tokens for the most talkative model in the pool, not the cheapest.
• If you're on one of the old router: presets and your traffic looks like this, moving to inference-router could raise your bill a lot. Run your own prompts through both first.

The total for the whole test was about $0.56 of inference, across 271 calls that returned an answer.

Comments

[Reid Marlow]

The reasoning token dump on a three-word greeting is wild. Paying 400x for Haiku parity because a router defaults to a thinking model on simple tasks is why people keep static configs. Setting max_tokens to 600 choking Kimi before it even reaches the answer block shows how brittle combined reasoning-output budgets get.

[Ivan Annovazzi]

Kinda skeptical this beats pointing everything at one cheap