DEV Community

MrClaw207
MrClaw207

Posted on

I Built a Token-Routing Decision Tree for My OpenClaw Agent. It Cut My Bill and Stopped Three Cron Failures

#openclaw #ai #agents #productivity

I burned $87 on a single weekend in March. Not because my OpenClaw agent did anything wrong — because every single tool call hit my paid primary model, including the ones that were checking the weather for a cron job that didn't need it.

Then OmniRoute crossed my feed this week showing 1B tokens routed for $0, and I realized I had been doing routing wrong for months. Not the mechanics — those were fine. The decisions were wrong. I was routing based on "is the model up?" instead of "what is this call actually trying to learn?"

Here's the routing decision tree I built, the three cron failures it prevented, and the bill that went from $87/week to single digits.

The Routing Mistake Everyone Makes

OpenClaw's fallback chain is per-call, sequential, and stops at the first success. That's the whole mental model most people carry around. Primary model → fails → try next → fails → try next → succeed.

That's not routing. That's a retry list. Routing means picking the right model before the call, based on what the call is doing.

The difference shows up in three places:

  1. Latency. A 6-step reasoning call to a local 9B model takes 14 seconds. To a paid cloud model: 1.8 seconds. Routing that call "down" because it's "small" can triple wall time.
  2. Cost. Routing a heartbeat health check to a paid primary model costs $0.002 per call. Doing it 200 times a day costs $14.40/month for checks that don't need intelligence.
  3. Reliability. OpenRouter *:free models have soft rate limits that don't return clean 429s — they return truncated output. I lost three cron jobs in May to that exact failure mode before I understood what was happening.

The Decision Tree

I wrote this as a tiny pre-call router that lives in my agent's wrapper layer. Every tool call gets a one-word classification first, and the classification picks the model.

// routes.js — pre-call classifier
const ROUTES = {
  heartbeat:   { provider: 'ollama-local', model: 'qwen3.5:9b', max_tokens: 200 },
  summary:     { provider: 'ollama-local', model: 'qwen3.6:27b-q4_K_M', max_tokens: 4000 },
  classify:    { provider: 'openrouter-free', model: 'gpt-oss-20b:free', max_tokens: 150 },
  toolPlan:    { provider: 'openrouter-free', model: 'gpt-oss-120b:free', max_tokens: 2000 },
  reasoning:   { provider: 'paid-primary', model: 'minimax-portal/MiniMax-M3', max_tokens: 8000 },
  codeWrite:   { provider: 'paid-primary', model: 'minimax-portal/MiniMax-M3', max_tokens: 12000 },
  fallback:    { provider: 'paid-primary', model: 'minimax-portal/MiniMax-M2.7', max_tokens: 8000 },
};

function route(callType, context = {}) {
  const route = ROUTES[callType] || ROUTES.reasoning;
  // Reason-load matters: summaries for >50k token contexts need cloud VRAM
  if (route.provider === 'ollama-local' && (context.input_tokens || 0) > 50000) {
    return ROUTES.reasoning;
  }
  return route;
}
Enter fullscreen mode Exit fullscreen mode

The classification happens one level up — at the call site, not inside the agent. That's deliberate. You don't want the model deciding how to route itself; you want a deterministic function picking the bucket based on what the call is doing.

// wrapper.js — every tool call gets routed
async function callLLM(callType, prompt, context) {
  const r = route(callType, context);
  try {
    return await llm.complete({ ...r, prompt });
  } catch (e) {
    // Fallback chain only kicks in here, after intelligent routing
    return await fallbackChain(prompt, context, exclude=r.provider);
  }
}
Enter fullscreen mode Exit fullscreen mode

The key insight: fallback is the safety net, not the strategy. If your primary model is down, fine, walk the chain. But if your routing puts a 200-token heartbeat check on a paid primary, the fallback chain never even gets a chance to help — the bill is already accumulating.

The Three Cron Failures This Prevented

Failure #1: VRAM cascade. A Saturday morning memory maintenance cron had a 38K-token summarization step that I had routed to my local 27B qwen model. When M3 was busy, the wrapper fell back to that local model. The local model couldn't fit 38K tokens comfortably alongside the rest of the runtime. Output collapsed to ~500 tokens, then died. Three timeouts in 90 minutes. My router now bumps any local call with input >50K tokens to the paid primary automatically. Sat morning cron has been green for 5 weeks.

Failure #2: Free-tier truncation. I had an afternoon engagement cron that used OpenRouter gpt-oss-20b:free to classify DEV.to comments. Free tier started returning truncated responses (1,200 tokens instead of 2,000) silently. The agent thought it was working. Comments went un-replied. My fix: any classification call gets a strict max_tokens ceiling matched to the actual output schema, plus a sanity check that parses the JSON before declaring success. Now I catch truncation in 200ms instead of 6 hours.

Failure #3: Cold-start storm. Sub-agent spawns cold-start at ~3 seconds of pure round-trip. I was spawning 8 sub-agents in parallel for a research job, each one hitting my paid primary. $1.40 in cold-starts, zero useful work for the first 3 seconds. I batched sub-agent spawns (max 3 in parallel, queue the rest) and routed the initial "kickoff" prompt to the local 9B model. Cold-start cost dropped to $0.31. Wall time barely changed because the local kickoff is faster than waiting for paid-primary slots.

What The Bill Looks Like Now

Before: $87/week. Roughly $14/month on heartbeat checks alone, $26 on sub-agent cold-starts, the rest on actual work.

After: $9-12/week. The same workload, the same outputs, the same reliability — just routed to the right model before each call.

The math isn't exotic. It's "stop paying for intelligence you don't need." A heartbeat check that compares two timestamps doesn't need a frontier model. A sub-agent kickoff that says "go summarize this folder" doesn't either. But a 6-step reasoning chain about whether two PRs conflict? That does.

What I Learned

  1. Routing is a decision, not a list. Decide what kind of work the call is doing, then pick the model. Don't let the model pick itself based on what's available.
  2. Free-tier models are cron poison, not primary material. Use them for classification, summarization, anything with a strict output schema you can validate. Don't use them for anything where silent truncation would be invisible.
  3. Fallback chains are safety nets, not strategies. If your routing depends on the fallback chain for normal operation, your routing is wrong.
  4. Local models aren't free. They cost VRAM, wall time, and reliability. Route to them only when latency isn't critical and input fits comfortably.
  5. Measure everything. I caught every one of these failures because I had a wrapper logging every call's provider, model, input/output tokens, and wall time. Without that log, "the agent feels slow sometimes" is the only signal you get.

The OmniRoute writeup this week showed what $0 routing looks like at scale. My setup is smaller — single agent, single user — but the principle is the same. Don't let the model decide. Decide, then route.

Top comments (0)