A 7M-parameter model that knows when to say no

Most requests sent to an LLM are boring. "What's the weather in Osaka", "remind me to call mom", "what's 47 times 23". A frontier model answers them beautifully — and every call costs real money and a network round-trip.

I built the opposite end of that trade-off: a 7.2M-parameter dispatcher that only does triage. It reads a request, picks a typed route, reports calibrated confidence — and when it isn't sure, it says so and hands off instead of guessing.

One call, one typed decision

POST /v1/route {"state": "remind me to call mom tomorrow at 6pm"}

→ {"action": "tool_call", "route": "set_reminder",
   "call": {"name": "set_reminder", "arguments": {...}},
   "confidence": 0.9999, "latency_ms": 6}

POST /v1/route {"state": "write me a 2000-word essay on the roman empire"}

→ {"action": "delegate", "reason": "route_escalate", "confidence": 0.96}

The gates are explicit: low confidence → delegate. Escalation head fires → delegate. Decision head and generative trunk disagree → delegate. A dispatcher that guesses wrong confidently is worse than no dispatcher.

The unusual part: zero ML framework

Tokenizer training, pretraining (AdamW/RoPE/RMSNorm/SwiGLU), fine-tuning, int8 export, and the OpenAI-compatible CPU server are all written in machin/MFL — a language designed to be written by machines. No PyTorch, no ONNX, no llama.cpp.

The shipped artifact is a static binary + an 8MB int8 model. The whole appliance tarball is 7.5MB — untar, ./start.sh, and you have typed decisions on :8097 in ~15ms per call on a laptop CPU.

Per-customer routing without fine-tunes

Decisions come from tiny linear heads (~7KB artifacts) trained on the frozen trunk's hidden state. A customer's routes are a config file of example phrases:

{"routes": {"reset_password": ["reset my password", "locked out"],
            "escalate": ["migrate our infrastructure"]}}

head_studio.py validates it, synthesizes train/eval data, trains the head, reports accuracy + ECE + confusion. Multi-tenant serving swaps heads per API key — same trunk, different router per customer.

There's a runnable proof that's a little silly and completely real: a dungeon crawler where every typed intent — light the torch, go north, equip the sword — is routed by a tenant head. The game is just another customer.

Calibration: measured, including the failure

The honest part: a head's out-of-box confidence is fit on synthetic validation data, and on off-distribution real traffic softmax saturates — in one shadow deployment, 65/72 decisions sat at conf ≥0.99 while only 61.5% were right.

So refit_temp.py ships in the repo: it refits the head's temperature on ~50+ labeled rows of your traffic, after which mean confidence tracks measured accuracy — and a head that hasn't earned trust degrades to honest abstention. "p=0.9 means right 90% of the time" is achievable on your data, not just ours.

Honest limits

• 7M params — it routes, it does not reason. Delegation is built in.
• noul/score answer fixed trained questions; arbitrary criteria text is the roadmap.
• Ambiguous human-judgment tasks need label volume, not a bigger model — a 0.6B web-pretrained backbone scored worse than this 7M trunk on one fuzzy 3-way task.
• Chat replies are TinyStories-grade. It's a dispatcher, not a chatbot.

Get it

• Code + appliance: github.com/javimosch/mtlm-router
• Weights: Hugging Face · ModelScope
• Runtime: machin-anvil

tar xzf mtlm-router-*.tar.gz && cd mtlm-router-* && ./start.sh
curl localhost:8097/v1/route -d '{"state":"remind me to stretch every hour"}'