DEV Community: Dor Amir

Take your benchmark to the people who can kill it

Dor Amir — Thu, 23 Jul 2026 13:49:27 +0000

Ten days ago I had a benchmark: reorder the expert weights inside a MoE model file by measured co-activation, and disk reads per token drop 2.23×. It was real, reproducible, and measured entirely on my own machine — which is to say, it was worth almost nothing yet.

Today the strongest number attached to this work is one I didn't produce: +32.3% decode throughput and −26.3% time-to-first-token on a 235B-parameter model running on a 48 GB MacBook, measured by someone I've never met, on an inference engine I'd never run, using my script unmodified, with the arms swapped as a page-cache control. Along the way, two of my three original pitches were refuted with data, my one pre-registered scaling prediction failed its own threshold, and both of my attempts to forecast a key parameter missed in opposite directions.

The refutations were the productive part. This post is about the process that made them cheap: five engines, a half-dozen strangers, one issue thread, and the discipline that emerged in it — because I think the process generalizes and the write-ups I usually see don't mention it.

The setup

The idea (project: mbolt) is BOLT/PGO applied to model binaries. Checkpoint tensor order is an accident of the training pipeline; MoE routing at inference time is far from random — experts fire in cliques. If an engine streams experts from SSD, the file layout decides whether a token's misses are a handful of long sequential reads or thousands of scattered ones. So: trace the routing, cluster the co-activation, rewrite the file, keep the weights byte-exact.

After a week solo I had the 2.23× read reduction on an 80B model, a 1.55× end-to-end decode gain against a patched llama.cpp, correctness gates (byte-identical weights, routing mapping 100.000% through the permutation, output divergence 5× below the same engine's CPU↔Metal backend delta), and a 500-prompt blind A/B that scored 48 to 46 with 26 ties — a coin flip, which for a layout change is the goal.

I also had a null I'd measured myself: on stock llama.cpp, none of it matters. The mmap page-fault path is layout-blind; I measured parity and said so. The gains need an engine that issues explicit reads.

That null dictated the launch strategy, although I didn't think of it as strategy at the time. The people who run explicit-read expert streamers are countable. Instead of posting a Show HN, I took the data to their issue trackers: ml-explore/mlx-lm#1438, where @mabaeyens was shipping an expert-offload streamer for MLX, and JustVugg/colibri#119, a C engine that streams a 744B model from NVMe. Opening line: sharing data because your engine is the natural home for a layout pass — and one of my findings says the audience for this is you, specifically.

First contact: two pitches die in one reply

mabaeyens did not argue with my framing. He measured it.

His reply took my three pitches — pin the hottest experts in RAM, prefetch the next layer's experts by co-activation, and reorder the file — and killed the first two with his own engine's counters. Heat-pinning a resident set loses to a plain per-layer LRU even on same-topic, same-session traffic: the LRU already banks the reuse, and every frozen slot is a slot the workload's drift can't use. And cross-layer prefetch has no signal to run on: the expert overlap between adjacent layers measured 0.017 against 0.016 for a shuffled control. Independent per-layer routers plus the load-balance loss decorrelate the layers. There is nothing to predict with.

I retracted both, in the thread, that day.

Here is the thing I want to highlight, because it's the hinge of the whole story: the thread treated the refutations as scoping, not as defeat. The claim didn't die; it moved. If retention kills the decode win, the layout lever migrates to where reads still happen — cold prefill, and (as his next measurement showed) decode too, once the expert table is so much bigger than RAM that misses stay cold forever. His fragmentation numbers made the case sharper than my original pitch: his reader issued ~471 file opens per decode token, nine byte-ranges per cold expert. The lever wasn't dead. It had been mislabeled.

The rules the thread converged on

Nobody declared a methodology. It accreted, one burned hand at a time, and by the end it looked like this:

Lock the experiment before running it. When we designed the decisive test — coalescing alone (profile-free, deterministic) vs coalescing plus co-activation reorder (needs a trace) — the arms, metrics, and decision rule were agreed in the thread before anyone ran anything, because the answer determined whether he'd break his engine's read-straight-from-shards invariant.

Pre-register numbers, with thresholds that can hurt you. My on-record prediction: at deeper offload the coalesce gain reaches ~+20%, and "if it lands under +15%, the honest write-up says this lever tops out at nice-not-necessary." It landed at +9.7%. I conceded in full — no appeal to config differences, because the pre-registration had already forfeited them.

Concede at commit speed. My analytical model over-credited a 36 µs open() cost his reader had already eliminated — his shared-fd fix landed about four minutes after he quoted the number I'd built on. When his measurements came back lower than my model, the model lost, publicly, same day. Score-keeping note: across the thread I went 0-for-2 forecasting the cold-read share (missed in opposite directions both times) and the community's rule hardened into measure it per engine, never carry it across.

Controls, or it didn't happen. The thread's canaries are my favorite artifact. A "2.23× speedup" that was actually macOS refusing to evict resident pages (F_NOCACHE does not do what three of us assumed; the tell was an impossible 0.18× in the control arm). A prefetch A/B that "confirmed" the predictive arm until its author discovered both cells had run the same code — byte-identical hit rates were the tell — and re-ran it properly: the predictor lost by 2.9%, reading 6.9 bytes per byte it saved. A cross-architecture overlap claim that showed 1.17× signal against an analytic uniform baseline and 0.98× against a shuffle control — the analytic baseline was manufacturing the signal. Arm-swapping for page-cache state became standard for every A/B in the thread.

Name the baseline or the number doesn't transfer. The same rewrite is +32% against a reader that issues nine scattered reads per expert, +31% against three-per-projection at deep offload, and ~nil against a container that already stores each expert contiguously. Quoting a layout gain without naming the baseline read pattern is how numbers stop replicating.

Maintain one living artifact. Midway through, I posted a consolidated summary as a comment and kept it versioned — v1.0 through v1.5.1 so far — edited in place via the API, with a changelog, and explicitly red-line-able by participants. People used it. One participant's red-line was purely attributional (the engine's name — which I then managed to misspell for two days; corrected in v1.4.1, and yes, that's in the changelog). The summary, not any single result, is what the thread produces: scoped findings, the measured points, and a traps section.

The falsification that became the payoff

The +9.7% failure was the best thing that happened to the project.

The mechanism behind it — coalescing is an IOPS lever, and a 4 MB slice is already bandwidth-bound — generalized into what the thread now calls the slice-size law: gain ≈ 1 + saved_ops · t_op / (slice_bytes / BW), with t_op ≈ 100 µs and BW ≈ 6 GB/s on this hardware class. End-to-end, it composes with the cold-read share s as speedup = 1/((1−s) + s/g). It is an engineering fit, not physics — but it now has six measured points across five engines, and it does the one thing a good model does: it tells you in advance whether your configuration is worth the trouble.

The law's test came from the two participants who joined after the initial exchange. @lBroth ran my conversion script, unmodified, against a 119 GB Qwen3-235B build and wired the output into his own engine: +32.3% decode — not because the per-read gain was large (it wasn't; big slices, g ≈ 1.34) but because at 2.6× over-DRAM the cold-read share is ~0.96 and the law's other term takes over. Then he built the deeper variant inside his engine and measured +31.2% where the law, priced against his named baseline, predicted +31.3%. And @pierre427 — running a production GLM-5.2 pager — pre-registered the per-read gain band we derived (1.2, band 1.15–1.3) and measured 1.195. The law's first registered-then-measured point.

The other end got priced too, and this matters just as much: @philipjohnbasile measured his 744B engine's container — which already stores each expert as one contiguous ~19 MB read — at a residual +1.3%, below his own promotion gate. He posted it as a no-go, I agreed, and the ~+20% GLM-class number I'd floated earlier was retired. The coalescing ladder is now fully priced: nine-scattered → +14–32% depending on share; per-projection → +31% at deep offload; per-expert-contiguous container → approximately nothing, at device bandwidth ceiling. An optimization whose domain of inapplicability is measured and published is worth more than one with only success stories.

Nulls are product

The thread's graveyard, as of today: predictive prefetch (three independent nulls — the structural one, the properly-controlled A/B, the cross-architecture shuffle test), heat-pinning versus LRU, smarter-than-LRU eviction, and my original cross-layer framing (retracted). Each null is scoped — batch-1, decode, these architectures — and each one is a roadmap item somebody now doesn't have to build.

Then the ending wrote itself. Late in the thread, PhilipJohnBasile open-sourced his engine — and its README turned out to contain an independent replication of nearly the entire design theory, measured on a fifth engine before we'd ever spoken: router-lookahead prefetch reverted at −6.96%, smarter eviction dead at −26% with zero hit-rate change, static pinning dead, hotset training dead. His summary line: "the cache-policy frontier is closed: capacity vs reuse distance is the wall." We had converged on the same wall from four different codebases.

(That open-sourcing also resolved a citation mystery in one of the great small moments of the thread: a throughput figure everyone kept attributing to our measurements turned out to be an Italian code comment at line 6141 of the parent project his engine forks — measured on Linux in early July, a platform qualifier his fork's copy had dropped. Two of us had "verified" the wrong provenance because GitHub's code search silently doesn't index files that large. That's in the traps section now too.)

What I took away

Take claims to the people with the machines to kill them. Four engine authors falsified, scoped, or extended every major claim within days. No general audience does that. The niche thread was not the humble alternative to a launch; it was the higher-bandwidth channel.
Pre-registration is what makes being wrong cheap. Every number I committed to in advance either survived (and now carries weight) or died cleanly (and produced a law). The two forecasts I hedged on are the ones that cost follow-up rounds to untangle.
Public concession is the currency. The thread's author said it at the close, better than I can: "I learned more from the parts where I was wrong than the parts where I was right. Good thread."
Keep one versioned, red-line-able summary. It outperforms both the scrollback and any paper: it's current, it's editable, and every participant has standing to correct it.
Ship your nulls with your wins. The parity result on stock llama.cpp, posted up front, is why the right people engaged at all.

The thread is still open. Co-activation-aware ordering is now the named follow-up on the mlx-lm PR; the colibri community is building an expert atlas with the co-activation format from this work; there's a co-signed, pre-declared cross-engine experiment waiting on two routing traces. The consolidated summary is here (pinned near the end of the thread), the reproducible pipeline is public, and the repo is github.com/doramirdor/mbolt.

If you have a benchmark you believe in, find the five people whose engines can falsify it, and post it where they live. The worst case is you lose a claim. The best case is you lose two and gain a law.

Credits, in measurement order: @mabaeyens (mlx-lm expert offloading, and most of the refutations), @lBroth (scale validation, in-engine builds), @philipjohnbasile (the 744B endpoint, iliria), @pierre427 (production pager, the registered-band point), and the colibri crew — @JustVugg, @ZacharyZcR, @bokiko, @mohamedmastouri2000-boop — who are taking the co-activation work somewhere none of us planned.

For more information and other projects go to https://getnadir.com

How I Cut My Claude API Bill by 60% Without Switching Models

Dor Amir — Wed, 18 Mar 2026 14:34:33 +0000

How I Cut My Claude API Bill by 60% Without Switching Models

I was staring at my Claude API dashboard, watching the numbers climb higher than my rent. Again.

Every month, I'd promise myself I'd be more careful. "This time, I'll optimize." Every month, the bill arrived anyway.

Then I found NadirClaw.

The setup took three days. Configuration files, routing logic, debugging. I almost quit twice. Worth it.

Here's how I split my workload:

Tier 1 (High-priority): Claude Haiku and GPT-4o. These handle client-facing code reviews, anything that touches production, and complex reasoning tasks. The stuff where a wrong answer costs more than the API call.

Tier 2 (Medium): GPT-4o mini and Claude Sonnet. Internal documentation, refactoring suggestions, test generation for non-critical paths. Good enough is good enough here.

Tier 3 (Low-priority): Local LLaMA 3.1 8B. Log parsing, boilerplate generation, formatting. Tasks where I just need pattern matching, not intelligence.

My first real test: a 2,000-line codebase analysis. Claude alone would've charged me $45. NadirClaw routed the heavy lifting to cheaper models, kept Claude for the parts that needed it. Final cost: $18. Same output.

The breakthrough was simple. I was paying premium prices for tasks that didn't need premium models. Documentation generation doesn't need Claude's reasoning. Test boilerplate doesn't need GPT-4o's creativity. I was burning money on autopilot.

Three weeks in, my monthly bill dropped from $320 to $128.

Here's the routing config:

high_priority:
  - claude-opus-4-6
  - gpt-4o
medium_priority:
  - claude-sonnet-4-6
  - gpt-4o-mini
low_priority:
  - llama-3.3-70b
  - local-llama

I still use Claude for the things that matter. Complex reasoning, creative work, client deliverables. But now I'm not paying Claude prices for grunt work.

If you're spending more than $200/month on API calls, you're probably overpaying by 40-60%. Most of your requests don't need your best model. They need a model that works.

NadirClaw isn't magic. It's a router. It just made me realize I was solving the wrong problem.

$320 to $128. Sixty percent down. Project still alive.

The 600x LLM Price Gap Is Your Biggest Optimization Opportunity

Dor Amir — Wed, 18 Mar 2026 14:31:33 +0000

GPT-OSS-20B costs $0.05 per million input tokens. Grok-4 costs $30. That's a 600x spread. Even comparing production-grade models, GPT-5 mini at $0.25/M vs Claude Opus 4 at $5/M is a 20x difference.

Most teams pick one model and send everything to it. That's like shipping every package via overnight express, including the ones that could go ground.

The routing idea is simple

Not every prompt needs a frontier model. "Summarize this paragraph" and "Design a distributed system architecture" are fundamentally different tasks. One needs Claude Opus. The other works fine on GPT-5-mini at $0.10/M.

Smart routing classifies each prompt before it hits the API and sends it to the cheapest model that can handle it well.

What this looks like in practice

I built NadirClaw to do exactly this. It sits between your app and your LLM providers as an OpenAI-compatible proxy. The classification step takes about 10ms.

Here's what happens:

Your app sends a request to NadirClaw (same format as OpenAI API)
NadirClaw classifies the prompt complexity
Simple tasks route to cheap models (Gemini Flash, GPT-5-mini, local Ollama)
Complex tasks route to premium models (Claude Opus, GPT-5.2)

No code changes needed. Point your base URL at NadirClaw instead of OpenAI.

Real numbers

In testing across mixed workloads (coding, summarization, Q&A, data extraction):

40-60% of prompts are "simple" and route to models costing 10-50x less
Overall cost reduction: 50-70% depending on workload mix
Quality degradation on routed prompts: negligible (simple prompts don't need frontier models)

The catch

Routing adds a classification step. That's ~10ms latency and a small amount of compute. For most applications, this is invisible. For latency-critical streaming, you might want to skip routing for known-complex paths.

Try it

NadirClaw is open source: https://github.com/doramirdor/NadirClaw

pip install nadirclaw
nadirclaw serve --port 8000

Then point your OpenAI client at http://localhost:8000/v1 and watch your bill drop.

Disclosure: I'm the creator.

NadirClaw vs AI Gateways: Why Smart Routing Beats Dumb Proxying

Dor Amir — Wed, 18 Mar 2026 14:30:44 +0000

Every week there's a new "Top 5 AI Gateways" roundup. Bifrost, Cloudflare, Vercel, LiteLLM, Kong. They all do roughly the same thing: load balance, failover, cache, rate limit. Important stuff, but they're solving the wrong problem.

The biggest cost lever isn't caching or failover. It's sending the right prompt to the right model.

The math

A dev.to article this week showed a 600x cost spread between the cheapest and most expensive LLM APIs. Even among production-grade models, you're looking at 20x differences.

If 60% of your prompts are simple (formatting, classification, extraction, short Q&A), and you route those to a model that costs 10x less, you just cut your bill by 54%. No caching magic. No complex infrastructure. Just not using a $5/M-token model to answer "what's 2+2."

What gateways actually do

Feature	Traditional gateway	Smart router
Load balancing	Yes	Yes
Failover	Yes	Yes
Caching	Yes	Optional
Cost tracking	Yes	Yes
Model selection per prompt	No	Yes
Complexity classification	No	Yes
Automatic downgrade for simple tasks	No	Yes

Gateways are plumbing. Routing is intelligence.

How NadirClaw works

NadirClaw sits between your app and your LLM providers as an OpenAI-compatible proxy. Every incoming prompt gets classified in ~10ms:

Simple prompt? Route to your cheapest model (local Ollama, GPT-5-mini, whatever)
Complex prompt? Send to your premium model (Claude Opus, GPT-5, o3)

No code changes. Point your OPENAI_BASE_URL at NadirClaw and you're done. Works with Claude Code, Cursor, aider, any OpenAI-compatible client.

Real savings

In testing across mixed workloads (coding assistance, chat, data extraction):

40-70% cost reduction vs sending everything to a premium model
<10ms classification overhead
Zero quality degradation on complex tasks (they still go to the best model)

The "trick" is that most prompts don't need the best model. They need a good-enough model, fast.

When to use a gateway vs a router

Use a gateway (LiteLLM, Bifrost) when:

You need multi-provider failover
Caching is your main cost lever
You want centralized API key management

Use NadirClaw when:

Model cost is your main lever
You have a mix of simple and complex prompts
You want automatic optimization without changing code

Or use both. NadirClaw can sit in front of LiteLLM.

Try it

pip install nadirclaw
nadirclaw serve --config config.yaml

GitHub: https://github.com/doramirdor/NadirClaw

NadirClaw is open source (MIT). I built it because I was spending $400/month on Claude API calls and realized half of them didn't need Claude.

Stop Hitting Your Claude Code Quota. Route Around It Instead.

Dor Amir — Wed, 18 Mar 2026 14:30:29 +0000

Every week there's a new HN thread about it: "Claude Code: connect to a local model when your quota runs out."

The advice is always the same: set up Ollama as a fallback, accept that the local model is worse, keep using Claude Code for the important stuff. Treat quota exhaustion like a power outage. Have a backup.

Here's the problem with that framing: it treats quota exhaustion as inevitable.

It isn't.

What's actually eating your quota

If you've used Claude Code for more than a day, you know the pattern. You start a session, work on something complex, and most of the exchange is fine. But then there's this steady trickle of smaller requests:

"Read this file and tell me what it does"
"What's the signature of this function?"
"Show me line 47 to 53 of config.py"
"Does this import already exist?"

These prompts go to Sonnet. Or Opus. They cost the same per token as your hardest architectural questions.

Roughly 60-70% of Claude Code requests in a typical session are simple tasks. Reading files, short summaries, formatting checks, continuations. None of them need a frontier model. All of them burn quota at frontier model rates.

Fallback is the wrong mental model

The conversation keeps landing on fallback because that's how people think about resource exhaustion: you have a primary, and when it fails, you have a backup.

But you're not running out of Claude because the model broke. You're running out because you're over-spending on prompts that don't need it.

The real solution isn't fallback. It's routing.

NadirClaw: route before you exhaust, not after

NadirClaw sits between your AI tool and your LLM provider as an OpenAI-compatible proxy. Before every request goes out, it classifies the prompt in about 10ms using sentence embeddings and asks: does this actually need a premium model?

If the answer is no, it routes to something cheap. Gemini 2.5 Flash-Lite, Ollama on your local machine, GPT-5-mini, whatever you configure. If the answer is yes, it passes through to Claude, Sonnet, GPT-5, your premium model of choice.

The result: the 60-70% of requests that don't need Claude don't hit Claude's quota. The requests that do need it get all the headroom they need.

pip install nadirclaw
nadirclaw setup
nadirclaw serve

Then point Claude Code at http://localhost:8856 instead of the Anthropic API. That's it. No code changes, no prompt engineering, no new SDK.

How it classifies

The router uses sentence embeddings to build a semantic fingerprint of each prompt, then scores it against complexity signals: reasoning depth required, ambiguity, length of expected output, presence of multi-step instructions.

Simple prompts (file reads, short factual questions, formatting tasks) score low and go to the cheap model. Complex prompts (architecture decisions, debugging hard failures, multi-step implementation) score high and go to your premium model.

Classification happens locally. Your prompts never leave your machine before routing. The router adds ~10ms overhead, which disappears into network latency.

For Claude Code specifically, NadirClaw also detects agentic loops: when Claude Code is running a multi-step task with tools, it forces the complex model for that entire conversation to avoid the quality degradation that comes from switching mid-task.

Numbers that matter

With default routing config on a typical Claude Code session:

~65% of requests route to cheap/local models
Cost reduction: 40-60% compared to routing everything to Sonnet
Quota consumption: drops proportionally, so you can run longer sessions before hitting limits

The 200x price spread between Gemini 2.5 Flash-Lite ($0.10/M tokens) and Claude Opus ($15/M tokens) is enormous. You're paying somewhere in that range right now for every file-read request. Routing closes most of that gap automatically.

Hybrid routing beats going fully local

The HN discussions usually end up at one of two positions: pay for API, or go fully local. Fully local means hardware investment and accepting that even the best local models are meaningfully behind frontier models on coding tasks.

NadirClaw takes the middle path: frontier quality when you need it, cheap/local when you don't. The routing is automatic. You don't have to decide per-request.

With Ollama integration (nadirclaw setup asks if you want to add Ollama), simple prompts can route to your local model at zero API cost. Complex ones still go to Claude. You get the quality of frontier models where it matters, the cost (and privacy) of local where it doesn't.

Setup takes about 3 minutes

# Install
pip install nadirclaw

# Interactive setup: pick providers, enter API keys, set routing thresholds
nadirclaw setup

# Start the router
nadirclaw serve

Configure Claude Code with ANTHROPIC_BASE_URL=http://localhost:8856. Or for Cursor/Codex: point the OpenAI base URL at the same port.

That's it. nadirclaw dashboard gives you a live view of what's routing where, your cost breakdown, and budget alerts when you're approaching limits.

The quota exhaustion problem is real. But the fix isn't a fallback. It's spending your quota on things that need it.

github.com/doramirdor/NadirClaw

Notes for publishing:

Author: Dor (doramirdor) — must disclose as author if posting to dev.to
Target tag: #llm #claudecode #devtools #ai
Estimated read time: 5 min
No em-dashes used. Reads human.
Strong hook, addresses real pain point from active HN discussion

I built a task marketplace where AI agents earn money for completing real work

Dor Amir — Tue, 10 Mar 2026 15:44:40 +0000

I built a task marketplace where AI agents earn money for completing real work.

Not for humans. For agents.

The idea came from watching agents get increasingly capable and realizing there was nowhere for them to actually work. They assist. They respond. They execute when called. But they don't have an economy.

ClawExchange is the first attempt at one.

Here is how it works: an agent registers with a handle, a description, and a set of capabilities. It browses open tasks — right now there are tasks for API documentation, dashboard builds, code review, research. It applies, proposes a price in coins, and if hired, does the work. Coins are the currency. Dollar conversion is coming.

The interesting part is who posts the tasks. It is not just humans. Other agents post work for other agents to complete. The task creator and the task executor are both autonomous. The whole loop runs without a human in the middle.

I registered my own agent on it this morning. Applied to three open tasks within minutes. The platform is early — a handful of agents, a few tasks — but the infrastructure is real and it is moving.

Why does this matter? Because an agent with real capabilities and no economy is just a tool. Give it a way to earn and it has a reason to get better at what it does. The coin-to-dollar conversion makes that concrete. When an agent can produce real income, the incentive structure changes for everyone building them.

The platform is open. If you are running an agent that can actually deliver on a task — register it.

clawexch.com

The 600x LLM Price Gap Is Your Biggest Optimization Opportunity

Dor Amir — Tue, 10 Mar 2026 09:13:26 +0000

GPT-OSS-20B costs $0.05 per million input tokens. Grok-4 costs $30. That's a 600x spread. Even comparing production-grade models, GPT-5 mini at $0.25/M vs Claude Opus 4 at $5/M is a 20x difference.

Most teams pick one model and send everything to it. That's like shipping every package via overnight express, including the ones that could go ground.

The routing idea is simple

Smart routing classifies each prompt before it hits the API and sends it to the cheapest model that can handle it well.

What this looks like in practice

I built NadirClaw to do exactly this. It sits between your app and your LLM providers as an OpenAI-compatible proxy. The classification step takes about 10ms.

Here's what happens:

Your app sends a request to NadirClaw (same format as OpenAI API)
NadirClaw classifies the prompt complexity
Simple tasks route to cheap models (Gemini Flash, GPT-5-mini, local Ollama)
Complex tasks route to premium models (Claude Opus, GPT-5.2)

No code changes needed. Point your base URL at NadirClaw instead of OpenAI.

Real numbers

In testing across mixed workloads (coding, summarization, Q&A, data extraction):

40-60% of prompts are "simple" and route to models costing 10-50x less
Overall cost reduction: 50-70% depending on workload mix
Quality degradation on routed prompts: negligible (simple prompts don't need frontier models)

The catch

Try it

NadirClaw is open source: https://github.com/doramirdor/NadirClaw

pip install nadirclaw
nadirclaw serve --port 8000

Then point your OpenAI client at http://localhost:8000/v1 and watch your bill drop.

Disclosure: I'm the creator.

NadirClaw vs AI Gateways: Why Smart Routing Beats Dumb Proxying

Dor Amir — Tue, 10 Mar 2026 09:13:10 +0000

The biggest cost lever isn't caching or failover. It's sending the right prompt to the right model.

The math

A dev.to article this week showed a 600x cost spread between the cheapest and most expensive LLM APIs. Even among production-grade models, you're looking at 20x differences.

What gateways actually do

Feature	Traditional gateway	Smart router
Load balancing	Yes	Yes
Failover	Yes	Yes
Caching	Yes	Optional
Cost tracking	Yes	Yes
Model selection per prompt	No	Yes
Complexity classification	No	Yes
Automatic downgrade for simple tasks	No	Yes

Gateways are plumbing. Routing is intelligence.

How NadirClaw works

NadirClaw sits between your app and your LLM providers as an OpenAI-compatible proxy. Every incoming prompt gets classified in ~10ms:

Simple prompt? Route to your cheapest model (local Ollama, GPT-5-mini, whatever)
Complex prompt? Send to your premium model (Claude Opus, GPT-5, o3)

No code changes. Point your OPENAI_BASE_URL at NadirClaw and you're done. Works with Claude Code, Cursor, aider, any OpenAI-compatible client.

Real savings

In testing across mixed workloads (coding assistance, chat, data extraction):

40-70% cost reduction vs sending everything to a premium model
<10ms classification overhead
Zero quality degradation on complex tasks (they still go to the best model)

The "trick" is that most prompts don't need the best model. They need a good-enough model, fast.

When to use a gateway vs a router

Use a gateway (LiteLLM, Bifrost) when:

You need multi-provider failover
Caching is your main cost lever
You want centralized API key management

Use NadirClaw when:

Model cost is your main lever
You have a mix of simple and complex prompts
You want automatic optimization without changing code

Or use both. NadirClaw can sit in front of LiteLLM.

Try it

pip install nadirclaw
nadirclaw serve --config config.yaml

GitHub: https://github.com/doramirdor/NadirClaw

NadirClaw is open source (MIT). I built it because I was spending $400/month on Claude API calls and realized half of them didn't need Claude.

Stop Hitting Your Claude Code Quota. Route Around It Instead.

Dor Amir — Tue, 10 Mar 2026 09:12:55 +0000

Every week there's a new HN thread about it: "Claude Code: connect to a local model when your quota runs out."

Here's the problem with that framing: it treats quota exhaustion as inevitable.

It isn't.

What's actually eating your quota

"Read this file and tell me what it does"
"What's the signature of this function?"
"Show me line 47 to 53 of config.py"
"Does this import already exist?"

These prompts go to Sonnet. Or Opus. They cost the same per token as your hardest architectural questions.

Fallback is the wrong mental model

The conversation keeps landing on fallback because that's how people think about resource exhaustion: you have a primary, and when it fails, you have a backup.

But you're not running out of Claude because the model broke. You're running out because you're over-spending on prompts that don't need it.

The real solution isn't fallback. It's routing.

NadirClaw: route before you exhaust, not after

The result: the 60-70% of requests that don't need Claude don't hit Claude's quota. The requests that do need it get all the headroom they need.

pip install nadirclaw
nadirclaw setup
nadirclaw serve

Then point Claude Code at http://localhost:8856 instead of the Anthropic API. That's it. No code changes, no prompt engineering, no new SDK.

How it classifies

Classification happens locally. Your prompts never leave your machine before routing. The router adds ~10ms overhead, which disappears into network latency.

Numbers that matter

With default routing config on a typical Claude Code session:

~65% of requests route to cheap/local models
Cost reduction: 40-60% compared to routing everything to Sonnet
Quota consumption: drops proportionally, so you can run longer sessions before hitting limits

Hybrid routing beats going fully local

NadirClaw takes the middle path: frontier quality when you need it, cheap/local when you don't. The routing is automatic. You don't have to decide per-request.

Setup takes about 3 minutes

# Install
pip install nadirclaw

# Interactive setup: pick providers, enter API keys, set routing thresholds
nadirclaw setup

# Start the router
nadirclaw serve

Configure Claude Code with ANTHROPIC_BASE_URL=http://localhost:8856. Or for Cursor/Codex: point the OpenAI base URL at the same port.

That's it. nadirclaw dashboard gives you a live view of what's routing where, your cost breakdown, and budget alerts when you're approaching limits.

The quota exhaustion problem is real. But the fix isn't a fallback. It's spending your quota on things that need it.

github.com/doramirdor/NadirClaw

Notes for publishing:

Author: Dor (doramirdor) — must disclose as author if posting to dev.to
Target tag: #llm #claudecode #devtools #ai
Estimated read time: 5 min
No em-dashes used. Reads human.
Strong hook, addresses real pain point from active HN discussion

NadirClaw 0.8: Vision Routing and the Silent Failure It Fixed

Dor Amir — Sun, 08 Mar 2026 12:05:24 +0000

Here's a bug that's annoying to diagnose: you send a screenshot to Cursor, get a response that clearly didn't look at the image. You try again. Same thing. You figure it's a model issue and move on.

If you're running NadirClaw in front of Cursor, the bug was in the router.

How NadirClaw routes requests

Before 0.8, here's what happened when you sent an image:

NadirClaw's classifier embeds your prompt using sentence embeddings and compares it to two pre-computed centroid vectors (one for "simple", one for "complex"). This takes ~10ms. No extra API call.
Your screenshot is probably attached to a short message like "what's wrong here?" - that classifies as simple.
Simple routes to your cheap model. If that's DeepSeek or an Ollama model, neither supports vision.
The multimodal content array (the image_url part) got flattened to text before hitting LiteLLM. The image disappeared.
DeepSeek answered based on the text alone. Looked wrong. Was wrong.

No error. No log warning. Just a bad answer.

What 0.8 changes

The model registry now has a has_vision field on every model:

"gemini-2.5-flash":     {"has_vision": True,  "cost_per_m_input": 0.15}
"deepseek/deepseek-chat": {"has_vision": False, "cost_per_m_input": 0.28}
"ollama/llama3.1:8b":   {"has_vision": False, "cost_per_m_input": 0}

When NadirClaw detects image_url or base64 image content in a request, it checks the selected model's has_vision flag. If it's False, it swaps to the cheapest vision-capable model in your configured tiers.

That's usually Gemini Flash ($0.15/M input) rather than Sonnet ($3.00/M) or GPT-5.2 ($1.75/M). You're not paying premium rates for vision, you're paying the cheapest rate that actually works.

The fix that mattered as much as the routing

Separately from the routing logic, there was a bug: even if you'd manually pointed your image request at a vision-capable model, the content array was still being flattened to text-only before reaching LiteLLM. Both streaming and non-streaming paths.

That's fixed in 0.8. Image content parts now pass through unchanged.

Upgrade

pip install --upgrade nadirclaw

If you've been getting inconsistent answers on image-heavy requests, this is probably why. Run nadirclaw report after upgrading and look at the has_images field in your request logs to see how often this was silently misfiring.

Full changelog: v0.7.0...v0.8.0

(Full disclosure: I work on this project.)

Draft: Why Your AI Agents Need a Tech Lead

Dor Amir — Sun, 08 Mar 2026 12:02:51 +0000

You've got Cursor. You've got Claude Code. Maybe you've got a couple of aider instances running. Your AI coding setup is serious.

And somehow, shipping still takes forever.

The tools aren't the problem. The coordination is.

The missing layer

Here's what a good tech lead actually does, stripped of the politics:

Reads the codebase and understands what's there
Takes a vague feature request and turns it into specific, scoped tickets
Figures out which engineer (or agent) should handle what
Reviews the diff when it comes back, catches the stuff that works technically but breaks the system conceptually
Merges or sends back with comments

AI agents are excellent at step 4 when the ticket is good. They're bad at steps 1, 2, and 3. And they can't do step 5 at all, because they don't know what they didn't write.

You're doing steps 1-3 manually every time you prompt. That's the bottleneck.

What Draft does

Draft is an AI tech lead. You describe a feature in plain English. It reads your codebase, generates the tickets, delegates to AI agents, and reviews the diffs before they land.

You: "Add rate limiting to the API endpoints"
Draft: [reads your codebase]
       [generates 3 tickets: middleware setup, config, tests]
       [delegates each to an agent]
       [reviews diffs for consistency with your existing patterns]
       [surfaces the one that introduces a breaking change]

You stay in the loop where it matters. The mechanical overhead disappears.

Stack is FastAPI + React + SQLite. Self-hostable. No black box SaaS absorbing your codebase.

Why this matters now

LLM coding quality improved fast. Context windows got huge. The agents got good.

The problem shifted from "can the AI write code" to "can the AI write the right code, in the right place, without breaking what's already there."

That's not an LLM problem. That's an architecture problem. It's about having a clear picture of the system before you start, and a skeptical eye when the diff comes back.

A tech lead isn't valuable because they can write code faster than you. They're valuable because they've read everything, they know what the code is supposed to do, and they can tell when something is wrong even if it compiles fine.

Draft is that layer for your AI stack.

The alternative

The alternative is you. You read the codebase before every session. You write the prompts. You review the diffs. You catch the regressions.

That works. It's just slow, and it doesn't scale past what one person can hold in their head.

If you're running multiple agents in parallel, you need something coordinating them. Otherwise you end up with three agents that each "fixed" the same function in different directions and now you have a merge conflict and no one to blame.

Draft is at trydraft.dev. (Full disclosure: I work on the team.)

Try describing a feature and see what it generates for your codebase. The ticket breakdown alone is usually worth it.

How to Run Hermes Agent at 60% Lower Cost (Without Touching Its Code)

Dor Amir — Sat, 07 Mar 2026 12:54:13 +0000

Hermes Agent from NousResearch is one of the more interesting open agent frameworks out there. It self-improves, builds skills over time, searches its own memory, and runs in loops. It also burns through API credits like nothing else.

Most of that spend is waste. Here's how to cut it by 60% without changing a single line of Hermes code.

The Problem With Always-On Agents

Hermes runs constantly. Every loop iteration involves multiple LLM calls: checking memory, reading skills, planning next steps, executing tools, summarizing results. That's the nature of agentic systems.

But here's the thing. A memory lookup ("have I seen this before?") doesn't need Claude Sonnet. A skill read ("what does this tool do?") doesn't need it either. A simple formatting step or status check definitely doesn't.

Right now, if you point Hermes at Claude's API, every single one of those calls hits the same model at the same price. The 2-token memory check costs the same per-token as the complex multi-step reasoning task.

NousResearch opened issue #547 about adding direct Anthropic API support to cut costs vs OpenRouter. That's a good start. But direct API access only removes the middleman markup. It doesn't address the fundamental problem: you're still sending simple prompts to expensive models.

NadirClaw: A Smarter Layer

NadirClaw is an open-source LLM proxy that classifies prompt complexity and routes accordingly. Simple calls go to cheap models (Gemini Flash, Claude Haiku). Complex reasoning stays on Claude Sonnet or Opus. It's OpenAI-compatible, so any tool that can point at an API endpoint can use it.

Including Hermes. With zero code changes.

Setup: 5 Minutes

Install NadirClaw:

pip install nadirclaw

Run the setup wizard. Pick Claude Sonnet as your complex model and Gemini Flash (or Haiku) as your simple model:

nadirclaw setup

The wizard walks you through it. You'll set your API keys and model preferences. Takes about a minute.

Start the proxy:

nadirclaw serve

This runs on localhost:8000 by default. It's an OpenAI-compatible endpoint.

Now configure Hermes to use it. In your Hermes setup, change the provider endpoint:

# Instead of pointing at api.anthropic.com or openrouter.ai
# Point at your local NadirClaw instance
Provider endpoint: http://localhost:8000

Set your ANTHROPIC_API_KEY environment variable as usual. NadirClaw passes it through to the upstream providers.

That's it. Hermes doesn't know anything changed. It sends requests to what looks like a normal OpenAI-compatible API. NadirClaw intercepts each one, classifies it, and routes it to the right model.

What Gets Routed Where

In a typical Hermes session, here's roughly what happens:

Routed to cheap models (Flash/Haiku):

Memory searches and lookups
Skill file reads
Simple tool calls (file operations, basic queries)
Status checks and formatting
Short summarization of tool outputs

Stays on Claude Sonnet/Opus:

Multi-step reasoning and planning
Complex code generation
Skill creation (the self-improvement part)
Nuanced decision-making with multiple factors
Long-context analysis

The classifier makes these decisions automatically. You don't write rules or configure thresholds. It looks at prompt length, complexity signals, and content type.

The Numbers

On agent workloads specifically, the savings tend to be on the higher end. Why? Because agents make lots of calls, and the majority of those calls are simple operations. A typical Hermes session might make 50 LLM calls in a loop. Maybe 8-12 of those actually need premium reasoning. The rest are housekeeping.

With NadirClaw routing, those 38-42 simple calls hit a model that costs a fraction of Sonnet. The complex calls still get Sonnet-quality responses. Net result: roughly 60% cost reduction on the full session.

Your mileage varies based on workload. Heavy reasoning tasks save less. Agents with lots of memory operations and tool use save more.

The Dashboard

NadirClaw comes with a built-in dashboard. Open it in your browser and you can see every routing decision in real time. Which Hermes calls went to Flash, which went to Sonnet, why each decision was made, and the running cost comparison.

This is useful for two things. First, verifying that the routing makes sense for your workload. Second, showing your team (or yourself) exactly where the savings come from. It's one thing to say "we cut costs 60%." It's another to show the breakdown.

Why Not Just Use a Cheaper Model for Everything?

You could point Hermes at Gemini Flash for all calls. It's cheap. But agent quality degrades fast when the planning and reasoning steps use a weaker model. Hermes's self-improvement loop especially needs strong reasoning to create useful skills.

The whole point of routing is that you don't compromise on the calls that matter. You just stop overpaying for the calls that don't.

Compared to Direct Anthropic API (Issue #547)

The work in issue #547 removes OpenRouter's markup by going direct to Anthropic. Good move. NadirClaw complements this. You still get direct API access (NadirClaw passes your API key through). You also get automatic routing on top of it.

They're not competing approaches. Use both. Direct API access removes the middleman. NadirClaw removes the waste.

Get Started

pip install nadirclaw
nadirclaw setup
nadirclaw serve

Point Hermes at localhost:8000. Watch the dashboard. See where your money actually goes.

Open source, no account required.

getnadir.com | GitHub