DEV Community: Glenn Allogho

Your fallback model only counts if the tool calls survive

Glenn Allogho — Tue, 03 Nov 2026 09:25:27 +0000

Anthropic's S-1 says open-weight models could win some workloads and pressure its prices. That doesn't mean Claude is losing. It does mean your agent workflow sits on commercial decisions you don't control, so a backup model is worth having.

The catch is that "backup" is usually just an endpoint. DeepSeek and Kimi both speak the Anthropic format, so pointing Claude Code at them takes minutes. Whether the agent still works is a different question. In one reported case, a small local Qwen model behind Ollama returned tool calls as plain text JSON instead of tool_use blocks, four times running. Valid responses, zero finished tasks.

Three checks before you trust a fallback:

The protocol carries tool calls, tool results and streaming intact.
Your real harness still enforces hooks and permissions on a throwaway repo. The harness does that, not the model.
About five real tasks, several runs each, with the patches actually validated.

Then compare cost per accepted task, including failed attempts and review time, instead of price per token.

Full breakdown, including the pricing math and the routing traps: https://levelup.gitconnected.com/anthropics-s-1-names-its-real-rival-cheap-open-weight-models-05d11f23d229?sk=d17020566f668a2d02482e5f20461815

Rust won't save your agent from a double charge

Glenn Allogho — Sat, 31 Oct 2026 09:16:35 +0000

An agent sends a payment request. The provider accepts it, the connection drops before the response arrives, the worker times out, the orchestrator retries. The customer is charged twice, and a Rust rewrite would not have prevented it.

Rust's ownership model removes use-after-free bugs and data races in safe code. It can't know whether a remote API finished an operation before the network failed. That takes durable state, idempotency and a recovery protocol that separates safe retries (a read-only search) from operations that need reconciliation (a payment).

Raw speed is a weak argument too. In an illustrative run of 4 model calls at about 800 ms, 6 tool calls at about 150 ms and 100 ms of orchestration, removing orchestration completely saves about 2.4%.

Where Rust earns its place is concurrency: how many agent sessions one worker sustains in a fixed memory budget at acceptable tail latency. Memory safety, concurrency correctness, distributed execution correctness. Rust covers the first, helps with the second, and leaves the third to your architecture.

Full piece, with the state store, queue, worker and recovery design: Why Rust might be the most important language for the next era of AI

A cheaper model with a better harness just beat the stronger one

Glenn Allogho — Thu, 29 Oct 2026 09:15:44 +0000

Google Cloud AI Research's Harness² keeps the model frozen and rewrites everything around it: system prompt, guardrails, memory, tool guidance, subagents, skills, scripts and plugins.

The loop proposes two edited harnesses, runs them against the current one on the same task with the same model, and keeps only the changes the traces support. Lessons get stored and retrieved per task, so one fix doesn't leak into unrelated work.

One round took GLM-5 under OpenCode from 64.2 to 75.7 on WorkBuddy-Bench, above base Gemini at 72.4.

The consequence for builders: the model is the part everyone can swap next month. The harness, and the evidence behind each piece of it, is the part that stays yours. The catch is that "better" here is behavioral evidence with no answer key, and the repo itself warns about unrestricted network access and a disabled sandbox in the Codex runs. Self-editing harnesses need controls in code.

Full breakdown, with the benchmarks and where it loses to the baseline: Google's Harness² suggests the next AI upgrade may not be a new model

Your proactive AI agent costs 4x the plan price

Glenn Allogho — Tue, 27 Oct 2026 09:21:22 +0000

Comparing Claude Duet, OpenAI's Dots, Meta's Muse and Grok Bot by monthly price misses the real bill: the human review every output still needs.

cost per accepted outcome = (plan cost + review minutes x hourly rate) / accepted outputs

A hypothetical: a $100 plan drafts 40 briefs, 32 get accepted. Sticker math says $3.13 each. Add 5 minutes of review per brief and 15 minutes to fix each reject, at $60 an hour, and it's about $13.13 each.

Two more filters that matter before model quality:

Permissions decide where work stops. Duet blocks trading and transfers even with approval. Dots hands transfers and password changes back to you. Muse routes actions through a separate allow, ask or deny authority. Grok Bot's background approvals expire after about 10 minutes.
Availability. Personal Dots accounts exclude the EEA, Switzerland and the UK. Duet is a waitlisted US preview.

Pilot two eligible agents on the same bounded recurring job, judge output before granting actions, and decide against a threshold you set in advance.

Full comparison table, quota math and the six-step pilot: Proactive AI agents: Claude Duet, Dots, Muse or Grok Bot?

Once tokens cost cents, your agent bill runs on minutes

Glenn Allogho — Sun, 25 Oct 2026 09:19:06 +0000

DeepSeek V4.1-Flash-1017 resolves a SWE benchmark task for just under 20 cents in tokens. At that price, the bill that matters is wall-clock time: sandboxes, CI runners, concurrency slots and people waiting.

cost per task ≈ token_cost + wall_clock_hours x hourly_rate_of_everything_waiting

Three things change once you price time:

The host matters as much as the model. Same MIT weights, OpenRouter hosts ranging from 24 to 290 tokens per second, prices barely tracking it.
Concurrency follows wall clock. At 1,000 tasks an hour, a 7.6 minute task (210 tok/s) needs about 127 slots. The same task at 42 tok/s takes 23.5 minutes and needs about 392.
Success rate divides everything. 8 minutes at 55% success is roughly even with 12 minutes at 85%. Compare cost per successful task, never per-token price.

Also worth knowing: DeepSeek's direct API is double price in weekday peak windows, and the deepseek-flash alias moves on release day, so pin versions.

The full worked example, the cost script and a Python harness to measure time and cost per task on your own agent traffic: DeepSeek V4.1-Flash-1017: Why Time Per Task Beats Token Price

Your LLM tracing tool isn't where the lock-in is

Glenn Allogho — Fri, 23 Oct 2026 08:17:30 +0000

Langfuse, Phoenix and LangSmith all accept OpenTelemetry spans over OTLP. If your app emits plain spans, the backend is a config choice:

OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:6006 python trace_demo.py

So tracing isn't the decision. Scores, datasets and prompts are, because no exporter setting moves them. That leaves four constraints worth picking on:

License. Langfuse is MIT outside ee/. Phoenix's server is Elastic License 2.0, so no hosting it for clients. LangSmith is closed, self-host is Enterprise only.
Ops. Self-hosted Langfuse is six containers. Phoenix is one process on SQLite.
Billing shape. Langfuse Cloud bills traces plus observations plus scores. Cheap at 4 spans per trace, more expensive than LangSmith at 40.
Eval loop. Wherever your evals run is where you'll stay.

The full comparison, a runnable trace_demo.py, and the pricing math at three traffic shapes: Langfuse vs Phoenix vs LangSmith: the decision is not tracing

Anthropic didn't leave Git because the repo got big

Glenn Allogho — Wed, 21 Oct 2026 08:23:37 +0000

Anthropic replaced Git in its monorepo with Scion, and the lazy reading is "Git doesn't scale." The real break was checkout volume. Almost every client was a CI job or an agent sandbox: always online, short-lived, created by the million. For those, a full local clone is pure overhead, and several hundred mirrors stopped helping.

Scion moves the work to the server. Clients ask for the exact objects they lack, and every query is also an API. Anthropic reports 99%+ cache hits, a 200x smaller footprint than the mirror fleet, and 25% of CI CPU freed.

You can't install it, but two ideas work on plain Git today. Stop cloning to ask questions: a small service that owns one mirror can answer "what changed?" and "what's the merge base?" for every job. And make checkout cost follow the change:

git clone --filter=blob:none --no-checkout --sparse "$REPO_URL" src
git -C src sparse-checkout set --cone services/payments build
git -C src checkout --detach "$HEAD_SHA"

The query service in about 100 lines of Python, where sparse checkouts break, and how Scion compares to Sapling, Piper and Scalar: Anthropic Scion vs Git: what agent-scale source control changes

Clef replaces your router's classifier call, not your router

Glenn Allogho — Mon, 19 Oct 2026 08:14:29 +0000

The weakest line in most agent routers is a small LLM told to "reply with only one word." Sometimes it replies with "coding" plus a paragraph, and a strict parser sends a coding prompt to the cheap model.

Cloudflare's Clef removes that failure. It scores the options you give it in one pass, generates no tokens, and bills input only: $0.038 per million on Clef-flash. In one public build it routed 200 of 200 test requests correctly.

That builder kept his 0.7 confidence threshold, his fallback flag and his "unsure goes to Sonnet" rule anyway. Clef replaced the classifier call. The policy is still yours to write and tune. And on Cloudflare's own RouterBench run it ties TypeSafe's Jev (79.7 vs 79.9), so the case for switching is price, the free tier, images or open weights, not better routing.

Prices, outside latency tests, the routing policy I'd start from and a four-step check on 200 of your own requests: Cloudflare Clef Makes Your Agent Router Obsolete

Meta's best model has no API price, and that's the strategy

Glenn Allogho — Sat, 17 Oct 2026 08:23:19 +0000

Muse Wonder costs exactly what Claude Opus 5.5 costs: $4 in, $20 out per million tokens. The price match got the headlines. The interesting part is underneath.

Meta's API speaks both OpenAI's and Anthropic's wire formats, and Meta publishes a seven-setting config that points Claude Code at muse-wonder-1.0. Switching in is a config change. But Wonder Full, the stronger model, has no per-token price. It only ships inside Muse Code, from $50 a month per seat, with training on your code on by default.

My read: easy in at Opus prices, subscription to reach the top. If you build your own harness, you can buy Anthropic's and OpenAI's flagships by the token and not Meta's. If Meta puts Wonder Full on the API, that read is wrong.

Configs, a per-task cost calculator and a five-task test to run before switching: Meta priced Muse Wonder like Claude, then kept the best model off the API

Open weights don't make a 1T model cheap to run

Glenn Allogho — Thu, 15 Oct 2026 08:13:29 +0000

Mistral Large 4 is 1.05T parameters with 52B active per token. The 52B sets your compute. The 1T sets your memory, because every expert has to stay resident in GPU memory for the router to pick it. At FP8 that is about a terabyte of weights before any KV cache.

That makes 8x B200 (1,440 GB) the first single node I'd plan FP8 on. Rented at about $53.52 an hour, it has to sustain roughly 7,100 output tokens per second all month to beat Mistral's preview API price, about 3,600 at list price. Most teams won't have that concurrency.

My read: the open weights matter most as an option that caps vendor leverage. Running them yourself pays off only for organisations that already run GPU clusters.

Memory tables, hardware tiers and the break-even chart: Mistral Large 4 is Open-Weight, But Not Easy to Run

Green tests are not evidence your coding agent finished the job

Glenn Allogho — Tue, 13 Oct 2026 08:56:36 +0000

A coding agent can ship a green diff that is not the job you asked for.

A vague brief leaves the agent to decide what is broken, what must stay stable, and how far the change reaches. Those are product and engineering decisions. The implementation makes them for you if you leave them implicit.

The missing ingredient is rarely a better prompt. It is four loops around the agent: bound the outcome, require a plan before edits, delegate only inspectable increments, and approve evidence before merge. Green tests say the assertions still pass. They do not say the increment matched the bound. A diff shows what changed. The agent's completion report shows what it thinks changed. A mismatch is a review target.

Full write-up with a task-brief template and before-merge checklist: The Four Loops that Keep Coding-Agent Work Reliable

A decision model that can't refuse will still answer

Glenn Allogho — Sun, 11 Oct 2026 08:23:49 +0000

OpenAI's Decisions API (public beta since Oct 6) and TypeSafe's Jev do the same job: closed questions in, typed answers with probabilities out, no prose.

Jev is cheaper ($0.042 vs $0.10 per million input tokens, no output charge on either) but text only. OpenAI takes images, offers US and EU residency, and can return a refusal. Jev always picks one of your options. Jev has a published calibration number; OpenAI has none yet.

My read: the price gap barely matters at agent volumes. What matters is what your code does when the model shouldn't be answering. A model that must choose gives you a confident label for input it didn't understand, so you need your own "insufficient" option and a human fallback either way.

I compared images, latency, pricing, residency, limits, and a few small tests that point in different directions here: OpenAI's Decisions API vs Jev: which one should you use?