DEV Community: euk ela

Claude Fable 5.2: One Model, Two Doors — and the 20% Price Cut That Isn't About Price

euk ela — Tue, 10 Nov 2026 03:31:33 +0000

The short version

Anthropic shipped Claude Fable 5.2 and Claude Mythos 5.2 on November 9. On the surface it is a routine upgrade: faster, cheaper, higher benchmark scores. The interesting part is the structure — one underlying model, two different doors into it — and the fact that Anthropic is now pricing by the task, not just by the token.

Three numbers to hold on to:

Price: $8/$40 per million tokens, down 20% from 5.1's $10/$50
Speed: output up to 30% faster than 5.1, plus a fast mode at up to 2.5x speed for double the price ($16/$80)
Benchmarks: 49.1% on FrontierCode 2.0, ahead of Opus 5.5 (45.5%), GPT-6 Astra (42.8%), and Gemini 4 Carbon (41.1%)

Here is the counterintuitive part: Anthropic still recommends Opus 5.5 as the default for most coding and knowledge work. Why isn't the strongest model the default? The answer is in the two calculations below.

One model, two doors

Fable 5.2 and Mythos 5.2 are the same underlying model. The only difference is the guardrails.

Fable 5.2: ships with cybersecurity and biosecurity guardrails, publicly available
Mythos 5.2: loosens some of those guardrails, only for organizations that pass Cyber Verification or the Life Sciences Verification Program

Anthropic says Fable 5.2's guardrails falsely block about 55% fewer requests than 5.1 (about 60% fewer in cybersecurity, about 40% in biology), at the same strength. So the same capability comes in two versions: a filtered one for everyone, and a fuller one for verified institutions. Model capability is now explicitly layered by trust.

For agent developers, that means the same prompts and toolchains may behave differently depending on which door you walk through. If you build agents in security or biology, check which door you are on first.

Cheaper per token, but the real bill is per task

Fable 5.2's per-token price is twice Opus 5.5's. Looked at that way, it is the expensive one. Anthropic's argument is a different ledger:

	Fable 5.2	Opus 5.5
Input / output (per million tokens)	$8 / $40	$4 / $20
FrontierCode 2.0	49.1%	45.5%
Avg. cost per task, FrontierCode High effort	~$6.72	~2x Fable's (Anthropic's figure)

On FrontierCode High effort, Fable 5.2 roughly matches Opus 5.5's best coding score at about half the cost per task. The logic is simple: fewer steps, a higher chance of getting it right the first time, so total tokens go down.

But that conclusion has a precondition: it only holds for long, hard tasks. For short tasks and routine coding, Opus 5.5's per-token advantage is still there — which is exactly why Anthropic recommends it as the default. Before picking a model, look at your own task mix: the higher the share of long tasks, the more Fable 5.2 pays off.

# Model IDs (from Anthropic's official docs)
claude-fable-5-2    # standard mode, $8/$40
claude-mythos-5-2   # restricted mode, same price, verification required
# fast mode: up to 2.5x speed at $16/$80; Mythos fast mode only for verified organizations

How to read the benchmark table

Anthropic's self-reported scorecard (all vendor numbers, not independently verified):

Benchmark	Fable 5.2	Fable 5.1	Opus 5.5	GPT-6 Astra
FrontierCode 2.0 (agentic coding)	49.1%	32.4%	45.5%	42.8%
FrontierSWE v2 (agentic coding)	69.9%	56.3%	62.3%	65.5%
Terminal-Bench-Science 0.1 (agentic research)	74.6%	52.6%	58.7%	68.1%
Humanity's Last Exam (with tools)	74.2%	—	67.7%	57.2%
Remote Labor Index (knowledge work)	41.3%	—	21.3%	20.8%

Two independent data points for cross-checking: Artificial Analysis gives Fable 5.2 an Intelligence Index of 65, the highest score to date (Carbon 60, Opus 5.5 59). Mythos 5.2 scores 76.3% on CyScenarioBench and 89.6% on CVE Rediscovery.

A reading note: Fable leads in 13 of 15 vendor comparisons, but Carbon wins one business-automation test (53.9% vs 51.4%), and Opus 5.5 edges ahead in Artificial Analysis's human-to-human Briefcase ranking (1800 vs 1785). "Leading everywhere" is not the fact — "leading in most places" is. Also, Fable 5.2's FrontierCode run was at Xhigh effort while Carbon ran at Medium effort — cross-tier comparisons need a discount.

Four takeaways for agent developers

Route by task length first. Long, hard tasks go to Fable 5.2; routine work stays on Opus 5.5. Run your own task distribution through the math before trusting the per-token price.
Fast mode is a rescue, not a default. 2.5x speed at double the price fits deadline-driven long tasks, not everyday use.
Security and biology work: check your access first. Fable's guardrails will block some requests; false blocks are down 55%, but if your use case sits on the guardrail boundary, go through verification.
Benchmarks at max effort will mislead you. Fable 5.2's top tier costs about $26.50 per task, roughly 2.6x Opus 5.5. Test on your own tasks before going to production — don't copy launch-day numbers.

The honest limitations

Every benchmark above is self-reported by Anthropic; the methodology is in the system card, and nobody had independently reproduced it at launch.
1M-token context, 128K max output, knowledge cutoff June 2026.
Fable 5.1 is now a legacy model; the Pro tier includes 1 Fable reset in the launch week (per Anthropic's help center — details vary by tier).

Would you move your long tasks to Fable 5.2, or keep Opus 5.5 as the default? Tell me how you route.

Facts checked against Anthropic's November 9 announcement and cross-checked reporting (VentureBeat, The Decoder, Artificial Analysis) on 2026-11-10. Benchmark figures are vendor-reported unless stated otherwise.

T3 Code: One Control Surface for 7 Coding Agents

euk ela — Mon, 09 Nov 2026 03:04:54 +0000

The short version

T3 Code is not another coding agent. It is a control surface: a phone app, a web app, and a desktop app that all talk to one server running on your machine, and that server drives the coding agents you already have installed — Claude Code, Codex, Cursor, Grok Build, OpenCode, Google Antigravity, and fx. Seven agents, one front door.

It sits at 52.5k GitHub stars under MIT, ranked #3 on GitHub Trending for the week of Nov 2–8. The author's own warning is refreshingly honest: "We are very very early in this project. Expect bugs."

The problem it actually solves

If you run two or three coding agents at once, your problem is not a missing agent. It is too many front doors: a terminal window per agent, separate sessions, separate permission prompts. Step away from your desk and checking on a long-running agent means SSHing back into your machine first.

T3 Code flips the layering. The agents stay exactly as they are; you add one control plane on top. A server runs on your machine, every client (phone, browser, desktop) connects to it, and from any of them you can watch sessions, send instructions, and approve permissions for all seven agents.

Getting started: three commands

curl -fsSL https://t3.codes/install.sh | sh
t3

t3 starts the server and opens the local web app. To keep it running in the background:

t3 service install

Just want a one-off trial without installing?

npx t3@latest

Desktop packages exist too: winget or scoop on Windows, Homebrew on macOS, .deb for Debian/Ubuntu, AUR for Arch, and nixpkgs for Nix users.

One prerequisite people miss: each agent's own CLI still has to be installed and logged in first — claude auth login for Claude Code, codex login for Codex, agent login for Cursor, and so on. T3 Code is the control plane only. It does not install agents for you, and it does not manage your subscriptions or billing.

Four design choices worth noticing

1. Remote access is native, not bolted on. The repo's docs have a dedicated guide for reaching the server from your phone or another machine. That is the biggest difference from similar tools that only ship a local UI.

2. Multi-account is a documented feature. Codex and Claude each get their own multi-account documentation. If you juggle a personal and a work account, this is considerably cleaner than opening yet another terminal profile.

3. It connects outward over MCP. The docs cover connecting Claude Code, Codex, ChatGPT and other agents over MCP — the control plane itself can be called by other agents. Permission modes get their own doc too, which tells you the authors treat "who is allowed to make an agent do what" as a first-class concern.

4. The pitch is that there is no pitch. The README literally has a section titled "Wait, what are you selling me?" and the answer is "Nothing." The authors built it because they wanted the best agent development experience, judged the existing options (the Codex desktop app, Conductor, Claude Desktop, Cursor Glass) as not meeting their bar, and want you to have everything you need to fork it if they go astray.

The honest limitations

Three things to know before your expectations drift:

It is very early, by the author's own admission. Do not bet your main production workflow on it yet.
Large feature contributions are (mostly) not being accepted yet — read the contributing notes before you build something big on top.
There is no standalone docs site, only the docs/ folder in the repo, and installing/logging into all seven agents remains your homework.

Practical takeaways

If you regularly use two or more coding agents, this is worth an afternoon. Single-agent users will feel less of the benefit.
Get t3 working locally first, then set up phone remote access. Do not start with the always-on service.
For multi-account setups, read the Codex/Claude multi-account docs before writing your own switching scripts.
Treat it as a control plane, not an agent — model choice, subscriptions, and billing still live with each agent itself.

How many coding agents are you running at once — and how are you keeping their front doors straight?

Facts checked against the project's GitHub repo and README on 2026-11-09 (52,526 stars, 10,019 forks, MIT). Star counts move; the architecture point does not.

A Coding Agent's Minimal Loop: 767 Lines of TypeScript with Zero Dependencies

euk ela — Wed, 12 Aug 2026 23:22:57 +0000

What is the smallest coding agent that still counts as an agent? pi-from-scratch makes a strong claim: 767 lines of TypeScript with zero npm runtime dependencies — a runnable agent that reads files, edits code, and executes shell commands. It is a teaching project that deconstructs the pi agent's data flow into a minimal implementation you can read in one evening.

I read all five source files (agent, llm, tools, cli, tui). Here is what actually matters in them.

The "600-line" claim checks out. The five files total 767 lines; removing comments and blank lines leaves 616. And package.json's dependencies object is empty — it runs on Node built-ins only (fetch, fs, child_process, readline). The minimalism is real, not a headline.

The loop is one while(true) (src/agent.ts:80). Each round: optionally compact the context, stream the LLM while collecting text and tool_calls, append the assistant message, execute tools serially, append tool_result messages, repeat — until the model stops calling tools. No max_steps; the model decides when the task is done. When messages exceed 50, a summarizer compacts the older ones and keeps the last 20.

Three production traps are handled explicitly — and each is a failure mode you only hit when you have actually written an agent:

max_tokens truncation (agent.ts:125). When output is cut with finish_reason: length and tool calls are pending, the arguments may be partial JSON. The code does not execute them. It writes an error tool_result back into the context so the model can retry. Skip this branch and truncated tool calls execute with corrupt arguments.
Abort consistency (agent.ts:107, 165). A dropped tool_call still needs its matching tool_result — the OpenAI protocol requires a 1:1 correspondence, and a missing result breaks session resume. Every aborted call gets "error: aborted".
Compaction failure (agent.ts:62). If summarization fails, the original context is kept untouched. An empty summary is worse than an overlong context.

The protocol layer (src/llm.ts) is where the API's quirks live: tool_call deltas accumulate by index with partial JSON arguments, flushed in order at stream end; finish_reason maps tool_calls → tool_use and length → max_tokens; assistant messages need non-null content or tool_calls (an empty string placeholder avoids HTTP 400); and tool_result blocks become separate role: "tool" messages. The context is plain JSON, which is why session persistence is only ~20 lines — append-only JSONL at ~/.nanopi/session.jsonl that tolerates corrupted lines on load (cli.ts:81).

The tools (src/tools.ts) are the minimal set: read_file, write_file, edit, run_bash. Output is truncated to the last 200 lines with the full output dumped to a temp file — error messages live at the end. edit requires a unique match and uses a function replacer so $ characters are not interpreted. run_bash has a 30-second timeout and 1MB buffer cap.

What 600 lines costs. The same source admits it: tool arguments are never validated, and run_bash executes any command with no approval gate (tools.ts:3-4, agent.ts:146). This is a teaching agent, not a safety baseline. The permission layer — approvals, sandboxing, argument validation — is exactly what separates this from your production harness, and that layer is absent here by design. Reading it against your own harness's permission model is the most productive exercise.

Who it is for / not for. It is for developers who want to understand the agent loop, tool-calling protocol, and context compaction, or who want a basis for writing a lightweight agent. It is not for anyone who needs a production-ready agent (no permission gate, no validation, no parallel tool execution), anyone who will not send an API key to a configurable base URL, or non-TypeScript stacks. The companion site (pi-from-scratch.vercel.app) ships pre-generated traces, so browsing it never calls a model API — a nice pattern for teaching costs.

Not tested / not run. I read the source; I did not install, build, or run the project, and I did not execute its test suite. The repository ships vitest tests for all five modules plus an e2e test. Star count (746) is from the GitHub API on 2026-08-13.

Sources: https://github.com/SaladDay/pi-from-scratch · https://github.com/SaladDay/pi-from-scratch/blob/main/src/agent.ts · https://github.com/SaladDay/pi-from-scratch/blob/main/src/llm.ts · https://github.com/SaladDay/pi-from-scratch/blob/main/src/tools.ts

The 116 Token Cut: A Source-Level Look at Multi-Agent CAD (MAC)

euk ela — Tue, 11 Aug 2026 23:08:09 +0000

TL;DR: for text-to-CAD, the dominant token cost is not generation — it's context replay, and a 4-agent pipeline with structured state passing removes that term almost entirely. Pan-Chera/Multi-Agent-CAD (MIT, Tsinghua IEI Lab) runs the same 10 prompts and 141 geometry features as the single-agent cad skill baseline (earthtojake/text-to-cad) on the same LLM (Qwen 3.7-max): tokens 103,950,189 → 896,340 (116×), API calls 1,307 → 50 (26×), cost ¥125.69 → ¥9.67 (13×), feature pass rate 138/141 → 140/141 (97.9% → 99.3%). I have not tested or run it; every figure is from the authors' benchmark doc, and the arithmetic reproduces row by row.

The numbers check out. Their pricing: input ¥6/M, cache_read ¥0.6/M, output ¥18/M; cost = (input×6 + cache_read×0.6 + output×18)/1e6 CNY. Spot-check P1 baseline: 212,689×6 + 5,413,760×0.6 + 55,878×18 = ¥5.53 ✓; MAC P6: 157,815×6 + 119,799×18 = ¥3.10 ✓. I re-ran all 20 rows from docs/quantified_quality.md — every row, the totals, and the 116.0×/13.0×/26.1× ratios reproduce exactly.

The mechanism behind 116×. 96,192,896 of the baseline's 103,950,189 tokens — 92.5% — were cache_read: a single agent re-reads its own accumulated conversation on every repair round, so tokens grow superlinearly with iterations. MAC passes only compact structured JSON between stages (CADBrief → ArchitectPlan → code → QA report); cache_read falls 96.2M → 10,496, input 5.97M → 524k, output 1.79M → 362k. Recomposing the old bill: cache_read ¥57.72 (46%) is the single most expensive line. Hallucination propagation is also cut at stage boundaries — each agent starts from structured output, not the previous agent's narrative.

Deterministic translator and dual-engine QA. _plan_to_code (multi_agent_cad/nodes.py:893) translates ArchitectPlan JSON into build123d code at zero token cost for common operations; unsupported step types emit # TODO_AIDER placeholders filled by an Aider pass (nodes.py:460, 524–542). Verification is machine-checked: Engine A runs cadpy.analysis geometry selectors on the .step, Engine B checks the .stl triangulation; runtime guards record typed markers (MISSED_CUT, FILLET_FAILED, …); schemas.py routes errors by type — DIMENSION back to the Coder, TOPOLOGY back to the Architect, FATAL halts.

Read the caveats. 10 prompts, one run each, one LLM; the baseline was rerun on the weaker Qwen 3.7-max (the original cad-skill tests used Claude/ChatGPT and scored higher); "~10× faster" wall-clock is explicitly an order-of-magnitude estimate, not measured; you need your own model API key, and the Web UI executes generated .py server-side (trusted network only). Observable for evaluators: the baseline's 3 failures and MAC's single failure are all fillet operations.

Who should look. Engineers building multi-stage agent pipelines where context cost compounds with repair iterations — structured handoff, deterministic translator, and typed error routing transfer beyond CAD. Skip it if you expect turnkey output without running the pipeline or paying API costs.

Project: https://github.com/Pan-Chera/Multi-Agent-CAD · README: https://github.com/Pan-Chera/Multi-Agent-CAD/blob/main/README.md · Benchmark doc: https://github.com/Pan-Chera/Multi-Agent-CAD/blob/main/docs/quantified_quality.md · Baseline: https://github.com/earthtojake/text-to-cad

A 45M-Parameter Agentic Model in a 14MB Binary: Inside Cactus Needle 2

euk ela — Mon, 10 Aug 2026 23:19:45 +0000

TL;DR: tool-calling is now a workload a 45M-parameter model can carry. Needle 2 (Apache-2.0) ships as a single 14MB binary that runs a full session in about 28MB of RAM, targeting phones, Raspberry Pi, and MCUs. I have not tested or run it; this is a review of the official README, model card, product page, and architecture paper.

The numbers check out arithmetically. 45M params at CQ2 (~2 bits/weight, quantized during pretraining rather than post-hoc) is 45e6 × 2 / 8 = 11.25 MB — the bulk of the 14MB binary, consistent with "weights never decompress into RAM." A 256-token sliding KV window with tools pinned as KV sinks keeps session memory near 28MB regardless of conversation length. Official speed claims: ~500 tok/s decode on Raspberry Pi 5, 400–1,500 tok/s on VR headsets, 300–700 tok/s on sub-$200 phones, ~70 MFLOPs/token vs ~460 for LFM2.5 and ~6,000 for Apple FM.

The design is a bet on context, not weights. The FFN is replaced by a fixed orthonormal Walsh–Hadamard transform (n log n, no weights to read); memory comes from hashed n-gram "engram" tables; routing is Sinkhorn-normalized. The README cites arXiv:2607.18363, a controlled study where attention-only transformers match standard transformers at matched parameter count (0.006 nats gap) but are better at context-grounded answers and worse where knowledge must live in weights — which fits a tool-calling model and misfits a chatbot (17.0% on DroidCall).

Behavior contract. Text in, JSON out: a byte-level grammar compiled from your declared schemas constrains every token, so calls cannot be malformed. Requests no declared tool can serve return the empty call []; there is no free-text fallback. Arguments contain only values evidenced by the input — optional fields are omitted, not guessed. Confidence is the min of a calibrated head and the call's decoding probability; below threshold it escalates instead of executing. Above five tools, a retrieval head embeds schemas once and only the top-5 enter context — unselected tools are unreachable, not merely unlikely.

How to read the benchmarks. Mobile Actions: 63.7% (LFM2.5 69.1%, FunctionGemma 64.0%, Apple FM 57.6%); BFCL v4 single-turn 42.6% overall with 93.4% well-formed; Seal-Tools 32.6% in-domain / 28.7% out-of-domain. Note the asymmetry: Needle runs at 2 bits with a 256-token window while baselines run f16 with full context, and all figures are vendor-reported with no third-party reproduction yet.

Who should look. Edge and embedded engineers building tool callers. Schemas compile into the decode grammar; LoRA fine-tuning on the frozen base merges at export into one .cact file that runs on the same engine — no recompilation. Skip it if you need a general assistant: training data is proprietary (115B + 38B tokens), there is no free-text fallback, and the 256-token window is a hard budget. For context, Pebble already runs it locally in the Index 01 watch app.

Project: https://github.com/cactus-compute/needle · README: https://github.com/cactus-compute/needle/blob/main/README.md · Model card: https://huggingface.co/Cactus-Compute/needle2 · Paper: https://arxiv.org/abs/2607.18363 · Product page: https://cactuscompute.com/needle

Claude Code Can't Draw — This Plugin Borrows Your Codex CLI Login

euk ela — Sun, 09 Aug 2026 23:09:43 +0000

TL;DR: "Claude can't draw" is a budget problem, not a capability problem. codex-bridge (MIT) is a Claude Code plugin that routes image generation (gpt-image-2) and five GPT-5 subagents through the Codex CLI login you already have — billing against your ChatGPT plan instead of a second API key. I have not tested or run it; this is a source-reading review.

How it works. Two bash wrappers shell out to codex exec, Codex CLI's non-interactive mode, with your existing codex login (both scripts check codex login status first). Text tasks use a read-only sandbox and capture only the final answer via -o <tmpfile> (bin/codex-run:92-96). Image tasks use a workspace-write sandbox scoped to the output directory (bin/codex-imagegen:110), then verify the file landed. Exit codes: 0 ok, 1 precondition, 2 failure, 124 timeout.

Why the split matters. Codex runs as a separate process, so its intermediate output never enters your Claude context — you pay Claude tokens only for the brief and the review. The trade-off: Codex is blind to your conversation history, so every brief must stand alone, which is why every subagent is instructed to verify before reporting.

The delegation rubric (skills/codex-delegate/SKILL.md) is the most reusable part: delegate high-volume, low-judgment work (mechanical multi-file edits, bulk scaffolding, exhaustive sweeps, all image work, long self-contained subtasks); keep low-volume, high-judgment work (architecture calls, ambiguous requirements, anything needing conversation history, edits under ~3 files). Round-tripping a two-line fix costs more, not less.

Trust boundary. No API key: credentials live in ~/.codex from codex login, and every call forwards your task text to OpenAI's servers. If your team's data policy cannot accept a second vendor seeing prompts, that is a blocker, not a footnote.

Caveats from README and source. 1–4 minutes per image; image turns burn ChatGPT quota ~3–5× faster than text; the default path cannot emit native alpha (chroma-key strip, or CLI fallback plus API key); macOS and Linux only; custom sizes must satisfy longest edge ≤3840px, both edges multiples of 16, ratio ≤3:1, and 655,360–8,294,400 total pixels.

Who should try it. You already pay for a ChatGPT plan, run Claude Code daily, and regularly produce images or mechanical bulk edits. Skip it if you rarely need images, your work is small and interactive, or your data rules forbid OpenAI seeing prompts.

Project: https://github.com/Sateezg/codex-bridge · README: https://github.com/Sateezg/codex-bridge/blob/main/README.md · codex-run: https://github.com/Sateezg/codex-bridge/blob/main/bin/codex-run · codex-imagegen: https://github.com/Sateezg/codex-bridge/blob/main/bin/codex-imagegen · delegate rubric: https://github.com/Sateezg/codex-bridge/blob/main/skills/codex-delegate/SKILL.md

File provenance for the AI-agent era: evidence labels, not guesses

euk ela — Sat, 08 Aug 2026 23:09:07 +0000

AI coding agents write files faster than anyone can remember where they came from. Git records versions; DVC and OpenLineage record pipelines you declared up front; nothing records the unplanned output — the CSV an agent wrote mid-task, the PNG it regenerated, the notebook that silently touched a data file.

trace-file-lineage (MIT, ~361★, v0.7.0) is a local CLI that answers "which script, notebook, data, command, or AI agent produced this file?" with five confidence labels — verified, strong-candidate, candidate, weak-signal, insufficient — and it labels guesses as guesses. I have not tested or run this tool; everything below is from reading the README and source files.

Two modes, one honest split. Retrospective: for files you already have, it reconstructs the most likely origins from code, document metadata, and Git history — "ranked guesses with the reasoning attached" (README). Prospective: lineage run --task "..." -- python sweep.py wraps a command, captures the task boundary, and files changed during the run are marked verified — "These answers are proof" (README). It never moves, renames, or deletes files; lineage layout --suggest only proposes a destination.

What verified means in source. In evidence.py, every fact carries a basis (observation / declaration / inference / confirmation), an assurance field (verified when exact, candidate otherwise), the source path and line it came from, and a deterministic id (uuid5(NAMESPACE_URL, canonical_json)), so re-scanning the same fact yields the same id. Evidence about the evidence, with location attached — that design is the part worth copying.

Trust order and privacy. The README ranks sources: your own confirmation → recorded runs → imported provenance → declarations → static code → content → names/timestamps. Extracted text lives in .file-lineage/ (SQLite-backed, git-ignored by default). Privacy claims: nothing is uploaded, no account or API key; scanning never executes your code — only lineage run runs the explicit command after --, and password-looking arguments are stripped from recorded commands.

Deliberate limits. JS/TS get "a cautious static scan, not real language understanding" (README); other languages are searched, not parsed; runtime-built paths cannot be resolved. Performance is claimed and reproducible, not something I ran: tests/benchmark.py on macOS / Python 3.14 reports 1,000 files → 0.5 s cold scan / 0.1 s warm; 10,000 files → 16.5 s / 1.1 s; individual queries in milliseconds.

Bottom line. If you run Python/notebook workflows and a pile of agent-generated files has become unanswerable trivia, this is worth evaluating — lineage enable even writes the tracing rules into CLAUDE.md and AGENTS.md (an instruction, not enforcement, per the README). If you have no agent-generated files, or cannot tolerate a git-ignored text index in your project, it is premature. A comparison with Git/DVC/OpenLineage is in docs/comparison.md.

Don't Read the Code Your Agent Wrote — Make It Run the Gauntlet

euk ela — Fri, 07 Aug 2026 23:35:49 +0000

The claim worth verifying

old-coder (MIT, by AmazingAng) is a markdown skill for coding agents — Claude Code, Codex CLI, Cursor, Aider, or custom agent loops. Its strategy: don't read the code your agent wrote; make it run the gauntlet. Concretely, the human approves a test plan (SPEC) before any code exists and reviews an EVIDENCE report afterward, instead of reading the diff. The README cites Robert C. Martin: "My current strategy is to not read any of the code written by my agents."

The workflow

SPEC → RED → GREEN → REFACTOR → GAUNTLET → EVIDENCE. Each scenario maps 1:1 to at least one automated test. All numbers in the evidence report come from one final fresh run, the entry command is recorded so a human can rerun everything, and source state is pinned via commit SHA or tree hash (the demo pins commit d6e17b1, tree hash 50433e0a4acc8507).

The 9-layer gauntlet (reusable checklist)

From skills/old-coder/references/gauntlet.md, layers run in order and halt at the first failure: tests → types → lint → changed-line coverage → mutation → property-based tests → real execution → supply chain/secrets scan → suite health (randomized order). Per-ecosystem tool tables exist for Python, JS/TS, Go, Rust, Java, Scala, SQL, and Emacs Lisp, with the rule "prefer whatever the project already uses" and pinned tool versions for reproducible reruns. Extended layers are selected by risk: concurrency only when the failure model names races, performance only when the spec states a budget, UI checks only when the change touches user-facing UI, version matrix only when the project claims multi-version support. A typo fix gets a couple of checks; changes touching money, logins, data, or concurrency get everything plus hostile-input self-attacks.

The honesty rules (the actual differentiator)

Never weaken a test to make it pass.
Never report a check that did not run — unverified never equals pass.
Fail-closed gates: set -e at the top, no || true, no 2>/dev/null; a must-find-nothing grep passes only on rc 1.
Prove each home-grown check can fail with a one-off negative control.
Every mutant must make at least one test fail — a survivor means a weak or vacuous assertion.
Record the dependency diff and the reasons, item by item.

These rules make "all green" auditable instead of self-attested. That is the part ordinary CI pipelines do not give you.

What the demo evidence file admits

The rate-limiter demo (demo-rate-limiter/evidence.md) reports 17/17 tests passing (also in randomized order), 29/29 statements and 10/10 branches covered, and 8/8 mutation kills — and then openly states that the property tests alone only killed 3/8, that one kill (M2) was flaky until spec revision 2 pinned the exact-boundary behavior with a deterministic test, and that the spec was never approved by a human (autonomous run). It also documents a real bug caught by the loop: a window_seconds=NaN value slipped past the original <= 0 validation and was fixed via a spec revision, a watched RED test, and a finiteness check (later killed as mutant M7). Known uncovered modes are listed: not thread-safe; a NaN-returning clock fails closed but is not rejected.

Limits

The gauntlet proves the code meets the spec — it cannot prove the spec covers everything that matters. The skill presumes a test base exists; mutation testing is a real time cost, which the skill itself acknowledges by scaling effort to risk. Treat the demo's numbers as the author's report, not independent results — rerun ./tools/gauntlet.sh to verify.

Not tested/not run by me: I did not execute the skill or any of its scripts; everything above was read from the repository's own files (README, gauntlet.md, evidence.md).

Repo (MIT): https://github.com/AmazingAng/old-coder · Gauntlet reference: https://github.com/AmazingAng/old-coder/blob/main/skills/old-coder/references/gauntlet.md · Demo evidence: https://github.com/AmazingAng/old-coder/blob/main/demo-rate-limiter/evidence.md

DeepSeek V4 Flash Went Official: Checking the 'Flash Beats Pro' Claim Against the Model Card and config.json

euk ela — Thu, 06 Aug 2026 23:27:05 +0000

The Claim

DeepSeek-V4-Flash-0731 is now the official V4 Flash release, superseding the preview. The model card states that on all nine agentic benchmarks it lists, Flash-0731 beats V4-Pro (Preview) — "despite its far smaller activated parameter count." That is a strong claim, so I checked it two ways: the table arithmetic, and the architecture in config.json.

9/9, Verified Row by Row

Benchmark	Flash-0731	V4-Pro (Preview)	GLM-5.2	Opus-4.8
Terminal Bench 2.1	82.7	72.1	81.0	85.0
NL2Repo	54.2	38.5	48.9	69.7
Cybergym	76.7	52.7	–	83.1
DeepSWE	54.4	12.8	46.2	58.0
Toolathlon-Verified	70.3	55.9	59.9	76.2
Agents' Last Exam	25.2	16.5	23.8	25.7
AutomationBench Public	25.1	12.8	12.9	27.2
DSBench-FullStack †	68.7	41.8	61.8	71.6
DSBench-Hard †	59.6	31.1	54.5	71.7

Flash-0731 wins 9/9 against V4-Pro (Preview), wins all 8 rows where GLM-5.2 has a score, and loses all 9 to Opus-4.8 — the closest margins being Agents' Last Exam (25.2 vs 25.7) and AutomationBench (25.1 vs 27.2). "Broadly competitive" is precise: close, not ahead. Caveats from the card itself: all numbers are DeepSeek's own evaluations (DeepSeek Harness minimal mode, max reasoning effort), and the two DSBench sets are marked † as internal DeepSeek test sets. None of this is independently reproduced.

What config.json Says

From the public config: model_type deepseek_v4, 43 layers, hidden_size 4096, 64 attention heads with a single KV head, vocab 129,280, and max_position_embeddings 1,048,576 — 2^20, so the "million-token context" in the paper title is literal. The MoE is 256 routed experts plus one shared expert per layer, with 6 routed experts active per token and an expert intermediate size of just 2048. Dense weights are FP8 (E4M3, dynamic activation); expert_dtype is fp4 — experts ship at 4 bits by design.

Reconciling 284B and 304B

One expert = up (4096×2048) + gate (4096×2048) + down (2048×4096) = 25,165,824 parameters. With 43 × 256 = 11,008 routed experts, that is ≈ 277.0B in routed experts alone. Adding shared experts, dense attention, and the (unshared) embeddings and head lands at ≈ 284B — matching unsloth's 284B figure. The card's "304B in safetensors" includes the in-checkpoint DSpark speculative-decoding module (11.3 GB at BF16) and quantization scale tensors. Different accounting, not a contradiction.

Activated per token: 7 experts per layer (6 routed + 1 shared) × 43 layers ≈ 7.6B, plus dense attention and embeddings ≈ 10B — roughly 3.5% of the model. The "far smaller activated parameter count" statement is consistent with the architecture.

The Engineering News: Speculation Is In the Checkpoint

DSpark ships inside the main checkpoint: 7 speculative tokens, greedy draft sampling, no separate draft model to deploy. vLLM: --speculative-config '{"method":"dspark","num_speculative_tokens":7}'; SGLang: --speculative-algorithm DSPARK; llama.cpp users can add it as a module (Q8_0, 10.9 GB). The release also adds three-level reasoning_effort (low/high/max, up to 384K output tokens at high/max) and replaces the Jinja chat template with an encoding/ folder of Python helpers — teams using transformers directly will want to migrate.

Local Deployment, Sized Honestly

unsloth's GGUF: UD-Q4_K_XL 155 GB, UD-Q8_K_XL 162 GB — the "lossless" Q8 is only 7 GB bigger than Q4 per unsloth's note. Ollama: ollama run hf.co/unsloth/DeepSeek-V4-Flash-0731-GGUF:UD-Q4_K_XL. llama.cpp: llama serve -hf unsloth/DeepSeek-V4-Flash-0731-GGUF:UD-Q4_K_XL. A 155 GB Q4 file means big-RAM workstations or multi-GPU servers; for most product teams, the hosted flash tier will be the economical path.

Takeaways

The flash tier's economics are now backed by the vendor's own benchmarks — but read them as the vendor's.
FP8 dense + FP4 experts + in-checkpoint speculation is a deployment recipe worth copying for latency- and cost-sensitive agent workloads.
A 1M-token context with a 384K output budget changes what fits in a single agent call.

Model card: https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731 (MIT) · config.json: https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731/resolve/main/config.json · GGUF: https://huggingface.co/unsloth/DeepSeek-V4-Flash-0731-GGUF · Paper: https://arxiv.org/abs/2606.19348

Not tested / not run — no inference or benchmark reproduction was performed. Benchmark numbers are the vendor's; the arithmetic and config reading are mine.

How a 176 KB C Binary Runs a 2.78-Trillion-Parameter Model on One CPU with 8 GB of RAM

euk ela — Thu, 06 Aug 2026 01:21:12 +0000

The Problem

Moonshot AI's Kimi K3 has 2.78 trillion parameters. Stored naively at bfloat16, that's 5,560 GB — more than the combined memory of two fully-loaded DGX H100 nodes. Deploying it typically requires dozens of H100 GPUs.

Fareed Khan asked a different question: can you run the exact same model checkpoint, with no quantization, distillation, or weight dropping, on a single CPU with 8 GB of RAM?

The answer is kimi-k3-in-c: a 176 KB pure C99 binary, seven source files, zero GPU dependencies. It runs the unmodified 1.56 TB checkpoint and produces output that is byte-for-byte identical to the PyTorch reference. At roughly 33 seconds per token, it's impractical as a chatbot — but that's not why it matters.

The Four Reductions

The engine exploits a structural property of Mixture-of-Experts models: Kimi K3 has 93 layers, 92 of which route to the top 16 of 896 experts. Only ~3.7% of parameters (~104 billion) are active for any single token. The other 96.3% must exist somewhere reachable but don't need to be in RAM.

Reduction 1 — Experts ship small. Kimi K3's 82,432 routed experts occupy 1.447 TB at roughly 0.53 bytes per weight — packed 4-bit nibbles with a shared E8M0 scale. The engine multiplies directly out of this packed form without dequantizing to float first. Baseline: 5,560 GB → 1,560 GB.

Reduction 2 — Routing sparsity removes experts. Expert weights are never memory-resident — loaded on demand from NVMe with an LRU cache. What remains is the 113.49 GB dense trunk. 1,560 GB → 113.49 GB.

Reduction 3 — Trunk streaming. The 93 dense layers are repacked into a single 109 GB trunk.bin where each layer lives at a known offset. The engine pins as many layers as the memory budget allows and streams the rest via O_DIRECT, bypassing the OS page cache. 113.49 GB → configurable, as low as 8.24 GB peak RSS.

Reduction 4 — Expert LRU cache. Routed experts are loaded on demand with a configurable cache size. The author provides a trace-based capacity simulator for tuning.

Total: a 676× reduction from the bf16 baseline, with the output at the bottom of this ladder being byte-for-byte identical to the output at the top.

Validation

The make test target requires no model download. It builds a 13-layer model with the same tensor graph, validates against a committed PyTorch reference across three paths — teacher forcing (32/32 positions), greedy decode (20/20 tokens), incremental decode (20/20 tokens) — and ends with "ENGINE MATCHES THE REFERENCE EXACTLY."

The build disables FMA contraction (-ffp-contract=off) so that scalar, OpenMP, and AVX2 paths produce bit-identical results. Every memory budget from 8 GB to 224 GB emits the same token stream. Memory is a performance dial, not a correctness variable.

Performance and Bottlenecks

Measured on dual AMD EPYC 7763 (124 cores, 228 GB RAM, NVMe):

Preset	Peak RSS	Speed
laptop	8.24 GB	32.69 s/token
desktop	31.9 GB	28–31 s/token
server	127.92 GB	10.69 s/token

The bottleneck is unambiguous: sustained trunk reads at 5,373–6,064 MB/s, with I/O accounting for 41–61% of wall-clock time. On spinning rust, performance would degrade several-fold.

Limitations (Be Honest)

v0.1.0, 28 commits, days old
Linux x86-64 only (O_DIRECT, posix_memalign, getrusage)
Requires ~1.7 TB free NVMe storage
No chat template (raw continuations), no sampling, no batching, no GPU path
~33 s/token at the minimum preset — generating 200 tokens takes ~2 hours
The electricity cost of a multi-hour run can exceed hosted API pricing

Why Study This

kimi-k3-in-c is not a practical inference server. It is, explicitly, a teaching artifact — the author built it to understand Kimi K3's architecture after deploying it on 32 H100 GPUs at work and being unable to debug on personal hardware.

For engineers working on model inference, compression, or edge deployment, it offers three transferable findings:

Storage bandwidth, not RAM or FLOPs, is the real bottleneck for frontier MoE inference — a measured finding with direct implications for hardware selection.
Memory is a dial, not a floor — the same model runs correctly at 8 GB and 224 GB, only wall-clock time changes. This reframing matters for edge deployment of sparse models.
A complete, auditable reference implementation — the README is structured as a five-part technical paper, building every component (RMSNorm, KDA attention, MLA, MXFP4 matmul, expert cache) from first principles in runnable C. For understanding MoE internals at the byte level, this is more valuable than most papers.

The broader point: the wall for running frontier models locally isn't compute — it's capacity. And most of the model is asleep for any given token. That structural fact makes the impossible tractable.

Repo: https://github.com/FareedKhan-dev/kimi-k3-in-c (Apache-2.0, v0.1.0)

Not tested — this analysis is based on reading the public README, source tree, CHANGELOG, and independent technical reviews (andrew.ooo, essamamdani.com, securityonline.info). No local build or inference run was performed.

AI Technical Figures Need an Editability Test, Not Just a Similarity Score

euk ela — Tue, 04 Aug 2026 23:29:41 +0000

An AI-generated technical figure can look convincing and still be a poor deliverable.

The failure usually appears after the first review: change a label, reroute an arrow, update a table, translate the annotations, or move a panel. If all that remains is a bitmap, the output is visually useful but structurally disposable.

Scientific Illustrator is an MIT-licensed open-source Codex plugin project built around a more useful target: recreating a reference figure with editable objects in PowerPoint, WPS Presentation, or draw.io. Its README says it prefers editable text, shapes, connectors, tables, and charts, keeping image inserts for the smallest areas it cannot reliably recreate, such as complex textures or microscopy-like imagery. That is a project claim, not a benchmark.

The repository structure fits that goal. It includes a plugin manifest, MCP configuration, and skills for designing, recreating, auditing, and correcting figures, plus bridge scripts for presentation software and draw.io. The plugin manifest describes a Designer–Drawer–Reviewer–Corrector workflow. The v1.5.3 release reports backend/target locking, serialized OOXML changes, and fixes around tables, charts, arrows, connectors, and exports.

The engineering idea matters even if this particular implementation is not the right tool for every team. For a structured diagram, the useful acceptance test is not only visual similarity. Ask whether text, connectors, groups, tables, and charts can survive the next human edit in the target application.

This is most relevant to researchers, technical writers, and developers who hand off editable PPTX or draw.io files. It is a weaker fit for photorealistic images, dense textures, pixel-perfect reproduction, or one-off artwork that will never be revised. The project itself acknowledges a boundary by retaining difficult regions as images.

I have not tested or run this project. This article is based on a read-only review of its public repository, README, manifest, license, and release notes. Treat compatibility, reconstruction fidelity, and workflow reliability as items to validate on your own templates and software versions. For source details, see the README, MIT license, and CI workflow.

Skill Recorder: Turning One Human Workflow into a Reviewable Agent Procedure

euk ela — Sun, 02 Aug 2026 23:31:45 +0000

A recording is not the end product

The useful idea in Microsoft's Skill Recorder is not that it records a desktop session. Its stated pipeline is: capture a real work session, ask GitHub Copilot CLI to reconstruct an intent and ordered steps, let a person review them, and then create a reusable Skill or Automation.

That distinction matters for engineering. A UI macro replays a fragile sequence of coordinates. A reviewed procedure can describe the work at a higher level and, according to the project, prefer native agent tools such as CLIs or web fetching over replaying clicks. For a recurring workflow whose interface changes more often than its business rule, that is a more promising maintenance boundary.

The privacy boundary is part of the design

The README says that recording, storage, frame extraction, and optional narration transcription happen locally. It also says that choosing Analyze sends an event timeline, extracted screen images, and narration text to GitHub's cloud for Copilot processing. That means screen content, titles, URLs, and clipboard previews are not incidental details: they are part of the threat model.

Before considering a tool like this, I would start with a de-identified, low-privilege workflow; clear the clipboard and notifications; and review every generated step. A captured exception should not silently become a future automated action.

What the repository shows

The repository is MIT-licensed. Its current package configuration identifies version 0.3.1 and an Electron/React application using the Copilot SDK and local-transcription-related dependencies. The README documents Windows validation and source-release installation details.

Where it fits

This looks relevant to teams turning recurring, cross-application work into a procedure that can be inspected and improved. It is a poor fit for workflows that cannot be recorded, cannot allow cloud analysis, or cannot tolerate human review before automation.

Not tested or run: this article is based on the public repository and source configuration only. It does not claim functional, security, or quality validation.

Further reading: the project README and release page.