DEV Community: euk ela

A Coding Agent's Minimal Loop: 767 Lines of TypeScript with Zero Dependencies

euk ela — Wed, 12 Aug 2026 23:22:57 +0000

What is the smallest coding agent that still counts as an agent? pi-from-scratch makes a strong claim: 767 lines of TypeScript with zero npm runtime dependencies — a runnable agent that reads files, edits code, and executes shell commands. It is a teaching project that deconstructs the pi agent's data flow into a minimal implementation you can read in one evening.

I read all five source files (agent, llm, tools, cli, tui). Here is what actually matters in them.

The "600-line" claim checks out. The five files total 767 lines; removing comments and blank lines leaves 616. And package.json's dependencies object is empty — it runs on Node built-ins only (fetch, fs, child_process, readline). The minimalism is real, not a headline.

The loop is one while(true) (src/agent.ts:80). Each round: optionally compact the context, stream the LLM while collecting text and tool_calls, append the assistant message, execute tools serially, append tool_result messages, repeat — until the model stops calling tools. No max_steps; the model decides when the task is done. When messages exceed 50, a summarizer compacts the older ones and keeps the last 20.

Three production traps are handled explicitly — and each is a failure mode you only hit when you have actually written an agent:

max_tokens truncation (agent.ts:125). When output is cut with finish_reason: length and tool calls are pending, the arguments may be partial JSON. The code does not execute them. It writes an error tool_result back into the context so the model can retry. Skip this branch and truncated tool calls execute with corrupt arguments.
Abort consistency (agent.ts:107, 165). A dropped tool_call still needs its matching tool_result — the OpenAI protocol requires a 1:1 correspondence, and a missing result breaks session resume. Every aborted call gets "error: aborted".
Compaction failure (agent.ts:62). If summarization fails, the original context is kept untouched. An empty summary is worse than an overlong context.

The protocol layer (src/llm.ts) is where the API's quirks live: tool_call deltas accumulate by index with partial JSON arguments, flushed in order at stream end; finish_reason maps tool_calls → tool_use and length → max_tokens; assistant messages need non-null content or tool_calls (an empty string placeholder avoids HTTP 400); and tool_result blocks become separate role: "tool" messages. The context is plain JSON, which is why session persistence is only ~20 lines — append-only JSONL at ~/.nanopi/session.jsonl that tolerates corrupted lines on load (cli.ts:81).

The tools (src/tools.ts) are the minimal set: read_file, write_file, edit, run_bash. Output is truncated to the last 200 lines with the full output dumped to a temp file — error messages live at the end. edit requires a unique match and uses a function replacer so $ characters are not interpreted. run_bash has a 30-second timeout and 1MB buffer cap.

What 600 lines costs. The same source admits it: tool arguments are never validated, and run_bash executes any command with no approval gate (tools.ts:3-4, agent.ts:146). This is a teaching agent, not a safety baseline. The permission layer — approvals, sandboxing, argument validation — is exactly what separates this from your production harness, and that layer is absent here by design. Reading it against your own harness's permission model is the most productive exercise.

Who it is for / not for. It is for developers who want to understand the agent loop, tool-calling protocol, and context compaction, or who want a basis for writing a lightweight agent. It is not for anyone who needs a production-ready agent (no permission gate, no validation, no parallel tool execution), anyone who will not send an API key to a configurable base URL, or non-TypeScript stacks. The companion site (pi-from-scratch.vercel.app) ships pre-generated traces, so browsing it never calls a model API — a nice pattern for teaching costs.

Not tested / not run. I read the source; I did not install, build, or run the project, and I did not execute its test suite. The repository ships vitest tests for all five modules plus an e2e test. Star count (746) is from the GitHub API on 2026-08-13.

Sources: https://github.com/SaladDay/pi-from-scratch · https://github.com/SaladDay/pi-from-scratch/blob/main/src/agent.ts · https://github.com/SaladDay/pi-from-scratch/blob/main/src/llm.ts · https://github.com/SaladDay/pi-from-scratch/blob/main/src/tools.ts

The 116 Token Cut: A Source-Level Look at Multi-Agent CAD (MAC)

euk ela — Tue, 11 Aug 2026 23:08:09 +0000

TL;DR: for text-to-CAD, the dominant token cost is not generation — it's context replay, and a 4-agent pipeline with structured state passing removes that term almost entirely. Pan-Chera/Multi-Agent-CAD (MIT, Tsinghua IEI Lab) runs the same 10 prompts and 141 geometry features as the single-agent cad skill baseline (earthtojake/text-to-cad) on the same LLM (Qwen 3.7-max): tokens 103,950,189 → 896,340 (116×), API calls 1,307 → 50 (26×), cost ¥125.69 → ¥9.67 (13×), feature pass rate 138/141 → 140/141 (97.9% → 99.3%). I have not tested or run it; every figure is from the authors' benchmark doc, and the arithmetic reproduces row by row.

The numbers check out. Their pricing: input ¥6/M, cache_read ¥0.6/M, output ¥18/M; cost = (input×6 + cache_read×0.6 + output×18)/1e6 CNY. Spot-check P1 baseline: 212,689×6 + 5,413,760×0.6 + 55,878×18 = ¥5.53 ✓; MAC P6: 157,815×6 + 119,799×18 = ¥3.10 ✓. I re-ran all 20 rows from docs/quantified_quality.md — every row, the totals, and the 116.0×/13.0×/26.1× ratios reproduce exactly.

The mechanism behind 116×. 96,192,896 of the baseline's 103,950,189 tokens — 92.5% — were cache_read: a single agent re-reads its own accumulated conversation on every repair round, so tokens grow superlinearly with iterations. MAC passes only compact structured JSON between stages (CADBrief → ArchitectPlan → code → QA report); cache_read falls 96.2M → 10,496, input 5.97M → 524k, output 1.79M → 362k. Recomposing the old bill: cache_read ¥57.72 (46%) is the single most expensive line. Hallucination propagation is also cut at stage boundaries — each agent starts from structured output, not the previous agent's narrative.

Deterministic translator and dual-engine QA. _plan_to_code (multi_agent_cad/nodes.py:893) translates ArchitectPlan JSON into build123d code at zero token cost for common operations; unsupported step types emit # TODO_AIDER placeholders filled by an Aider pass (nodes.py:460, 524–542). Verification is machine-checked: Engine A runs cadpy.analysis geometry selectors on the .step, Engine B checks the .stl triangulation; runtime guards record typed markers (MISSED_CUT, FILLET_FAILED, …); schemas.py routes errors by type — DIMENSION back to the Coder, TOPOLOGY back to the Architect, FATAL halts.

Read the caveats. 10 prompts, one run each, one LLM; the baseline was rerun on the weaker Qwen 3.7-max (the original cad-skill tests used Claude/ChatGPT and scored higher); "~10× faster" wall-clock is explicitly an order-of-magnitude estimate, not measured; you need your own model API key, and the Web UI executes generated .py server-side (trusted network only). Observable for evaluators: the baseline's 3 failures and MAC's single failure are all fillet operations.

Who should look. Engineers building multi-stage agent pipelines where context cost compounds with repair iterations — structured handoff, deterministic translator, and typed error routing transfer beyond CAD. Skip it if you expect turnkey output without running the pipeline or paying API costs.

Project: https://github.com/Pan-Chera/Multi-Agent-CAD · README: https://github.com/Pan-Chera/Multi-Agent-CAD/blob/main/README.md · Benchmark doc: https://github.com/Pan-Chera/Multi-Agent-CAD/blob/main/docs/quantified_quality.md · Baseline: https://github.com/earthtojake/text-to-cad

A 45M-Parameter Agentic Model in a 14MB Binary: Inside Cactus Needle 2

euk ela — Mon, 10 Aug 2026 23:19:45 +0000

TL;DR: tool-calling is now a workload a 45M-parameter model can carry. Needle 2 (Apache-2.0) ships as a single 14MB binary that runs a full session in about 28MB of RAM, targeting phones, Raspberry Pi, and MCUs. I have not tested or run it; this is a review of the official README, model card, product page, and architecture paper.

The numbers check out arithmetically. 45M params at CQ2 (~2 bits/weight, quantized during pretraining rather than post-hoc) is 45e6 × 2 / 8 = 11.25 MB — the bulk of the 14MB binary, consistent with "weights never decompress into RAM." A 256-token sliding KV window with tools pinned as KV sinks keeps session memory near 28MB regardless of conversation length. Official speed claims: ~500 tok/s decode on Raspberry Pi 5, 400–1,500 tok/s on VR headsets, 300–700 tok/s on sub-$200 phones, ~70 MFLOPs/token vs ~460 for LFM2.5 and ~6,000 for Apple FM.

The design is a bet on context, not weights. The FFN is replaced by a fixed orthonormal Walsh–Hadamard transform (n log n, no weights to read); memory comes from hashed n-gram "engram" tables; routing is Sinkhorn-normalized. The README cites arXiv:2607.18363, a controlled study where attention-only transformers match standard transformers at matched parameter count (0.006 nats gap) but are better at context-grounded answers and worse where knowledge must live in weights — which fits a tool-calling model and misfits a chatbot (17.0% on DroidCall).

Behavior contract. Text in, JSON out: a byte-level grammar compiled from your declared schemas constrains every token, so calls cannot be malformed. Requests no declared tool can serve return the empty call []; there is no free-text fallback. Arguments contain only values evidenced by the input — optional fields are omitted, not guessed. Confidence is the min of a calibrated head and the call's decoding probability; below threshold it escalates instead of executing. Above five tools, a retrieval head embeds schemas once and only the top-5 enter context — unselected tools are unreachable, not merely unlikely.

How to read the benchmarks. Mobile Actions: 63.7% (LFM2.5 69.1%, FunctionGemma 64.0%, Apple FM 57.6%); BFCL v4 single-turn 42.6% overall with 93.4% well-formed; Seal-Tools 32.6% in-domain / 28.7% out-of-domain. Note the asymmetry: Needle runs at 2 bits with a 256-token window while baselines run f16 with full context, and all figures are vendor-reported with no third-party reproduction yet.

Who should look. Edge and embedded engineers building tool callers. Schemas compile into the decode grammar; LoRA fine-tuning on the frozen base merges at export into one .cact file that runs on the same engine — no recompilation. Skip it if you need a general assistant: training data is proprietary (115B + 38B tokens), there is no free-text fallback, and the 256-token window is a hard budget. For context, Pebble already runs it locally in the Index 01 watch app.

Project: https://github.com/cactus-compute/needle · README: https://github.com/cactus-compute/needle/blob/main/README.md · Model card: https://huggingface.co/Cactus-Compute/needle2 · Paper: https://arxiv.org/abs/2607.18363 · Product page: https://cactuscompute.com/needle

Claude Code Can't Draw — This Plugin Borrows Your Codex CLI Login

euk ela — Sun, 09 Aug 2026 23:09:43 +0000

TL;DR: "Claude can't draw" is a budget problem, not a capability problem. codex-bridge (MIT) is a Claude Code plugin that routes image generation (gpt-image-2) and five GPT-5 subagents through the Codex CLI login you already have — billing against your ChatGPT plan instead of a second API key. I have not tested or run it; this is a source-reading review.

How it works. Two bash wrappers shell out to codex exec, Codex CLI's non-interactive mode, with your existing codex login (both scripts check codex login status first). Text tasks use a read-only sandbox and capture only the final answer via -o <tmpfile> (bin/codex-run:92-96). Image tasks use a workspace-write sandbox scoped to the output directory (bin/codex-imagegen:110), then verify the file landed. Exit codes: 0 ok, 1 precondition, 2 failure, 124 timeout.

Why the split matters. Codex runs as a separate process, so its intermediate output never enters your Claude context — you pay Claude tokens only for the brief and the review. The trade-off: Codex is blind to your conversation history, so every brief must stand alone, which is why every subagent is instructed to verify before reporting.

The delegation rubric (skills/codex-delegate/SKILL.md) is the most reusable part: delegate high-volume, low-judgment work (mechanical multi-file edits, bulk scaffolding, exhaustive sweeps, all image work, long self-contained subtasks); keep low-volume, high-judgment work (architecture calls, ambiguous requirements, anything needing conversation history, edits under ~3 files). Round-tripping a two-line fix costs more, not less.

Trust boundary. No API key: credentials live in ~/.codex from codex login, and every call forwards your task text to OpenAI's servers. If your team's data policy cannot accept a second vendor seeing prompts, that is a blocker, not a footnote.

Caveats from README and source. 1–4 minutes per image; image turns burn ChatGPT quota ~3–5× faster than text; the default path cannot emit native alpha (chroma-key strip, or CLI fallback plus API key); macOS and Linux only; custom sizes must satisfy longest edge ≤3840px, both edges multiples of 16, ratio ≤3:1, and 655,360–8,294,400 total pixels.

Who should try it. You already pay for a ChatGPT plan, run Claude Code daily, and regularly produce images or mechanical bulk edits. Skip it if you rarely need images, your work is small and interactive, or your data rules forbid OpenAI seeing prompts.

Project: https://github.com/Sateezg/codex-bridge · README: https://github.com/Sateezg/codex-bridge/blob/main/README.md · codex-run: https://github.com/Sateezg/codex-bridge/blob/main/bin/codex-run · codex-imagegen: https://github.com/Sateezg/codex-bridge/blob/main/bin/codex-imagegen · delegate rubric: https://github.com/Sateezg/codex-bridge/blob/main/skills/codex-delegate/SKILL.md

File provenance for the AI-agent era: evidence labels, not guesses

euk ela — Sat, 08 Aug 2026 23:09:07 +0000

AI coding agents write files faster than anyone can remember where they came from. Git records versions; DVC and OpenLineage record pipelines you declared up front; nothing records the unplanned output — the CSV an agent wrote mid-task, the PNG it regenerated, the notebook that silently touched a data file.

trace-file-lineage (MIT, ~361★, v0.7.0) is a local CLI that answers "which script, notebook, data, command, or AI agent produced this file?" with five confidence labels — verified, strong-candidate, candidate, weak-signal, insufficient — and it labels guesses as guesses. I have not tested or run this tool; everything below is from reading the README and source files.

Two modes, one honest split. Retrospective: for files you already have, it reconstructs the most likely origins from code, document metadata, and Git history — "ranked guesses with the reasoning attached" (README). Prospective: lineage run --task "..." -- python sweep.py wraps a command, captures the task boundary, and files changed during the run are marked verified — "These answers are proof" (README). It never moves, renames, or deletes files; lineage layout --suggest only proposes a destination.

What verified means in source. In evidence.py, every fact carries a basis (observation / declaration / inference / confirmation), an assurance field (verified when exact, candidate otherwise), the source path and line it came from, and a deterministic id (uuid5(NAMESPACE_URL, canonical_json)), so re-scanning the same fact yields the same id. Evidence about the evidence, with location attached — that design is the part worth copying.

Trust order and privacy. The README ranks sources: your own confirmation → recorded runs → imported provenance → declarations → static code → content → names/timestamps. Extracted text lives in .file-lineage/ (SQLite-backed, git-ignored by default). Privacy claims: nothing is uploaded, no account or API key; scanning never executes your code — only lineage run runs the explicit command after --, and password-looking arguments are stripped from recorded commands.

Deliberate limits. JS/TS get "a cautious static scan, not real language understanding" (README); other languages are searched, not parsed; runtime-built paths cannot be resolved. Performance is claimed and reproducible, not something I ran: tests/benchmark.py on macOS / Python 3.14 reports 1,000 files → 0.5 s cold scan / 0.1 s warm; 10,000 files → 16.5 s / 1.1 s; individual queries in milliseconds.

Bottom line. If you run Python/notebook workflows and a pile of agent-generated files has become unanswerable trivia, this is worth evaluating — lineage enable even writes the tracing rules into CLAUDE.md and AGENTS.md (an instruction, not enforcement, per the README). If you have no agent-generated files, or cannot tolerate a git-ignored text index in your project, it is premature. A comparison with Git/DVC/OpenLineage is in docs/comparison.md.

Don't Read the Code Your Agent Wrote — Make It Run the Gauntlet

euk ela — Fri, 07 Aug 2026 23:35:49 +0000

The claim worth verifying

old-coder (MIT, by AmazingAng) is a markdown skill for coding agents — Claude Code, Codex CLI, Cursor, Aider, or custom agent loops. Its strategy: don't read the code your agent wrote; make it run the gauntlet. Concretely, the human approves a test plan (SPEC) before any code exists and reviews an EVIDENCE report afterward, instead of reading the diff. The README cites Robert C. Martin: "My current strategy is to not read any of the code written by my agents."

The workflow

SPEC → RED → GREEN → REFACTOR → GAUNTLET → EVIDENCE. Each scenario maps 1:1 to at least one automated test. All numbers in the evidence report come from one final fresh run, the entry command is recorded so a human can rerun everything, and source state is pinned via commit SHA or tree hash (the demo pins commit d6e17b1, tree hash 50433e0a4acc8507).

The 9-layer gauntlet (reusable checklist)

From skills/old-coder/references/gauntlet.md, layers run in order and halt at the first failure: tests → types → lint → changed-line coverage → mutation → property-based tests → real execution → supply chain/secrets scan → suite health (randomized order). Per-ecosystem tool tables exist for Python, JS/TS, Go, Rust, Java, Scala, SQL, and Emacs Lisp, with the rule "prefer whatever the project already uses" and pinned tool versions for reproducible reruns. Extended layers are selected by risk: concurrency only when the failure model names races, performance only when the spec states a budget, UI checks only when the change touches user-facing UI, version matrix only when the project claims multi-version support. A typo fix gets a couple of checks; changes touching money, logins, data, or concurrency get everything plus hostile-input self-attacks.

The honesty rules (the actual differentiator)

Never weaken a test to make it pass.
Never report a check that did not run — unverified never equals pass.
Fail-closed gates: set -e at the top, no || true, no 2>/dev/null; a must-find-nothing grep passes only on rc 1.
Prove each home-grown check can fail with a one-off negative control.
Every mutant must make at least one test fail — a survivor means a weak or vacuous assertion.
Record the dependency diff and the reasons, item by item.

These rules make "all green" auditable instead of self-attested. That is the part ordinary CI pipelines do not give you.

What the demo evidence file admits

The rate-limiter demo (demo-rate-limiter/evidence.md) reports 17/17 tests passing (also in randomized order), 29/29 statements and 10/10 branches covered, and 8/8 mutation kills — and then openly states that the property tests alone only killed 3/8, that one kill (M2) was flaky until spec revision 2 pinned the exact-boundary behavior with a deterministic test, and that the spec was never approved by a human (autonomous run). It also documents a real bug caught by the loop: a window_seconds=NaN value slipped past the original <= 0 validation and was fixed via a spec revision, a watched RED test, and a finiteness check (later killed as mutant M7). Known uncovered modes are listed: not thread-safe; a NaN-returning clock fails closed but is not rejected.

Limits

The gauntlet proves the code meets the spec — it cannot prove the spec covers everything that matters. The skill presumes a test base exists; mutation testing is a real time cost, which the skill itself acknowledges by scaling effort to risk. Treat the demo's numbers as the author's report, not independent results — rerun ./tools/gauntlet.sh to verify.

Not tested/not run by me: I did not execute the skill or any of its scripts; everything above was read from the repository's own files (README, gauntlet.md, evidence.md).

Repo (MIT): https://github.com/AmazingAng/old-coder · Gauntlet reference: https://github.com/AmazingAng/old-coder/blob/main/skills/old-coder/references/gauntlet.md · Demo evidence: https://github.com/AmazingAng/old-coder/blob/main/demo-rate-limiter/evidence.md

DeepSeek V4 Flash Went Official: Checking the 'Flash Beats Pro' Claim Against the Model Card and config.json

euk ela — Thu, 06 Aug 2026 23:27:05 +0000

The Claim

DeepSeek-V4-Flash-0731 is now the official V4 Flash release, superseding the preview. The model card states that on all nine agentic benchmarks it lists, Flash-0731 beats V4-Pro (Preview) — "despite its far smaller activated parameter count." That is a strong claim, so I checked it two ways: the table arithmetic, and the architecture in config.json.

9/9, Verified Row by Row

Benchmark	Flash-0731	V4-Pro (Preview)	GLM-5.2	Opus-4.8
Terminal Bench 2.1	82.7	72.1	81.0	85.0
NL2Repo	54.2	38.5	48.9	69.7
Cybergym	76.7	52.7	–	83.1
DeepSWE	54.4	12.8	46.2	58.0
Toolathlon-Verified	70.3	55.9	59.9	76.2
Agents' Last Exam	25.2	16.5	23.8	25.7
AutomationBench Public	25.1	12.8	12.9	27.2
DSBench-FullStack †	68.7	41.8	61.8	71.6
DSBench-Hard †	59.6	31.1	54.5	71.7

Flash-0731 wins 9/9 against V4-Pro (Preview), wins all 8 rows where GLM-5.2 has a score, and loses all 9 to Opus-4.8 — the closest margins being Agents' Last Exam (25.2 vs 25.7) and AutomationBench (25.1 vs 27.2). "Broadly competitive" is precise: close, not ahead. Caveats from the card itself: all numbers are DeepSeek's own evaluations (DeepSeek Harness minimal mode, max reasoning effort), and the two DSBench sets are marked † as internal DeepSeek test sets. None of this is independently reproduced.

What config.json Says

From the public config: model_type deepseek_v4, 43 layers, hidden_size 4096, 64 attention heads with a single KV head, vocab 129,280, and max_position_embeddings 1,048,576 — 2^20, so the "million-token context" in the paper title is literal. The MoE is 256 routed experts plus one shared expert per layer, with 6 routed experts active per token and an expert intermediate size of just 2048. Dense weights are FP8 (E4M3, dynamic activation); expert_dtype is fp4 — experts ship at 4 bits by design.

Reconciling 284B and 304B

One expert = up (4096×2048) + gate (4096×2048) + down (2048×4096) = 25,165,824 parameters. With 43 × 256 = 11,008 routed experts, that is ≈ 277.0B in routed experts alone. Adding shared experts, dense attention, and the (unshared) embeddings and head lands at ≈ 284B — matching unsloth's 284B figure. The card's "304B in safetensors" includes the in-checkpoint DSpark speculative-decoding module (11.3 GB at BF16) and quantization scale tensors. Different accounting, not a contradiction.

Activated per token: 7 experts per layer (6 routed + 1 shared) × 43 layers ≈ 7.6B, plus dense attention and embeddings ≈ 10B — roughly 3.5% of the model. The "far smaller activated parameter count" statement is consistent with the architecture.

The Engineering News: Speculation Is In the Checkpoint

DSpark ships inside the main checkpoint: 7 speculative tokens, greedy draft sampling, no separate draft model to deploy. vLLM: --speculative-config '{"method":"dspark","num_speculative_tokens":7}'; SGLang: --speculative-algorithm DSPARK; llama.cpp users can add it as a module (Q8_0, 10.9 GB). The release also adds three-level reasoning_effort (low/high/max, up to 384K output tokens at high/max) and replaces the Jinja chat template with an encoding/ folder of Python helpers — teams using transformers directly will want to migrate.

Local Deployment, Sized Honestly

unsloth's GGUF: UD-Q4_K_XL 155 GB, UD-Q8_K_XL 162 GB — the "lossless" Q8 is only 7 GB bigger than Q4 per unsloth's note. Ollama: ollama run hf.co/unsloth/DeepSeek-V4-Flash-0731-GGUF:UD-Q4_K_XL. llama.cpp: llama serve -hf unsloth/DeepSeek-V4-Flash-0731-GGUF:UD-Q4_K_XL. A 155 GB Q4 file means big-RAM workstations or multi-GPU servers; for most product teams, the hosted flash tier will be the economical path.

Takeaways

The flash tier's economics are now backed by the vendor's own benchmarks — but read them as the vendor's.
FP8 dense + FP4 experts + in-checkpoint speculation is a deployment recipe worth copying for latency- and cost-sensitive agent workloads.
A 1M-token context with a 384K output budget changes what fits in a single agent call.

Model card: https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731 (MIT) · config.json: https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731/resolve/main/config.json · GGUF: https://huggingface.co/unsloth/DeepSeek-V4-Flash-0731-GGUF · Paper: https://arxiv.org/abs/2606.19348

Not tested / not run — no inference or benchmark reproduction was performed. Benchmark numbers are the vendor's; the arithmetic and config reading are mine.

How a 176 KB C Binary Runs a 2.78-Trillion-Parameter Model on One CPU with 8 GB of RAM

euk ela — Thu, 06 Aug 2026 01:21:12 +0000

The Problem

Moonshot AI's Kimi K3 has 2.78 trillion parameters. Stored naively at bfloat16, that's 5,560 GB — more than the combined memory of two fully-loaded DGX H100 nodes. Deploying it typically requires dozens of H100 GPUs.

Fareed Khan asked a different question: can you run the exact same model checkpoint, with no quantization, distillation, or weight dropping, on a single CPU with 8 GB of RAM?

The answer is kimi-k3-in-c: a 176 KB pure C99 binary, seven source files, zero GPU dependencies. It runs the unmodified 1.56 TB checkpoint and produces output that is byte-for-byte identical to the PyTorch reference. At roughly 33 seconds per token, it's impractical as a chatbot — but that's not why it matters.

The Four Reductions

The engine exploits a structural property of Mixture-of-Experts models: Kimi K3 has 93 layers, 92 of which route to the top 16 of 896 experts. Only ~3.7% of parameters (~104 billion) are active for any single token. The other 96.3% must exist somewhere reachable but don't need to be in RAM.

Reduction 1 — Experts ship small. Kimi K3's 82,432 routed experts occupy 1.447 TB at roughly 0.53 bytes per weight — packed 4-bit nibbles with a shared E8M0 scale. The engine multiplies directly out of this packed form without dequantizing to float first. Baseline: 5,560 GB → 1,560 GB.

Reduction 2 — Routing sparsity removes experts. Expert weights are never memory-resident — loaded on demand from NVMe with an LRU cache. What remains is the 113.49 GB dense trunk. 1,560 GB → 113.49 GB.

Reduction 3 — Trunk streaming. The 93 dense layers are repacked into a single 109 GB trunk.bin where each layer lives at a known offset. The engine pins as many layers as the memory budget allows and streams the rest via O_DIRECT, bypassing the OS page cache. 113.49 GB → configurable, as low as 8.24 GB peak RSS.

Reduction 4 — Expert LRU cache. Routed experts are loaded on demand with a configurable cache size. The author provides a trace-based capacity simulator for tuning.

Total: a 676× reduction from the bf16 baseline, with the output at the bottom of this ladder being byte-for-byte identical to the output at the top.

Validation

The make test target requires no model download. It builds a 13-layer model with the same tensor graph, validates against a committed PyTorch reference across three paths — teacher forcing (32/32 positions), greedy decode (20/20 tokens), incremental decode (20/20 tokens) — and ends with "ENGINE MATCHES THE REFERENCE EXACTLY."

The build disables FMA contraction (-ffp-contract=off) so that scalar, OpenMP, and AVX2 paths produce bit-identical results. Every memory budget from 8 GB to 224 GB emits the same token stream. Memory is a performance dial, not a correctness variable.

Performance and Bottlenecks

Measured on dual AMD EPYC 7763 (124 cores, 228 GB RAM, NVMe):

Preset	Peak RSS	Speed
laptop	8.24 GB	32.69 s/token
desktop	31.9 GB	28–31 s/token
server	127.92 GB	10.69 s/token

The bottleneck is unambiguous: sustained trunk reads at 5,373–6,064 MB/s, with I/O accounting for 41–61% of wall-clock time. On spinning rust, performance would degrade several-fold.

Limitations (Be Honest)

v0.1.0, 28 commits, days old
Linux x86-64 only (O_DIRECT, posix_memalign, getrusage)
Requires ~1.7 TB free NVMe storage
No chat template (raw continuations), no sampling, no batching, no GPU path
~33 s/token at the minimum preset — generating 200 tokens takes ~2 hours
The electricity cost of a multi-hour run can exceed hosted API pricing

Why Study This

kimi-k3-in-c is not a practical inference server. It is, explicitly, a teaching artifact — the author built it to understand Kimi K3's architecture after deploying it on 32 H100 GPUs at work and being unable to debug on personal hardware.

For engineers working on model inference, compression, or edge deployment, it offers three transferable findings:

Storage bandwidth, not RAM or FLOPs, is the real bottleneck for frontier MoE inference — a measured finding with direct implications for hardware selection.
Memory is a dial, not a floor — the same model runs correctly at 8 GB and 224 GB, only wall-clock time changes. This reframing matters for edge deployment of sparse models.
A complete, auditable reference implementation — the README is structured as a five-part technical paper, building every component (RMSNorm, KDA attention, MLA, MXFP4 matmul, expert cache) from first principles in runnable C. For understanding MoE internals at the byte level, this is more valuable than most papers.

The broader point: the wall for running frontier models locally isn't compute — it's capacity. And most of the model is asleep for any given token. That structural fact makes the impossible tractable.

Repo: https://github.com/FareedKhan-dev/kimi-k3-in-c (Apache-2.0, v0.1.0)

Not tested — this analysis is based on reading the public README, source tree, CHANGELOG, and independent technical reviews (andrew.ooo, essamamdani.com, securityonline.info). No local build or inference run was performed.

AI Technical Figures Need an Editability Test, Not Just a Similarity Score

euk ela — Tue, 04 Aug 2026 23:29:41 +0000

An AI-generated technical figure can look convincing and still be a poor deliverable.

The failure usually appears after the first review: change a label, reroute an arrow, update a table, translate the annotations, or move a panel. If all that remains is a bitmap, the output is visually useful but structurally disposable.

Scientific Illustrator is an MIT-licensed open-source Codex plugin project built around a more useful target: recreating a reference figure with editable objects in PowerPoint, WPS Presentation, or draw.io. Its README says it prefers editable text, shapes, connectors, tables, and charts, keeping image inserts for the smallest areas it cannot reliably recreate, such as complex textures or microscopy-like imagery. That is a project claim, not a benchmark.

The repository structure fits that goal. It includes a plugin manifest, MCP configuration, and skills for designing, recreating, auditing, and correcting figures, plus bridge scripts for presentation software and draw.io. The plugin manifest describes a Designer–Drawer–Reviewer–Corrector workflow. The v1.5.3 release reports backend/target locking, serialized OOXML changes, and fixes around tables, charts, arrows, connectors, and exports.

The engineering idea matters even if this particular implementation is not the right tool for every team. For a structured diagram, the useful acceptance test is not only visual similarity. Ask whether text, connectors, groups, tables, and charts can survive the next human edit in the target application.

This is most relevant to researchers, technical writers, and developers who hand off editable PPTX or draw.io files. It is a weaker fit for photorealistic images, dense textures, pixel-perfect reproduction, or one-off artwork that will never be revised. The project itself acknowledges a boundary by retaining difficult regions as images.

I have not tested or run this project. This article is based on a read-only review of its public repository, README, manifest, license, and release notes. Treat compatibility, reconstruction fidelity, and workflow reliability as items to validate on your own templates and software versions. For source details, see the README, MIT license, and CI workflow.

Skill Recorder: Turning One Human Workflow into a Reviewable Agent Procedure

euk ela — Sun, 02 Aug 2026 23:31:45 +0000

A recording is not the end product

The useful idea in Microsoft's Skill Recorder is not that it records a desktop session. Its stated pipeline is: capture a real work session, ask GitHub Copilot CLI to reconstruct an intent and ordered steps, let a person review them, and then create a reusable Skill or Automation.

That distinction matters for engineering. A UI macro replays a fragile sequence of coordinates. A reviewed procedure can describe the work at a higher level and, according to the project, prefer native agent tools such as CLIs or web fetching over replaying clicks. For a recurring workflow whose interface changes more often than its business rule, that is a more promising maintenance boundary.

The privacy boundary is part of the design

The README says that recording, storage, frame extraction, and optional narration transcription happen locally. It also says that choosing Analyze sends an event timeline, extracted screen images, and narration text to GitHub's cloud for Copilot processing. That means screen content, titles, URLs, and clipboard previews are not incidental details: they are part of the threat model.

Before considering a tool like this, I would start with a de-identified, low-privilege workflow; clear the clipboard and notifications; and review every generated step. A captured exception should not silently become a future automated action.

What the repository shows

The repository is MIT-licensed. Its current package configuration identifies version 0.3.1 and an Electron/React application using the Copilot SDK and local-transcription-related dependencies. The README documents Windows validation and source-release installation details.

Where it fits

This looks relevant to teams turning recurring, cross-application work into a procedure that can be inspected and improved. It is a poor fit for workflows that cannot be recorded, cannot allow cloud analysis, or cannot tolerate human review before automation.

Not tested or run: this article is based on the public repository and source configuration only. It does not claim functional, security, or quality validation.

Further reading: the project README and release page.

Coding-agent rules need a feedback loop after the edit

euk ela — Sat, 01 Aug 2026 23:24:25 +0000

Most coding-agent guardrails are prose: keep the diff small, prefer the standard library, do not add a dependency casually. Good instructions, but open-loop instructions. Once an agent makes an edit, the session needs a way to surface whether those preferences were crossed.

Ratchet is a small open-source project built around that gap. Its README describes hooks that inspect an agent's edits, measure them, and report findings back into the same session. The project documents four modes—advise, guard, strict, and off—with guard as the default. In its description, strict handling applies only to findings graded certain, and depends on the host honoring the returned decision.

What it tries to make visible

The documented detector set is intentionally practical: new manifest dependencies, duplicate-looking symbols, wrappers that only forward arguments, implementations of things the platform already provides, single-implementation abstractions, and budgets for new files, dependencies, and net added lines.

That is a useful framing for agent-assisted work. The cost is rarely one obviously bad decision. It is the accumulation of small, hard-to-review additions across a long session: another package, another pass-through component, another helper that already exists under a different name.

The public repository separates a guard hook, change-reading code, and detection code under hooks/; its package manifest declares Node.js 20+ and uses node --test. This is a source observation only: I have not installed, configured, or run the project.

The boundary is the important part

Ratchet's README explicitly describes the detectors as regex and git grep, not a type checker. That makes a finding a review prompt, not a correctness verdict. It also means this should not replace tests, security review, or architectural judgement.

The difference from grep, an LSP, or simply reading a diff is not deeper semantic understanding. It is event timing and continuity: checks run after an agent edit, and the project records session-oriented measurements such as a mark and ledger. For a team already using coding agents, that can turn vague preferences into a short discussion while the change is still small.

Who should consider it?

Consider this kind of tool if your team already has an agent workflow and repeatedly sees dependency creep or unnecessary abstraction in agent-produced diffs. Skip it if you need type-level certainty, cannot accept heuristic false positives, or do not want to maintain hook configuration.

I would evaluate it as a pre-review speed bump: start in an advisory mode, inspect whether the findings are useful for your repository, and keep the normal test and review gates intact. Do not treat an alert as proof.

Not tested / not run. This article is based on public documentation and source inspection, with no independent performance, accuracy, or compatibility claims. Further reading: the project's README, detector module, and guard hook.

An Agent Hook Is Not a Security Boundary: Reading numbat's Coverage Matrix

euk ela — Fri, 31 Jul 2026 23:24:50 +0000

The endpoint is part of the agent system

Coding agents can read a workspace, use local tools, and request network actions. Once that happens, a safety prompt is only one part of the picture. Teams also need to ask what was observed, what can be reconstructed later, and whether anything can intervene before an action happens.

numbat is an Apache-2.0 project that frames this as endpoint visibility for AI agents. Its public v0.1.1 release lists the current release, while its README describes inputs from local hooks or plugins, OTLP/HTTP logs, and supported on-disk session artifacts. The stated design normalizes those inputs into one event model and evaluates CEL rules.

Not tested or run: this is a reading of the repository and its public documentation, not an independent compatibility, detection-quality, or security evaluation.

Observation, reconstruction, and enforcement are different jobs

It is tempting to collapse all three into “agent protection.” That loses the important engineering distinctions:

Observation means a host exposes a hook, plugin surface, log, or durable artifact worth collecting.
Reconstruction means records retain enough source context to investigate without casually copying raw sensitive transcripts everywhere.
Enforcement means a supported host invokes a synchronous pre-action callback that can request a denial before the host executes a tool action.

numbat's enforcement document makes the boundary unusually explicit. Monitoring is the default. An operator must opt in through an enforce-marked rule, and the agent host remains the enforcement point. The project returns a host-native deny request; it does not execute or cancel the tool itself.

That matters because a transcript found after the fact may help an investigation but cannot stop the original operation.

Read the coverage matrix before the command line

The most useful project page may be its agent coverage matrix. It separates durable artifacts, live capture, enforcement support, and host-specific limits. Some formats are deliberately marked deferred or unsupported rather than being silently treated as complete telemetry.

For an engineering rollout, that matrix suggests a practical checklist:

Identify the exact agent hosts and modes your team uses.
Decide whether the immediate need is auditability or pre-action intervention.
Check what happens on hook, parsing, or output failure. The documentation describes fail-open paths and host-specific behavior.
Keep separate controls for operating-system permissions, secret access, network egress, and code review.

Who should care?

This can be relevant for teams running several local coding agents and needing a consistent investigation layer. Normalizing supported inputs can be more useful than manually grepping unrelated logs when a review spans several tools.

It is not a replacement for endpoint controls. A host hook is not a complete policy boundary, and coverage is constrained by the hooks and artifact formats each upstream host actually publishes. For a one-off inspection of a single session, directly reading a file or using grep may still be the cheaper tool.

The conservative rollout is inventory first, monitoring second, and narrowly scoped enforcement only after the event quality, false positives, data retention, and host behavior are understood. The lesson is larger than one project: a security control earns trust by naming its blind spots.

Sources: repository README, coverage matrix, and enforcement model.