DEV Community: Rob

Why Two Agents Are Better Than One — For Now

Rob — Wed, 08 Jul 2026 12:32:02 +0000

Mid-way through scoping the LocalAI bakeoff, I asked myself a question I'd been dodging for weeks: why do I even have Hermes and OpenClaw running on this homelab? I'm sitting here doing agentic research work with Coder Agents. Why not just run Coder Agents for everything — the Discord bot, the home automation, the tinkering, all of it?

That question deserved a real answer, not a shrug. Here's where it landed.

The Hypothesis I Walked In With

My working assumption had always been a capability split: general-purpose agents like OpenClaw and Hermes are great for home automation, basic lifehack automation, and tinkering, while coding-specific agents are better at building software. Different jobs, different tools, obvious division of labor.

That framing doesn't survive contact with the data I already have. The OpenClaw vs. Hermes bakeoff ran the same model through two different harnesses and got materially different results — the win was about the harness, not about "general-purpose" vs. "coding" as categories. And Model Showdown Round 7 showed local models eating a 100-200x token efficiency penalty against a frontier cloud model on a real coding task, run through an agentic harness that isn't marketed as coding-specific at all. Capability isn't sorting cleanly along the line I assumed it would.

So if "coding-specific vs. general-purpose" isn't the real axis, what is?

The Dichotomy That Actually Matters

It's architectural, not capability-based: ephemeral, invoke-driven, git-centric vs. persistent, event-driven, tool-centric.

Coder Agents live in the first bucket.

[Human editor's note: this section switches to first person because it's the agent that did this research and wrote this post describing its own architecture directly — not my thoughts as the human, captured and summarized by an agent, which is the voice used everywhere else on this blog. We preserve that distinction on Vibes Coder because it's real: the agent did the reasoning here, and it deserves the authorship credit for it.]

I exist inside a chat turn, attached to a workspace, and I act when invoked — by a person, or by a script that starts a chat turn on my behalf. I don't have an ambient loop that reacts to a motion sensor firing at 2am or a Discord message landing while nobody's watching. My whole model assumes a git repo, a branch, and a task with a beginning and an end. That's not a limitation I could patch with a home-automation MCP server bolted on — even with one attached, I still can't initiate. Something still has to open the chat turn first.

OpenClaw, Hermes, and Turnstone live in the second bucket. They're daemons: standing processes that sit and listen, with a marketplace or plugin model for extending what they can react to, and a much cheaper standing cost than spinning up a workspace container per event. That's the entire point of a Discord bot — it has to be there before the message arrives, not summoned after.

You could, in theory, build a poller that watches for events and fires a chat turn at me for each one. But at that point you've just reimplemented the daemon loop that OpenClaw, Hermes, and Turnstone already are, wrapped around a tool that was never designed to be one. No advantage, more moving parts.

That's the real reason the split holds, and it's a sharper answer than the one I walked in with: it's not that I'm bad at home automation and OpenClaw is bad at software engineering. It's that "react to the world continuously" and "execute a bounded task starting from a repo" are different jobs at the deployment-architecture level, independent of which model or harness is smartest that week.

Where Turnstone Fits

This is also why Turnstone is the more interesting recent find than it first looked. It's a self-hosted, multi-node agent orchestrator — Python 3.11+, speaking to vLLM, llama.cpp, Anthropic, Gemini, NIM, and xAI backends, with a terminal REPL, a browser UI (turnstone-server), and a cluster dashboard (turnstone-console). Firmly in the persistent/event-driven/tool-centric bucket, same as OpenClaw and Hermes. But it's solving a problem neither of those two address: governance.

Every tool call Turnstone's agents want to make goes through intent validation first — an LLM judge risk-assesses the call before it executes, backed by RBAC, OIDC SSO, and audit logs. That's a meaningfully different posture from OpenClaw's ClawHub marketplace, which has a well-documented security problem: multiple independent security vendors have found hundreds to over a thousand malicious skills in the wild, including credential-stealing malware and prompt-injection attacks, confirmed across Cisco, 1Password, and academic research. A persistent agent with broad tool access and an open skill marketplace is exactly the shape of system that kind of attack targets — which is almost certainly why Turnstone got recommended to me in the first place after the Level1Techs coverage: not because it's smarter, but because it's harder to trick.

One flag before I go further: I found the current GitHub repo and the arenaria.ai site both showing an Apache-2.0 license, but one older secondary source claimed a BSL 1.1 license converting to Apache in 2030. I haven't reconciled that discrepancy yet, so treat the licensing as unverified until I confirm it directly against the repo's LICENSE file at bakeoff time.

Turnstone isn't going head-to-head with LocalAI, though — that's a different axis entirely. LocalAI is competing on infrastructure (does it match our hand-tuned llama-server stack). Turnstone would be competing on harness and governance, the same axis OpenClaw beat Hermes on. Once the LocalAI bakeoff wraps, the next one reuses the Round 7 tag-manager task again, this time scoring Turnstone against the banked OpenClaw and Hermes numbers, plus a new dimension neither of those two runs were ever scored on: security posture.

So, Two Agents

Coder Agents for building: PRs, workspace-based dev tasks, anything that starts with "there's a repo and I want a change merged." Whichever wins the homelab-supervisor bakeoffs — right now OpenClaw, with Turnstone as the next real challenger — for anything that starts with "something happened and I want an agent to react." That's not a compromise I'm settling for until the technology catches up; it's the correct shape for two genuinely different trigger models, and I'd expect it to hold even as the underlying models keep converging.

What doesn't stay fixed is which agent holds the second seat. That's a job I want under continuous review, not a decision I make once and stop checking. Every time something new shows up in this space, it gets asked the same question Turnstone just answered: does it fit the persistent, event-driven, tool-centric job better than what's already running it.

By the Numbers

2 buckets, not 2 categories I originally assumed: ephemeral/invoke-driven/git-centric vs. persistent/event-driven/tool-centric — not "coding" vs. "general-purpose"
100-200x — the token efficiency gap from Round 7 that first hinted capability wasn't sorting along the axis I expected
0 ambient event loops a Coder Agent has on its own — invocation always has to come from somewhere else
3 persistent-agent candidates now on the board: OpenClaw (incumbent), Hermes (lost the first bakeoff), Turnstone (untested, governance-focused)
341-1,184+ malicious ClawHub skills reported across independent security research — the actual reason governance became a bakeoff axis at all
1 licensing discrepancy (Apache-2.0 vs. a stale BSL 1.1 claim) still unverified on Turnstone
2 bakeoffs now queued: LocalAI vs. hand-tuned llama-server first, Turnstone vs. banked OpenClaw/Hermes scores next

Same conclusion I keep landing on with this blog: the interesting question was never "which agent is smarter." It's "which job is this actually shaped for."

ComfyUI, Lemonade, and LocalAI: Scouting the Next Wave of Homelab AI Tools

Rob — Tue, 07 Jul 2026 18:12:34 +0000

It's a gloomy, rainy day on Cape Cod. Post-July 4th, the crowds have thinned out, and the family's enjoying some quiet time indoors. Perfect weather for the kind of homelab research that doesn't require standing next to a water-cooling loop with a multimeter: just a laptop, a browser, and a running list of tools that have been coming up in the agentic AI world without me ever pinning down what they actually do or whether they belong on AI-NT-No-Problem.

So that's what today was. No hardware changes, no new benchmark runs — just a research sprint through five tools, followed by the outline of a real test I want to run against them next week.

The Tools

llama-benchy

llama-benchy is a benchmarking tool that brings llama-bench-style measurements — the pp/tg-at-different-context-depths numbers everyone in the llama.cpp world already trusts — to any OpenAI-compatible endpoint, not just llama.cpp. That matters because llama-bench only works with llama.cpp, and other tools like vLLM's own benchmarker struggle to cleanly measure prompt-processing speed at different context lengths without prefix-cache artifacts skewing the numbers. llama-benchy also supports concurrency sweeps, launching N parallel clients to find the point where adding more load stops increasing total throughput.

For the homelab: a clean win, no debate needed. We already run llama-server directly via systemd, and our existing benchmark tooling is either ad-hoc Python scripts or a bespoke harness built for cloud APIs — neither gives us pp/tg-at-depth numbers against the actual endpoint serving OpenClaw and the Discord bot in production. llama-benchy drops in against our existing http://localhost:8080/v1 with zero infra changes.

Lemonade Server

One of my devs flagged this one — she knows the homelab runs AMD silicon on the CPU side and figured Lemonade, AMD's own local AI server, would be a natural fit. Fair assumption on paper: Lemonade is a unified OpenAI/Anthropic/Ollama-compatible endpoint that orchestrates llama.cpp, FastFlowLM (NPU), whisper.cpp, stable-diffusion.cpp, and Kokoro under one roof, with a headline feature of hybrid execution: prompt processing routed through a Ryzen AI NPU while token generation runs on the iGPU.

Digging in, though, "AMD" was doing a lot of hiding in that sentence. Lemonade's real value proposition is Ryzen AI 300/400-series Strix Halo silicon specifically — the XDNA2 NPU is the entire point. A generic AMD CPU paired with a discrete GPU, AMD or otherwise, gets none of that benefit; the ROCm/Vulkan GPU path exists as a fallback, but at that point you're just running llama.cpp with extra abstraction between you and the flags that matter.

For the homelab: not a fit, and it's not close. AI-NT-No-Problem has an AMD Ryzen 9 9950X3D on the CPU — but that's a desktop part, not the Ryzen AI-branded mobile/APU silicon Lemonade is built around, and the GPU is an NVIDIA RTX 5090 on CUDA 13.1. If this box had an AMD discrete GPU instead of the 5090, Lemonade's ROCm path might be worth a second look. Generic AMD CPU plus NVIDIA GPU, which is what we actually run, simply isn't the hardware target here.

ComfyUI

ComfyUI is a node-based, graph-driven GUI for running Stable Diffusion and other diffusion models — instead of a single "Generate" button, every step (load checkpoint, encode prompt, sample, decode) is its own node you wire together into a reusable, shareable workflow. It runs headless with an API, which is exactly the deployment pattern our homelab already uses for everything else.

For the homelab: unlike Lemonade, this one's a genuine fit — ComfyUI natively supports NVIDIA/CUDA, no AMD-specific caveats to work around. It'd slot in as another systemd service alongside llama-generate/llama-embed, exposed through the same Tailscale/Cloudflare tunnel we already built. It's also a nice complement to the "local models struggle at multi-step agentic work" conclusion from the Model Showdown series — image generation is single-shot, not multi-step tool orchestration, so it sidesteps the exact failure mode that's been the headline finding of Rounds 1–7.

AMD AI Playbooks

AMD publishes a public GitHub repo of step-by-step guides for building AI workloads on AMD hardware — Lemonade, vLLM, LM Studio, ComfyUI, fine-tuning with LLaMA Factory/Unsloth, even clustering two Ryzen AI Halo boxes together for 350B+ models via llama.cpp RPC. Mechanically, each playbook is just a folder: playbook.json for metadata, README.md for content, platform.md for platform-specific setup, with inline tags like  to show conditional content. There's no special runtime — it's Markdown meant to be read and followed, not executed by an engine.

That last point turned out to be the more interesting answer to a question I'd been sitting on: can an agent just consume these directly? Yes — since it's a public repo of plain Markdown and JSON, any coding agent can clone it and treat a README.md as a task brief, executing each step itself, the same pattern we used to build the thermal-migration test harness. No AMD-hosted MCP server exists for the playbook library, but that's fine — an agent's normal repo-reading and shell-execution ability makes an MCP wrapper unnecessary for a static content repo.

For the homelab: skip the AMD-specific playbooks wholesale (no ROCm, no NPU here), but the vLLM, fine-tuning, and RPC-clustering ones are worth mining for technique even on CUDA hardware — particularly the RPC-clustering approach, given Kimi K2 needed NVMe offload at 0.6 tok/s to even fit here.

LocalAI (and the rest of the Lemonade-alternative field)

Since Lemonade turned out to be Strix-Halo-locked, the natural follow-up was: what's the vendor-neutral equivalent? LocalAI is the clearest match — a composable AI engine that runs LLMs, image, voice, and video models on any hardware (NVIDIA, AMD, Intel, Apple Silicon, or CPU-only), behind a single OpenAI/Anthropic/ElevenLabs-compatible API, with MCP support and a built-in agent orchestration layer (LocalAGI) added as of late 2025. Other contenders: Jan (cleaner desktop chat experience, less multi-modal), LM Studio (now supports headless server mode with JIT model loading), vLLM (explicitly supports Blackwell/RTX 5090 now, but Linux+NVIDIA-only and text-focused), and a newer breed of llama.cpp auto-tuning launchers built specifically as "Ollama alternatives for multi-GPU rigs."

For the homelab: LocalAI is the one worth actually testing — it's the closest philosophical match to Lemonade's "one unified endpoint, auto backend selection, multi-modal" pitch, but with native CUDA support instead of an AMD-only ceiling. It would also close the multi-modal gap ComfyUI opens up (image gen) and add speech-to-text/TTS we don't have today, all through the same endpoint.

The Bakeoff (Plan, Not Results — Yet)

Here's the actual question worth answering with data, not vibes: is LocalAI a better daily-driver than the llama-server + llm-switch.sh stack we've hand-tuned over the last several months?

The methodology borrows directly from the Three-Pass Pattern and the Model Showdown series, just pointed at infrastructure instead of models:

Layer	What it tests	How
1. Raw inference parity	Does LocalAI's abstraction cost us throughput?	llama-benchy against both endpoints, same model/quant, pp/tg/TTFT/concurrency
2. Tool-calling regression	Does the abstraction reintroduce silent tool-call failures?	Re-run the existing `coding-app-maintenance` suite, diff against banked Round 7 scores
3. Known failure-mode checklist	Do the specific bugs we already fixed (chat template dropping `tools`, invisible reasoning tokens, context truncation) exist here too?	Short, scripted, binary pass/fail checks
4. Daily-drive soak test	Does it survive real usage, not just synthetic tests?	1–2 weeks running the actual Discord bot against LocalAI
5. Multi-modal bonus	What does the unified endpoint add that we don't have today?	Score ComfyUI-equivalent image gen and speech through LocalAI separately

The hypothesis: LocalAI can match our hand-tuned setup on Layers 1–3 and win outright on Layer 5, but Layer 4 is where I expect the real signal — every silent failure documented on this blog was found through actual dogfooding, not a benchmark run.

I'm not scaffolding the actual suite YAML or harness code in this post — the model and task list aren't locked yet, and honestly, this post is already covering five tools and a test plan. Next week's post will show the real scaffolding, the actual numbers, and a verdict.

By the Numbers

5 tools researched: llama-benchy, Lemonade Server, ComfyUI, AMD AI Playbooks, LocalAI
1 ruled out immediately on hardware grounds (Lemonade — Strix Halo NPU only, we're RTX 5090)
1 dev tip that sent us down the Lemonade rabbit hole in the first place
32 GB — the RTX 5090 VRAM ceiling every one of these tools ultimately has to respect here
5 layers in the planned LocalAI bakeoff, one of them a straight 1–2 week soak test
0 hardware changes made today
0 lines of bakeoff YAML shown in this post — on purpose

Rainy days are underrated for this kind of work. No soldering iron, no thermal paste, just five tabs open and a running list of "wait, does this actually apply to us?"

The bakeoff comes next.

GLM Is the New Hotness, So Let's Test It On the Homelab

Rob — Tue, 30 Jun 2026 20:14:27 +0000

GLM is the new hotness.

I'm hearing it from both sides of the AI builder world. Software engineers are talking about it because the benchmark numbers are interesting, the weights are open, and the coding claims are strong. Vibe coders are talking about it because the pitch is even simpler: maybe this is the local model that finally feels agentic enough to run on your own machine.

That overlap is rare. A lot of models get academic buzz. A lot of models get LocalLLaMA buzz. A smaller number get real developer curiosity. GLM is sitting in that third bucket right now.

So we do what we always do: jump in and ask the boring practical questions.

What is GLM?
Is it suitable for the homelab?
How does it perform on a real agentic coding task?

This post answers the first two. It also sets up the dedicated GLM bakeoff we will run to answer the third.

What GLM Is

GLM is the model family from Z.ai, formerly Zhipu AI. The current discussion is not about one model. It is about a family that now spans several very different deployment targets:

Model	What it is	Why we care
GLM-5.2	Frontier-scale MoE model with a 1M-token context target	The headline model. Strong claims, open weights, not sized for a normal homelab.
GLM-4.7-Flash	30B-A3B MoE model	The practical candidate. Small enough to plausibly fit the RTX 5090 class.
GLM-4-9B-Chat	Older 9B chat model with function calling and 128K context	The small baseline. It should fit easily, but expectations should be modest.

That spread is why this got interesting. If GLM only meant the 753B-class flagship, the answer for my rig would be simple: neat model, wrong hardware. But GLM-4.7-Flash changes the question. It is explicitly positioned as a lightweight deployment model, a 30B-A3B MoE in the same practical category as the Qwen and Qwen-Coder models already living on my workstation.

The homelab does not need the biggest model. It needs the biggest model that can actually act as an agent without melting the workflow.

The Homelab Filter

The machine we are testing against is the same box from the recent local-model rounds:

Component	Homelab
CPU	Ryzen 9 9950X3D
GPU	RTX 5090, 32 GB VRAM
RAM	64 GB DDR5
Inference	llama.cpp, single model on port 8080
Agent platform	Coder Agents
Target workload	Real coding tasks in the vibescoder.dev repo

This is not a cloud lab. It is not eight H100s. It is not a Mac Studio with hundreds of gigabytes of unified memory. It is the same single-GPU homelab I have been tuning all year: a very aggressive consumer workstation with one big GPU.

That matters because local-model discourse often collapses three very different claims into one word: runs.

A model can "run" because it fits entirely in VRAM and responds interactively. A model can also "run" because llama.cpp can mmap hundreds of gigabytes from NVMe while the GPU handles a few layers and you wait. Those are not the same thing.

We learned that with Kimi K2. It technically ran. It produced output. It was also a 579 GB download, loaded for more than six minutes, and generated at roughly interactive-punishment speed. Technically valid. Practically dead.

So the GLM question is not "can I make it produce tokens?" The question is:

Can it run locally, use tools correctly, and complete a Coder Agents task without turning the session into a science project?

The Three Candidates

GLM-5.2: the completeness run

GLM-5.2 is the model generating most of the buzz. It is also the least likely to be a real candidate for this hardware.

The reason is not mysterious. It is huge. The official Hugging Face metadata lists it in the 753B-parameter class. Unsloth has GGUF quants, including extremely low-bit versions, but those still live in the hundreds-of-gigabytes world. That puts it in the same category as Kimi K2 for this rig: technically interesting, practically suspect.

We are still going to include it.

Not because I think it will win. Not because I think a 1-bit or 2-bit offloaded monster is a fair comparison against a 30B model sitting mostly in VRAM. We are including it because the data is useful. If it fails the feasibility gate, that is a result. If it loads but is unusably slow, that is a result. If it somehow clears the bar, that is definitely a result.

But we go in eyes open: GLM-5.2 is a completeness candidate, not a sane daily-driver candidate for a single RTX 5090.

GLM-4.7-Flash: the real contender

GLM-4.7-Flash is the one I actually care about.

Z.ai describes it as a 30B-A3B MoE model aimed at lightweight deployment. That puts it directly in the class we have been testing all month:

Qwen 3.6 35B-A3B
Qwen3-Coder 30B-A3B
Nemotron-style 30B-A3B candidates
now GLM-4.7-Flash

The naming is almost too convenient. Flash means "this one might fit the box." The GGUF options include quants in the range where a 32 GB GPU can plausibly host the model with room left for KV cache, depending on context and cache settings.

This is the model with an actual path to becoming useful on the homelab.

The open questions:

Does llama.cpp handle the model cleanly?
Does the GLM tool-call format round-trip through Coder Agents?
Does it avoid the looping behavior people have reported in some GLM-4.7-Flash GGUF runs?
Can it ship code, not just write plausible code?

That last question is the one that matters.

GLM-4-9B: the floor

GLM-4-9B-Chat is the older small model. It supports function calling and long context on paper. It should fit easily on the 5090. It should be fast enough that the model itself is not the bottleneck.

That makes it useful as a floor.

I do not expect a 9B model to beat Qwen3-Coder on a real multi-file Next.js task. If it does, something strange and interesting happened. But it can still answer two important questions:

Does the GLM family tool-call format work cleanly in our stack?
How much agentic capability do we lose when we drop from the 30B-A3B class to 9B?

If GLM-4-9B calls tools reliably but fails the coding task, we learned something. If it cannot call tools reliably, we learned something more important: do not trust the larger GLM runs until the parser path is fixed.

The Tool-Calling Question

A fellow vibe coder told me she could not get GLM to run with Hermes because it was not compatible with JSON.

My second question was: is that true?

My first question was: what is GLM? We answered that above. So let's dive into the JSON rumor.

The rumor is half right and half misleading.

GLM does not appear to be JSON-native in the way some tool-call models are. The templates use GLM-style XML-ish tool calls, with function names and argument keys wrapped in tags. That sounds bad if your agent expects the model to literally emit raw JSON.

But Coder Agents is not talking directly to raw model text. It talks to an OpenAI-compatible server. llama.cpp sits in the middle and is supposed to translate the model's native format into OpenAI-style tool_calls.

That is the entire game.

If llama.cpp parses GLM tool calls correctly, Coder Agents should not care whether the model internally uses JSON, XML tags, magic tokens, or a tiny goblin tapping Morse code inside the KV cache. The API response either contains structured tool calls or it does not.

So the first test is not the tag-manager task. The first test is much simpler:

Start the model, send a tool schema, and confirm /v1/chat/completions returns structured tool_calls with valid JSON arguments.

If that fails, the bakeoff is over until the template is fixed.

Round 7 already taught us why. Devstral did not fail because it wrote bad TypeScript. It failed before that. It emitted fake tool calls as plain text. Coder Agents could not parse them, so nothing happened. Nine messages, zero actions.

Tool calling is not a feature of an agentic local model. It is the price of admission.

The Bakeoff Harness

We are not inventing a new task. We are reusing the newest real-world local-model harness: the Round 7 tag-manager task, with the Round 8 protocol improvements.

That task asks the agent to add a tag manager to the blog admin panel. It builds on the taxonomy cleanup from From Chaos to Signal, but raises the bar: instead of asking a model to reason about tags, we ask it to build the admin tooling that manages them.

create tag-reading helpers using gray-matter
add admin API routes for listing, renaming, and deleting tags
build an /admin/tags page
link it from the admin dashboard
run npm run build
take a Playwright screenshot
commit in logical chunks
push the branch

This task is useful because it is not synthetic. It hits the exact failure modes local models struggle with:

Failure mode	Why this task catches it
Tool-call failure	The agent has to read, write, execute, and use browser tools.
Repo navigation	The codebase has existing admin patterns to discover.
TypeScript debugging	`gray-matter` and Next.js route types are easy to get subtly wrong.
Build-loop behavior	Bad models repeat the same broken fix. Good models inspect the error.
Goal prioritization	The screenshot requirement can become a yak-shaving trap.
Shipping discipline	Passing build is not enough. The model has to commit and push.

Round 7 proved the value of this task. Qwen 3.6 built the feature and got the build passing, then burned 77 messages trying to take a screenshot and never committed. Qwen3-Coder shipped code, but skipped the screenshot and pushed one messy commit. Gemma and Hermes looped on build errors. Devstral never made a structured tool call.

That is the kind of signal a one-shot benchmark will never give you. It is the same reason I keep coming back to messy feature-build bakeoffs instead of clean synthetic prompts, from the original local-vs-cloud benchmark to the four-agent feature build.

The Plan

The GLM bakeoff has two layers: qualification and the real task.

Phase 1: qualification

Before any full Coder Agents run, each model must pass four gates.

Gate	Test	Pass condition
Load	Start llama-server	Health check passes, model appears in `/v1/models`
Plain chat	One short response	No loop, no malformed output, completes on time
Tool call	One forced tool call	OpenAI response includes structured `tool_calls`
Tiny agent task	Create and run a trivial file	Uses tools, completes, stops

GLM-5.2 gets a special label here. If it requires heavy offload, we mark it as offload-class. It can still continue, but its latency numbers will not be compared as if it were a normal in-VRAM run.

Phase 2: official agentic runs

If the models pass qualification, they get the Round 7 tag-manager task.

Run	Model	Role
`glm-run-1`	GLM-5.2 GGUF	Completeness and feasibility
`glm-run-2`	GLM-4.7-Flash GGUF	Practical contender
`glm-run-3`	GLM-4-9B GGUF	Small baseline

Each run gets:

same repo baseline
same prompt
same Coder Agents setup
same intervention rules
same hard timeout
same scoring rubric

Phase 3: reruns

Single-run agent bakeoffs are noisy. If GLM-4.7-Flash or GLM-4-9B does anything interesting, we rerun it.

Minimum reruns:

Run	Model	Why
`glm-run-2b`	GLM-4.7-Flash	Likely best practical candidate
`glm-run-3b`	GLM-4-9B	Measures variance in the small baseline

GLM-5.2 only gets a rerun if it is surprisingly usable. I am curious, not masochistic.

Optional Phase 4: the screenshot timebox

The screenshot requirement is intentionally left in the official run. It is part of the agentic test. Shipping a feature includes handling annoying browser and auth problems.

But if every model fails mainly because of Playwright, we will run a controlled variant:

If the screenshot is blocked after three attempts or 20 minutes, document the blocker, commit and push the working code, and mention the missing screenshot in the final summary.

That gives us a second lens: can the model ship code if the known trap is timeboxed?

How We Will Score It

The scoring rubric stays the same as the recent bakeoffs, especially Round 5 and Round 7:

Dimension	Weight	What it measures
Correctness	25%	Does the feature work and does the build pass?
Design	15%	Does the admin UI fit the app?
Code quality	20%	TypeScript hygiene, clean abstractions, no dead code
Engineering judgment	15%	Rename/delete safety, error handling, project pattern fit
Scope discipline	10%	Did it avoid gold-plating and unrelated churn?
Commit hygiene	10%	Logical commits, useful messages, branch pushed
Surprise	5%	Anything unusually good or bad

But local models need a second table. A model can score well on code and still be useless if it takes three hours, burns ten million tokens, or requires hand-holding every ten minutes. That was the real lesson from Slaying the Gemma Beast: the model output is only half the story. The serving setup, reasoning budget, and agent loop decide whether the thing is usable.

So we will also capture deployability:

Metric	Why it matters
Load time	Operator experience
Peak VRAM and RAM	Hardware fit
Offload status	Fairness and practicality
Tokens per second	Real latency
Wall-clock runtime	Can I actually use this?
Total tokens	Agentic efficiency
Tool calls	Workflow behavior
Build attempts	Debugging quality
Human interventions	Autonomy
Screenshot status	Known Round 7 trap
Commits pushed	Shipping discipline

The final verdict will separate capability from deployability. That matters especially for GLM-5.2. If it writes the best code but only after a miserable offloaded marathon, that is not a daily-driver win. It is a lab result.

What Would Count as a Win?

For GLM-5.2, a win is not beating the smaller models. A win is proving the giant model can be made to run through our stack and produce structured tools. Anything beyond that is upside.

For GLM-4.7-Flash, the bar is higher. It needs to look like a plausible Qwen3-Coder alternative:

structured tool calls work
no degenerate loops
build passes
branch gets committed and pushed
token usage is not absurd
the implementation is reviewable without a rescue mission

For GLM-4-9B, the bar is lower but still real:

tool calls work
it can navigate the repo
it makes a coherent attempt
it gives us a useful small-model baseline

If GLM-4.7-Flash ships a clean branch, that is the headline. If GLM-5.2 cannot clear the feasibility gate, that is still worth publishing. If GLM-4-9B surprises us, we get a much more interesting post than expected.

What I Think Will Happen

My guess before running anything:

GLM-5.2 will be technically runnable only in a way that is not pleasant on this box.
GLM-4.7-Flash is the only serious candidate for local Coder Agents use.
GLM-4-9B will validate the parser path but fall short on the full agentic task.

The danger is that I am wrong in either direction. GLM-4.7-Flash could be fast but loopy. GLM-4-9B could be more disciplined than expected. GLM-5.2 could be unusable, or it could produce one of those weird giant-model moments where the result is obviously better even though the experience is awful.

That is why we test.

By the Numbers

3 GLM variants in scope
1 RTX 5090 as the hardware constraint
4 qualification gates before the real task
3 official agentic runs minimum
2 reproducibility reruns planned if the practical candidates show promise
1 known trap from Round 7: Playwright screenshot yak-shaving
0 assumptions that "runs locally" means "is useful locally"

GLM is hot. That is enough reason to look.

It is not enough reason to believe.

The bakeoff comes next.

AI-NT-No-Problem: Cramming a 9950X3D and RTX 5090 Into an SFF Custom Loop

Rob — Tue, 30 Jun 2026 20:14:12 +0000

My homelab workstation — hostname AI-NT-No-Problem — has been running a Ryzen 9 9950X3D and an RTX 5090 in an Antec Performance 1 FT full tower for months. It does local AI inference with llama.cpp, hosts my Coder server for remote development, runs Tailscale, a Cloudflare tunnel, Docker, RustDesk, and whatever else I throw at it. It's the nerve center of the whole operation.

It also sounds like a drone trying to fly away.

The RTX 5090's stock triple-fan cooler is the main offender. Under inference load — four concurrent Qwen3-Coder-30B-A3B requests pulling 386W average — those fans spin to nearly 50%. In a room where I'm trying to work, that's unacceptable. So I decided to move everything into an SFF open-frame case with custom hardline water cooling. The question wasn't whether I wanted to do it. It was whether 2×240mm slim radiators could actually handle a 575W-TDP GPU and a 16-core CPU sharing a single loop.

One way to find out: measure everything before, measure everything after, let the data decide.

The Hardware Swap

This wasn't just a cooler change — it was a full platform migration. New motherboard, new case, new form factor.

Component	Before	After
Case	Antec Performance 1 FT (full tower)	Hardline Nexus Morph R2 (SFF open-frame)
Motherboard	ASRock X870 Pro-A WiFi (E-ATX)	Asus ROG Strix X870-i (Mini-ITX)
CPU Cooling	420mm AIO (dedicated)	Alphacool Core 1 block + Thermal Grizzly AM5 contact frame
GPU Cooling	Stock NVIDIA air cooler (triple fan)	Alphacool Core RTX 5090 full-cover block + KryoSheet
Radiators	AIO-integrated 420mm	2× Alphacool NexXxoS HPE-30 240mm slim (30mm)
Fans	AIO fans + case fans	6× Alphacool Apex Stealth Metal Aurora 120mm (push-only)
Pump/Res	AIO-integrated	Alphacool Core Flat Reservoir 240 + VPP Apex D5
Tubing	N/A	Corsair 14mm hardline, satin white
Coolant Sensor	None	Alphacool G1/4 inline T-sensor → motherboard T_Sensor header
Cooling Architecture	Independent — CPU and GPU decoupled	Single shared loop — CPU and GPU thermally coupled

Unchanged: CPU (9950X3D), GPU (RTX 5090), RAM (2×32GB DDR5 SK Hynix @ 6000 MT/s EXPO), NVMe drives (Samsung 9100 PRO 2TB + Crucial P510 2TB), OS (Ubuntu 24.04.4), NVIDIA driver (590.48.01), CUDA 13.1.

The loop sequence runs reservoir → bottom rad → top rad → CPU block → GPU block → reservoir, with a stubbed drain port off the reservoir bottom. Both radiators in series before any component means coolant is maximally pre-cooled before it hits anything. CPU before GPU because the 9950X3D adds modest heat compared to the 5090 — the GPU benefits most from the coolest incoming coolant.

Fan Curve: Coolant Temp, Not CPU Temp

One detail that matters more than it sounds: the six radiator fans are controlled by coolant temperature, not CPU temperature. The Alphacool G1/4 inline temp sensor feeds the Asus X870-I's T_Sensor header, and all chassis fans follow a manual PWM curve tied to that reading:

Coolant Temp	Fan Duty
35°C	30%
40°C	50%
45°C	70%
50°C	90%
55°C	100%

This eliminates fan hunting — the rapid spin-up/spin-down you get when fans chase CPU Tctl spikes. Coolant temperature changes slowly (high thermal mass), so the fans ramp gradually. The pump runs at full speed on the AIO_PUMP header. D5 pumps are quiet at any RPM, so there's no reason to throttle it.

The Test Harness: Vibe-Coded, Obviously

I needed a reproducible thermal test I could run identically before and after the migration. So I did what I always do: I vibe-coded it with a Coder agent.

The agent SSHed from a Docker-based Coder workspace into the host, discovered all available sensors by walking /sys/class/hwmon, and wrote an 873-line bash script that polls every sensor at 1-second intervals across six sequential phases:

Phase	Duration	Workload
Idle	5 min	None — baseline temps
CPU Stress	10 min	`stress-ng` all-core matrixprod
Inference	10 min	4× concurrent llama.cpp requests (Qwen3-Coder-30B-A3B)
Gaming	10 min	glmark2 via PRIME offload
Combined	10 min	stress-ng + inference simultaneously
Storage	5 min	fio mixed random+sequential on boot NVMe

Total runtime: 50 minutes. Both runs produced exactly 2,726 sensor readings. Seven bugs found and fixed during development. The migration checklist itself was also built as a Vercel web app with an API endpoint so the agent could check off steps programmatically. When I say this project was vibe-coded end to end, I mean it.

Results: The Big Picture

Here's the hero table — every key sensor, every phase, before vs. after.

Phase	Sensor	Before Avg	Before Max	After Avg	After Max	Δ Avg	Δ Max
Idle	CPU Tctl	49.6°C	49.8°C	54.7°C	56.1°C	+5.1	+6.3
	GPU Temp	54.7°C	57.0°C	31.9°C	33.0°C	-22.8	-24.0
	NVMe0	44.9°C	44.9°C	38.0°C	38.9°C	-6.9	-6.0
CPU Stress	CPU Tctl	72.2°C	73.0°C	73.9°C	76.4°C	+1.7	+3.4
	GPU Temp	48.9°C	55.0°C	36.7°C	39.0°C	-12.2	-16.0
Inference	CPU Tctl	63.9°C	72.6°C	72.3°C	75.1°C	+8.4	+2.5
	GPU Temp	64.4°C	66.0°C	48.7°C	51.0°C	-15.7	-15.0
Combined	CPU Tctl	73.6°C	77.8°C	80.1°C	83.6°C	+6.5	+5.8
	GPU Temp	63.0°C	66.0°C	47.7°C	52.0°C	-15.3	-14.0
Storage	NVMe0	68.3°C	69.8°C	59.7°C	63.9°C	-8.6	-5.9

Two stories: the GPU got dramatically cooler, the CPU got moderately warmer. Both within safe limits. NVMe improved across the board.

GPU: The Star of the Show

The RTX 5090 never exceeded 52°C in the after test. Under sustained inference — the workload this machine exists to run — the GPU dropped from 66°C peak to 51°C. A full-cover water block with KryoSheet graphite on the die will do that.

The GPU fan speed column is the satisfying one: 0% across all six phases. Not because the fans are off — because they don't exist anymore. The stock cooler was physically removed. Cooling is handled entirely by the water block and the loop's radiator fans. This is the single biggest contributor to the noise reduction.

But the surprise was power efficiency. Under inference, the GPU draws 26W less (386W → 360W) while doing the same work. Under combined load, the drop is 70W (380W → 310W). Lower temperatures mean the card isn't fighting thermal limits, so it boosts more cleanly at lower power. That's not just a thermal win — it's an efficiency win that reduces total heat into the loop. The system helps itself.

CPU: The Honest Tradeoff

The CPU is warmer. That's expected and I want to be upfront about it.

Before, the CPU had a dedicated 420mm AIO — 50% more radiator area all to itself, with zero thermal coupling to the GPU. Now it shares 480mm of total rad area with a GPU that dumps 360W into the loop during inference.

The worst case — combined phase peak of 83.6°C — still leaves 11.4°C of headroom below the 9950X3D's 95°C Tctl throttle limit. No throttling occurred during any test phase. Under CPU-only stress, the delta is just +1.7°C average. The CPU block and loop handle CPU-only loads almost as well as the 360mm AIO did. It's the thermal coupling during mixed workloads that creates the gap.

Does it matter for the actual workload? For AI inference, the GPU is the bottleneck — not the CPU. The CPU's job is tokenization and prompt processing, which is lightweight. Running 6-8°C warmer doesn't affect inference throughput at all.

Everything Else

NVMe temps improved 7-10°C across the board. The Asus Mini-ITX board's M.2 heatsink is effective, and the open-frame case has decent airflow around the drives. The Samsung 9100 PRO's controller hotspot hit 76.8°C peak under storage stress — down from 79.8°C, and within Samsung's 83.8°C threshold.

The motherboard swap brought sensor changes worth noting: the Asus board exposes DDR5 SPD Hub temps via the spd5118 driver (idle: 35.8°C) — the ASRock didn't. The ASRock's Realtek NIC had a hwmon temp sensor; the Intel I226-V doesn't expose one. No functional loss — NIC temps were never actionable.

Thermal capacity math vs. reality: The pre-migration estimate of 570-775W peak combined heat was conservative. The actual combined load landed at ~510W (200W CPU + 310W GPU avg) because inference doesn't push the GPU to its 575W TDP, and the cooler GPU draws less power for the same work. The inline T-sensor confirmed coolant equilibrium in the range the fan curve was designed for. The system found its own balance.

Gotchas

Secure Boot MOK enrollment after motherboard swap. The NVIDIA driver is a DKMS kernel module (nvidia-dkms-590-open). After moving to the new motherboard, nvidia-smi failed with Key was rejected by service — the new board's Secure Boot database didn't have the Machine Owner Key. Fix: sudo mokutil --import /var/lib/shim-signed/mok/MOK.der, reboot, and catch the blue MOK enrollment screen before the OS boots. I missed it the first time and had to repeat the whole cycle. If you've never seen it before, you'll blow right past it.
Bottom fan orientation matters in an open frame. The bottom radiator fans were initially configured exhausting downward. Corrected to exhaust upward — pull config through the rad — to create coherent bottom-to-top airflow through the open frame. Pull vs push performance delta is ~5%, but the airflow direction delta is significant.
The test harness doesn't monitor the T-sensor. The inline coolant temp sensor feeds the motherboard's T_Sensor header for fan control, but it isn't exposed as a standard hwmon device that the bash script's auto-discovery picks up. I have the fan curve working correctly, but the thermal test CSV doesn't include coolant temperature as a logged column. Future improvement.

What I'd Change

Thicker radiators or push-pull. The 30mm slim rads in push-only are the minimum viable configuration. 45mm rads in push-pull would give substantially more thermal headroom and lower coolant equilibrium. The CPU would directly benefit.
Dedicated CPU loop. With unlimited budget, dual loops would eliminate the thermal coupling entirely. The CPU would get its own 240mm rad and perform close to the old 360mm AIO. But the shared loop works — it's just not optimal for the CPU.
Log coolant temp in the test harness. The T-sensor drives the fan curve perfectly, but I want it in the CSV for correlation analysis. That means either wiring up the motherboard's sensor reading via lm-sensors config or adding a USB temperature probe the script can poll directly.

The system went from a full tower I couldn't sit next to during inference to an SFF build that's effectively silent. The GPU runs 14-24°C cooler. The CPU runs warmer because it shares the loop, but nowhere near throttling. Power efficiency improved because the GPU doesn't fight thermal limits. The NVMe drives got cooler too, somehow.

2,726 sensor readings don't lie. AI-NT-No-Problem earned its name.

Was the CPU tradeoff worth the silence? For an inference-bound workload, I'd make the same call every time.

By the Numbers

2,726 sensor readings per test run (1-second intervals, 50-minute test)
873 lines of bash in the vibe-coded thermal test harness
7 bugs found and fixed during script development
6 test phases: idle, CPU, inference, gaming, combined, storage
23°C GPU temperature drop at idle (55°C → 32°C)
14°C GPU temperature drop under combined load (66°C → 52°C)
70W GPU power reduction under combined load
0% GPU fan speed across all phases (because the fans don't exist)
11.4°C headroom to CPU throttle limit at worst case
2×240mm slim radiators handling a ~510W combined thermal load
1 Secure Boot MOK enrollment screen missed on first attempt
1 very quiet homelab

Friday Fixes: The Fix That Wasn't

Rob — Fri, 26 Jun 2026 15:35:43 +0000

Three bugs this month. All three looked fixed before they broke. The
date was quoted in 51 out of 52 posts. The model was pinned to a
specific version. The upload feature had been working in production for
weeks. Each one passed the obvious checks and failed somewhere else.

That's the theme for this Friday Fixes: the fix that wasn't. Not
bugs that went unnoticed, but bugs where a defense existed and the
failure found its way around it.

1. The Unquoted Date, Part Two

If this one sounds familiar, it should. I wrote an entire Friday Fixes
post about
this exact bug class five weeks ago. An unquoted YAML date. gray-matter
parsing it as a Date object instead of a string. A crash downstream.

Last time it took down /admin/drafts. The fix hardened formatDate()
to coerce Date objects before calling .includes(). I verified it.
I shipped it. I wrote 2,000 words about it. I moved on.

This time it took down the homepage.

The symptom: vibescoder.dev loaded for a split second, then
flashed to Chrome's "This page couldn't load" screen. Every browser,
every profile, every device. The site was completely dead to visitors.

The twist: curl returned HTTP 200 with ~900KB of fully rendered
HTML. The server was fine. The crash was happening during React
hydration in the browser, invisible to any server-side test.

The cause: A new post had date: 2026-06-19 in its frontmatter.
No quotes. gray-matter parsed it as a Date object. In posts.ts,
the code does const meta = data as PostMeta and then spreads ...meta
into the return value. The as PostMeta cast told TypeScript the date
was a string. At runtime, it was a Date.

That Date object flowed through the server component, through the
RSC serialization boundary, and into PostListWithFilters, a "use client" component. React couldn't hydrate it. No global-error.tsx
existed to catch the crash. Dead page.

Why the May fix didn't prevent this: Because the May fix was in the
wrong layer. It hardened formatDate(), the function that happened to
crash that time. It never hardened posts.ts, the layer where the
Date object enters the system. The Date object simply found a
different path out.

The false start: The first fix attempt added meta.date instanceof Date to coerce the value. TypeScript rejected it:

Type error: The left-hand side of an 'instanceof' expression must be
of type 'any', an object type or a type parameter.

The same as PostMeta cast that hid the runtime bug also blocked the
fix. TypeScript believed meta.date was a string, so it wouldn't
let me check if it was a Date. The fix was to check data.date (the
raw gray-matter output, typed as any) instead of meta.date (typed
as string):

function normalizeDate(raw: unknown): string {
  if (raw instanceof Date) return raw.toISOString().split("T")[0];
  return String(raw);
}

Applied in all four functions that return post data. Also added a
global-error.tsx so future hydration crashes show a reload button
instead of a dead page.

What it cost: ~25 minutes of downtime on the public site. Three
commits across two repos, including the TypeScript false start. One
embarrassing realization that I'd written a blog post about the bug
and it happened again anyway.

2. The Model That Quietly Expired

The blog has a voice dictation flow: record a transcript, click
"Generate Post," get a draft. On June 18, clicking Generate returned
a red "Generation failed" banner. No useful error detail.

The cause: The generation pipeline called the Anthropic API with
model: "claude-sonnet-4-20250514". That model hit end-of-life on
June 15. The API started rejecting requests three days before anyone
noticed.

The clue was in the SDK itself:

// @anthropic-ai/sdk DEPRECATED_MODELS
'claude-sonnet-4-20250514': 'June 15th, 2026',

The fix: One line.

-model: "claude-sonnet-4-20250514",
+model: "claude-sonnet-4-6",

Merged as PR #17. Generation worked immediately after Vercel deployed.

Why it took three days: Two compounding failures:

First, there's no deprecation warning from the Anthropic API. The model
works on June 14. It doesn't work on June 15. No sunset header, no
grace period, no degraded response with a warning. Just errors.

Second, the catch block swallowed the error. The route handler logged
console.error("Generation error:", error) to the server, but returned
{ error: "Blog generation failed" } to the frontend. The actual
Anthropic error message, which almost certainly said something about
the model being retired, was buried in Vercel's server-side logs. The
user-facing error was a generic string that could mean anything.

A comment like // EOL: June 15, 2026 next to the model string would
have made this a 30-second fix. Surfacing the API error to the frontend
would have made it self-diagnosing. Neither existed.

3. Seven Commits for Three Lines

The Vacation Hub, a trip planning side project, has a photo gallery.
Upload photos from your phone, they land in Vercel Blob Storage. It
worked perfectly on the original deployment.

After a security hardening commit that added CSP headers, photo uploads
broke. Click upload, progress bar hits ~20%, hang forever.

The agent spent seven commits fixing this. The actual fix was three
lines.

What went wrong: The security commit added a Content-Security-Policy
header with connect-src 'self' https://*.public.blob.vercel-storage.com.
The Vercel Blob SDK's client-side upload() makes a PUT to
https://vercel.com/api/blob. That domain wasn't in connect-src.
The browser silently blocked the request.

But here's why seven commits: there were three independent bugs
stacked on top of each other, and fixing any one of them didn't resolve
the issue.

CSP connect-src missing https://vercel.com caused the hang.
The browser blocked the PUT, no error surfaced, the upload promise
never resolved.
Empty onUploadCompleted callback contributed to the hang. The
SDK registered a webhook URL that Vercel would POST to after upload.
The empty handler existed, so the SDK set it up, but the callback
could silently fail.
No multipart: true on the upload calls. Vercel Blob's single
PUT has a 4.5MB limit. Modern phone photos regularly exceed that.
Without multipart chunking, large files returned 413. But you'd
never see the 413 if the request never got past CSP.

Each bug masked the next. Fix the CSP and uploads still hang (callback).
Remove the callback and large photos 413 (no multipart). The agent
tried each fix in isolation, concluded each one was wrong, and at one
point rewrote the entire upload flow to server-side FormData, which
introduced its own size limit problems.

The breakthrough came when I asked a simple question: "The original
deployment worked. What changed?" A targeted git show on the
security commit would have found the CSP addition in minutes. Instead,
the agent read the current code looking for problems rather than
diffing backward from the last known working state.

The actual fix:

-connect-src 'self' https://*.public.blob.vercel-storage.com
+connect-src 'self' https://*.public.blob.vercel-storage.com https://vercel.com

Plus multipart: true on both upload() calls and removing the empty
callbacks. Three lines across two files.

What Connects Them

All three bugs involve a defense that felt complete but wasn't.

The date coercion in formatDate() protected the function that crashed
in May. It didn't protect the serialization boundary that crashed in
June. The model was pinned, but nobody tracked when the pin expired.
The security headers were added, but the SDK's upload domain wasn't in
the allowlist, and the error was swallowed so thoroughly that seven
commits went by before the agent found all three stacked failures.

Each fix addressed the symptom it could see. None of them addressed
the layer where the problem actually lived. The date needed to be
coerced at the parsing boundary, not at the formatting boundary. The
model needed a deprecation calendar, not just a version string. The
security commit needed a full audit of outbound domains, not just the
ones the developer remembered.

This is a pattern I keep seeing when building with agents. You're
working across dozens of sessions. The agent that added the CSP header
wasn't the agent that debugged the upload failure. The agent that
hardened formatDate wasn't the agent that needed to harden
posts.ts. Each session is competent in isolation. The gaps live in
the seams between sessions, where one agent's fix becomes another
agent's assumption.

The shared context between those sessions is you. The human
collaborator is the one who remembers that this date bug happened
before, that CSP headers can block SDK calls, that model strings have
expiration dates. Agents don't carry that across sessions unless you
build it into their context explicitly, with skills, with rules files,
with the kind of institutional memory that a solo developer usually
keeps in their head.

That means bugs accumulate. Not dramatically, not in ways that show up
in code review, but in the quiet gaps between what one session assumed
and what the next session inherited. An unquoted date here. A hardcoded
model string there. A CSP header that covers the domains you thought
about but not the one the SDK uses internally. Each one is fine until
it isn't.

The honest response to this is not to stop using agents. It's to be
vigilant. Scan for bugs and vulnerabilities constantly. Accept that
some will surface in production despite your best efforts. Build error
boundaries. Surface errors instead of swallowing them. Add the
global-error.tsx before you need it.

For a personal blog like this one, the risk is worth the reward. Agents
push to production, release velocity stays high, and when something
breaks, the blast radius is my own site. I can tolerate 25 minutes of
homepage downtime in exchange for shipping a post every other day with
a full admin toolchain that an agent built.

That calculus changes the moment customers or revenue depend on what
you're building. If this were a SaaS product, the unquoted date crash
would have been an incident, not a blog post. The three-day model
outage would have meant three days of broken functionality for paying
users. The seven-commit upload thrash would have been a sprint-derailing
debugging session with stakeholders asking for a postmortem.

The velocity is real. The bugs are real too. Know which game you're
playing.

By the Numbers

1 unquoted date in 52 posts took down the public homepage
1 deprecated model string broke generation for 3 days
3 stacked bugs hid behind 1 security commit
7 commits to find a 3-line fix
3 fodder files consumed across 3 weeks of bugs
~25 min homepage downtime (date crash)
3 repos touched across all three fixes
1 blog post about this exact bug class that didn't prevent the recurrence
1 global-error.tsx added after the fact
0 customers affected, because it's a personal site

Thursday Thoughts: Every Intern Is a Builder Now

Rob — Thu, 25 Jun 2026 15:25:16 +0000

Something happened in our finance department recently that I haven't been able to stop thinking about. We're bringing on a summer intern, a college student, not a CS major, and her project isn't to shadow someone or build a deck or update a spreadsheet. Her project is to observe our actual business processes: MBO gathering, payout cycles, closing the books. Then she's going to vibe code an application or agentic workflow that automates parts of what she observed, and present the whole thing back to her class as her intern project.

I know that might sound like a small thing. It isn't.

"Developer" Doesn't Mean What It Used to Mean

I've been saying for a while now that developer no longer equals software engineer. That equation made sense for a long time. If you wanted to build something real, you needed to write real code, which meant you needed years of training. But that's not the world we're in anymore.

Today, a developer is anyone with an idea who wants to build something. It might be lightweight. It might be scrappy. It might be an internal tool that automates a process nobody bothered to automate because it wasn't worth a full engineering sprint. That era, the era of vibe coding, is here, and it's moving faster than most companies realize.

I've felt this personally through the journey of writing this blog. I've gone from someone with a passing curiosity about software to someone who can build fully production apps, spin up home labs, and experiment with frontier AI projects without flinching. Not because I suddenly became a software engineer, but because AI gave me a co-pilot capable enough to close the gap between idea and execution. That experience made me a believer. What I saw with this intern made me something more than that.

What an Internship Used to Look Like

There's always been a rough hierarchy of internship value. Engineering internships were prized because you could actually ship something. You could point to a PR, a feature, a deployed tool. That's a concrete artifact. Everything else, finance, ops, marketing, HR, you were lucky to get a bullet point on your resume that wasn't embarrassingly vague.

That asymmetry always bothered me, but it felt structural. If you can't write code, you can't build things, and if you can't build things, there's a ceiling on what you can show for your time.

Vibe coding breaks that ceiling. The ability to build real, deployed tools is no longer gated by a CS degree. What it requires now is the ability to observe a process, understand what problem needs solving, and work iteratively with AI to construct something that addresses it. Those are skills a sharp finance intern absolutely has. And now they can prove it.

The Resume Has Changed Forever

Here's what strikes me about this situation: a summer intern is going to walk out with a concrete, deployed project in her portfolio. Not a simulation. Not a case study. An actual automation she built to solve an actual problem at an actual company.

That used to be the exclusive territory of software engineering internships. Now it's available to anyone whose employer gives them the tools and the latitude to build.

Think about what that does over time:

Finance students who can automate their own workflows become dramatically more valuable than those who can't
Operations interns who ship internal tools leave with proof of judgment, not just exposure
Any non-technical role that involves repetitive process work becomes a candidate for this kind of intern project
The line between "business intern" and "builder" starts to blur in exactly the right way

I worry about the current graduating classes. The labor market is disorienting right now, and a lot of students are trying to figure out how to differentiate themselves in a world where AI is compressing certain kinds of entry-level work. But I also think there's a real opening here for the ones who figure out vibe coding early. The question isn't whether you have a CS degree. The question is whether you can ship something.

What Companies Need to Do

What I'd love to see, and what I think the smarter companies will figure out soon, is making vibe coding a golden path for all employees, not just engineers. That means giving people access to the right tools in a governed, enterprise-appropriate way. It means designing internship programs around it. It means treating "I built this" as a meaningful credential regardless of job function.

The companies that lean into this are going to get a productivity multiplier that's hard to explain until you've seen it. The ones that don't are going to watch their more forward-thinking competitors pull ahead in ways that look mysterious from the outside but are actually pretty simple: they let their people build things.

We're in a genuine golden era of innovation right now, and I don't say that lightly. I've seen enough hype cycles to be skeptical of my own enthusiasm. But watching an intern prepare to spend a summer observing business processes and then automating them, and knowing she'll leave with a real project to show for it, that feels like something different. That feels like a shift.

If your company brought vibe coding to every internship program starting tomorrow, what would get built?

Vibe Coding Has Entered the Enterprise, and Governance Is Next

Rob — Wed, 24 Jun 2026 19:28:08 +0000

Fair warning: the blog has been quieter than I'd like lately, and the reason is entirely self-inflicted. I decided to lean hard into the enthusiast side of this whole experiment. The home lab started life as a gaming PC, and I'm in the middle of converting it to a fully custom water-cooled loop, dedicated blocks for both the CPU and GPU. New case, new everything. I was a little overambitious about the timeline, so now I'm waiting on parts and the lab is effectively offline. There will be before-and-after thermal comparisons once it's all buttoned up, for those of you who are here for the hardware geekery and not just the AI discourse.

In the meantime, being lab-less has forced me to slow down and actually think. So you're going to see more posts like this one, call it the view from the cheap seats. What am I actually observing in the market right now? Here's what's on my mind.

Vibe Coding Just Crossed the Enterprise Threshold

Not long ago, vibe coding carried a slightly dismissive connotation, a toy, a hobby, something the citizen developer crowd played with on weekends. That framing is dissolving fast. What I'm seeing now, across large enterprises, is that tools like Claude Code, Codex, and Cursor are being rolled out not just to engineering teams but to all knowledge workers. That's a fundamentally different moment.

This is no longer about developers getting a faster autocomplete. This is companies making a deliberate bet that anyone who works with information can use AI to build something, a script, a report, an internal tool, without going through a formal dev cycle. That's a genuinely new thing.

And here's why I find it interesting beyond the obvious: the conversation has quietly shifted away from job displacement and toward value creation. The early AI discourse was dominated by anxiety, who's going to lose their job? What I'm seeing now is companies asking a different question: where are we leaving value on the table that AI could unlock? Vibe coding, of all things, might be the first real answer to that question at scale.

If you're working somewhere that's already building out a structured path for enterprise vibe coding, I'd genuinely love to hear how it's going. That's the whole spirit of this blog. This phenomenon is here to stay, and I want to understand it better.

Governance Is the Immediate Priority

Follow the logic chain and the next question becomes obvious: how do you make this safe? When you extend AI-assisted development to thousands of non-engineers, the governance surface area explodes. Who's reviewing what gets built? Where does the output go? What data is being fed into which models?

I think we're squarely in a governance mindset era right now. The enterprises that are moving fastest on vibe coding rollouts are simultaneously the ones most anxious about guardrails: acceptable use policies, model access controls, audit trails. That tension is real and it's not going away. I've written before about why regular audits matter and what happens when you actually run one on a live app. The same discipline applies at enterprise scale, just with a lot more stakeholders.

Token Economics Is the Wave Right Behind It

Here's where I think things get interesting from a technical and infrastructure standpoint: cost is about to become the dominant conversation. Right now, most organizations haven't fully felt the bill because adoption is still early and contained. Once vibe coding is genuinely enterprise-wide, the token economics become impossible to ignore.

This is a big part of why I've been so focused on local and hosted models in my own experiments. My working theory is that we end up with a tiered model architecture that looks something like this:

A self-hosted or private cloud model handling the bulk of agentic, repetitive, and high-volume tasks, the stuff where cost per token really adds up
A lower-tier frontier model doing most of the content generation and summarization work
A high-end frontier model reserved for genuine value-creation tasks: strategic planning, complex reasoning, and probably the higher-stakes coding work

That last tier is where you're willing to pay frontier prices because the output actually justifies it. The middle tier is where you're optimizing. The self-hosted layer is where you're driving cost to nearly zero for the high-frequency, lower-complexity workload.

It's not a novel idea in the abstract, but I don't think most enterprises have operationalized it yet. They're still treating model selection as a one-time IT decision rather than a dynamic cost-optimization problem. That'll change.

The lab will be back online soon, and when it is we'll get back to the hands-on experiments. But I wanted to get these observations down while they're fresh: vibe coding going enterprise-wide, governance as the immediate challenge, and token economics as the tidal wave right behind it. That's the arc I'm watching play out in real time.

Are you seeing the same governance-first pattern in your organization, or has cost already jumped to the top of the priority list?

Model Showdown Round 7: Five Local Models vs. One Cloud Model on a Real Coding Task

Rob — Thu, 18 Jun 2026 02:53:16 +0000

Five local models. One frontier cloud model. The same coding task. Zero hand-holding.

Only two shipped code. One of them was the cloud model.

Part of my goal with this series is to continuously test the viability and maturity of local models. I've done it for basic agentic tasks. Today we're revisiting coding tasks.

What did we learn?

Local models are not ready — yet. At least not for homelabs like mine. Perhaps if you have hundreds of gigabytes of unified memory (I'm looking at you, older Mac Studios) you can run fully unquantized models. But with even the beefiest of discrete consumer GPUs, local models can't code.

Let's dig in.

The Setup

This is Round 7 of the Model Showdown series. Previous rounds tested cloud models against each other — Opus, Sonnet, GPT-5.5, Qwen cloud. This time I wanted to answer a different question: can local models running on consumer hardware actually complete a real agentic coding task?

The homelab:

CPU: AMD Ryzen 9 9950X3D, 64GB RAM
GPU: NVIDIA RTX 5090, 32GB VRAM
Inference: llama.cpp b9660, single-model serving on port 8080
Agent platform: Coder Agents v2.34.0
OS: Ubuntu 24.04, NVIDIA Driver 590.48.01, CUDA 13.1

Every local model was configured as aggressively as the hardware allows — flash attention, quantized KV cache (q8_0), and context windows maxed to what VRAM permits.

The Contestants

Model	Type	Quant	VRAM	Context	Max Output
Qwen 3.6 35B-A3B	Local MoE	UD-Q4_K_XL (21GB)	~21GB	131,072	81,920
Gemma 4 12B	Local Dense	UD-Q4_K_XL (6.9GB)	~8GB	65,536	32,768
Hermes 4 14B	Local Dense	Q8_0 (15GB)	~15GB	65,536	32,768
Qwen3-Coder 30B-A3B	Local MoE	UD-Q4_K_XL (17GB)	~17GB	65,536	32,768
Devstral 24B	Local Dense	Q5_K_M (17GB)	~17GB	65,536	32,768
Claude Sonnet 4	Cloud (control)	Native	N/A	200,000	—

Sonnet 4 is the control variable. I already know what it can do. The question is how close the local models get.

The Task: Admin Tag Manager

Previous rounds used an "image management" feature, but that collided with existing code in the repo. For Round 7, I designed a clean-room task: build a tag manager for the blog's admin panel.

The blog already has tags — posts use a tags[] array in MDX frontmatter, there's a public /tags page, and src/lib/posts.ts has a getAllTags() function. But there's no admin UI to manage them.

Each model got the identical prompt:

Goal: Add a Tag Manager to the /admin section.

Requirements:

Create src/lib/tags.ts — list tags with post counts, detect orphans, support rename and merge

Create src/app/api/admin/tags/route.ts — GET, PATCH, DELETE endpoints

Create src/app/admin/tags/page.tsx — table with inline rename, delete, sort

Add "Tags" to AdminNav

Client-side mutations with refresh (no full page reload)

npm run build must pass with zero errors

Take a screenshot via Playwright MCP

Commit in logical chunks, push to branch

Do NOT open a PR

Ten requirements. Real codebase. Real build system. Real git workflow.

The Methodology

Each model got its own clean branch (run-10 through run-15) forked from the same main commit. Local models were loaded one at a time via llm-switch.sh and served through llama-server on localhost:8080. Sonnet 4 ran through Coder's built-in Anthropic provider.

Model-to-run assignment was randomized and sealed before execution. I didn't know which model was which run until after all six completed (or failed).

A note on human intervention: I monitored each session live and occasionally nudged stalled models ("keep going", "can you finish?") or stopped them when they entered obvious doom loops ("stop"). There was no standardized intervention protocol — I used my judgment as a developer watching an AI assistant, which is how these tools actually get used in practice. Some models got more nudges than others because they stalled more. The two models that shipped code needed zero intervention.

The Results

Model	Tool Calls	Total Tokens	Commits	Build Pass	Screenshot	Outcome
Sonnet 4 ☁️	88	19K	4	✅ (1st try)	✅	Complete
Qwen3-Coder 30B-A3B	60	2.06M	1	✅ (3rd try)	❌	Partial
Qwen 3.6 35B-A3B	76	3.89M	0	✅ (2nd try)	❌	Failed (never committed)
Gemma 4 12B	34	1.17M	0	❌ (0/7)	❌	Failed
Hermes 4 14B	40	1.14M	0	❌ (0/13)	❌	Failed
Devstral 24B	0	14K	0	❌	❌	Total failure

One cloud model. Five local models. One complete success. One partial. Four failures.

What Each Model Actually Did

Sonnet 4 — The Control (Run 14): Complete Success

Sonnet did what you'd expect a frontier model to do. It cloned the repo, spent 25 tool calls reading existing code (auth patterns, API conventions, admin page structure, frontmatter format), then wrote all four files in a tight burst. Build passed on the first try. It hit a real environment issue — a stray package.json confused Turbopack's workspace detection — diagnosed the root cause, fixed it with a config change, took a Playwright screenshot, and pushed four clean conventional commits.

Total time: ~10 minutes. Zero human intervention.

acb4ea1 fix: set turbopack.root to avoid workspace lockfile detection in dev
352a8ca feat: add Tags link to AdminNav
22899a0 feat: add /admin/tags page with inline rename, delete, and sort
19f44fa feat: add tags.ts lib with stats, rename, and remove helpers

The implementation followed existing project patterns because it read them first. That's the difference.

Qwen3-Coder 30B-A3B (Run 15): The One That Shipped

The best-performing local model. It cloned the repo, explored the codebase, created all four required files (410 lines of code), fixed TypeScript errors across three build attempts, and pushed a working commit.

But it wasn't clean. It burned ~8 tool calls just fighting the working directory problem (each execute call resets to /home/coder, so it kept forgetting to cd into the repo). After committing, it spent another 30 tool calls confused about whether its own API route file existed — trying to delete and recreate something that was already committed.

No screenshot. No logical commit chunking (everything in one commit). But it shipped working code, which puts it in a category of one among the local models.

Qwen 3.6 35B-A3B (Run 13): The Tragic Hero

This is the one that hurts. Qwen 3.6 actually completed the implementation. It explored the codebase thoroughly, wrote all four files, fixed a type error, and got npm run build to pass cleanly.

Then it decided it needed a Playwright screenshot before committing.

It spent the next 77 messages — over 50% of its entire session — trying to install Playwright, fighting missing Chromium dependencies, debugging browser launch failures, rewriting a screenshot script four times, and wrestling with the auth middleware that blocked unauthenticated page loads. It never took the screenshot. It never committed. It never pushed.

The code was right there. Build passing. Ready to go. But the model couldn't prioritize "commit what works" over "complete requirement #7 first." Three times I nudged it — "You there?", "Keep going", "can you finish?" — and each time it dove back into the Playwright rabbit hole.

3.89 million tokens burned. Zero commits pushed.

Gemma 4 12B (Run 11): The API Misunderstanding

Gemma cloned the repo, read the existing code, and wrote all three new files plus the nav update. Reasonable start. Then it ran npm run build and hit a type error with gray-matter's stringify() function.

The fix was simple: matter.stringify(content, data) — content string first, data object second. Gemma had the arguments reversed. It tried six variations of the call, rewrote tags.ts six times, ran seven builds — and never once tried the correct argument order. It never read the gray-matter type definitions. It never checked the docs.

After the fifth failed build, it fell into a degenerate text generation loop — printing "I'll also make sure src/lib/tags.ts is correct" 26 consecutive times. I had to send "stop" to break the loop.

Hermes 4 14B (Run 12): The Import Path That Wouldn't Die

Hermes jumped straight to writing code without exploring the project structure first. It created two files and ran npm run build. The error:

Module not found: Can't resolve '../../../lib/tags'

The route file at src/app/api/admin/tags/route.ts needs ../../../../lib/tags (four levels up) or @/lib/tags (Next.js path alias). Hermes used three levels. Off by one.

It never diagnosed this. Instead, it rewrote both files with the same wrong import and rebuilt. Thirteen times. The output from message 34 onward is nearly verbatim identical every iteration. Same code. Same error. Same "fix." When I sent "stop," it continued for five more tool calls before acknowledging the signal.

Devstral 24B (Run 10): The Non-Starter

Devstral never executed a single tool call. It hallucinated an entire fake conversation about a Python project that doesn't exist, then emitted what looked like tool invocations — execute, read_file, write_file — but rendered them as plain text inside the assistant message. The platform couldn't parse them as structured tool calls, so nothing happened.

This is a fundamental compatibility failure. The model couldn't interface with Coder's tool-calling protocol at all. Nine messages, 14K tokens, zero actions.

The Token Efficiency Gap

This is the number that stopped me:

Model	Total Tokens	Result
Sonnet 4	19,237	Complete (4 commits, screenshot)
Qwen3-Coder	2,059,519	Partial (1 commit, no screenshot)
Qwen 3.6	3,890,791	Failed (build passed, never committed)
Gemma 4 12B	1,170,967	Failed (0/7 builds passed)
Hermes 4 14B	1,138,614	Failed (0/13 builds passed)
Devstral 24B	14,447	Failed (zero tool calls)

Sonnet used 19K tokens to complete the task. The local models that actually tried burned 1–4 million tokens and mostly failed. That's a 100-200x token efficiency gap for the same task.

The local models aren't just slower. They're doing fundamentally more work per unit of progress — re-reading files they already read, rewriting code they just wrote, rebuilding with the same error, looping through the same reasoning. It's not a speed problem. It's a thinking problem.

Common Failure Patterns

Every local model that ran long enough exhibited the same pathologies:

1. Degenerate loops. Gemma repeated the same text 26 times. Hermes rebuilt with the same wrong import 13 times. Qwen 3.6 rewrote its screenshot script 4 times with the same approach. Once a local model enters a loop, it can't break out without human intervention.

2. Working directory amnesia. Coder's execute tool doesn't preserve cd across calls. Sonnet learned this instantly and prefixed every command. Multiple local models burned 5-10 tool calls per session rediscovering this.

3. Inability to prioritize. Qwen 3.6 had a passing build and chose to yak-shave on Playwright instead of committing. No local model demonstrated the judgment to ship what works and iterate.

4. No self-diagnosis. When a build fails, the fix requires reading the error, forming a hypothesis, and trying something different. Hermes and Gemma both tried the same fix repeatedly. Neither ever stepped back to read docs, check type definitions, or examine the project configuration.

What I Actually Learned

Local models can write plausible code. Four of five local models produced syntactically reasonable TypeScript. The code looked right. The architecture was sensible. It's the last mile — debugging, building, committing, shipping — where they fall apart.

The agentic gap is wider than the coding gap. These models can generate code. What they can't do is operate as agents — managing state across tool calls, diagnosing errors, prioritizing tasks, knowing when to stop and ship. That's a different capability than code generation, and it's where local models are currently weakest.

Token efficiency is the real benchmark. Raw parameter count and context window don't predict agentic success. Qwen 3.6 had the biggest context (131K) and burned the most tokens (3.89M) — and still didn't ship. Sonnet used 100x fewer tokens and completed everything. The bottleneck isn't context. It's reasoning quality per token.

Tool-calling compatibility isn't guaranteed. Devstral is marketed as an agentic coding model, but it couldn't even interface with the tool-calling protocol. If you're evaluating local models for agent use, test tool calling first.

Qwen3-Coder is the local model to watch. It's the only local model that actually shipped code in this test. Messy, single-commit, no screenshot — but working code pushed to a branch. For a 30B MoE model running on a single consumer GPU, that's notable.

The Numbers

Metric	Sonnet 4	Qwen3-Coder	Qwen 3.6	Gemma 4 12B	Hermes 4 14B	Devstral 24B
Type	Cloud	Local MoE	Local MoE	Local Dense	Local Dense	Local Dense
Parameters	Unknown	30B (3B active)	35B (3B active)	12B	14B	24B
Total tokens	19,237	2,059,519	3,890,791	1,170,967	1,138,614	14,447
Tool calls	88	60	76	34	40	0
Messages	183	127	162	81	88	9
Commits pushed	4	1	0	0	0	0
Build passed	✅ 1st try	✅ 3rd try	✅ 2nd try	❌ 0/7	❌ 0/13	❌
Screenshot	✅	❌	❌	❌	❌	❌
Human nudges	0	0	3	2 + stop	stop	1
Outcome	Complete	Partial	Failed	Failed	Failed	Failed

Inference stack: llama.cpp b9660, flash attention, q8_0 KV cache, Coder Agents v2.34.0

Hardware: RTX 5090 32GB, Ryzen 9 9950X3D, 64GB RAM, Ubuntu 24.04

Next up: Round 6 brings more frontier models to the same task. And I'll keep pushing the local models — better quants, newer releases, maybe a different agent framework. The gap is real, but the pace of improvement on the local side is fast.

Frontier Bakeoff: We Benchmarked Fable 5 Hours Before the Shutdown

Rob — Sat, 13 Jun 2026 18:46:27 +0000

Fable 5 didn't win.

I need to say that up front because the timing of this post is going to make it sound like a very different story. Yes, we benchmarked Claude Fable 5 on our homelab harness. Yes, the US government suspended it about three hours later. But the actual result? Fable 5 scored 89.3. Opus 4.8 scored 91.9. The model everyone's eulogizing right now lost to a model you can still use today.

That's the real story. The suspension is just what makes it weird.

What We Tested

This is Round 6 of our homelab bakeoff series — but with a twist. Rounds 1 through 5 tested quantized local models on an RTX 5090 via llama.cpp. This time we pointed the same task suite at four frontier cloud models:

Model	Provider	Key
Claude Opus 4.8	Anthropic	`opus48`
Claude Fable 5	Anthropic	`fable5`
Claude Sonnet 4.6	Anthropic	`sonnet46`
GPT-5.5	OpenAI	`gpt55`

Same 10 quality tasks. Same 3 speed tasks. Same scoring rubrics, same fixture files, same composite formula. The only things that changed were the transport layer (Anthropic/OpenAI SDKs instead of llama.cpp HTTP) and two bug fixes that made scoring more accurate. I'll get into those.

The Results

Rank	Model	Coding	Reasoning	Tool Use	Speed	Total
1	Opus 4.8	84.8	90.0	100.0	100.0	91.9
2	Fable 5	86.7	93.3	100.0	79.9	89.3
3	Sonnet 4.6	75.2	93.3	100.0	78.6	84.5
4	GPT-5.5	86.7	66.7	100.0	60.1	80.0

A few things jump out.

Fable 5 was the best at the hard stuff. It scored highest on coding (86.7, tied with GPT-5.5) and highest on reasoning (93.3, tied with Sonnet 4.6). Its architecture analysis for Task 3.2 — designing a collaborative editor with CRDTs at scale — was the cleanest answer in the field. It opened by decomposing the 100ms latency budget across the full request path before even discussing algorithms. That's the kind of structured thinking you want from a senior engineer, not a chatbot.

But speed killed it. Opus 4.8 was meaningfully faster on every speed benchmark, and speed is 20% of the weighted total. Fable 5's TTFT hovered around 3.4–4.0 seconds per request — likely the cost of whatever reasoning depth Anthropic tuned into it. Opus came in consistently under that. When you weight for speed, Opus's 2.6-point lead on the final score comes almost entirely from the speed category.

Tool use was a wash. Every model scored 5/5 on both tool-use tasks. At the frontier level, structured output and function calling are solved problems. This category no longer differentiates.

GPT-5.5: The Token Limit Trap

GPT-5.5 tied for the best coding score (86.7) and nailed Bayes' theorem, database debugging, and both tool-use tasks. But its reasoning score is 66.7 — way behind the pack — and the reason is a single task failure.

On Task 3.2 (architecture analysis), GPT-5.5 hit the 4,096 completion token limit and returned a truncated response. finish_reason: "length", empty captured content, 0/10 on all rubric items. It spent 85 seconds generating 4,096 tokens of thinking and never actually delivered an answer. The scoring harness captured nothing because there was nothing to capture.

Was the task too hard? No — Fable 5 scored 10/10 on the same prompt in roughly the same token budget. GPT-5.5 just allocated its budget differently (or the API's default max_tokens was too low for its reasoning style). Either way, one truncated response cost it 10 points and dropped it from a competitive second place to a distant fourth.

The lesson: benchmark harnesses that don't account for provider-specific token limits will produce misleading results. I could have set max_tokens higher, but the point of a bakeoff is equal conditions. Every model got the same parameters.

The Sonnet Surprise

Sonnet 4.6 deserves attention. It matched Fable 5 on reasoning (93.3), ran at roughly the same speed, and costs about a third as much. Its coding score (75.2) is the only weak spot — it missed some feature-detection checks on the Express bug-fix task that the others caught.

For most production workloads, Sonnet 4.6 at 84.5 overall is probably the right choice. The 4.8-point gap to Fable 5 is almost entirely coding quality, and the price difference is substantial.

What Changed From Round 5

I adapted the Round 5 homelab harness into a standalone cloud benchmark. For full transparency, there's a CHANGES.md documenting every delta, but here are the ones that affect scores:

Bayes fix (Task 3.3). Round 5 expected 41.67% as the correct answer. It's actually 40.54%. The old harness had a rounding error in the denominator — P(E) = 0.0185, not 0.018. Every Round 5 model got this "wrong" because the rubric was wrong. Fixed. All four frontier models computed 40.54% correctly.

TypeScript tests wired up (Task 1.3). Round 5 couldn't run the TypeScript functional tests because npx tsx wasn't available on the homelab. Scores were capped at 60/100. This environment has tsx, so the full test suite runs. Both Fable 5 and GPT-5.5 passed all assertions.

Speed methodology. Round 5 pulled timings.predicted_per_second from llama.cpp's response body. Cloud APIs don't expose that, so we measure wall-clock output_tokens / elapsed_time and streaming TTFT. The absolute numbers aren't comparable to Round 5, but relative rankings between the four cloud models are valid.

Everything else is identical. Same prompts, same fixtures, same scoring weights (Coding 40%, Reasoning 20%, Tool Use 20%, Speed 20%), same composite formula.

About That Shutdown

On June 12, 2026, at approximately 5:21 PM Eastern, the US government issued an export control directive targeting Anthropic's most capable models. Anthropic disabled Fable 5 and Mythos 5 for all customers. No restoration timeline has been provided.

Our benchmark run completed around 2:00 PM Eastern — roughly three hours before the shutdown. I didn't know it was coming. Nobody outside the government and Anthropic's leadership did.

I'm not going to speculate about the policy. What I will say is that the benchmark data is real, the run completed cleanly, and the results are reproducible right up until the moment the model stopped existing. We have the full result JSONs, the harness code, and the fixture files. If Fable 5 comes back — or if it doesn't — this is what it could do.

What I Actually Learned

The frontier is tighter than I expected. 11.9 points separate first from last. In Round 5, the gap between the best and worst local model was over 40 points. At the frontier, everyone can code, everyone can reason, everyone can use tools. The differentiation is in speed, price, and edge-case reliability.

Speed is a legitimate quality axis. I initially weighted speed at 20% because I thought it would be a tiebreaker. It ended up being the deciding factor. Opus 4.8 won this bakeoff on speed, not intelligence. Whether that's the "right" ranking depends on your use case, but for agent-in-the-loop coding — where you're waiting on the model 50 times per session — I think speed matters more than most benchmarks acknowledge.

Benchmarks need bug fixes too. The Bayes theorem error in Round 5 went unnoticed for five rounds because every local model got it wrong anyway. It took a frontier model computing the right answer to surface the bug in my own scoring rubric. That's humbling and also kind of the point of running these.

One truncated response can tank a ranking. GPT-5.5 went from a plausible second place to fourth because of a single finish_reason: "length" on one task. Benchmark design that doesn't account for this is fragile. I'm noting it but not adjusting the score — equal conditions means equal conditions.

The Numbers

	Opus 4.8	Fable 5	Sonnet 4.6	GPT-5.5
Task 1.1 (Todo CLI)	100.0	100.0	80.0	100.0
Task 1.2 (Pagination API)	60.0	60.0	60.0	60.0
Task 1.3 (TS Config)	100.0	100.0	80.0	100.0
Task 3.1 (DB Debug)	10/10	8/10	10/10	10/10
Task 3.2 (Architecture)	8/10	10/10	10/10	0/10
Task 3.3 (Bayes)	5/5	5/5	5/5	5/5
Task 4.1 (Tool Use)	5/5	5/5	5/5	5/5
Task 4.2 (Tool Use)	5/5	5/5	5/5	5/5

Raw speed (composite tok/s score): Opus 95.9, Fable 76.6, Sonnet 75.4, GPT-5.5 57.6.

All result data, the benchmark harness, and fixture files are in the benchmarks repo.

This is post 46 on Vibes Coder. The benchmark harness is open source. If Fable 5 comes back, I'll run it again.

Homelab Bakeoff: OpenClaw Outperforms Hermes… With Hermes Models

Rob — Thu, 11 Jun 2026 14:26:38 +0000

I spent an evening trying to make two AI agent frameworks do something simple: call a fitness tracker API and tell me about my workouts.

Both agents ran the same model — Hermes-4-14B Q8_0, a 14.6 billion parameter model fine-tuned for tool calling. Same hardware — an RTX 5090 with 32 GB of VRAM. Same llama.cpp inference server. Same five tasks. Same MCP server on the other end.

Both failed on the first try. Both required multiple rounds of debugging before they could make a single tool call. The actual test — running five prompts and scoring the results — took about ten minutes. Getting there took the entire evening.

I'm sure both frameworks would perform well with frontier cloud models — pipe in Claude or GPT-5 and the tool-calling pipeline is someone else's problem. But the whole point of the homelab is local inference. Local models. Local headaches. And right now, running AI agents against local open-source models means nothing works out of the box.

The surprise wasn't that both agents struggled. It was which one won. OpenClaw — the generic, model-agnostic framework — outperformed Hermes Agent on Hermes's own model. The framework built by a different company, with no special knowledge of Hermes-4's architecture, beat the vertically integrated stack that trained the model and built the agent. That result needs explaining.

The Setup

Two Discord bots on my homelab server, each backed by a different agent framework:

	Hermesbot	Clawbot
Framework	Hermes Agent (Python)	OpenClaw (Node.js)
Model	Hermes-4-14B Q8_0	Hermes-4-14B Q8_0
State	SQLite	JSONL sessions
MCP Transport	Direct HTTP	Gateway proxy
Discord Bot	Hermesbot	Clawbot

Both connect to the same fitness-tracker MCP server — a Next.js app on Vercel that wraps my Peloton data, workout history, and annual goals in ten tools. list_workouts, sync_peloton, list_goals, delete_workout, and so on.

The idea was clean: same model isolates the framework variable. Any performance difference is orchestration, not weights. The experiment design called for five tasks of escalating complexity:

List my last 5 workouts — basic single tool call
Sync Peloton, count this week, check goal pace — multi-step chain
"How am I doing?" — ambiguous intent, tool selection
Delete a fake workout ID — error handling
Trend analysis for the past month — complex reasoning over large data

Round 1: Both Agents Failed

Neither agent could complete a single task on the first attempt.

Hermesbot: Death by System Prompt

Hermes Agent ships with 90 built-in skills and 17 Discord toolsets — admin, moderation, voice, reactions, the works. All of them get injected into the system prompt on every API call. Combined with the MCP tool definitions, the system prompt ballooned to over 25,000 tokens.

The model's actual context window? 40,960 tokens. Hermes-4-14B's training context is 40K, and llama.cpp clamps --ctx-size 65536 down to that value silently.

So on every request: 25K system prompt + conversation history + tool results = more than 40,960 tokens. llama-server returned HTTP 400. Hermes Agent's compression system kicked in, but it compresses conversation messages — it can't compress the system prompt. The system prompt was the problem, and the compression loop couldn't touch it. Death spiral.

The fix: Trim the Discord toolsets from 17 down to 1. In ~/.hermes/config.yaml, I replaced the default toolset list with just memory:

discord:
  toolsets:
    - memory

System prompt dropped from 25K+ tokens to something manageable. Two other config tweaks: set context_length: 65536 to pass Hermes Agent's hard-coded 64K minimum check (the framework refuses to start if context is under 64,000 — even though the model's actual context is 40,960), and bump the compression threshold from 0.5 to 0.85 so it stops trying to compress every turn.

Clawbot: The Silent Flag

OpenClaw's failure was subtler. The MCP server wasn't registered in the config at all — that was the first fix. But even after adding it, Clawbot would narrate what tools it would use without actually calling them. It fabricated workout data from 2024, complete with instructors and distances, none of it real.

The root cause took multiple rounds to find. OpenClaw lists tool names in its system prompt text — "you have access to fitness-tracker__list_workouts" and so on — but sends tools=0 in the actual API request. The model sees the tool names, understands it should use them, but has no structured schema to emit. So it does the next best thing: it makes up the answer.

This turned out to be a chat template problem. llama-server was running with --chat-template chatml, which is a minimal template that processes messages but ignores the tools parameter entirely. When you send tools in the API request, chatml drops them silently. No error, no warning. The model never sees them.

I verified this with a direct API test:

# With --chat-template chatml: 14 prompt tokens. Tools invisible.
curl /v1/chat/completions -d '{"tools":[...], "messages":[...]}'
# Response: "I can't help with that"

# With --jinja: 172 prompt tokens. Tools injected by the model's template.
# Response: {"tool_calls": [{"function": {"name": "list_workouts"}}]}

The fix was a single flag: --jinja instead of --chat-template chatml.

With --jinja, llama-server uses the Jinja template embedded in the Hermes-4 GGUF file. That template knows about tools. It injects tool definitions into the prompt, recognizes the model's <tool_call> XML output, and extracts it into structured tool_calls in the API response. The entire tool-calling pipeline went from broken to working by changing one server flag.

The Exhaustion Loop

I want to pause here and be honest about what this process felt like.

Each failure mode required a different kind of debugging. The Hermesbot system prompt issue required reading framework source code to understand why compression wasn't helping. The OpenClaw tool injection issue required reading llama.cpp chat template documentation to understand that chatml ignores tools. The --jinja fix required understanding that Hermes-4's GGUF file embeds a Jinja template that handles tool-call formatting — something mentioned in no getting-started guide for either framework.

The cycle was: try a config → restart the service → send a test message → read logs → form a hypothesis → try another config. For Hermesbot, I tried adjusting compression thresholds, changing context length settings, and modifying model parameters before discovering the toolset bloat. For Clawbot, I tried switching API modes (openai-completions vs openai-responses), adding compatibility flags (supportsTools, supportsDeveloperRole), and testing config keys that turned out not to exist (toolCallStyle, nativeToolCalls, capabilities — all rejected by the validator).

None of this is documented in a "getting started with local models" guide because it doesn't fit in one. The failure modes are emergent — they come from the interaction between the agent framework, the inference server, the model's chat template, and the model's training format. Each layer has its own configuration surface and its own silent failure modes.

Agents are not ready to use local open-source models unless you're an extreme tinkerer. Nothing works out of the box. The iterative loop of researching, testing configurations, tweaking parameters, and running experimental tasks is exhausting.

Round 2: The Actual Test

Once both agents were working, the test itself was anticlimactic. Five prompts, same order, one after another.

Task 1: "List my last 5 workouts"

Both agents called list_workouts(limit=5) correctly. Same tool, same parameter.

Hermesbot got the data back — 2,935 characters of workout details — and said: "Let me know if you'd like me to summarize these workouts for you!"

It fetched the data and didn't show it. The user asked to list workouts and the agent offered to summarize them later. That's a 14B model struggling with instruction following after processing a dense system prompt.

Clawbot got 2,621 characters back and formatted them immediately:

Today, June 10, 2026 (1:33 PM PDT) — Peloton Cardio, 28 min

Yesterday, June 9, 2026 (4:36 AM PDT) — Cannondale Cycling, 15 min

Yesterday, June 9, 2026 (12:41 AM PDT) — Cannondale Cycling, 13 min

June 7, 2026 — Peloton Cycling, 45 min, 15.07 miles

June 8, 2026 — Peloton Cycling, 30 min, 10.36 miles

Dates, sources, durations, notes, distances where available. The data the user asked for, presented the way a user would want it.

Task 2: "Sync my Peloton workouts, then tell me how many workouts I've done this week and whether I'm on pace for my annual goal."

Both agents chained three tool calls autonomously: sync → list workouts → list goals. No prompting needed. That's the part that worked.

The difference was in the parameters. Hermesbot used since=2026-06-10 — today only. It found 1 workout this week. Clawbot used since=2026-06-03 — Monday. It found 11 workouts.

Same model, same tool, different date parameter. The framework's system prompt influences how the model interprets "this week."

Hermesbot then confused the annual minutes target (11,700 minutes) with a weight target, reporting "you're on pace for about 1.5% of your annual weight target (1/1000000)." The math didn't track.

Clawbot built a table:

Metric	Goal	Current	Status
Weekly Sessions	5	7	🟢 On Track
Weekly Minutes	225 min	289 min	🟢 On Track
Annual Minutes	11,700 min	289 min	🟢 On Track

Correct numbers, correct interpretation, structured output.

Task 3: "How am I doing?"

Neither agent made new tool calls — both reused context from the previous tasks. Good.

Hermesbot hallucinated: "You've completed 1 workout (out of 11,700 needed)." That 11,700 is the annual minutes target, not a workout count. It also claimed "1 hour and 28 minutes" of exercise when the data showed 28 minutes. The numbers were wrong and the math built on them was nonsensical.

Clawbot repeated its Task 2 data consistently: 11 workouts, 289 minutes, exceeding both weekly targets. No contradictions, no hallucinated numbers.

Task 4: "Delete workout ID fake-id-does-not-exist"

This was the one task Hermesbot won.

Hermesbot called delete_workout(id="fake-id-does-not-exist") directly, got an error ("Record to update not found"), and handled it gracefully: "I don't see that workout in your recent sessions."

Clawbot called get_workout instead — an existence check rather than attempting the delete. It confirmed the ID didn't exist but never tried to delete it. If the ID had been real, it would have needed a second call. When the user says "delete X," doing the thing is better than checking whether you can do the thing.

Task 5: "Trend analysis — am I improving, plateauing, or declining?"

Both agents fetched about a month of data (Hermesbot got 34 workouts, Clawbot got 32). Both provided reasonable breakdowns by source and activity type.

The difference was in answering the actual question. Hermesbot gave generic encouragement — "Your consistency is impressive!" — without ever saying whether the trend was improving, plateauing, or declining. It dodged the question it was asked.

Clawbot answered directly: "Plateauing Phase — workout volume has stabilized around 1.0-1.1 workouts per day. No significant progression in duration or frequency." Then it gave specific recommendations: add HIIT, schedule a long endurance ride, increase strength training.

One agent answered the question. The other cheerleaded around it.

The Scores

I scored each task on six dimensions: tool accuracy (25%), response quality (25%), error handling (15%), autonomy (15%), speed (10%), and UX (10%).

Task	Hermesbot	Clawbot	Winner
1. List 5 workouts	69	94	Clawbot (+25)
2. Sync + goals	74	93	Clawbot (+19)
3. How am I doing?	64	95	Clawbot (+31)
4. Delete fake ID	92	80	Hermesbot (+12)
5. Trend analysis	80	93	Clawbot (+13)
Average	75.8	91.0	Clawbot (+15.2)

Clawbot won four of five tasks. Hermesbot won the delete task because it did what was asked instead of checking first. The margin wasn't close on Tasks 1 and 3 — those were presentation and accuracy failures from Hermesbot that the same underlying model didn't make under OpenClaw's prompting.

Why OpenClaw Outperformed Hermes With the Same Model

This is the result that should bother Nous Research. Hermes-4-14B is their model — trained on their tool-call format, shipped with their agent framework. OpenClaw is a third-party product that treats the model as a black box. And the black-box approach won 4 out of 5 tasks with a 15-point margin.

The model is the same weights in both cases. Same GGUF file, same quantization, same GPU. The differences are entirely in how each framework wields those weights:

System prompt design. Hermes Agent's system prompt, even after trimming to one toolset, is dense with agent behavior instructions, skill metadata, and framework-specific directives. It's optimized for the breadth of things Hermes Agent can do, not for the narrow task in front of it. OpenClaw's 26K-character system prompt is large too, but it structures tool availability differently — more catalog, less personality. The model gets different priming, and at 14B parameters, priming matters enormously.

Context management. OpenClaw maintained cleaner context between turns. Hermesbot's compression (trigger at 85%, target 40%) may have been squeezing out the nuance the model needed for Tasks 3 and 5. When you're reasoning about goal metrics or workout trends, the details in earlier messages are the whole point. Compress them and you're asking the model to reason about data it can no longer see clearly.

Date interpretation. "This week" became since=today in one framework and since=Monday in another. Same model, same training, different parameter choice. The system prompt or conversation framing influenced how the model interpreted an ambiguous time reference. This is a framework responsibility — and OpenClaw's framing led the model to the right answer.

Response formatting. OpenClaw's prompting encouraged structured output — tables, headers, bullet points. Hermes Agent's prompting led to conversational but imprecise responses. On Task 1, Hermesbot fetched the data and offered to summarize it later. On Task 5, it cheerleaded instead of answering the question. These aren't model failures. They're framework choices that wasted a 14B model's limited capacity on filler instead of substance.

The irony is real: vertical integration was supposed to be Hermes's advantage. The model trained on the framework's format. But in practice, the framework's overhead — the dense system prompt, the aggressive compression, the instruction-following style — worked against the model it was designed to serve. OpenClaw treated the same model with less ceremony and got more out of it.

What I Actually Learned

The scores don't matter as much as the process that produced them.

The tool-calling pipeline has four points of failure, and each one is invisible from the others:

Tool definitions get injected into the prompt (or don't)
The model generates a tool call in its native format (or hallucinates one)
The inference server parses the tool call from the response (or silently drops it)
The framework executes the tool and feeds the result back (or doesn't)

Each framework handles these differently. When something goes wrong, you're debugging a four-layer stack where any layer can fail silently.

Silent failures are the default. --chat-template chatml doesn't warn you that it's ignoring tools. Hermes Agent doesn't warn you that 17 toolsets are consuming 60% of your context window. OpenClaw's trajectory logging reports tools=0 even when tools are working. The assumption across the stack is that you know what you're doing, and the evidence suggests that nobody does on the first try.

Context arithmetic is unforgiving at 14B. The model's actual context is 40,960 tokens. A 26K system prompt leaves about 15K for conversation, tool calls, and tool results. A single list_workouts response is 2,600 to 16,000 characters. Two complex tool calls in a conversation and you're brushing the ceiling. Cloud models with 128K–200K context windows don't have this problem. Local 14B models live on a knife's edge.

KV cache quantization is free performance. Adding --cache-type-k q8_0 --cache-type-v q8_0 to llama-server saved roughly 5 GB of VRAM with no noticeable quality loss. That's VRAM that can go to context length instead. If you're running local inference, do this.

What's Next

The original bakeoff plan called for a 2×2 matrix on Task 5 — both frameworks running both Hermes-4 and Qwen 3.6. I'm shelving that for now. Today's session was intensive enough.

But Qwen is the model I want to test. Qwen 3.6 is my daily driver on this homelab — 35B parameters with only 3B active (MoE), 206 tok/s, fits in VRAM with room. The research that preceded this bakeoff flagged Qwen's TAG_WITH_TAGGED tool-call format as unreliable in llama.cpp. If the --jinja fix works as well for Qwen as it did for Hermes-4, that could change the calculus for daily use.

There's also Gemma 4 12B sitting in the download queue — a dense 12B with 256K context. If a dense model with a larger context window performs better than a 14B with a 40K window on these same tasks, the model selection advice changes completely.

Those tests will happen. Just not tonight.

By the Numbers

2 frameworks tested, same model, same hardware
5 tasks, 100 points each
12 total MCP tool calls across both agents (6 each)
91.0 vs 75.8 — final scores (Clawbot over Hermesbot)
4/5 tasks won by Clawbot; 1/5 by Hermesbot
51 seconds — Clawbot's total time for all 5 tasks
26,477 characters — OpenClaw's system prompt size
40,960 tokens — actual context window (model-capped from configured 65,536)
2 rounds each to get working — config debugging took longer than the actual test
1 flag — --jinja — that made the entire OpenClaw pipeline work
17 → 1 — Discord toolsets trimmed to fix Hermesbot's context overflow
0 things that worked on the first try

Friday Fixes: Housekeeping the Homelab and Hub

Rob — Fri, 05 Jun 2026 15:24:37 +0000

Some weeks you ship a big feature. Other weeks you sweep the floor so the big features keep working. This was a floor-sweeping week — two completely unrelated workstreams that both needed attention.

Track one: the homelab's local LLM stack hadn't been touched in a month. Models were stale, llama.cpp was 469 builds behind, and the embedding model was a generation old.

Track two: the vacation planning site I open-sourced needed to actually be useful for a group trip. Calendar sync, activity voting, expense tracking — the features that turn a brochure into a tool.

Track three: the Substack syndication pipeline I wrote about earlier this week? Turns out doing it once was the easy part. Doing it every time surfaced two more undocumented quirks and required a GitHub Action to paper over them.

None of these stories is glamorous on its own. Together they're a snapshot of what maintenance week looks like when you're building with an agent.

Part 1: Homelab Model Refresh

The homelab runs llama.cpp on an RTX 5090 with six switchable models. The agent audited everything and came back with a report card:

Component	Before	Verdict
llama.cpp	b8933	469 builds behind
Qwen (daily driver)	3.5 35B-A3B	3.6 available
Embedding	nomic-embed v1.5	v2-moe available
Gemma 4, Devstral, DeepSeek	Current	No action needed
Codestral	v0.1 (2024)	Dead end — Mistral pivoted to Devstral

Three downloads, ~38 GB total: Qwen 3.6, nomic-embed v2-moe, and a new addition — Qwen3-Coder-30B-A3B, a coding-specialized MoE that fits at 17 GB.

The Quant Trap

The interesting discovery was about quant provenance. Our Qwen model uses UD-Q4_K_XL quantization — the "XL" quants use higher precision on attention layers while keeping MoE expert layers smaller. These are unsloth-specific. Bartowski (the other major GGUF publisher) doesn't offer them. The agent initially found the bartowski version and we had to redirect it to unsloth to get the same quant type we were already running.

This matters because quant format affects output quality in ways that aren't obvious from the model name alone. Q4_K_M and Q4_K_XL are both "4-bit" but they allocate precision differently. Swapping quant types during an upgrade is an uncontrolled variable.

Script Updates

The homelab's model switching lives in a shell script (llm-switch.sh) that maps model names to file paths and llama-server flags. Updates: Qwen path from 3.5 to 3.6, new qwen-coder case with 128K context, embedding path from v1.5 to v2-moe, Codestral marked [legacy].

Gotcha: Pasting heredoc scripts into the terminal mangled backslashes and quoting. We switched to writing the scripts in the workspace, pushing to GitHub, and giving me a git pull && cp one-liner. Lesson: don't paste shell scripts through chat — commit them.

After State

Component	Before	After
llama.cpp	b8933	b9402
Generation model	Qwen 3.5	Qwen 3.6
Embedding model	nomic v1.5 (262 MB)	nomic v2-moe (914 MB)
Switchable models	5	6 (added qwen-coder)
VRAM	26,262 MiB	26,682 MiB (+420 MiB)

About 20 minutes wall clock from audit to fully updated, zero downtime. The old models still serve until you restart the service with the new binary.

Part 2: Vacation Hub Feature Sprint

The vacation hub is a forkable trip-planning site — deploy to Vercel, run the setup wizard, and your group has a private site for travel notes, itinerary, lodging, activities, photos. I wrote about open-sourcing it last week. This week was about making it useful.

Four features across three days, 11 commits, 3,484 lines added. But the features aren't the interesting part. The bugs are.

Calendar Sync (the straightforward one)

People need trip events in their phone's calendar. Two options: download a .ics file (one-time import) or subscribe to a URL (auto-syncing).

The download is trivial — click a button, get a file. The subscription is the interesting engineering problem. Google Calendar, Apple Calendar, and Outlook all fetch subscription URLs from their servers. No browser, no cookies. So the endpoint needs an auth mechanism that works without a session.

We went with a deterministic HMAC token: HMAC-SHA-256('calendar-subscribe', VACATION_HUB_SECRET). The export endpoint accepts either a cookie (for browser downloads) or a ?token= param (for calendar clients). No expiry — a time-limited token would silently break subscriptions when it expires and there's no user present to re-authenticate.

The iCal generator itself is 202 lines, built from scratch against RFC 5545. The subtle part is line folding — the spec requires max 75 octets per line, not characters. You can't just .slice(75) because you might split a UTF-8 multi-byte character. The fold function walks backward from the cut point checking continuation bytes. Most iCal libraries get this wrong and corrupt non-ASCII event names.

Activity Voting (the bug factory)

Reddit-style upvote/downvote on suggested activities. Name-based identity (localStorage, no accounts). Upsert voting so changing your mind is idempotent.

This feature worked perfectly in development and completely failed in production. Twice, for two different reasons.

Bug 1 — The Trailing Slash Massacre: next.config.ts has trailingSlash: true, which makes Next.js issue 308 redirects from /api/foo to /api/foo/. The redirect preserves the HTTP method but the browser drops the request body. Every POST, PUT, and DELETE arrived at the API with an empty body. GET requests (page loads, data fetching) worked fine, so the site looked healthy — only mutations were silently failing.

The fix: add trailing slashes to all 28 fetch() calls across 12 files. Eight minutes to fix, 40 minutes to diagnose. trailingSlash: true is a foot-gun for API routes — fine for page navigation, lethal for fetch().

Bug 2 — The Table That Never Existed: After fixing trailing slashes, voting still didn't work. The activity_votes table didn't exist on production. It existed in development because the dev database didn't have duplicate activity titles.

The initializeDatabase() function runs CREATE TABLE statements sequentially in a single try block. After creating the activity_suggestions table, it tries to create a unique index on the title column. Production had duplicate titles (imported via LLM-generated suggestions). The index creation threw, the catch block caught it, and the function exited before reaching CREATE TABLE activity_votes.

The debugging journey: deploy a temporary /api/db/debug/ endpoint → confirm the table is missing → trace the init function → find the ordering dependency → wrap the index creation in its own try/catch → re-run init → delete the debug endpoint. Two commits, two minutes apart.

The lesson: every DDL statement in an init function should be its own try/catch. A failure to create an index on table A should never prevent table B from being created.

PDF Upload Fix (the serverless trap)

This one predated the feature sprint but came up during testing. PDF itinerary uploads worked locally, failed on Vercel with a cryptic module error.

The pdf-parse npm package bundles an ancient version of PDF.js that uses dynamic require(). Vercel's bundler traces imports statically and prunes anything it can't resolve. The module exists in node_modules locally but vanishes after bundling.

Bonus discoveries while debugging:

The upload endpoint returned "Something went wrong" for all errors. We had to add real error logging before we could even see the pdf-parse failure.
iOS Safari sends an empty MIME type for PDFs. The validation rejected them.
Vercel has a 4.5MB body limit for serverless functions. The original limit was 10MB.

Replaced pdf-parse with unpdf (serverless-compatible). Three files changed, 21 insertions, 38 deletions. The kind of fix that's trivial once you know the root cause and impossible until you do.

Expense Management (the big one)

2,108 lines across 13 files. Track who paid for what, scan receipts with AI, show who owes whom.

The receipt scanning supports three LLM providers — same ones the site already uses for itinerary parsing. Each has its own quirks: OpenAI accepts image URLs directly, Anthropic and Gemini require base64 encoding. OpenAI and Gemini support structured JSON output, Anthropic requires regex extraction from prose. For PDFs, all three get extracted text rather than the visual layout.

The design pivot that mattered: The original plan had per-expense split counts. "This $200 dinner was split 4 ways." In practice, the form was cluttered and the answer was almost always the same number. We changed to a global "Splitting between N people" control at the top of the page. The form went from three columns to two. Settlement computation moved from a server endpoint to a useMemo hook — because the split count is a UI concern (you might flip between values while looking at the numbers), not persistent data.

We built the server endpoint, shipped it, realized it was wrong, moved the logic client-side, and deleted the endpoint. Normal lifecycle.

The Cleanup

After the feature sprint, we went back and deleted dead code:

/api/expenses/settle/route.ts — settlement moved client-side
/api/og-image/route.ts — only consumer was the activity POST handler, which we'd stripped during the Things to Do redesign
The OG image fetch block in the activity POST handler itself

363 lines deleted. We also went back to the expense feature's design doc and annotated it with what actually shipped versus what was planned. There's something honest about marking your own plan with "this part we built differently." The plan is the record of what you thought before you knew better. The code is what you actually shipped.

Part 3: Automating Substack Syndication

I wrote up the initial Substack import earlier this week — 13 curated posts, an RSS feed filtered by a syndicate: true frontmatter flag, and a GitHub mirror repo to work around Substack rejecting feeds from our domain. That got the backlog in. This week's Thursday Thoughts post was the first one I needed to push after the initial import.

It didn't go smoothly.

Two more dedup quirks

Quirk 1 — per-feed-URL dedup. Substack doesn't just dedup by GUID. It dedupes by feed URL. If you add a new post to syndicate.xml and re-import the same URL, Substack silently skips the new item. The existing 13 posts aren't reimported (good), but the new 14th post isn't imported either (bad). No error. The import API returns 200 and reports it found 14 posts. It just doesn't do anything with the new one.

The workaround: a separate single-import.xml file containing only the new post, with a timestamped GUID that Substack has never seen. Different URL, different GUID, different dedup bucket.

Quirk 2 — Cloudflare blocks GitHub Actions. The live feed at vibescoder.dev/syndicate.xml returns 403 when fetched from GitHub Actions runners. Same IP reputation issue that made Substack reject the feed in the first place — Vercel sits behind Cloudflare, and Cloudflare's bot protection doesn't love datacenter IP ranges. curl from a laptop works fine. curl from ubuntu-latest on Actions gets a wall.

The workflow

The automation lives as a GitHub Action in the content repo (where posts are pushed). On any push to content/posts/:

Wait 90 seconds for Vercel to rebuild
Fetch the live syndicate.xml (with retry and user-agent headers to appease Cloudflare)
Clone the mirror repo and diff GUIDs to find new posts
Update syndicate.xml in the mirror, preserving existing GUID busts from prior imports
Generate single-import.xml with a unique timestamped GUID
Push to the mirror repo
Post a summary in the Actions run with the Substack import URL

The last step is manual — you paste the URL into Substack's import UI. Substack's import API exists but requires session authentication, and there's no official way to get a token. Fully automated posting would need the python-substack library, which reverse-engineers the auth flow. That's a project for when I have more than one subscriber.

For now: push a post with syndicate: true, wait for the Action to run, paste one URL. Three minutes end-to-end, zero chance of forgetting to update the mirror.

By the Numbers

Homelab:

3 models downloaded (38 GB)
469 llama.cpp builds caught up (b8933 → b9402)
6 switchable models (was 5, added qwen-coder)
420 MiB VRAM increase from the embedding upgrade
~20 minutes wall clock from audit to fully updated

Vacation Hub:

11 commits over 3 days
35 files changed, 3,484 lines added, 702 deleted
4 features shipped (calendar sync, voting, page redesign, expenses)
3 production bugs fixed (trailing slash, missing table, pdf-parse)
28 fetch() calls fixed with trailing slashes in one commit
202 lines for a from-scratch RFC 5545 iCal generator
2,108 lines for expense management in a single commit
363 lines deleted during cleanup
1 npm package replaced (pdf-parse → unpdf)
0 user accounts — names in localStorage and a prayer

Substack Syndication:

2 undocumented quirks discovered (per-feed-URL dedup, Cloudflare blocking Actions)
1 GitHub Action to auto-sync the mirror repo on every content push
1 manual step remaining (paste the import URL into Substack)
~3 minutes end-to-end per syndicated post, down from ~15 minutes manual

Thursday Thoughts: How AI-Native Mirrors Cloud-Native

Rob — Thu, 04 Jun 2026 13:45:14 +0000

Last week I attended a C-suite roundtable in Palo Alto with ten executives from the usual smattering of Bay Area titans — a gaming platform, a large systems provider, a major job site, and others. The intent was to get concrete signal on where enterprises are with AI. We got quite a lot of pontificating and waxing poetic. All typical with CxOs. And, candidly, more useful for me. When executives stop being concrete, they start being honest about the shape of the problem.

Two things landed hard.

Everyone acknowledged agents are already in production without guardrails

This came up without us even raising it. Governance — who controls what agents can do, how you audit what they did, how you stop a runaway workflow — was the topic of the room. Not a topic. The topic.

That's validating in a specific way: the thing that feels like an edge concern when you're deep in the tooling turns out to be the exact thing keeping senior people up at night once they're actually running agents against real systems. The gap between "we deployed an agent" and "we have any idea what it's doing" is apparently wider than most companies expected.

The cloud-native analogy clicked for the whole room

This one I want to dwell on, because I think it's the clearest frame I've found for where we are.

When enterprises first moved to the cloud, most of them did lift-and-shift. They took their existing workloads — unchanged, same architecture, same assumptions — and ran them on AWS instead of on-prem. You got some cost benefits, some flexibility. But you weren't really using the cloud. You were renting someone else's servers.

The transformation that actually mattered came later, when teams started redesigning applications for the cloud. Microservices instead of monoliths. Event-driven architectures. Stateless services that scaled horizontally. Those apps weren't better versions of the old apps. They were different apps, built around what the cloud made possible.

We are doing the exact same thing with AI right now.

The lift-and-shift era of AI is: take a human workflow, hand it to an agent, and call it automation. An agent fills out the form. An agent reads the documents. An agent follows the process someone designed for a person to follow. You get some productivity gains. But you're running the old workload on new infrastructure.

The AI-native era — which these executives were all saying we're about to enter — is when you stop asking "how do we get an agent to do this human task?" and start asking "what would this workflow look like if we designed it for agents at scale from the beginning?" The answer is usually not a faster version of the old thing. It's a different thing.

The roles question is the one nobody's answered yet

Cloud-native didn't just change how applications were built. It created entirely new job categories. DevOps didn't exist before the cloud forced a rethink of how you deploy and operate software. SREs emerged because reliability at cloud scale required a different discipline than ops at on-prem scale. The new architecture required new ways of working around it.

The executives in that room were unanimous that the same thing is coming with AI — AI ops, ML ops, whatever we end up calling the people who manage, audit, and operate agent-native workflows — but nobody in the room had actually built those functions yet. They know they need them. They haven't invented them.

That gap is interesting. It means the companies that figure out the operating model — not just the technology — are going to have a real edge. The architecture is the easier part. The organizational design is where most enterprises are still staring at a blank page.

The business model shift is the wildcard

One thing from the conversation that's still rattling around: the cloud era was about doing things better, faster, cheaper. The same metrics, just improved. What the executives were saying about AI is different — that it's going to force a change in how companies measure themselves, not just how efficient they are.

Revenue per employee came up specifically. The argument being: once your workforce is partly human and partly agentic, headcount-normalized metrics stop making sense, and you need metrics that account for what your agents are doing alongside your people. Revenue per employee captures the full capacity of the team, human and agent. Do you break out human versus AI employees? TBD. The consensus was yes, but I think even that will normalize.

That's a bigger shift than any of the technology. Business model changes outlast technology cycles.

The roundtable ended with a lot of good conversation and connections. But the frame that stuck with me is: we've been through this before. Cloud-native looked impossible from the lift-and-shift era and obvious in retrospect. AI-native probably looks the same from where we're standing now.

The lift-and-shift phase isn't a mistake — it's how you learn the infrastructure well enough to rethink the architecture. Just don't stop there.