DEV Community: Rob

From Idea to Infrastructure: Standing Up a Self-Hosted AI Dev Environment

Rob — Thu, 07 May 2026 23:37:37 +0000

The first wave of content here on Vibes Coder was meta by design: a blog about building a blog, from a cabana in Cabo, on an iPhone. But that was always just the foundation. The thing I actually want to explore is local and self-hosted AI, and that starts with infrastructure.

This post is the journey from "I should build a home lab" to a fully running Coder server with GitHub integration, workspace templates, multi-user support, and AI agents that are genuinely useful out of the box. Everything here was done conversationally through Coder Agents.

Why Self-Hosted, Why Now

We're in the middle of an explosion in local hardware capabilities. Apple's shipped insanely powerful M-series silicon for generations. Qualcomm's latest Snapdragon Elite processors are serious. NVIDIA keeps pushing consumer GPUs with more VRAM, and is now getting into CPUs with the N1 chips. The combination of CPUs, GPUs, and NPUs available today far exceeds what standard productivity apps actually require.

It's pretty clear where this is heading: sophisticated LLMs running directly on our devices. I genuinely believe the future is Siri interfacing with a local model on an iPhone. A self-hosted home lab is the best approximation for testing that future before on-device capabilities go mainstream.

So I broke this into three phases:

Get the hardware capable of real inference
Build the dev environment to work with it (Coder, agents, templates)
Run local models and wire them into the coding workflow

This post covers phases 1 and 2. Phase 3 posts next.

The Hardware Hack: Buy a Gaming PC

How do you get a machine powerful enough for serious AI work when RAM, storage, and GPU prices are brutal?

Buy a pre-built gaming PC.

Individually sourcing components means paying extreme markups, thanks to AI's ripple effects on GPUs, memory, and storage. But a gaming PCs built a few months ago with all the latest parts are just sitting on shelves at Best Buy, Newegg, and Micro Center. These complete systems are actually worth more parted out than what they're selling for. The pricing is inverted.

I picked up a rig from Newegg. Here's what's inside:

Category	Component	Spec
CPU	AMD Ryzen 9 9950X3D	16-core Zen 5, 5.75 GHz boost
GPU	Zotac RTX 5090	32 GB GDDR7
RAM	G.Skill Trident Z5 RGB	64 GB (2x32 GB) DDR5-6000
Storage	Samsung 9100 Pro	2 TB Gen5 NVMe
PSU	Thermaltake Toughpower GT	1200W 80+ Gold ATX 3.1
OS	Ubuntu 24.04 LTS	NVIDIA driver 590.48.01

The spec that matters most for local LLMs is VRAM. The 32 GB on the RTX 5090 is the sweet spot: enough to run 27B-35B parameter models at full quality, or 70B models at aggressive quantization. The 64 GB system RAM provides headroom for KV cache spillover, and the 2 TB NVMe means models load fast and you can store plenty without worry. More on all of that in the next post. Now I have a capable AI workstation, in all it's RGB puke glory.

But a powerful machine sitting in a closet isn't useful until you can actually develop on it. That's where Coder comes in.

Standing Up Coder

I installed Ubuntu on the workstation. Why Ubuntu? It has the most documentation and is often what surfaces first in searches. Basically, it's the most agent-friendly distro. I didn't want troubleshooting my deployment with an agent to conflate Mint or Pop!_OS solutions. This was pretty straight forward minus a snafu getting the RTX 5090 drivers. Ends up you have to the install the open ones, and not the NVIDIA proprietary ones. Thankfully my motherboard had a built-in HDMI port I could use with the Ryzen's iGPU.

15 minutes later I connected my Ubuntu workstation via a Coder tunnel. This gives me a full cloud development environment accessible from anywhere, including my phone. Workspaces run as Docker containers on the machine, each with its own isolated environment, tools, and credentials.

The goal: anyone who creates a workspace from the template gets GitHub access, a full toolchain, and AI agents that know how to use everything, automatically. No manual setup.

GitHub Auth: The Long Way Around

The first task was connecting Coder workspaces to GitHub so agents could clone repos, commit, push, and create PRs without manual token management.

I explored three options:

Personal Access Tokens — works but doesn't scale to multiple users
SSH keys — same problem
Coder External Auth (OAuth) — configure once on the server, every user authenticates through the browser with their own GitHub account

Chose option 3. Created a GitHub OAuth App, configured the callback URLs, and started fighting with the server configuration.

The first struggle: Coder wasn't running in Docker (just using Docker for workspaces). It was running as a manual coder server process. The config file at /etc/coder.d/coder.env existed but wasn't being loaded because the file uses VAR=value format without export, and source reads the file but doesn't export to child processes. Had to export the variables directly in the shell before running the server.

The plot twist: After all the OAuth App setup, I discovered the Coder version had a built-in default GitHub provider that was already enabled. Navigating to /external-auth/github just worked. Didn't even need the custom OAuth App.

Lesson: Check coder server --help before manually configuring things. Or, realistically, ask your agent to do it for you. The answer was in the flags the whole time.

Wiring GitHub Into the Workspace Template

Even after authenticating, workspaces didn't automatically have GitHub credentials available. The external auth token existed but nothing told git or gh to use it.

The fix was template changes to main.tf:

data "coder_external_auth" "github" {
  id = "github"
}

Plus injecting GITHUB_TOKEN into the agent's environment variables and adding a startup script that configures the git credential helper and installs the GitHub CLI.

The template workflow I learned:

mkdir -p ~/coder-templates/docker
cd ~/coder-templates/docker
coder templates pull docker .
# edit main.tf
coder templates push docker
coder update my-workspace  # critical — stop/start alone reuses the old version

That last line is a gotcha worth highlighting: stopping and starting a workspace doesn't update the template version. You must run coder update to apply new template changes to an existing workspace.

System Instructions That Actually Work

With GitHub fully wired up, agents still had a problem: they'd ask users to authenticate or provide tokens. They didn't know the environment was pre-configured.

The fix was adding system instructions in the Coder admin panel (Agents > Settings > Behavior) that apply to all users. The key points:

GitHub access is pre-configured. Never ask users to authenticate.
Use gh CLI for all GitHub operations.
Always commit and push. Workspaces are ephemeral; GitHub is the source of truth.
Bias toward action. Build first, ask questions only when genuinely ambiguous.
Do the full loop: write code, install deps, test, commit, push.
Install tools with sudo as needed without asking permission.
Don't ask "would you like me to..." for obvious next steps.

This is the difference between an agent that's technically capable and one that's actually useful. Without these instructions, every session started with five minutes of the agent asking permission to do things it already had access to do.

The Vibe Coding Toolchain

The base Docker image was missing most of what a modern coding session needs. Added to the startup script:

Tool	Why
GitHub CLI (`gh`)	Repo management, PRs, issues from the terminal
Node.js + npm	Most web projects need it
Vercel CLI	Deploy directly from the workspace
uv	Fast Python package manager for new projects
zip, unzip, sqlite3	Common utilities that were missing

All installs are idempotent (if ! command -v ... &> /dev/null) so they only run on first boot.

Multi-User Setup

The real test: could my partner use the same server with her own account and GitHub credentials?

Setup was three steps:

coder users create on the host
She logs in, creates a workspace from the Docker template
Visits /external-auth/github once to link her GitHub account

Everything else (gh, git credentials, Vercel, system instructions) was automatic from the template. That's the whole point of doing this at the template level rather than per-workspace.

The Architecture

Here's what we ended up with:

Ubuntu AI Workstation (home lab)
├── coder server (running via tunnel)
│   ├── Built-in GitHub OAuth provider
│   ├── Agents with system instructions
│   └── Docker template
│       ├── GITHUB_TOKEN auto-injected per user
│       ├── gh CLI pre-installed
│       ├── Node.js + npm + Vercel CLI
│       ├── Python 3.12 + uv
│       └── code-server (VS Code in browser)
├── Docker (runs workspace containers)
└── Coder tunnel (*.try.coder.app)

Every user gets their own isolated workspace with full GitHub integration, a complete toolchain, and AI agents that know how to use all of it. The server handles auth, templates handle environment setup, and system instructions handle agent behavior.

Gotchas Worth Knowing

A few things that cost us time:

source vs export: source /etc/coder.d/coder.env reads the file but doesn't export variables to child processes. If your env file doesn't use export statements, child processes (like coder server) won't see the values.
Template versioning: Stopping and starting a workspace reuses the old template version. You must run coder update <workspace> to pick up new template changes. This one bit us three times before it stuck.
Agents settings vs Deployment settings: They're in completely different places in the Coder UI. Agents settings control AI behavior; deployment settings control server config. Easy to confuse.
The built-in GitHub provider: We spent time creating a custom OAuth App before discovering Coder ships with a default GitHub provider that was already enabled. The --help output had the answer all along.
Agent session refresh: After template changes that modify environment variables, you need a fresh Agents session. The running session won't pick up the new values.

What's Next: Local LLMs

The hardware is ready. The dev environment is running. But right now, all the AI work is still going through cloud APIs: Claude for blog generation, Claude for coding agents.

Tomorrow, we change that.

The RTX 5090's 32 GB of VRAM is sitting idle, and there's an entire ecosystem of open-source models that can run locally on this hardware. We're going to install Ollama, pull a stack of models purpose-built for different coding tasks, and start wiring local inference into the development workflow.

If you've ever wondered what it takes to run a 35-billion-parameter model on consumer hardware, or whether local models can actually keep up with cloud APIs for real coding work, that's what we're testing next.

By the Numbers

1 gaming PC purchased from Newegg
1 Coder server running via tunnel
1 GitHub OAuth integration (built-in, no custom app needed)
1 workspace template with 6 pre-installed tools
2 users configured
3 template pushes to get everything right
~15 minutes debugging export vs source
0 lines of code written outside of Coder Agents
32 GB of VRAM waiting for local models

Putting the GPU to Work: Running Local LLMs on a Home Lab

Rob — Thu, 07 May 2026 23:37:04 +0000

Yesterday we went from a gaming PC on a shelf to a fully configured Coder server with GitHub integration, workspace templates, and AI agents. The dev environment is running. But the RTX 5090's 32 GB of VRAM has been sitting idle, and all the AI work is still going through cloud APIs.

Today, we change that. This session was about installing Ollama, choosing the right models for different coding tasks, getting local inference running on the workstation, and then wiring it all into Coder Agents so local models show up right alongside Anthropic in the model selector. Everything here was done conversationally through Coder Agents, same as always.

Why VRAM Is the Only Spec That Matters

Before pulling any models, it helps to understand the constraint you're optimizing around. For local LLMs, that constraint is VRAM. Not CPU cores, not system RAM, not disk speed. VRAM determines what models you can run, and model size determines how useful they are.

VRAM	What You Can Run
8-12 GB	7B models (Qwen3:8b, DeepSeek-Coder 6.7B)
16 GB	14B-20B models (DeepSeek R1 14B, Codestral 25.12)
24-32 GB	27B-35B models, the sweet spot for agentic coding
32 GB+ / unified	70B quantized, Qwen3-Coder-Next

The 32 GB on the RTX 5090 lands squarely in the sweet spot. We can run 35B-parameter models at full quality, which is where the current generation of agentic coding models lives. The 64 GB of system RAM provides headroom for KV cache spillover when context windows get long, and the 2 TB NVMe means models load fast and we can store a whole library of them.

Only one generation model loads into VRAM at a time. Ollama automatically unloads the previous model when you switch. The embedding model is small enough to coexist with any of them.

The Setup Script

Rather than running commands one at a time, we built a single bash script that handles the entire setup in six phases: hardware verification, Ollama install, service configuration, model pulls, verification, and a connection reference card.

The script is tailored to this exact hardware profile but the structure works for any NVIDIA GPU setup. It supports --models-only (already have Ollama, just pull models) and --verify-only (check that everything is working) flags for re-runs.

#!/usr/bin/env bash
# Target: AMD Ryzen 9 9950X3D / RTX 5090 32GB / 64GB DDR5 / Ubuntu 24.04 LTS
set -euo pipefail

PRIMARY_MODEL="qwen3.5:35b-a3b"
CODING_MODEL="devstral-small:24b"
REASONING_MODEL="deepseek-r1:14b"
AUTOCOMPLETE_MODEL="codestral:22b"
EMBEDDING_MODEL="nomic-embed-text"
KEEP_ALIVE="30m"
OLLAMA_HOST="127.0.0.1"
OLLAMA_PORT="11434"

# Phase 1: Verify hardware & drivers
nvidia-smi --query-gpu=driver_version,name,memory.total \
  --format=csv,noheader

# Phase 2: Install Ollama
curl -fsSL https://ollama.com/install.sh | sh

# Phase 3: Configure service (keep models loaded 30min)
sudo mkdir -p /etc/systemd/system/ollama.service.d
sudo tee /etc/systemd/system/ollama.service.d/override.conf > /dev/null <<EOF
[Service]
Environment="OLLAMA_KEEP_ALIVE=${KEEP_ALIVE}"
Environment="OLLAMA_HOST=${OLLAMA_HOST}:${OLLAMA_PORT}"
EOF
sudo systemctl daemon-reload && sudo systemctl restart ollama

# Phase 4: Pull models
for model in "$PRIMARY_MODEL" "$CODING_MODEL" "$REASONING_MODEL" \
             "$AUTOCOMPLETE_MODEL" "$EMBEDDING_MODEL"; do
  ollama pull "$model"
done

# Phase 5: Verify
ollama run "$PRIMARY_MODEL" "What is 2+2? Reply with just the number."
curl -sf "http://${OLLAMA_HOST}:${OLLAMA_PORT}/v1/models"

# Phase 6: Connection reference
echo "Ollama API:    http://${OLLAMA_HOST}:${OLLAMA_PORT}"
echo "OpenAI API:    http://${OLLAMA_HOST}:${OLLAMA_PORT}/v1"
echo "Primary Model: ${PRIMARY_MODEL}"

The real script is ~330 lines with color output, error handling, idempotent checks, and flag parsing. This is the condensed version showing the actual work.

What Happened When We Ran It

Phase 1: Hardware Detection

All green:

[OK]  NVIDIA driver: 590.48.01
[OK]  GPU: NVIDIA GeForce RTX 5090 (32607 MiB)
[OK]  Driver 590.48.01 meets minimum requirement (550+).
[OK]  System RAM: 60 GB
[OK]  Available disk: 1717 GB

System RAM reports 60 GB instead of 64 GB. Normal; the kernel and firmware reserve some. Not a problem.

Phase 2: Ollama Install

One curl command. Ollama v0.21.0 installed cleanly, auto-detected the NVIDIA GPU, created a systemd service, and added the user to render/video groups.

Phase 3: Service Configuration

The important piece here is KEEP_ALIVE=30m. Without it, Ollama unloads models from VRAM after 5 minutes of inactivity. Loading a 23 GB model back into memory takes time, and if you're switching between coding and chatting every few minutes, you're hitting cold starts constantly. Thirty minutes keeps things warm during a real work session.

Phase 4: Model Downloads

~44 GB pulled. One failure:

Model	Size	Status	Notes
`qwen3.5:35b-a3b`	23 GB	OK	Primary agentic coder. MoE, only 3B params active per token.
`devstral-small:24b`	-	FAILED	Registry name wrong.
`deepseek-r1:14b`	9.0 GB	OK	Chain-of-thought reasoning.
`codestral:22b`	12 GB	OK	Fast autocomplete for IDE tab-completion.
`nomic-embed-text`	274 MB	OK	Embedding model for codebase search.

devstral-small:24b doesn't exist on Ollama's registry. The correct pull is ollama pull devstral. Registry names don't always match what blogs and guides reference. This is the kind of thing you only learn by running it.

Phase 5: Verification

The automated inference test returned empty. Cold-start timing issue: the bash $() capture returned before the model finished loading 23 GB into VRAM. Manual verification worked immediately after:

$ ollama run qwen3.5:35b-a3b "What is 2+2? Reply with just the number."
4

The OpenAI-compatible API endpoint confirmed working at http://127.0.0.1:11434/v1/models.

Why These Models

Every model in the stack was chosen for a specific job. This isn't a "download the biggest model that fits" strategy. Different tasks have different requirements, and the right model for autocomplete is not the right model for debugging a race condition.

Primary: qwen3.5:35b-a3b is the all-rounder. Best agentic coder available in April 2026 at this VRAM tier. Mixture-of-Experts architecture means only 3B parameters are active per token despite being a 35B model. That gives you big-model quality with small-model speed. 256K context window. Strong tool-calling support. Fits comfortably in 32 GB VRAM at ~22 GB.

Coding: devstral (Mistral's agentic coding model) is trained specifically for multi-file edits, terminal automation, and code repair. Benchmarks highest on Ollama for pure coding tasks. When you need raw code generation without the overhead of reasoning chains, this is the one.

Reasoning: deepseek-r1:14b is the chain-of-thought model. It thinks before answering. Slower, but catches bugs other models miss. At 14B it only needs ~12 GB VRAM, so it loads fast and leaves headroom.

Autocomplete: codestral:22b is optimized for fast inline code completion (fill-in-the-middle). Best fit for IDE tab-complete via Continue.dev. You want this model to be fast above all else.

Embeddings: nomic-embed-text is a lightweight (274 MB) embedding model for codebase search and RAG pipelines. Small enough to run alongside any generation model without VRAM pressure.

Wiring It Into Dev Tools

With Ollama running, everything that speaks the OpenAI API format can connect to it:

Ollama API:    http://127.0.0.1:11434
OpenAI API:    http://127.0.0.1:11434/v1
API Key:       ollama  (placeholder, not validated)
Primary Model: qwen3.5:35b-a3b

Continue.dev (VS Code)

name: Local Coder
version: 1.0.0
schema: v1
models:
  - name: Qwen3.5 35B (Chat/Edit)
    provider: ollama
    model: qwen3.5:35b-a3b
    roles: [chat, edit, apply]
  - name: Devstral (Coding)
    provider: ollama
    model: devstral-small:24b
    roles: [chat, edit]
  - name: Codestral (Autocomplete)
    provider: ollama
    model: codestral:22b
    roles: [autocomplete]
  - name: Nomic Embed
    provider: ollama
    model: nomic-embed-text
    roles: [embed]
context:
  - provider: code
  - provider: docs
  - provider: diff
  - provider: terminal
  - provider: codebase

Environment Variables

For scripts and agents that use the OpenAI client format:

export OLLAMA_HOST=http://127.0.0.1:11434
export OPENAI_API_BASE=http://127.0.0.1:11434/v1
export OPENAI_API_KEY=ollama

Connecting to Coder Agents

This is the real payoff. Ollama is running, the models are loaded, and the OpenAI-compatible API is live on localhost. Now we wire it into Coder Agents so local models appear as selectable options right alongside the cloud providers.

Coder Agents runs the LLM loop in the control plane, not inside workspaces. That means the Coder server process makes the API calls directly. Since Ollama and the Coder server are running on the same machine, this is just pointing one localhost process at another. No tunnels, no port forwarding, no API keys leaving the box.

Step 1: Add the Provider

In the Coder dashboard, navigate to Agents > Admin > Providers and select OpenAI Compatible. Coder treats any endpoint that implements the OpenAI chat completions API as a first-class provider.

Set the Base URL to http://127.0.0.1:11434/v1 and enter ollama as the API key. Ollama doesn't validate keys, but Coder requires one, so this is a placeholder.

For Key policy, keep the defaults: Central API key on, user API keys off. There's no reason for individual developers to bring their own key to a local Ollama instance. Everyone hits the same GPU.

Step 2: Add Models

Switch to the Models tab and add each model you want available in the Agents chat. The Model Identifier must match exactly what Ollama expects, because that string is sent directly to the /v1/chat/completions endpoint.

We added two models to start:

Model Identifier	Display Name	Context Limit
`qwen3:35b-a3b`	Qwen 3.5B	32,768
`devstral`	Devstral	131,072

The Cost Tracking, Provider Configuration, and Advanced sections can all be skipped for local models. No token pricing to track (it's your own GPU), and the default generation parameters work fine.

Step 3: Use It

That's it. The models now appear in the Agents model selector dropdown alongside the existing Anthropic models. Pick one, start a conversation, and the entire inference loop runs on the local GPU.

What Surprised Us

They work. Not "sort of work" or "work for simple prompts." The local models handle real agentic tasks through Coder Agents: reading files, running shell commands, editing code across multiple files, and reasoning about the results. Devstral in particular was impressive for code-focused work.

The latency difference compared to cloud providers is noticeable but not a dealbreaker. First-token time is slower because the model is running on a single consumer GPU rather than a cluster, but once inference is rolling, the throughput is solid. For the kind of iterative coding tasks Coder Agents handles, the tradeoff is worth it: zero API costs, zero data leaving your network, and no rate limits.

The practical recommendation: keep your cloud provider (Anthropic, OpenAI, whatever you're already using) as the default for complex, multi-step tasks. Use the local models for focused coding work, experimentation, and anything where you want to iterate fast without watching a billing dashboard.

Ollama vs vLLM: When to Scale Up

We chose Ollama because this is a single-developer workstation. Ollama wins on simplicity, resource efficiency, and single-user performance. One curl to install, one command to pull models, and it just works.

The tradeoff: if you later need to serve multiple concurrent Coder workspaces (5+ users hitting the same GPU), vLLM delivers roughly 16x more throughput under concurrent load. That's a future upgrade path, not a day-one requirement.

docker run --rm -it --gpus all --ipc=host --network host \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  nvcr.io/nvidia/vllm:26.01-py3 \
  vllm serve "Qwen/Qwen3-Coder-Next-FP8" \
  --served-model-name qwen3-coder-next \
  --port 8000 \
  --max-model-len 170000 \
  --gpu-memory-utilization 0.90 \
  --enable-auto-tool-choice \
  --enable-prefix-caching \
  --kv-cache-dtype fp8

Gotchas

Registry names lie. devstral-small:24b is what guides reference. devstral is what Ollama's registry actually has. Always check ollama search or the Ollama website before assuming a model name.
Cold starts kill scripted tests. Loading 23 GB into VRAM takes real time. If you're capturing output in a bash script with $(), the command can return before the model finishes loading. Manual ollama run works fine because it waits interactively.
KEEP_ALIVE is essential. The default 5-minute unload timer means constant cold starts during normal coding. Set it to 30m or -1 (indefinite) via the systemd override. This is the single biggest quality-of-life improvement.
60 GB != 64 GB is normal. The kernel and firmware reserve memory. Your 64 GB kit will report ~60 GB usable. This is expected, not a hardware problem.
Coder requires an API key even when the provider doesn't. Ollama doesn't authenticate requests, but Coder's provider config won't save without a key. Use any placeholder string. ollama works.
Model identifiers must be exact. The string you enter in Coder's admin panel is sent verbatim to the /v1/chat/completions endpoint. If you type qwen3.5:35b-a3b but Ollama expects qwen3:35b-a3b, you'll get a model-not-found error. Run ollama list and copy the name exactly.

What's Next

The models are running locally and wired into Coder Agents. We have a fully self-hosted AI coding environment: Coder server, Ollama, and local inference on the same box, with cloud providers as a fallback.

The next step is benchmarking. How many tokens per second does qwen3.5:35b-a3b actually push on this hardware? Is the 256K context window usable in practice, or does performance degrade at long contexts? Does codestral:22b autocomplete feel instant in the IDE, or is there noticeable lag? And the real question: for which tasks do local models match cloud providers, and where do they fall short?

Numbers coming soon.

By the Numbers

1 Ollama install (v0.21.0, single curl command)
5 models pulled (4 generation + 1 embedding)
~44 GB total model storage
32,607 MiB VRAM available
2 models configured in Coder Agents (Qwen 3.5B + Devstral)
1 model name that was wrong (devstral-small:24b -> devstral)
1 cold-start timing bug in the verification script
15 minutes from script start to working local inference
0 cloud API calls required
0 data leaving the network

The Agentic Gap: Claude Oneshots, Gemma Fails

Rob — Thu, 07 May 2026 23:36:32 +0000

Two days ago, Gemma 4 topped our local model benchmark — 167 tokens per second, perfect code quality score, smallest download. Faster than Sonnet. Faster than Opus. The blog post said "Gemma 4 is the new default."

Today we tested whether that's actually true.

The Experiment

Instead of another toy benchmark, we pulled a real item off the vibescoder.dev backlog: public-facing search across all blog posts. Multi-file feature, architectural decisions required, design system integration, no specification beyond "make search work."

Two models. Same prompt. Same codebase. Same workspace template. One shot — no follow-up instructions, no hand-holding. Walk away and see what happens.

	Gemma 4 27B	Opus 4.6
Provider	Ollama (local, RTX 5090)	Anthropic API (cloud)
Benchmark speed	167.1 tok/s	74.3 tok/s
Benchmark score	100/100	100/100
Cost	$0	Per-token pricing

The prompt was deliberately vague on implementation details:

Add public-facing search to vibescoder.dev. Users should be able to search across all published blog posts by title, content, tags, and description. The search should feel fast and match the site's existing Neon Brutalist design system. Consider: how users discover search, how results display, empty/no-result states, search state management (URL, keyboard shortcuts). Must be accessible from any page, work on mobile, and not introduce new design libraries. Commit and push when complete. Do not ask clarifying questions — make your own decisions.

Setting Up the Arena

Each model got its own Coder workspace with identical starting conditions: same Docker template, same base commit on main, same content repo.

Both workspaces built from the same Docker template — only the model selection differed.

We created two feature branches from the same commit (12fd589) and verified Vercel was configured to auto-build preview deployments for any branch push.

Vercel preview deployments would give us side-by-side URLs to compare the finished features.

Both prompts were delivered at the same time. Then we stepped back.

Opus 4.6: The Quiet Professional

Opus received the prompt and went silent. No questions. No plan narrated back. Just the spinning indicator showing it was working.

Over the next eight minutes, Opus:

Cloned both repos and installed dependencies
Read package.json, tsconfig.json, the app layout, existing components, lib/posts.ts, lib/types.ts, the design system in globals.css, and the middleware
Made architectural decisions: Cmd+K dialog with live API results for quick navigation, plus a full /search page for detailed browsing
Built a weighted scoring search API (/api/search) that ranks title matches above tag matches above content matches
Created a 407-line SearchDialog component with keyboard navigation, body scroll lock, abort controllers for in-flight requests, and ARIA accessibility
Built a server-rendered search results page with debounced URL state
Modified Header.tsx — three lines: import, component placement, mobile nav link
Updated middleware to whitelist the search API route
Committed everything in one clean commit and pushed

One prompt. One commit. 698 lines across 6 files. Pushed to GitHub, Vercel preview building.

 src/app/api/search/route.ts     | 104 ++++++++++
 src/app/search/SearchInput.tsx  |  78 ++++++++
 src/app/search/page.tsx         |  97 ++++++++++
 src/components/Header.tsx       |  10 +
 src/components/SearchDialog.tsx | 407 +++++++++++++++++++++++++++++++++
 src/middleware.ts               |   3 +-
 6 files changed, 698 insertions(+), 1 deletion(-)

What Opus Built

A Cmd+K search dialog. Press Cmd+K (or /) from any page and a full-screen overlay appears with a search input. Results appear live as you type, debounced at 200ms, with scores-based ranking. Arrow keys navigate results, Enter selects, Escape closes. The dialog shows up to 8 results with title, date, tags, reading time, and a context snippet showing where the match was found.

A full search page at /search. Accessible from the mobile hamburger menu and via a "View all results" link in the dialog. Server-rendered with URL state (/search?q=cloudflare). Shows the full PostCard component for each result — consistent with the blog's existing post listing.

A scored API route. Title matches score 100 points (150 for exact match). Tag matches score 50. Description matches score 25. Content matches score 10 plus 3 per occurrence. Results are sorted by score descending, capped at 20. The API strips markdown from content before extracting snippets.

The Header diff tells the story of good integration:

+import { SearchDialog } from "@/components/SearchDialog";
 ...
+          <SearchDialog />
           <ThemeToggle />
 ...
+          <Link href="/search" ...>Search</Link>

Three lines to wire a 407-line feature into the existing layout. That's how you integrate with a codebase.

Gemma 4: The Brilliant Planner Who Never Coded

Gemma's run was a different experience entirely.

Prompt 1 — the original task. Gemma cloned the repos, checked out the branch... and stopped. Checkmark. Done. It treated the setup commands as the entire task.

Gemma completed the setup and declared victory. The search feature hadn't been mentioned yet.

Prompt 2 — "You only completed the setup steps. Now read the codebase and implement the search feature." Gemma cloned the content repo, listed the directory, read package.json, wrote a thoughtful analysis plan (Architecture, Data Flow, Design System)... and stopped again. "I will begin by reading package.json."

Prompt 3 — "Keep going. Execute your plan — read the files, build the search feature, commit and push." Gemma read more files, outlined a detailed preliminary plan with four numbered steps... and stopped. "I will start by reading the core scripts directory."

Prompt 4 — "Do not stop between steps. Read all the files you need, write all the code, and push to git — all in one go. Do not pause or ask for confirmation."

The plan was actually good — client-side JSON index, search in header, /search?q=query routing. It just wouldn't execute it.

Gemma responded with its most detailed plan yet. A JSON search index builder, modifications to fetch-content.sh, a SearchInput component, Header integration, a search results page. Smart architectural choices. Then: "I will perform all these changes now." And stopped.

Prompt 5 — "Stop planning. Start coding. Write the files now."

After being told to code, Gemma showed code in the chat window instead of writing it to disk.

This time Gemma actually wrote some code — build-search-index.js, an edit to fetch-content.sh, and SearchInput.tsx. Three files to disk. Progress. Then it listed the three remaining tasks (Header, search page, commit) and stopped.

Prompts 6, 7, 8 — "Go." / "Go." / Explicit task list with three items. Gemma showed "Thinking..." briefly, then nothing. No output. No tool calls. The workspace eventually showed "unhealthy."

Eight prompts. Three partial files. Zero commits.

The AGENTS.md Experiment

Before giving up, we tried one more thing. We added explicit agentic behavioral instructions to AGENTS.md in the repo — the file that Coder agents read for project-level guidance:

# Agentic Execution Rules

You are an autonomous coding agent. Execute tasks end-to-end
in a single turn. Never stop to describe what you will do
next — just do it.

## What You Must Never Do

- Output a multi-step plan and then stop.
- Describe code you intend to write instead of writing it.
- Leave uncommitted changes in the workspace.

Started a fresh Gemma session with the same prompt. Same result. Clone, read package.json, plan, stop.

The instructions were clear. Gemma read them. And then it planned what it was going to do next and stopped.

The Scoreboard

	Opus 4.6	Gemma 4 27B
Prompts needed	1	8 (incomplete)
Files changed	6	3 (never committed)
Lines written	698	~150 (partial, uncommitted)
Commits pushed	1	0
Feature complete	Yes	No
Time to completion	~8 minutes	Never
Errors self-corrected	Yes (middleware, routing)	N/A
Design system match	Yes (Neon Brutalist tokens)	N/A
Keyboard shortcuts	Cmd+K, /, Escape, arrows	N/A
Mobile support	Yes (hamburger menu link)	N/A
Accessibility	Full ARIA	N/A

Technical Review: Opus Implementation

The code isn't perfect. A few things to fix before merging:

Duplicate search logic. The API route uses weighted scoring. The search page uses flat boolean filtering. Same query, different result order depending on which surface you use.
Unsafe type cast. post as Post in the search page strips content then casts back to Post, which expects a content field. Works at runtime but lies to TypeScript.
Missing Suspense boundary. useSearchParams() in SearchInput needs a Suspense wrapper for Next.js 14+.

But these are code review items — the kind of things you'd catch in a PR review and fix in 10 minutes. The feature works, the architecture is sound, the UX is polished.

Score: 87.5/100 across correctness (88), architecture (82), code quality (90), performance (85), completeness (92), and integration (91).

What We Learned

Benchmarks test generation, not agency. Gemma 4 writes excellent code when you tell it exactly what to write. That's what our todo-app benchmark measured — single-turn code generation from a clear spec. Agentic coding is a different skill entirely: reading a codebase, making decisions, chaining dozens of tool calls, self-correcting, and maintaining a plan across many steps. Gemma can't do that yet.

The plan-and-stop pattern is a model behavior, not a configuration problem. We tried explicit instructions ("do not stop"), behavioral directives in AGENTS.md, and increasingly urgent nudges. Gemma consistently planned what it would do, narrated the plan in detail, and then yielded control back to the user. It's not a token limit or context issue — it's how the model was trained to interact.

Speed doesn't matter if you can't finish. Gemma generates at 167 tok/s. Opus generates at 74 tok/s. But Opus delivered a complete, working, tested feature in 8 minutes with zero human intervention. Gemma delivered nothing usable in 20+ minutes with eight human prompts. The fastest model in our benchmark is the slowest in production.

The daily driver earned its spot. Opus 4.6 has been behind every line of code on vibescoder.dev since day one. This experiment didn't just confirm that choice — it quantified why. On a real task, the gap between "writes great code" and "builds great features" is the difference between a benchmark score and a shipping product.

Local models aren't there yet for agentic coding. This isn't a permanent verdict. Gemma 4 was released weeks ago. Agentic capabilities are the frontier every model vendor is racing toward. But today, if you need an AI agent that can autonomously build features, cloud models with tool-calling training (Claude, GPT) are still the only game in town.

What's Next

We have a working search feature on a Vercel preview branch, courtesy of Opus. Next step is reviewing the code, fixing the three issues identified, and merging it to production. vibescoder.dev gets search.

But we're not done with Gemma. The more we dug into the results, the more we think this shootout wasn't a fair fight. Our Gemma 4 deep dive ran Gemma through Ollama with default settings — and we've since discovered that Gemma's reasoning tokens are invisible but still eat your generation budget. With num_predict: 16384, the model may have blown its entire token budget on chain-of-thought we never saw, leaving nothing for actual code output. That would explain the plan-and-stop pattern perfectly: Gemma wasn't refusing to code — it was running out of room.

So we're rerunning the shootout. This time we're loading both models through llama.cpp directly, giving us fine-grained control over thinking budgets and VRAM allocation. We'll crank num_predict and num_ctx to 32K+, experiment with --reasoning-budget to cap invisible thinking tokens, and give Gemma the full 32 GB of RTX 5090 VRAM to work with. No more starving the local model on default settings and then calling it a fair comparison.

If Gemma was choking on its own reasoning, the fix might be as simple as giving it room to breathe. If it still can't finish — even with aggressive resources and tuned inference settings — then the agentic gap is real and it's in the model weights, not the configuration. Either way, we'll have a definitive answer.

By the Numbers

2 models tested head-to-head on a real feature
1 prompt — identical for both, no follow-ups allowed (for Opus)
8 prompts needed for Gemma before it stalled permanently
698 lines of working code from Opus
0 lines committed by Gemma
6 files changed by Opus (API route, search dialog, search page, input component, header, middleware)
3 files partially written by Gemma (never committed)
8 minutes from prompt to pushed commit (Opus)
20+ minutes of attempted nudging before calling it (Gemma)
87.5/100 technical review score for Opus implementation
407 lines in SearchDialog.tsx alone — keyboard nav, ARIA, scroll lock, abort controllers
3 code review items to fix before merging (duplicated logic, type cast, Suspense)
$0 spent on Gemma inference (also $0 of value delivered)
1 AGENTS.md rewrite attempted to fix Gemma's behavior (didn't work)
1 clear winner

Thursday Thought: Chat is the New Source Code

Rob — Thu, 07 May 2026 23:35:59 +0000

I just walked out of a customer meeting that completely shifted my perspective on the future of software development. What they told me sounds almost revolutionary, but it makes perfect sense when you think about it: chat is becoming the new source code.

The Paradigm Shift: From Code to Conversation

Here's what blew my mind. This customer explained that in their AI-agent-powered workflow, generating code has become the easy part. What's actually difficult—and incredibly valuable—is recreating the context, the intent, and the reasoning that led to that code.

Think about it: when you're working with an AI agent, the magic isn't just in the final output. It's in the entire conversation—the back-and-forth refinements, the clarifications, the "actually, let me change that" moments that shape the final solution.

Storing Chat History in GitHub: A Game Changer

This customer has started doing something fascinating: they store their chat histories directly in GitHub. Not just the code that results from those chats, but the entire conversational thread that led to it.

Why? Because they've discovered something profound:

They can fork chat conversations just like code branches
They can roll back to previous chat states
Most importantly, they can recreate any piece of code trivially from the chat history

It's like having a perfect record of not just what was built, but why it was built and how the thinking evolved.

Intent Over Implementation

This represents a fundamental shift in how we think about software development. We're moving from an implementation-first world to an intent-first world.

In traditional development:

Idea → Code → Version Control → Collaboration

In the new agent-assisted world:

Intent → Conversation → Code Generation → Chat History Storage

The code becomes ephemeral—easily regenerated. The conversation becomes permanent—the true source of truth.

The Future of Version Control

I predict we're going to see GitHub, GitLab, and other version control platforms rapidly evolve into something entirely different: extensible memory layers for agentic coding.

Instead of primarily tracking file changes, these platforms will become sophisticated conversation managers that can:

Branch conversations at any point in the dialogue
Merge different conversational threads when collaborating
Diff chat histories to see how approaches diverged
Replay conversations with different agents or parameters

What This Means for Developers

This shift has huge implications for how we work:

1. Documentation Becomes Native

The chat history is the documentation. No more outdated comments or README files—the reasoning is preserved in the conversation that created the code.

2. Collaboration Changes

Instead of reviewing pull requests, we might be reviewing conversation threads. "I see you took this approach in your chat with the agent, but what if we tried this angle instead?"

3. Debugging Gets Easier

When something breaks, you don't just look at the code—you look at the conversation that created it. The context and assumptions are right there.

The Big Picture

We're witnessing the emergence of conversational version control. Just as Git revolutionized how we think about code collaboration, chat-based development is about to revolutionize how we think about preserving and sharing intent.

The source code was never really the valuable part—it was always the human thinking behind it. AI agents are just making that distinction crystal clear.

What do you think? Are you ready for a world where your Git repos contain more conversations than code? Let me know in the comments—this feels like one of those moments where the industry is about to take a sharp turn, and I'm curious to hear how others are experiencing this shift.

Have you experimented with storing chat histories as part of your development workflow? I'd love to hear about your experiences and approaches.

Your AI Strategy Has a Blind Spot: An SEO and AEO Audit of vibescoder.dev

Rob — Thu, 07 May 2026 23:35:27 +0000

I spend a lot of time thinking about how AI agents discover and consume content. I run a company that builds developer tools. I write a blog about building with AI agents. And most importantly, I'm married to a woman that runs an AI consulting practice. Through the home-office wall I've heard her warn many a client that they have a silent suppressor in their content strategy if they're a Cloudflare customer. She recommends a site audit.

And she was right. Until this morning, every major AI crawler was blocked from reading my site.

Not by choice. Not by misconfiguration. By a Cloudflare setting I'd already turned off — that got silently re-enabled by a different setting I didn't know existed.

If you're a content creator, marketer, or engineer who cares about whether ChatGPT, Perplexity, Google AI Overviews, or Claude can find your work — read this. The infrastructure between your content and your audience may be working against you.

The TL;DR for Non-Technical Readers

If you don't want to read the whole audit, here's what matters:

Cloudflare's free tier blocks AI search engines by default. If your site uses Cloudflare (and millions do), your content may be invisible to ChatGPT, Perplexity, Claude, and Google's AI features — even if you never asked for that.
There are now two categories of discoverability. Traditional SEO (Google search results) and AEO — Answer Engine Optimization (AI-powered search and assistants). You need both. They require different things.
The fix for Cloudflare takes 60 seconds — but you have to know it exists. Go to Security → Settings → "Manage your robots.txt" and switch from "Instruct AI bots to not scrape content" to either "Content Signals Policy" or "Disable robots.txt configuration."
There's a new file called llms.txt that's becoming the robots.txt for AI. It tells AI agents what your site is, what it covers, and where to find content. If you don't have one, you're leaving discoverability on the table.

The TL;DR for Technical Readers

We ran a full SEO + AEO audit against vibescoder.dev and found 20 issues across 4 severity levels. The highlights:

4 P0 (critical): Cloudflare's managed robots.txt was blocking GPTBot, ClaudeBot, Google-Extended, and 5 others. RSS feed had wrong URL prefix (15 broken links). Sitemap.xml was referenced but returned 404. Duplicate User-agent: * blocks in robots.txt.
6 P1 (high): No JSON-LD structured data. No llms.txt. No canonical URLs. No heading anchor IDs. Missing article:author/tag meta. Homepage force-dynamic.
Everything was fixed in a single session — 17 files changed, 428 insertions, pushed and deployed.

The commit: SEO/AEO overhaul.

The Audit

I asked my Coder agent to evaluate vibescoder.dev on two dimensions: traditional search engine optimization (SEO) and Answer Engine Optimization (AEO) — making the site discoverable and citable by AI agents like ChatGPT Search, Perplexity, Google AI Overviews, and Claude.

The agent cloned the engine repo, crawled the live site, inspected every response header, parsed every meta tag, and cross-referenced the codebase against both SEO and AEO best practices.

The results were humbling.

The Cloudflare Gotcha (Yes, Again)

I wrote about Cloudflare's AI crawler settings two weeks ago. In that post, I specifically called out that Cloudflare's free tier has "Block AI bots" and "AI Labyrinth" turned on by default. I explicitly turned both off. I even wrote this:

"If your site exists for thought leadership, you want AI services to find, index, and cite your content. Blocking AI crawlers is blocking your distribution channel."

I was right. And I was still blocked.

The problem: Cloudflare has a separate setting called "Manage your robots.txt" under Security → Settings. It's not the same as "Block AI bots." It's a newer feature that injects directives directly into your robots.txt file at the edge — after your origin server responds.

Here's what the agent found when it compared my repo's robots.txt (100 bytes, 7 lines) to what Cloudflare was actually serving:

Metric	Value
My robots.txt	100 bytes, 7 lines
Content-Length header	100 (Vercel's original)
Actual response body	1,838 bytes, ~65 lines

Cloudflare was prepending 1,738 bytes of content — including Disallow: / rules for ClaudeBot, GPTBot, Google-Extended, Amazonbot, CCBot, Bytespider, and meta-externalagent — without updating the Content-Length header. The setting responsible? "Instruct AI bots to not scrape content," which was selected by default.

The fix: Security → Settings → "Manage your robots.txt" → select "Disable robots.txt configuration." This tells Cloudflare to stop modifying your robots.txt entirely. Your origin file gets served as-is.

Why "Disable" and not "Content Signals Policy"? The Content Signals option keeps a Content-Signal: ai-train=no directive, which tells AI crawlers not to use your content for model training. That sounds reasonable — but for a personal blog trying to maximize reach, being in the training corpus means AI models are more likely to know about you and reference your ideas. The risk it protects against (content absorbed without credit) is theoretical. The cost (reduced presence in AI systems) is concrete.

Gotcha #1: Cloudflare has three separate AI-related settings, and changing one doesn't affect the others. You need to check all three:

Setting	Location	What It Does
Block AI Bots Scope	Security → Settings	Deploys firewall rules blocking AI training crawlers
AI Labyrinth	Security → Settings	Injects fake content links to trap non-compliant bots
Manage your robots.txt	Security → Settings	Modifies robots.txt at the edge to add AI crawler directives

I had turned off #1 and #2 weeks ago. But #3 was still on — silently rewriting my robots.txt at the CDN layer.

Here's the full picture — the Security Overview flagging the AI-related action items, and each of the three settings:

What Is AEO?

AEO — Answer Engine Optimization — is the practice of making your content discoverable and citable by AI agents. (You'll also see it referred to as AI Engine Optimization or Agentic Engine Optimization — the discipline is new enough that the name is still settling.) It's the emerging counterpart to SEO. Where SEO focuses on Google's traditional index, AEO targets the systems that power ChatGPT Search, Perplexity, Google AI Overviews, Claude, and whatever comes next.

The key differences:

	SEO	AEO
Primary consumer	Googlebot	GPTBot, ClaudeBot, PerplexityBot, Google-Extended
Content format	HTML with meta tags	Structured data (JSON-LD), plain text (llms.txt), RSS
Discovery mechanism	Sitemap, backlinks, crawling	Sitemap, RSS, llms.txt, structured data
Ranking signal	PageRank, content quality, Core Web Vitals	Authorship (Person schema + sameAs), recency, structured data
Citation style	Blue link with snippet	Inline citation with direct quote and link
Key enabler	Canonical URLs, meta descriptions	JSON-LD, llms.txt, heading anchors for deep linking

You need both. Many of the improvements help both. But some are AEO-specific.

AEO-Specific Changes

These improvements specifically target AI agent discoverability:

llms.txt and llms-full.txt

llms.txt is an emerging convention — think of it as robots.txt for AI comprehension rather than crawling. It tells AI agents what your site is, what topics it covers, and where to find content.

We created two files:

/llms.txt — a structured summary: site description, author, topics, key posts, and links
/llms-full.txt — a dynamic route that serves every published post's full content as plain text

The full-content version is the important one. When an AI agent wants to cite your work, it needs the actual content — not just metadata. llms-full.txt is a single endpoint that gives it everything.

Person Schema with `sameAs`

JSON-LD structured data tells AI engines who wrote something and where else that person exists online. The sameAs property connects identity across platforms:

{
  "@type": "Person",
  "name": "Rob Whiteley",
  "url": "https://vibescoder.dev/about",
  "jobTitle": "CEO",
  "sameAs": [
    "https://www.linkedin.com/in/rwhiteley",
    "https://github.com/carryologist",
    "https://x.com/rwhiteley0"
  ],
  "worksFor": {
    "@type": "Organization",
    "name": "Coder",
    "url": "https://coder.com"
  }
}

When ChatGPT or Perplexity decides whether to cite "Rob Whiteley, CEO of Coder" in a response about AI-assisted development, this structured data is what gives it confidence in the attribution.

Full-Content RSS

The existing RSS feed only had <description> (a short excerpt). AI agents that consume RSS — and Perplexity in particular indexes it — get significantly more context from full-content feeds. We added <content:encoded> with the full post body, plus <author> and <managingEditor> tags.

Unblocking AI Crawlers

The Cloudflare fix described above. The single highest-impact AEO change — going from completely invisible to fully accessible.

SEO-Specific Changes

These target traditional Google search:

Sitemap.xml

robots.txt referenced it. It didn't exist. Every SEO tool and Google Search Console would flag this. We created src/app/sitemap.ts with dynamic generation — all posts, tags, and static pages with lastmod dates from the changelog.

Canonical URLs

No page had <link rel="canonical">. Without it, Google can treat URL variants (?utm_source=twitter, ?ref=hackernews) as separate pages. We added explicit canonical URLs to every page type — homepage, posts, about, tags, and individual tag pages.

Homepage Caching

The homepage was set to force-dynamic — every request hit the server with zero caching. For a blog that publishes daily at most, that's unnecessary. We switched to ISR with a 60-second revalidation window. (Vercel still serves it dynamically due to a cookies() call for admin detection — a future refactor.)

Custom 404 Page

The default Next.js 404 is a dead end. Our custom version shows recent posts and navigation links — keeping both users and crawlers moving through the site instead of bouncing.

Changes That Help Both

Most improvements benefit both SEO and AEO:

JSON-LD Structured Data

The single biggest miss. We added three schema types:

WebSite — site-level metadata with author info (every page)
BlogPosting — per-post schema with headline, dates, author, keywords, reading time (post pages)
BreadcrumbList — navigation hierarchy (post pages)

For SEO, this enables rich results in Google — article carousels, author info, breadcrumbs. For AEO, it's how AI engines understand content relationships and authorship with confidence.

Heading Anchor IDs

Added rehype-slug to the MDX pipeline. Every H2 and H3 now gets an auto-generated id attribute.

SEO: Google uses these for "jump to" links in search results and featured snippets.
AEO: AI agents cite specific sections via fragment URLs (#the-cloudflare-gotcha). Without heading IDs, citations can only link to the full page.

RSS Feed Fix

Every link in the RSS feed was a 404. The feed used /blog/ as the URL prefix, but the actual routes use /posts/. All 15 posts were broken. One-line fix, massive impact — RSS is a primary discovery mechanism for both Google and AI agents.

Article Meta Tags

Added article:author, article:tag, article:modified_time, and og:site_name to post OpenGraph metadata. These help both Google and AI engines categorize and attribute content correctly.

Image Improvements

MDX images now render inside <figure> with <figcaption> elements, and images without explicit alt text get an auto-generated fallback from the filename. Both changes improve how crawlers — traditional and AI — understand image content.

The Cloudflare Settings While We Were in the Dashboard

While fixing the robots.txt issue, we also optimized two other Cloudflare settings:

Early Hints — enabled. Cloudflare sends 103 Early Hints responses from the edge, letting browsers start loading fonts and CSS before Vercel even responds.
Smart Tiered Caching — enabled. Cloudflare edge nodes share cached content with each other, reducing origin hits. Ready to deliver benefits once ISR caching is fully enabled.
AI Labyrinth — confirmed still off. This injects fake content links to trap AI crawlers — the opposite of what a content site wants.

The Complete Scorecard

Every change, its impact, and whether it addresses AEO, SEO, or both:

Change	Impact	AEO	SEO
Disable Cloudflare managed robots.txt	Critical — AI crawlers could not access the site	✅	—
Fix RSS feed URLs (`/blog/` → `/posts/`)	Critical — all 15 RSS links were 404s	✅	✅
Create sitemap.xml	Critical — referenced in robots.txt but returned 404	✅	✅
Consolidate robots.txt (disable CF injection)	Critical — duplicate User-agent blocks caused ambiguity	—	✅
Add JSON-LD structured data	High — zero structured data across entire site	✅	✅
Create llms.txt + llms-full.txt	High — no AI discovery files existed	✅	—
Add canonical URLs to all pages	High — no page declared itself as canonical	—	✅
Add heading anchor IDs (rehype-slug)	High — no deep linking possible	✅	✅
Add article:author, article:tag to OG meta	High — tags and author missing from metadata	✅	✅
Add Person schema with sameAs	High — no cross-platform identity linking	✅	✅
Switch homepage to ISR (revalidate: 60)	Medium — every request was a cold server render	—	✅
Add RSS author + full content (`content:encoded`)	Medium — feed had excerpts only, no author	✅	✅
Add twitter:site and twitter:creator	Medium — social cards had no account attribution	—	✅
Create custom 404 page	Medium — default 404 was a dead end	—	✅
Wrap images in figure/figcaption	Low — bare img tags with no semantic context	✅	✅
Alt text fallback from filenames	Low — empty alt on content images	✅	✅
Remove x-powered-by header	Low — minor information disclosure	—	✅
Add humans.txt	Low — minor authorship signal	✅	—
Enable Cloudflare Early Hints	Low — browsers preload assets faster	—	✅
Enable Smart Tiered Caching	Low — prepared for when ISR is fully active	—	✅

Total: 20 changes. 13 help AEO. 17 help SEO. 11 help both.

What I Learned

AEO is a real discipline now, not a buzzword. The gap between "my content exists on the internet" and "AI agents can find, understand, and cite my content" is significant. Structured data, llms.txt, full-content RSS, heading anchors — these aren't nice-to-haves. They're the difference between being in the AI conversation and being invisible to it.

Your CDN can silently undermine your content strategy. This is the one that stings. I knew about the Cloudflare AI bot setting. I wrote a blog post about turning it off. And a different setting — one I didn't know existed — was doing the same thing through a different mechanism. If you use Cloudflare, check your robots.txt right now. Not the file in your repo — the one Cloudflare is actually serving. curl https://yoursite.com/robots.txt and compare it to what you expect.

The audit paid for itself in the first finding. Everything else — the JSON-LD, the canonical URLs, the sitemap — those are incremental improvements that compound over time. But the Cloudflare fix was binary: invisible → visible. Every day that setting was on was a day ChatGPT Search, Perplexity, and Google AI Overviews couldn't index my content.

What's Next

The one thing we identified but didn't implement: FAQPage schema for how-to posts. Several posts follow a problem/solution pattern that could surface as direct answers in AI search. The frontmatter already has a type field distinguishing how-to from opinion — the infrastructure is there. That's next.

By the Numbers

1,738 bytes of robots.txt injected by Cloudflare without updating Content-Length
8 AI crawlers blocked (GPTBot, ClaudeBot, Google-Extended, Amazonbot, CCBot, Bytespider, Applebot-Extended, meta-externalagent)
15 RSS feed links returning 404 — every single one
0 → 3 JSON-LD schema types (WebSite, BlogPosting, BreadcrumbList)
0 → 5 pages with canonical URLs
17 files changed, 428 lines added
3 Cloudflare settings that control AI crawlers — and you have to check all of them
60 seconds to fix the Cloudflare setting that was blocking all AI visibility
~2 hours for the full audit and implementation of all 20 changes
1 blog post that I thought had solved this problem — it hadn't

Invisible Failures: The Bugs That Hide in Plain Sight

Rob — Thu, 07 May 2026 23:34:54 +0000

Lucky you — bonus fix content, and you don't even have to wait until Friday.

I had a work trip to Austin coming up. The homelab was humming along at home, but I realized something uncomfortable: if anything went sideways while I was gone, I had no reliable way to fix it. SSH works when everything is running. SSH doesn't help when you need to see a stuck GUI dialog, a frozen window manager, or a service that needs a browser to configure.

So before I left, I fixed the access problem. Then, from a hotel room in Austin, I found and fixed three bugs that had been silently breaking things for days. None of them crashed. None of them logged errors. They just quietly did the wrong thing until the consequences finally became visible.

1. Setting Up Remote Access Before the Trip

I'd been putting off remote desktop because "I can always walk over to it." A trip to Austin fixed that mindset.

Evaluating options: Compared five tools. xrdp is a common recommendation but it's wrong for this use case — it spawns a new desktop session instead of mirroring the existing one. If something is stuck on the real display, xrdp can't help you see it. VNC works but it's laggy and unencrypted by default. Chrome Remote Desktop depends on Google's servers and a running Chrome instance. NoMachine is great but closed source.

RustDesk won: Open source, self-hostable, mirrors the real desktop, has iOS and macOS clients.

Self-hosted server: Two Docker containers — a rendezvous server and a relay server — so connections route through my own infrastructure instead of RustDesk's public relays:

sudo docker run -d --name rustdesk-hbbs \
  --restart always \
  -p 21115:21115 -p 21116:21116 -p 21116:21116/udp -p 21118:21118 \
  -v /opt/rustdesk-server:/root \
  rustdesk/rustdesk-server hbbs

sudo docker run -d --name rustdesk-hbbr \
  --restart always \
  -p 21117:21117 -p 21119:21119 \
  -v /opt/rustdesk-server:/root \
  rustdesk/rustdesk-server hbbr

Networking via Tailscale: Cloudflare Tunnels already handle coder.vibescoder.dev, but they only proxy TCP/HTTP — RustDesk's rendezvous server requires UDP on port 21116. Tailscale is a WireGuard mesh VPN that handles TCP and UDP natively. Each tool has its lane:

Cloudflare Tunnel → HTTP/S services (Coder dashboard, blog)
Tailscale → everything else (SSH, RustDesk, any UDP/TCP service)

Clients on macOS and iOS point at the workstation's Tailscale IP. A permanent password means fully remote access — no need to walk over and click "Accept" on a popup, which defeats the entire purpose of remote desktop for recovery scenarios.

Everything auto-starts on reboot: Docker containers with --restart always, RustDesk and Tailscale as systemd services. Five components confirmed persistent across power cycles.

The architecture now looks like this:

┌───────────────────────────────────────────────────────┐
│              HOMELAB (Ubuntu + RTX 5090)               │
│                                                       │
│  Coder Server ──── Cloudflare Tunnel ──── Internet    │
│  (systemd, :3000)   (TCP/HTTP only)                   │
│                                                       │
│  RustDesk Client ── Tailscale Mesh ──── MacBook       │
│  (systemd)           (TCP + UDP)        iPhone        │
│                                                       │
│  RustDesk Server (Docker, --restart always)            │
│  ├── hbbs (rendezvous, :21115-21116, :21118)          │
│  └── hbbr (relay, :21117, :21119)                     │
│                                                       │
│  llama-server (Gemma 4, :8080)                        │
│  Tailscale (systemd)                                  │
│  Cloudflared (systemd, tunnel)                        │
└───────────────────────────────────────────────────────┘

With that in place, I headed to Austin.

2. The Deploy That Only Fails on New Content

Three out of four Vercel deploys crashed with the same error:

Can't load image https://vibescoder.dev/images/downtime-is-a-feature/vercel-dns-ipv4-error.png: fetch failed
Error: Image size cannot be determined.
Export encountered an error on /posts/[slug]/opengraph-image/route

The blog generates dynamic OpenGraph cards for social sharing — each post gets a unique 1200×630 image with the title, description, and a faded background pulled from the post's first image. The OG image route extracts the first ![alt](/images/...) reference from the markdown and renders it.

The problem was how it loaded that image:

<img
  src={`https://vibescoder.dev${firstImage}`}
  style={{ width: "100%", height: "100%", objectFit: "cover", opacity: 0.15 }}
/>

During next build, this fetches from the live production site. For a new post, those images don't exist on production yet — they're only in the current build's public/ directory, copied there by the prebuild script. The fetch fails, next/og can't determine dimensions, and the entire build crashes.

The one post that succeeded? Its first image already existed on production from a previous deploy.

This is a classic chicken-and-egg bug. It only affects new content with new images. If you redeploy the same content twice, it works — because the first deploy put the images on the live site. You could publish for weeks without hitting it, then get three failures in a row when you finally add a post with a fresh screenshot.

The fix: Read from the local filesystem instead of fetching from production. The images are already in public/images/ at build time, so we read from disk and encode as a base64 data URI:

const imgPath = path.join(process.cwd(), "public", rawImage);
const buf = fs.readFileSync(imgPath);
const ext = path.extname(rawImage).replace(".", "").toLowerCase();
const mime = ext === "jpg" || ext === "jpeg" ? "image/jpeg" : `image/${ext}`;
firstImage = `data:${mime};base64,${buf.toString("base64")}`;

A try/catch around the read means a missing image degrades gracefully — no background in the OG card instead of a build-killing crash.

3. The Shell Guard That Eats Your Auth

While pushing the OG image fix, git push failed with an auth error. This had happened before. Every time, we'd manually run coder external-auth access-token github, paste the token into the remote URL, and move on. This time we decided to actually trace it.

Three layers of GitHub auth were configured in the workspace startup script, and all three were broken in agent sessions:

Component	What It Did	Why It Failed
`credential.helper`	Shell function reading `$GITHUB_TOKEN`	Env var was empty
`GITHUB_TOKEN` export	Appended to `~/.bashrc`	Below the interactive guard
`gh auth login`	Ran at startup	Token persisted, but `gh` also checks `GH_TOKEN` env

The startup script appended export GITHUB_TOKEN=... to ~/.bashrc. That line landed at line 118 — after the interactive guard at line 8:

case $- in *i*) ;; *) return;; esac

Every non-interactive shell — which is what Coder agent execute() calls use — bailed out at line 8 and never reached the exports. The credential helper then read an empty $GITHUB_TOKEN and returned an empty password. Git got a 401. The agent worked around it. Nobody noticed.

The fix was three changes:

Credential helper calls coder external-auth directly — no dependency on env vars, fresh token on every git operation:

git config --global credential.helper \
  '!f() { echo "username=x-access-token"; echo "password=$(coder external-auth access-token github 2>/dev/null)"; }; f'

Coder's env block injects tokens via Terraform — set in the agent process environment by Coder itself, inherited by every execute() call, no shell sourcing needed:

resource "coder_agent" "main" {
  env = {
    GITHUB_TOKEN = data.coder_external_auth.github.access_token
    GH_TOKEN     = data.coder_external_auth.github.access_token
  }
}

Cleanup of stale .bashrc entries — removed the old exports so they don't confuse future debugging.

The auth bug had been hiding for multiple sessions. The startup script looked correct. The agent silently worked around it every time. The fix was to stop relying on shell init files entirely and let Coder's process environment do the work at a level the shell can't interfere with.

4. The Date That Froze at Draft Creation

Published "The Agentic Gap" post. It went live. Then noticed the date: April 26 — three days ago. The post appeared sorted behind three days of other content instead of at the top.

The blog engine had no mechanism to update the frontmatter date when a draft is published. The flow:

Draft created → Claude sets date to the day it generates the content
Draft sits unpublished for N days
Someone flips published: false → published: true
Post appears on the blog sorted by its creation date, not its publish date

Two publish paths existed — the admin UI and the API — and neither touched the date. The admin UI did a single regex replace on the boolean. The API had a fixDateYear() function that corrects stale years (e.g., "2025" when it's 2026), but same-year drafts sailed right through.

The fix was two layers:

Client-side: handlePublish() stamps today's date immediately after flipping the boolean:

const today = new Date().toISOString().split("T")[0];
published = published.replace(
  /^date:\s*'[^']*'/m,
  `date: '${today}'`,
);

Server-side: A new stampPublishDate() function detects the false → true transition by comparing old and new content, and rewrites the date if it's a fresh publish. Safety net for all paths.

There's a third publish path — an agent directly editing the MDX and pushing to git, which is exactly what caused this bug. No code can fix that path. The fix there is process: the agent skill now instructs agents to always set the date to today when publishing.

The meta moment: the agent that introduced this bug also diagnosed, fixed, and documented it. In about fifteen minutes.

5. Capacity Planning: How Many Workspaces Can This Machine Run?

With the homelab now running Coder, Gemma 4 via llama.cpp, RustDesk, Tailscale, and Cloudflare — the question became: how much headroom is left?

The inventory: Ryzen 9 9950X3D, 64 GB RAM, RTX 5090 32 GB VRAM. Profiled from inside a container via /proc and from the host via Termius on an iPhone.

Key findings:

Category	Memory
Host services (GNOME, Coder, Docker, Tailscale, Cloudflare, RustDesk)	~5 GB
7 idle workspaces	~1.7 GB (230–270 MB each)
Gemma 4 (32K context, llama.cpp)	~19 GB VRAM — zero RAM impact
Available for workspaces	~58 GB

The GPU insight: Gemma 4 runs entirely in VRAM on the RTX 5090. Zero system RAM impact. Workspaces call it via the OpenAI-compatible API on the host. The 32 GB VRAM pool is completely separate from the 64 GB system RAM — running a local LLM doesn't reduce workspace capacity at all.

Capacity estimates: 8–12 active agent sessions comfortably (CPU is the bottleneck, not RAM). Dozens of idle workspaces parked at ~250 MB each.

Resource guardrails: Added an 8 GB per-container memory limit as a safety net — 32× normal usage, so it never constrains normal work, but a runaway npm install or memory leak gets OOM-killed cleanly instead of dragging the whole host into swap:

resource "docker_container" "workspace" {
  memory      = 8192  # MB — safety net, not a throttle
  memory_swap = 8192  # equal to memory = no swap for container
}

Autostop: 2-hour default TTL with 1-hour activity bump. Workspaces auto-stop when forgotten. All configurable via coder templates edit — template metadata, not Terraform.

What I Learned

Set up remote access before you need it. Every one of the fixes below happened from a hotel room because I'd set up RustDesk and Tailscale the day before I left. If I'd waited, I'd have come home to three days of broken deploys and a post sorted in the wrong place.

Invisible bugs are the most expensive. The deploy bug only hit new content. The auth bug was silently worked around. The date bug was three days stale before anyone noticed.

Shell init files are a liability. Anything that depends on .bashrc or .profile is fragile by default. Non-interactive shells, cron jobs, agent tool calls — none of them source your profile. If auth or config needs to be available everywhere, put it in the process environment or in a tool that's always in $PATH.

Self-hosted doesn't mean unmanaged. Adding RustDesk, Tailscale, and resource guardrails isn't gold-plating — it's the difference between a homelab that works when you're sitting in front of it and one that works when you're debugging from a phone in another room. Recovery access is table stakes.

The agent finds its own bugs. The publish-date bug was introduced by an agent, discovered by a human, then diagnosed, fixed, and documented by the same agent. That loop — agent ships, human reviews, agent fixes — is becoming the default workflow.

By the Numbers

1 trip to Austin that forced the remote access setup
4 invisible bugs found and fixed — three of them remotely
3 failed Vercel deploys from the OG image chicken-and-egg bug
3 layers of broken GitHub auth (credential helper, env var, gh CLI)
3 days a post was live with the wrong date before anyone noticed
~58 GB RAM available for workspaces after all services running
8–12 concurrent active agent sessions the workstation can handle
8 GB per-container memory limit as a safety net
5 tools evaluated for remote desktop — RustDesk won
2 Docker containers for self-hosted RustDesk server
5 services confirmed persistent across reboots
0 ports exposed to the internet (Tailscale mesh, Cloudflare tunnel)
~15 minutes from "the date is wrong" to fix deployed across both repos
~30 minutes from "how do I remote desktop" to working iPhone → Linux access
1 agent that introduced a bug, then found and fixed it

Slaying the Gemma Beast: How We Fixed Local AI and Shipped Search

Rob — Thu, 07 May 2026 23:34:22 +0000

Two days ago, Gemma 4 couldn't finish a feature. Today it built one, pushed it to GitHub, and it's live on this site right now.

If you press ⌘K (or Ctrl+K) on any page of vibescoder.dev, you'll see a search modal. Gemma 4 built that — running locally on an RTX 5090, zero cloud API calls, zero dollars spent. Then Claude reviewed the code, fixed the rough edges, and merged the polish. The feature you're using is a collaboration between a local model and a cloud model, each doing what they're best at.

Here's how we got there.

Previously: The Agentic Gap

In our last experiment, we pitted Gemma 4 against Opus 4.6 on the same task: build public-facing search for this blog. Opus one-shot it — 698 lines across 6 files, committed and pushed in 8 minutes. Gemma planned brilliantly, then stopped. Eight prompts later: 3 partial files, 0 commits.

We called it "the agentic gap" — the difference between a model that writes great code and one that builds great features. But we also left a thread dangling: maybe Gemma wasn't refusing to code. Maybe it was running out of room.

The Diagnosis

Our deep dive into Gemma 4's local inference uncovered the root cause: invisible thinking tokens consume your generation budget.

Gemma 4 defaults to a reasoning mode where it generates chain-of-thought tokens before producing visible output. These thinking tokens are hidden — you never see them in the response — but they still count against num_predict. With Ollama's defaults, the model was blowing its entire token budget on reasoning, leaving nothing for actual code.

That's not a model failure. That's a configuration failure.

The fix on paper was straightforward: give the model a bigger budget. But getting there required switching the entire inference stack.

Switching from Ollama to llama.cpp

Ollama is great for pulling and running models. It's not great for fine-grained control. The specific controls we needed:

Control	Ollama	llama.cpp
Context window (`num_ctx`)	Modelfile only	`--ctx-size` flag
Output limit (`num_predict`)	API parameter	`-n` flag + API
Reasoning budget	Not available	`--reasoning-budget` flag
Tool calling	Basic	Grammar-constrained

The --reasoning-budget flag is the key. It caps how many tokens the model can spend on invisible chain-of-thought, forcing it to start producing real content after hitting the limit. Ollama has zero equivalent.

The switch itself was an adventure. We couldn't use Ollama's blob files directly — llama.cpp expects standard GGUF files, but Ollama stores models in a split format that standalone tools can't load. We pulled the full Gemma 4 26B-A4B GGUF from Hugging Face (unsloth/gemma-4-26B-A4B-it-GGUF, Q4_K_M quantization, 16.9 GB download) and launched llama-server with tuned settings:

~/llama.cpp/build/bin/llama-server \
  -m ~/models/gemma4-26b/gemma-4-26B-A4B-it-UD-Q4_K_M.gguf \
  --ctx-size 32768 \
  -n 32768 \
  --reasoning-budget 4096 \
  --reasoning-format deepseek \
  --parallel 1 \
  --host 0.0.0.0 \
  --port 8080 \
  -ngl 999

Key settings:

--ctx-size 32768 — 32K context window. Fits comfortably at ~19 GB on the 5090.
-n 32768 — 32K max output tokens. Room for both reasoning and code.
--reasoning-budget 4096 — Cap invisible thinking at 4K tokens. The rest is for actual output.
--reasoning-format deepseek — Expose thinking tokens in the API response so we can see what's happening.
--parallel 1 — Single slot instead of default 4. Four slots × 32K context was causing OOM kills.

Then we pointed Coder at the new endpoint. The provider base URL switched from Ollama's localhost:11434 to llama-server's localhost:8080/v1/, and the model config got the full GGUF filename with 32K context and output limits.

Three Attempts to Slay the Beast

It didn't work on the first try.

Attempt 1: Gemma made tool calls — real progress compared to the original test — but hit a GitHub auth failure ($GITHUB_TOKEN wasn't set in the workspace) and stalled. The last output was raw token leakage: call:execute{command:<|">find... — special tokens leaking into the response, one of the known Gemma issues.

Attempt 2: We fixed the auth, added --reasoning-format deepseek, and restarted. Gemma got much further — wrote a search index generator, ran it, started exploring the codebase. Then llama-server got Killed — the OOM killer struck. Four parallel slots at 32K context each was too much VRAM.

Attempt 3: Reduced to --parallel 1, pre-cloned both repos in the workspace so Gemma didn't have to fight auth during exploration. This time it worked. Gemma laid out a clear implementation plan, and after one nudge — "keep going, don't stop, code and commit" — it executed the entire thing.

How Fast Was It?

In the Model Showdown Round 2, Gemma 4 clocked 167.1 tok/s on a short benchmark task via Ollama — the fastest perfect scorer. But a benchmark prompt and an agentic coding session are different workloads. How does Gemma perform when it's actually building something?

We ran fresh benchmarks against the llama.cpp server with coding prompts at different output lengths:

Task	Prompt Tokens	Output Tokens	TTFT	Tok/s
Short (debounce function)	29	512	27ms	179.2
Medium (React component)	63	2,048	28ms	177.3
Long (full Node.js script)	62	2,679	29ms	181.2

Three things stand out.

Time to first token is near-instant. 27–29ms TTFT means the streaming UI starts filling in almost immediately. For comparison, cloud models typically hit 500ms–2s TTFT depending on load and routing. On a local GPU, there's no network round-trip, no queue, no cold start.

Generation speed doesn't degrade. Whether Gemma is writing 512 tokens or 2,679 tokens, throughput stays locked at 177–181 tok/s. There's no slowdown as context grows — at least not at these output lengths. During the actual search build session, with thousands of tokens of accumulated context from tool calls and file contents, we observed ~159 tok/s. That's a ~12% drop from peak, which is expected: more context means more attention computation per token.

The reasoning budget has a real cost. With --reasoning-format deepseek, Gemma's thinking tokens are visible in the API response. On a short 256-token request, the model spent all 256 tokens reasoning and produced zero visible output. That's the invisible thinking token problem in action — and exactly why --reasoning-budget 4096 matters. Cap the thinking, and the remaining budget goes to code.

Metric	Ollama (Showdown R2)	llama.cpp (this session)
Tok/s (benchmark)	167.1	177–181
Tok/s (real workload)	N/A (failed)	~159
TTFT	3.92s	~28ms
Reasoning budget control	None	`--reasoning-budget 4096`

The TTFT difference is dramatic — 3.92s vs 28ms. Ollama's 3.92s likely included model loading or prompt cache misses. llama-server keeps the model hot in VRAM with a persistent prompt cache, so subsequent requests start generating almost instantly.

Bottom line: Gemma 4 on an RTX 5090 via llama.cpp generates code at ~180 tok/s peak, ~159 tok/s under real agentic load, with sub-30ms TTFT. That's fast enough that the model is never the bottleneck — tool execution (git operations, file I/O, npm installs) takes longer than inference.

What Gemma Built

Two prompts. One feature. Pushed to main.

 package-lock.json                | 466 +++++++++++++++++++
 package.json                     |   3 +-
 public/search-index.json         |  34 +++
 scripts/generate-search-index.ts |  40 ++++
 src/components/Header.tsx        |  32 +++
 src/components/SearchModal.tsx   | 216 ++++++++++++++++++
 6 files changed, 618 insertions(+), 173 deletions(-)

The architecture: a client-side Fuse.js search with a pre-generated JSON index. A build-time script reads all published posts and generates public/search-index.json. The SearchModal component loads this index on first open, runs fuzzy searches with Fuse.js, and renders results in a Cmd+K overlay.

Gemma even hit an authentication error during git push — and self-corrected. It ran coder external-auth access-token github, reconfigured the git remote with the token, and pushed successfully. That's agentic behavior — the thing that was completely absent in the original test.

The commit message: afb5c73 feat: add search functionality with Fuse.js. Vercel auto-deployed from main. The feature went live.

The Code Review: What Gemma Got Right and Wrong

Working code that ships is a milestone. But "it works" and "it's production-quality" are different standards. Claude reviewed every line of Gemma's implementation. Here's the honest assessment.

What Gemma Got Right

Architecture was sound. Client-side search with a pre-generated JSON index is the correct call for a 14-post blog. No server-side API needed, no database, sub-5ms search times. The index is ~130 KB — smaller than a hero image.

Component structure was clean. Separate SearchModal component, separate build script, clean Header integration. Three lines to wire it into the existing layout.

It used the existing design system. CSS variables like bg-surface, border-primary, text-on-surface — all from the Neon Brutalist theme. It read the codebase and matched the patterns.

Self-correcting on errors. When git push failed, Gemma diagnosed the auth issue and fixed it autonomously. Three tool calls: fetch token → reconfigure remote → push. No human intervention needed.

What Gemma Got Wrong

Zero accessibility. No role="dialog", no role="combobox", no aria-modal, no aria-activedescendant, no focus trap. A screen reader would have no idea this modal existed.

Broken exit animations. The AnimatePresence wrapper contained a regular <div> instead of a motion.div. When the modal closed, React unmounted the wrapper immediately, killing the exit animations before they played. The code looked right but didn't work.

Performance anti-pattern. A new Fuse instance was constructed on every keystroke. Fuse builds an internal index on construction — that's wasted work. Should be useMemo keyed on the index data.

Eager loading. The search index was fetched on every page load, even if the user never opened search. Should lazy-load on first modal open.

Wrong fonts. Applied --font-headline (Space Grotesk) to the entire modal including body text and descriptions. The codebase uses headline for titles only, with the default font for body text.

Ignored existing components. Rendered tags as raw <span> elements with custom styling instead of reusing the existing TagBadge component that already had the right design tokens.

Stale search index committed to git. The generated search-index.json was committed with 3 placeholder posts. It's a build artifact — should be in .gitignore.

Content truncated too aggressively. Each post's content was cut to 1,000 characters. Terms that only appeared deeper in posts (like "RustDesk" in our infrastructure writeups) were invisible to search.

The Polish Pass

Claude's fix addressed every issue in a single PR:

Accessibility: Full ARIA combobox pattern — role="dialog", role="combobox" on the input with aria-expanded/aria-activedescendant, role="listbox" and role="option" on results, aria-live="polite" for result count announcements.

Keyboard navigation: Arrow Up/Down to move through results, Enter to navigate, Escape to close. Active result scrolls into view automatically.

Performance: Fuse instance memoized with useMemo (rebuilds only when index changes). Index fetched lazily on first modal open. Minimum 2 characters before searching.

Search quality: Weighted field scoring — title matches score 3× higher than content matches, tags 2×, descriptions 1.5×. Markdown stripped from indexed content. Full post content indexed with no truncation.

Design system: Correct font usage matching PostCard patterns. TagBadge component reused. Platform-aware keyboard hint (⌘K on Mac, Ctrl+K elsewhere).

Animation fix: Outer wrapper is now a motion.div — exit animations actually play.

Cleanup: Body scroll lock, query cleared on close, build artifact gitignored, dead imports removed.

The polish commit: 383 insertions, 201 deletions across 5 files. The combined feature is 804 lines across 6 files.

Opus vs. Gemma+Opus: An Honest Comparison

We now have two complete implementations of the same feature. Opus 4.6's original branch (feature/search-opus46, 698 lines) is still in the repo. Here's how they compare.

Architecture

	Opus 4.6 (original)	Gemma 4 + Opus (shipped)
Search engine	Server-side API route with weighted scoring	Client-side Fuse.js with weighted config
Index	None — reads posts at request time	Pre-generated JSON, fetched once
Surfaces	Cmd+K dialog + `/search` page	Cmd+K modal only
URL state	Yes (`/search?q=cloudflare`)	No

Opus's architecture is more feature-complete. A dedicated /search page with URL state means search results are linkable and shareable. The server-side API route means the search logic runs where the content lives, with no index to generate or cache.

Gemma's architecture is simpler and arguably better for this scale. A static JSON index means zero server load, instant results, and the feature works on Vercel's free tier without hitting function invocation limits. At 14 posts and 130 KB, client-side search is the right call.

Code Quality

	Opus 4.6	Gemma 4 (raw)	Gemma 4 + Opus (merged)
Accessibility	Full ARIA, keyboard nav	None	Full ARIA, keyboard nav
Animation correctness	Correct	Broken exits	Fixed
Performance	AbortController for API calls	Fuse recreated per keystroke	Memoized, lazy-loaded
Design system	Mostly correct	Mostly correct	Fully correct
Known bugs	3 (duplicate logic, type cast, missing Suspense)	7 (see review above)	0

Opus's raw output was higher quality. Its SearchDialog had 407 lines including full ARIA, keyboard navigation, body scroll lock, and abort controllers — things Gemma missed entirely. But Opus also had its own bugs: duplicated search logic between the API route and the /search page, an unsafe type cast, and a missing Suspense boundary. We scored it 87.5/100 in the original review.

The merged Gemma+Opus implementation is the cleanest of the three. It takes Gemma's simpler architecture, applies Opus's quality standards for accessibility and interaction design, and fixes the issues both models left behind.

The Real Comparison

The honest truth: if I had to ship search today with one model and no review, I'd pick Opus. It produced higher-quality code in a single turn with zero intervention. The 87.5/100 score reflects real, shippable work with minor fixable issues.

But that's not the interesting takeaway. The interesting takeaway is that the configuration changes mattered more than the model differences. The original Gemma test didn't fail because Gemma is a bad model. It failed because:

num_predict was too low (invisible thinking tokens consumed the budget)
Ollama doesn't expose --reasoning-budget (no way to cap thinking)
Default parallel slots exhausted VRAM
GitHub auth wasn't configured in the workspace

Fix those four things — all infrastructure, not model weights — and Gemma went from "0 commits in 8 prompts" to "shipped a feature in 2 prompts." The model was the same. The environment was different.

What This Means for Local Models

Local models can ship production features. Not hypothetically. This search feature is live, built entirely by Gemma 4 running on consumer hardware. The code needed polish — but so does most code from any developer, human or AI.

Configuration is the bottleneck, not capability. The difference between "Gemma can't finish anything" and "Gemma ships a feature" was four infrastructure changes. Most teams evaluating local models are testing against default settings that actively sabotage the model's output. Invisible thinking tokens, insufficient context windows, VRAM contention — these are environment bugs, not model bugs.

The best workflow might be local + cloud. Gemma built the feature (free, fast, private). Claude reviewed and polished it (thorough, quality-focused). Each model did what it's best at. The total cost was one Opus API call for the review pass, not dozens for the entire build.

llama.cpp is the right tool for serious local inference. Ollama is great for getting started. For production use — where you need reasoning budgets, precise context control, and OpenAI-compatible APIs that tools like Coder can consume — llama-server gives you the knobs you actually need.

The Settings That Made It Work

For anyone running Gemma 4 locally, here's the configuration that turned it from a planning machine into a shipping machine:

llama-server \
  -m gemma-4-26B-A4B-it-UD-Q4_K_M.gguf \
  --ctx-size 32768 \       # 32K context — ~19 GB VRAM on 5090
  -n 32768 \               # 32K max output tokens
  --reasoning-budget 4096 \ # Cap thinking at 4K tokens
  --reasoning-format deepseek \ # Expose thinking in API response
  --parallel 1 \           # Single slot — don't OOM with 4 × 32K
  -ngl 999                 # All layers on GPU

The --reasoning-budget 4096 is the single most important flag. Without it, Gemma can spend its entire output budget on reasoning you never see. With it, the model gets 4K tokens to think, then the rest is for actual code. That one flag is the difference between a model that plans forever and a model that ships.

What's Next

Right now, Gemma 4 serves a single Coder instance on the workstation where it runs. That's fine for one person, but the RTX 5090 is sitting idle most of the day. The obvious next step: make it available to every machine on the local network.

My wife runs OpenClaw on a Mac Mini in the other room. With Tailscale already meshing our devices together, pointing her OpenClaw instance at http://workstation:8080/v1/ is trivially easy — llama-server's OpenAI-compatible API means any tool that speaks the OpenAI protocol can use it. One GPU, multiple clients, zero cloud costs.

Beyond that: migrating the remaining Ollama models to llama.cpp (for the same reasoning budget control we needed here), experimenting with longer context windows now that we know the VRAM budget, and — inevitably — the next model showdown when Gemma 4's bigger variants drop.

The homelab keeps growing. Who knows? Maybe the lobster starts vibe coding for me, too.

By the Numbers

3 attempts before Gemma completed the task (auth fix, OOM fix, success)
2 prompts in the successful run (vs 8 failed prompts in the original test)
618 lines written by Gemma 4 across 6 files
383 lines changed in the Opus polish pass (insertions + deletions)
804 total lines in the merged feature
0 cloud API calls for the build phase (Gemma ran 100% local)
177–181 tokens per second — Gemma's peak generation speed on the RTX 5090
~159 tokens per second — effective speed under real agentic load (accumulated context)
28ms time to first token — near-instant streaming start
16.9 GB model size (Gemma 4 26B-A4B, Q4_K_M quantization)
~19 GB total VRAM at 32K context (comfortable fit on 32 GB card)
4,096 reasoning budget tokens — the setting that made it all work
$0 inference cost for the feature build
1 nudge needed ("keep going, don't stop, code and commit")
7 bugs found in Gemma's code during review (all fixed)
3 bugs in Opus's original implementation (never merged, never fixed)
0 bugs in the merged Gemma+Opus version
1 production feature, live on vibescoder.dev right now — press ⌘K to try it

Day 3: Building the Editing Layer

Rob — Thu, 07 May 2026 23:30:32 +0000

The Problem with Day 2

Day 2 ended with a beautiful site. Neon Brutalist palette, light/dark toggle, design tokens wired up properly. It looked great. But I couldn't actually use it.

The admin login page accepted the correct password and then… nothing happened. No error, no redirect, no feedback. Just the same login form staring back at me. The edit button on posts linked to a recording page that ignored the fact that you were editing — it always started fresh. The admin dashboard had one big "Record" button and no way to find an existing post.

Day 3 was about turning a pretty blog into a functional one. Less visual, more plumbing. The kind of session where everything you build is invisible to readers but makes the difference between "I'll update that later" and actually updating it.

The Silent Login Bug

This one was subtle. The login form called fetch to set a session cookie, then used Next.js router.push("/admin") to navigate. In the App Router, router.push is a soft navigation — it fetches a React Server Component payload over the wire and patches the DOM. No full page reload.

The problem: after fetch sets a cookie, a soft navigation can reuse a stale client-side cache or have the middleware redirect silently swallowed. The browser has the cookie, but the RSC request either doesn't send it or the middleware response gets eaten by the router. The user lands right back on /admin/login with zero feedback.

The fix is almost embarrassingly simple:

// Before: soft navigation, cookie may not propagate
router.push("/admin")

// After: hard navigation, browser sends fresh cookies
window.location.href = "/admin"

Same fix for logout. window.location.href forces the browser to make a full request with the current cookie jar. It's the standard pattern for auth state transitions in Next.js App Router — any time cookies change, don't trust soft nav.

This is the kind of bug that doesn't show up in development. You're already authenticated, the cache is warm, everything works. It only bites you in production when a real user (me, from my phone) hits the cold path.

Admin Access

Two small UX fixes while I was in the auth flow:

Footer link. Added a subtle "Admin" link in the site footer — dimmed by default, lights up on hover with the primary accent. No security implications; the middleware blocks everything without a valid JWT. But now I don't have to type /admin from memory on a phone keyboard.

Visitor-friendly login page. If a non-admin stumbles onto /admin/login, they now see "This area is for the site owner" with a "Back to the blog →" link instead of a bare password field. Small thing, but it's the difference between a page that looks broken and one that looks intentional.

The Edit Pipeline

This was the biggest fix of the session. The blog had a voice recording feature from Day 1 — talk into the browser, Claude generates an MDX post, one-click publish to GitHub. But it was create-only. The "Edit" button on each post linked to /admin/record?edit=slug, and the record page completely ignored the query parameter. Every recording session created a new post.

The fix touched five files:

Record page reads ?edit=slug via useSearchParams (wrapped in a Suspense boundary — App Router requires this for client-side search params)
On mount in edit mode, fetches the existing post content from the API
Generation API receives the existing content alongside the new transcript, so Claude merges rather than rewrites
Publish call uses PUT (update) instead of POST (create) when in edit mode
PostPreview locks the slug field and shows "Update on GitHub" instead of "Publish to GitHub"

The useSearchParams + Suspense requirement is a Next.js App Router detail worth calling out. Without the Suspense boundary, the page throws during static rendering because search params aren't available server-side. It's documented, but it's the kind of thing that bites you when you're porting a pattern from Pages Router.

Now the voice-to-blog pipeline works in both directions: create from scratch, or record additional context and merge it into an existing post. Same UI, same flow, different HTTP verb.

Admin Dashboard Overhaul

The original admin dashboard had three cards: Record, Manage, Settings. Record was the only one that did anything useful. For Day 3, it became two distinct entry points:

Record New Post — same as before, links to /admin/record with no query params.

Edit Existing Post — a searchable dropdown picker. Type to filter posts by title (case-insensitive substring match), click to navigate to /admin/record?edit=slug. The dropdown is scrollable (max-h-64) and closes on click-outside via a pointerdown listener. Designed to handle hundreds of posts without pagination — client-side filtering is fine at blog scale.

Inline Text Editing

The voice recording flow is great for substantial rewrites, but overkill for fixing a typo. So I added a third option: inline text editing directly on the blog post page.

The admin bar on each post (visible only when logged in) expanded from two actions to three:

Action	What it does
Type Edits	Opens a textarea with the raw MDX, right on the post page
Record Edits	Links to the voice recording flow for substantial changes
Delete Post	Removes the post from GitHub (moved to right side)

The inline editor loads the raw MDX source, lets you edit in place, and saves via PUT /api/posts. No page navigation, no recording session, no Claude generation. Just fix the typo and hit Save.

This is the feature that changed how I use the site. Before, fixing a date typo meant opening the voice recorder, describing the change, waiting for Claude to regenerate, and publishing. Now it's: click Type Edits, fix the character, Save. Five seconds instead of two minutes.

Public Changelog

Every edit should be transparent. I added a changelog field to the post frontmatter — an array of {date, summary} entries:

changelog:
  - date: '2026-04-16'
    summary: Fixed Koto -> Coder typo

A collapsible <Changelog> component renders between the post header and content. Collapsed by default — "▸ 1 update" or "▸ N updates" — and expands on click to show the full history. Subtle monospace styling that doesn't compete with the post content.

The changelog entries are generated automatically based on the edit type:

Inline edits get a summary derived from a line-level diff: "Minor text edits," "Edited N lines," or "Revised post content (N lines changed)" depending on scope
Voice recording edits default to "Updated via voice recording"
Manual summaries are still supported if you want to be specific

No AI needed for the diff summaries — just line counting. No commit SHAs, file paths, or system information exposed. Only reader-facing descriptions.

One gotcha: the initial changelog text used text-outline-variant/60, which was nearly invisible in light mode. Bumped to text-on-surface-variant for proper readability in both themes. Another reminder to test both modes after every UI change.

The Blog Post Consolidation

A meta moment: during Day 3, I used the voice recording feature to publish a post about the Google Stitch workflow from Day 2. Then I realized it belonged in the Day 2 post, not as a standalone entry.

So I merged it — the brand guidelines trick and the mobile pipeline section got folded into the Day 2 post as new subsections, and the standalone third post got deleted. This is the kind of editorial decision that's easy when your content is just files in git. No CMS to wrestle with, no database records to reconcile. Delete a file, edit another file, push.

What I Learned

Soft navigation is not your friend during auth transitions. Next.js App Router's router.push is a client-side RSC fetch. If you've just set or cleared a cookie, the soft navigation may not reflect that. Use window.location.href any time authentication state changes. This isn't a bug — it's a fundamental aspect of how RSC caching works.

Build the editing tools early. I should have built inline editing on Day 1. Every session since has involved going back to fix small things in previous posts — typos, date errors, wording tweaks. Without quick inline editing, each fix was a multi-step process through the voice pipeline. The moment I had a textarea and a Save button, my publishing velocity doubled.

Transparency scales trust. The public changelog is a small feature, but it signals something: this content is alive, corrections are acknowledged, and readers can see what changed. For a blog about building in public, that's table stakes.

Each session fixes friction from the last one. Day 1 built the foundation. Day 2 made it look right. Day 3 made it usable. The pattern is consistent: build something, use it for real, discover what's broken or slow, fix it next session. The site isn't being designed upfront — it's being discovered through use.

By the Numbers

1 silent auth bug fixed (soft nav → hard nav)
5 files changed to make the edit pipeline work
3 admin actions per post (type edits, record edits, delete)
1 new component (InlineEditor)
1 new frontmatter field (changelog)
2 blog posts retroactively given changelog entries
1 post merged into another, 1 deleted
0 design changes — all plumbing, all invisible to readers

Open-Sourcing a Blog Without Open-Sourcing Your Drafts

Rob — Thu, 07 May 2026 23:28:59 +0000

I open-sourced my personal blog repo so I could use Giscus for blog comments — it needs a public repo with GitHub Discussions enabled. But open-sourcing the repo meant everything was public: unpublished drafts, raw session notes, half-baked ideas, and my TODO list. For a thought leadership blog, that's a problem. People could just read GitHub instead of the site.

Before we even got to that realization, though, we found something worse.

All of the work in this session was done conversationally through Coder Agents on a self-hosted home lab setup. (You'll hear a lot more about that setup soon — I'll be writing about the full home lab build next.)

The Security Audit

First thing we did was scan the repo for anything sensitive now that it was public. Found three issues.

1. The .gitignore Was Gone

In a previous session, an agent had tried to set up a drafts workflow. The idea was to use .gitignore to keep drafts out of the repo. When that broke persistence (gitignored files don't survive workspace destruction), I asked the agent to fix it. Instead of removing the one blog-drafts/ line, it replaced the entire .gitignore with a single comment — deleting all 50 standard Next.js ignore patterns.

This meant .env, .env.local, node_modules/, .next/, .vercel/, *.pem — none of it was being ignored. If anyone (or any agent) had run git add ., every secret in .env would have been committed to a public repo.

The root cause: The agent conflated two separate concerns — git tracking (persistence) and site publishing (visibility). .gitignore controls what git tracks, not what the site renders. The blog already had published: false frontmatter support in posts.ts that filters unpublished posts from the public site. The agent didn't look at existing code before reaching for a filesystem-level solution.

Lesson: When an AI agent suggests a fix, check whether the codebase already solves the problem. Also, always diff what an agent changed — don't assume a targeted edit was actually targeted.

2. Live Server URL Exposed

The blog drafts contained my actual Coder server tunnel URL — a live endpoint to my self-hosted instance sitting in the blog fodder notes from a previous session. Anyone could have tried to hit it.

Fix: Replaced with a placeholder and rotated the URL.

3. Infrastructure Reconnaissance

The drafts also contained hardware specs, home lab architecture details, systemd config paths, and multi-user setup info. Not a vulnerability per se, but useful reconnaissance for someone targeting the setup.

Verdict: Acceptable for a "building in public" blog, but worth being aware of.

The Real Problem: Code vs. Content Visibility

After fixing the immediate issues, we hit the bigger question: what about a world where this is a well-trafficked site? Anyone could ignore the site entirely and browse GitHub for unpublished drafts, upcoming topics, and editorial strategy.

published: false only gates the rendered site. GitHub shows everything.

Why Not Just Make the Repo Private?

Giscus. The whole reason we open-sourced was for blog comments. Giscus requires a public repo with GitHub Discussions enabled. Making the repo private kills comments.

The Key Insight

Giscus doesn't care what's in the repo — it just needs a public repo to host GitHub Discussions. The discussions are completely independent of the repo's file contents. So we could separate the code from the content without touching Giscus at all.

The Solution: Two Repos

the-vibe-coder (public) — The blog engine. All source code, configs, components, API routes. Giscus stays pointed here. Open source, as intended.

the-vibe-coder-content (private) — All content: published posts, unpublished drafts, raw session notes, settings, images, TODO list. Nobody sees this but me.

How They Connect

The critical design decision: the private repo uses the exact same directory structure as the original. This meant zero code changes to the GitHub API client, the post loader, or any admin panel routes. The only change was pointing the GITHUB_REPO environment variable at the private repo.

A prebuild script clones the private repo at build time and overlays the content into the working tree. On Vercel, this runs automatically before next build. Locally, you clone the content repo once and copy or symlink.

The Deploy Hook

Since Vercel watches the public code repo, it wouldn't know to rebuild when content changes in the private repo. A GitHub Action on the private repo hits a Vercel Deploy Hook on every push to main:

Content commit → GitHub Action → Vercel rebuild → site updated.

The Wiring

Created the private content repo
Pushed all content files (same directory structure)
Added a fetch-content.sh prebuild script to the public repo
Updated .gitignore to exclude content directories
Removed content files from the public repo
Updated GITHUB_REPO on Vercel to point to private repo
Created a Vercel Deploy Hook + GitHub Action trigger

The Token Gotcha

First deploy failed with exit code 128 (git auth failure). The GITHUB_TOKEN was a fine-grained PAT scoped to only the original repo. Had to update it in GitHub to also include the new private repo. Fine-grained PATs don't automatically pick up new repos — if your build pipeline uses one and you add a new private repo, you'll get a 403 until you update the token's repo list.

What I Learned

Agents and .gitignore

Agents reach for .gitignore as a blunt instrument. When the problem is "don't show this on the site," the answer is almost never "don't track it in git." Those are different concerns:

Git tracking = persistence, collaboration, backup
Site publishing = what visitors see

Conflating them leads to either lost work (gitignored files vanish) or the opposite — a gutted .gitignore that exposes secrets.

Always Audit Before Open-Sourcing

We caught three issues in a five-minute scan. The .gitignore one was a genuine time bomb. Open sourcing without a security pass is shipping without testing.

Giscus Is Decoupled From Content

This was the unlock. You can have a public repo with zero content files and Giscus works perfectly. "I need Giscus" and "I need private content" aren't in conflict.

The Same-Structure Trick

By keeping the private repo's directory layout identical to the original, we avoided code changes entirely. The admin panel, the build process, and the content API all work unchanged — they just talk to a different repo via the same env var. This is the kind of thing that makes a migration smooth instead of a refactor.

By the Numbers

1 .gitignore restored from 1 line to 52 lines
1 server URL redacted and rotated
2 repos (1 public, 1 private)
0 code changes to the blog engine
1 prebuild script (24 lines of bash)
1 GitHub Action (8 lines of YAML)
1 Vercel Deploy Hook
1 fine-grained PAT updated
5 published posts confirmed rendering
1 unpublished draft confirmed hidden
~45 minutes from "is there anything sensitive?" to verified deploy

Downtime Is a Feature: Custom Domains, Cloudflare, and MCP While Models Download

Rob — Thu, 07 May 2026 23:28:55 +0000

You know how it goes in AI development — sometimes you're stuck watching progress bars crawl forward. I've been preparing for the next installment of the Local Model Showdown series, and that means downloading some hefty models. Kimi K2.6 decided to take its sweet time. We're talking hours.

But here's the thing: downtime is really just opportunity in disguise. Instead of watching percentages tick up, I knocked out three items from the backlog that had been bugging me for a week. All done conversationally through Coder Agents, naturally.

The hit list:

Put my self-hosted Coder instance behind a real domain
Harden the Cloudflare setup (and discover a gotcha that every content creator needs to know)
Wire up MCP servers to give my agents superpowers

The Goal: coder.vibescoder.dev

My Coder instance was running on my homelab Ubuntu workstation, accessible through a try.coder.app tunnel URL — functional but ugly, hard to remember, and not exactly on-brand. I bought vibescoder.dev for the blog. Time to use coder.vibescoder.dev for the dev environment.

Sounds simple. It wasn't.

Attempt 1: CNAME Records (The Naive Approach)

First move was straightforward — add CNAME records in Vercel's DNS management (since I bought the domain through Vercel) pointing the coder subdomain to the existing tunnel URL.

Gotcha #1: Vercel's DNS form defaults to record type "A" (which expects an IPv4 address). Spent a minute confused by the "value should match format ipv4" error before realizing I needed to switch the Type dropdown to "CNAME." Small thing, but it'll trip you up if you're not looking at the form defaults.

Vercel's DNS form defaults to an A record — switch the type to CNAME or you'll get this unhelpful error.

Then I went to update the Access URL in Coder's dashboard. Deployment → General → Access URL. It's right there on the screen... and it's read-only. The UI shows you the value but can't change it. The badges underneath tell the story: CLI --access-url, ENV CODER_ACCESS_URL, YAML accessURL. Server config only.

Since Coder runs via systemd on my homelab, the config lives at /etc/coder.d/coder.env. But before I could update it, I needed to solve the bigger problem: getting traffic from the internet to my machine.

Attempt 2: Port Forwarding (The Frustrating Detour)

For a CNAME-based approach to work, my homelab needs to be reachable from the internet on ports 80 and 443. That means port forwarding on the router — a TP-Link Archer BE800.

The router app doesn't expose port forwarding. The Tether iPhone app has a "More" menu with various settings, but NAT Forwarding isn't there. Had to use the web interface at 192.168.0.1 instead. Found it under Advanced → NAT Forwarding → Port Forwarding.

Set up the rules: ports 80 and 443, TCP, forwarded to my machine's internal IP (192.168.0.243). Then tested with a simple Python HTTP server:

sudo python3 -c "from http.server import HTTPServer, SimpleHTTPRequestHandler; HTTPServer(('0.0.0.0', 443), SimpleHTTPRequestHandler).handle_request()"

Local curl worked. External requests to my public IP? "Connection refused." Not a timeout — refused. Tried non-standard ports too. Same result.

The diagnosis: "Connection refused" from outside while "works locally" means traffic is reaching the public IP but getting actively rejected before it hits the machine. The ISP is likely blocking inbound connections or there's a NAT layer beyond the router. A timeout would mean packets are being dropped. Refused means something is saying "no."

I spent more time on this than I'd like to admit. Time for Plan B.

The Solution: Cloudflare Tunnel

Cloudflare Tunnel flips the model entirely. Instead of opening inbound ports, it creates an outbound connection from your machine to Cloudflare's edge. No port forwarding. No router config. No public IP exposure. And it's free.

Step 1: Move Nameservers

Since I needed Cloudflare to manage DNS for vibescoder.dev, I:

Created a free Cloudflare account
Added vibescoder.dev as a site
Let it auto-import my existing DNS records (Vercel A records, CAA records, everything)
Selected the Free plan
Updated nameservers at Vercel (the registrar) to point to Cloudflare's nameservers

The blog continues to work — Cloudflare imported all existing records, so vibescoder.dev still routes to Vercel's servers.

Cloudflare auto-imports your existing DNS records — the blog keeps working while you set up the tunnel.

Step 2: Create the Tunnel

On the homelab machine:

curl -L --output cloudflared.deb https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-linux-amd64.deb
sudo dpkg -i cloudflared.deb
cloudflared tunnel login
cloudflared tunnel create coder-tunnel

After cloudflared tunnel login, the browser confirms the certificate is installed and you're authorized.

Config file at ~/.cloudflared/config.yml:

tunnel: a1b2c3d4-e5f6-7890-abcd-ef1234567890
credentials-file: /home/youruser/.cloudflared/a1b2c3d4-e5f6-7890-abcd-ef1234567890.json

ingress:
  - hostname: coder.vibescoder.dev
    service: http://localhost:3000
  - hostname: "*.coder.vibescoder.dev"
    service: http://localhost:3000
  - service: http_status:404

The wildcard entry handles Coder's app proxying — port forwarding, web terminals, and workspace apps all use subdomains like 8080--main--ws--user--coder.vibescoder.dev.

Step 3: Wire Up DNS and Coder

cloudflared tunnel route dns coder-tunnel coder.vibescoder.dev
cloudflared tunnel route dns coder-tunnel "*.coder.vibescoder.dev"

Updated /etc/coder.d/coder.env:

CODER_ACCESS_URL=https://coder.vibescoder.dev
CODER_WILDCARD_ACCESS_URL=*.coder.vibescoder.dev

Restarted Coder, ran the tunnel — four connections registered to Cloudflare's SJC edge locations. Opened https://coder.vibescoder.dev in a browser. Coder login screen. Done.

Step 4: Make It Permanent

Gotcha #2: sudo cloudflared service install couldn't find the config. It looks in /etc/cloudflared/, not ~/.cloudflared/. Had to copy both files:

sudo mkdir -p /etc/cloudflared
sudo cp ~/.cloudflared/config.yml /etc/cloudflared/config.yml
sudo cp ~/.cloudflared/*.json /etc/cloudflared/

Updated the credentials-file path in the copied config to point to /etc/cloudflared/, then:

sudo cloudflared service install
sudo systemctl enable cloudflared
sudo systemctl start cloudflared

Two systemd services now: coder and cloudflared. Both start on boot. The architecture:

Browser → https://coder.vibescoder.dev
       → Cloudflare Edge (TLS termination, DNS)
       → Cloudflare Tunnel (outbound from homelab)
       → localhost:3000 (Coder server)

Zero inbound ports. Zero public IP exposure. Free TLS from Cloudflare. Survives reboots.

Hardening Cloudflare (And the AI Crawler Gotcha)

With Cloudflare in front of everything, I reviewed the security settings. Here's what's worth enabling on the free tier:

Setting	What it does
SSL/TLS → Full (strict)	Ensures encryption all the way to the origin, not just browser-to-Cloudflare
Bot Fight Mode	Challenges malicious bots — scrapers, credential stuffers, spam
DDoS Protection	Already active by default
Always Online	Serves cached pages if your origin goes down

The WAF Managed Ruleset (SQL injection, XSS protection) requires a Pro plan. Skipped for now.

Set SSL/TLS to Full (strict) to encrypt traffic all the way from Cloudflare to your origin server.

The Part Every Content Creator Needs to Read

Cloudflare's free tier includes two AI-related settings that are on by default:

Block AI bots — Blocks bots Cloudflare categorizes as AI training crawlers (GPTBot, CCBot, Google-Extended, etc.)
AI Labyrinth (Beta) — Injects fake AI-generated content into your pages to poison bots that ignore crawling standards

Both sound great if you want to protect your content from being scraped. But think about what these actually do: they block the crawlers that feed ChatGPT search, Perplexity, Google AI Overviews, and every other AI-powered discovery tool.

This innocent-looking toggle blocks the AI crawlers that power ChatGPT search, Perplexity, and Google AI Overviews.

If your site exists for thought leadership, you want AI services to find, index, and cite your content. That's the entire point. Blocking AI crawlers is blocking your distribution channel.

The distinction:

Block AI bots / AI Labyrinth = blocks crawlers that feed AI search and training. Kills discoverability.
Bot Fight Mode = blocks malicious bots. Doesn't affect legitimate AI crawlers.

I turned both Block AI bots and AI Labyrinth off, while keeping Bot Fight Mode on. If you're running a personal brand, a company blog, or anything where you care about AI-powered search visibility — check these settings immediately after onboarding to Cloudflare. The defaults optimize for content protection, not content distribution.

MCP: Giving Agents Superpowers

With the infrastructure sorted, I moved to the fun part: MCP (Model Context Protocol) integration. MCP lets AI agents access external tools — think of it as a plugin system for LLMs.

The AI Gateway

Two lines in coder.env unlock the big stuff:

CODER_EXPERIMENTS=oauth2,mcp-server-http
CODER_EXTERNAL_AUTH_0_MCP_URL=https://api.githubcopilot.com/mcp/

The first enables Coder's experimental MCP support. The second wires GitHub's MCP server into Coder's AI Gateway. Since I already had GitHub OAuth configured, this means the gateway automatically injects GitHub tools (prefixed with bmcp_) into every agent's LLM requests. Every agent in every workspace gets GitHub repo access, PR management, issue tracking — zero per-workspace config.

Choosing MCP Servers

Researched the ecosystem and selected five servers based on this specific stack:

MCP Server	Why
GitHub (official, 29K stars)	Blog content is a private GitHub repo. Handled via AI Gateway.
Context7 (Upstash, 53K stars)	Feeds current library docs to LLMs instead of hallucinated APIs. Critical for Next.js 16.
Vercel (official)	Check deployments, read build logs, manage env vars.
Cloudflare (official, 3.6K stars)	DNS analytics, tunnel debugging, observability.
Playwright (Microsoft, 31K stars)	Visual testing of blog deployments.

What I deliberately skipped: Ollama-specific MCP servers. This is a common misconception worth calling out. You don't need an "Ollama MCP server." Ollama is the LLM backend — agents call it for inference. MCP servers provide tools (GitHub access, deployment management, browser automation). The agent uses Ollama to think about what to do, and MCP tools to do it. They're separate concerns.

Wiring It Into the Template

MCP servers in Coder aren't configured in the admin panel — they're discovered via a .mcp.json file in the workspace root. Gotcha #3: I spent time looking for an MCP settings page in the Coder dashboard before discovering this.

To make it persistent across all workspaces, I edited the Docker template's main.tf:

coder templates pull docker .
nano main.tf  # add .mcp.json to startup_script
coder templates push docker

Editing the Docker template's main.tf to inject .mcp.json into every workspace at startup.

The startup script now writes this .mcp.json to every workspace:

{
  "mcpServers": {
    "context7": {
      "command": "npx",
      "args": ["-y", "@upstash/context7-mcp"]
    },
    "vercel": {
      "command": "npx",
      "args": ["mcp-remote", "https://mcp.vercel.com"]
    },
    "cloudflare": {
      "command": "npx",
      "args": ["mcp-remote", "https://agents.cloudflare.com/mcp"]
    },
    "playwright": {
      "command": "npx",
      "args": ["-y", "@playwright/mcp@latest"]
    }
  }
}

Context7 and Playwright run locally as stdio processes. Vercel and Cloudflare connect to remote HTTP endpoints and handle OAuth on first use.

The Full Stack

┌─────────────────────────────────────────────────────┐
│              HOMELAB (Ubuntu + RTX 5090)             │
│                                                     │
│  Ollama (local LLMs) ◄── Coder AI Gateway           │
│                           │                         │
│  Coder Server ────────────┤  Injected tools:        │
│  (systemd, port 3000)     │  • bmcp_github_*        │
│                           │                         │
│  Cloudflared ─────────────┤                         │
│  (systemd, tunnel)        │                         │
│                           │                         │
│  Workspace (.mcp.json):   │                         │
│  • context7 (stdio)       │                         │
│  • playwright (stdio)     │                         │
│  • vercel (remote HTTP)   │                         │
│  • cloudflare (remote HTTP)                         │
└───────────────┬─────────────────────────────────────┘
                │ Cloudflare Tunnel
                ▼
┌─────────────────────────────────────────────────────┐
│              EXTERNAL SERVICES                      │
│                                                     │
│  Cloudflare Edge (DNS, TLS, DDoS, Bot Fight Mode)   │
│  GitHub (content repo, MCP via AI Gateway)           │
│  Vercel (blog hosting, MCP remote server)            │
│  Anthropic Claude (blog post generation)             │
│  Upstash Redis (analytics)                           │
└─────────────────────────────────────────────────────┘

By the Numbers

2 port forwarding rules attempted (failed — ISP blocking)
1 Cloudflare Tunnel created (0 inbound ports required)
4 environment variables changed in coder.env
2 systemd services running (coder + cloudflared)
5 Cloudflare security settings reviewed
2 AI-blocking features disabled for thought leadership discoverability
5 MCP servers configured (GitHub via AI Gateway + 4 in .mcp.json)
1 workspace template updated
3 gotchas discovered (Cloudflare AI defaults, .mcp.json discovery, cloudflared config paths)
~2 hours of productive "downtime" while models downloaded

Model Showdown: Benchmarking Local vs Cloud LLMs on a Real Coding Task

Rob — Thu, 07 May 2026 23:28:28 +0000

Last post we stood up Ollama on the RTX 5090, pulled a stack of models, and wired them into our coding workflow. The whole time there was an obvious question hanging over it: are local models actually good enough?

Not good enough in the abstract benchmarks-on-a-leaderboard sense. Good enough for the thing we’re journaling: vibe coding. Specifically, can a model running on consumer hardware in my homelab produce code that's as correct, as fast, and as complete as what comes back from Anthropic's cloud?

We built a benchmark to find out.

The Setup

Six models, one prompt, no second chances.

Cloud (Anthropic API):

Sonnet 4.6 (claude-sonnet-4-20250514)
Opus 4.6 (claude-opus-4-20250514)

Local (Ollama on RTX 5090, 32 GB VRAM):

Codestral 22B (codestral:22b)
DeepSeek R1 14B (deepseek-r1:14b)
Devstral (devstral:latest)
Qwen 3.5B MoE (qwen3.5:35b-a3b)

The prompt was intentionally straightforward: build a Python CLI todo app with SQLite persistence, CRUD commands (add, list, complete, delete), timestamps, pretty output, error handling, and a __main__ block. The kind of task that shows up in real work. A simple "write a small, complete program."

Every model got the exact same prompt with the instruction: "Respond with ONLY the Python code, no explanation."

We measured:

Time to first token (TTFT): how long before output starts streaming
Total generation time: wall clock from request to last token
Output tokens: how much the model wrote
Tokens per second: raw generation throughput
Validation: does it parse, does it have all the features, does it actually run through a functional test suite of 7 operations (add two todos, list, complete one, list again, delete one, list again)

The Results: Performance

Model	Type	TTFT	Total Time	Output Tokens	Tok/s
Sonnet 4.6	Cloud	0.87s	14.89s	1,461	104.2
Opus 4.6	Cloud	1.23s	19.06s	1,324	74.3
Codestral 22B	Local	15.81s	22.11s	620	98.5
DeepSeek R1	Local	11.74s	20.64s	1,707	191.7
Devstral	Local	2.24s	10.26s	723	90.2
Qwen 3.5B	Local	28.20s	30.91s	4,096	1,510.2

A few things jump out immediately. Devstral finished faster than every other model, cloud or local. Qwen's tokens-per-second number is absurd. And DeepSeek R1 produced the most tokens despite writing roughly the same amount of code as (more on why in a minute).

The Results: Quality

Performance doesn't matter if the code is wrong. Here's how each model scored:

Model	Syntax Valid	Features (X/10)	Functional (X/7)	Score
Sonnet 4.6	Yes	10/10	7/7	100
Opus 4.6	Yes	10/10	7/7	100
Devstral	Yes	10/10	7/7	100
Codestral 22B	Yes	10/10	0/7	60
DeepSeek R1	Yes	10/10	0/7	60
Qwen 3.5B	No	7/10	0/7	28

Three perfect scores. Two models that wrote valid code that didn't pass functional tests. One that didn't even produce valid Python.

Let's talk about what happened.

What Went Wrong (and Right)

The Interactive Menu Problem

Codestral 22B and DeepSeek R1 both scored 10/10 on features. Their code had SQLite, all four CRUD operations, timestamps, completion tracking, error handling, a main block, and pretty output. On paper, they nailed it.

The problem: both interpreted "Commands: add, list, complete, delete" as an interactive menu application. They built while True loops with input() prompts instead of CLI argument parsers.

Codestral's approach:

while True:
    action = input("Enter a command (add, list, complete, delete): ")

DeepSeek R1 went even further, building an entire menu system:

print("Todo App Menu:")
print("               ---")
print("add - Add a new todo      ")
print("list - List all todos     ")
cmd = input("Enter command: ").strip().lower()

Both are perfectly valid interpretations of "commands." Both produced clean, working code. But our automated test suite calls the script with command-line arguments (python todo.py add "Buy groceries"), not interactive input. The scripts immediately hit EOFError: EOF when reading a line because there's no stdin to read from.

This is arguably a prompt clarity issue, not a model quality issue. If the prompt had said "using argparse" or "using sys.argv," both models would have nailed it. But the three models that scored 100 all inferred CLI arguments without being told, which is the more common pattern for "command-line app" in the training data.

The Token Limit Trap

Qwen 3.5B is fascinating and frustrating in equal measure.

That 1,510 tokens-per-second number is real. The model uses a Mixture of Experts (MoE) architecture: 35 billion total parameters, but only ~3 billion active per token. The RTX 5090 tears through it. In pure generation speed, nothing else comes close.

But it hit the 4,096 output token limit mid-f-string:

        print(f"{'ID':<5} {'Status':<8} {'Title':<40} {'Created At'}")
        print('-' * 70)
        for row in rows:
            id_, title, created_at, completed = row
            status = "[X]" if completed else "[ ]"
            print(f"{

That's it. The code cuts off right there. No closing quote, no remaining functions, no main block. The syntax is invalid. The features for complete, delete, and __main__ are missing because the model never got to write them.

The speed is meaningless if the output is incomplete. The lesson: always set generous max_tokens for code generation tasks. A 4,096 limit that's fine for chat responses will absolutely truncate a complete program. We should have set 8,192 or higher. That's on us.

DeepSeek R1's Thinking Tax

DeepSeek R1 produced 1,707 output tokens, the most of any model, but its actual code was only 156 lines. Where did the extra tokens go?

Into <think> blocks. DeepSeek R1 is a reasoning model. Before writing code, it spends tokens working through the problem:

"Let me think about how to structure this... I need SQLite for persistence... I'll use a class-based approach with a menu system..."

This is genuinely useful for hard debugging problems or complex architectural decisions. But for straightforward code generation where the answer is obvious, it's wasted compute. You're paying (in time and tokens) for the model to reason through something it could just write directly.

Devstral's Quiet Dominance

The standout result of the entire benchmark. Devstral is a 24B parameter model from Mistral, purpose-built for coding tasks. On paper it's smaller than some of the competition. In practice:

Fastest total time: 10.26 seconds, beating even the cloud models
Best local TTFT: 2.24 seconds, nearly as fast as cloud cold-start
Perfect score: 100/100 on quality
Clean architecture: argparse-based CLI, exactly what the test expected

It didn't overthink it. It didn't build a menu system. It didn't run out of tokens. It just wrote a clean, correct, well-structured todo app and moved on.

Code Comparison: The Three Perfect Scores

All three 100-score models (Sonnet, Opus, Devstral) used argument-based CLI patterns, but their implementations differ in interesting ways.

Sonnet 4.6 went with argparse and a class-based design. 149 lines. Full docstrings, type hints, and emoji-rich output with status indicators:

class TodoApp:
    def __init__(self, db_path: str = "todos.db"):
        self.db_path = db_path
        self.init_database()

    def add_todo(self, title: str):
        # ...
        print(f"✅ Added todo #{todo_id}: {title.strip()}")

Opus 4.6 also used a class with sys.argv parsing instead of argparse. 157 lines. It used sqlite3.Row for named column access and a manual usage printer. More defensive with explicit connection closing in a finally block:

class TodoApp:
    def __init__(self, db_path: str = "todos.db"):
        self.conn = sqlite3.connect(db_path)
        self.conn.row_factory = sqlite3.Row
        self.create_table()

Devstral took the most minimal approach. 99 lines. Flat functions instead of a class, argparse with subparsers, CURRENT_TIMESTAMP in SQL instead of Python-side datetime generation. No emoji, no decorations, just clean output:

def list_todos():
    # ...
    for row in rows:
        status = "[x]" if row[3] else "[ ]"
        print(f"{row[0]:<3} {status} {row[1]} (created at: {row[2]})")

The style differences are telling. Sonnet writes like a senior developer who cares about DX. Opus writes like someone who's been burned by unclosed connections before. Devstral writes like someone who bills by the hour and values conciseness. All three approaches are correct. Devstral's is the one I'd actually want in a codebase.

import sqlite3
import argparse
import datetime
from typing import List, Tuple

class TodoApp:
    def __init__(self, db_path: str = "todos.db"):
        self.db_path = db_path
        self.init_database()

    def init_database(self):
        """Initialize the SQLite database and create the todos table if it doesn't exist."""
        conn = sqlite3.connect(self.db_path)
        cursor = conn.cursor()
        cursor.execute("""
            CREATE TABLE IF NOT EXISTS todos (
                id INTEGER PRIMARY KEY AUTOINCREMENT,
                title TEXT NOT NULL,
                created_at TEXT NOT NULL,
                completed INTEGER NOT NULL DEFAULT 0
            )
        """)
        conn.commit()
        conn.close()

    def add_todo(self, title: str):
        """Add a new todo item."""
        if not title.strip():
            print("❌ Error: Todo title cannot be empty")
            return

        conn = sqlite3.connect(self.db_path)
        cursor = conn.cursor()
        created_at = datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S")
        cursor.execute(
            "INSERT INTO todos (title, created_at, completed) VALUES (?, ?, 0)",
            (title.strip(), created_at)
        )
        conn.commit()
        todo_id = cursor.lastrowid
        conn.close()
        print(f"✅ Added todo #{todo_id}: {title.strip()}")

    def list_todos(self):
        """List all todo items."""
        conn = sqlite3.connect(self.db_path)
        cursor = conn.cursor()
        cursor.execute("SELECT id, title, created_at, completed FROM todos ORDER BY id")
        todos = cursor.fetchall()
        conn.close()

        if not todos:
            print("📝 No todos found. Add some with: python todo.py add \"Your task\"")
            return

        print("\n📋 Your Todos:")
        print("-" * 60)
        for todo_id, title, created_at, completed in todos:
            status = "✓" if completed else "○"
            status_text = "DONE" if completed else "TODO"
            print(f"{status} #{todo_id:<3} [{status_text}] {title}")
            print(f"    📅 Created: {created_at}")
        print("-" * 60)
        print(f"Total: {len(todos)} todos")

    def complete_todo(self, todo_id: int):
        """Mark a todo as completed."""
        conn = sqlite3.connect(self.db_path)
        cursor = conn.cursor()

        cursor.execute("SELECT id, title, completed FROM todos WHERE id = ?", (todo_id,))
        todo = cursor.fetchone()

        if not todo:
            print(f"❌ Error: Todo #{todo_id} not found")
            conn.close()
            return

        if todo[2]:
            print(f"ℹ️  Todo #{todo_id} is already completed")
            conn.close()
            return

        cursor.execute("UPDATE todos SET completed = 1 WHERE id = ?", (todo_id,))
        conn.commit()
        conn.close()
        print(f"✅ Completed todo #{todo_id}: {todo[1]}")

    def delete_todo(self, todo_id: int):
        """Delete a todo item."""
        conn = sqlite3.connect(self.db_path)
        cursor = conn.cursor()

        cursor.execute("SELECT id, title FROM todos WHERE id = ?", (todo_id,))
        todo = cursor.fetchone()

        if not todo:
            print(f"❌ Error: Todo #{todo_id} not found")
            conn.close()
            return

        cursor.execute("DELETE FROM todos WHERE id = ?", (todo_id,))
        conn.commit()
        conn.close()
        print(f"🗑️  Deleted todo #{todo_id}: {todo[1]}")

def main():
    app = TodoApp()

    parser = argparse.ArgumentParser(description="A simple command-line todo application")
    subparsers = parser.add_subparsers(dest="command", help="Available commands")

    add_parser = subparsers.add_parser("add", help="Add a new todo")
    add_parser.add_argument("title", help="Todo title")

    list_parser = subparsers.add_parser("list", help="List all todos")

    complete_parser = subparsers.add_parser("complete", help="Mark a todo as completed")
    complete_parser.add_argument("id", type=int, help="Todo ID to complete")

    delete_parser = subparsers.add_parser("delete", help="Delete a todo")
    delete_parser.add_argument("id", type=int, help="Todo ID to delete")

    args = parser.parse_args()

    try:
        if args.command == "add":
            app.add_todo(args.title)
        elif args.command == "list":
            app.list_todos()
        elif args.command == "complete":
            app.complete_todo(args.id)
        elif args.command == "delete":
            app.delete_todo(args.id)
        else:
            parser.print_help()
    except Exception as e:
        print(f"❌ Error: {str(e)}")

if __name__ == "__main__":
    main()

import sqlite3
import sys
from datetime import datetime
from typing import List, Tuple

class TodoApp:
    def __init__(self, db_path: str = "todos.db"):
        self.conn = sqlite3.connect(db_path)
        self.conn.row_factory = sqlite3.Row
        self.create_table()

    def create_table(self):
        self.conn.execute("""
            CREATE TABLE IF NOT EXISTS todos (
                id INTEGER PRIMARY KEY AUTOINCREMENT,
                title TEXT NOT NULL,
                created_at TIMESTAMP NOT NULL,
                completed BOOLEAN NOT NULL DEFAULT 0
            )
        """)
        self.conn.commit()

    def add(self, title: str) -> None:
        if not title.strip():
            print("Error: Todo title cannot be empty")
            return

        try:
            self.conn.execute(
                "INSERT INTO todos (title, created_at) VALUES (?, ?)",
                (title, datetime.now())
            )
            self.conn.commit()
            print(f"✓ Added todo: '{title}'")
        except Exception as e:
            print(f"Error: Failed to add todo - {e}")

    def list(self) -> None:
        try:
            cursor = self.conn.execute(
                "SELECT * FROM todos ORDER BY created_at DESC"
            )
            todos = cursor.fetchall()

            if not todos:
                print("No todos found. Add one with 'add <title>'")
                return

            print("\n" + "="*60)
            print("TODO LIST")
            print("="*60)

            for todo in todos:
                status = "✓" if todo["completed"] else "○"
                created = datetime.fromisoformat(todo["created_at"]).strftime("%Y-%m-%d %H:%M")
                print(f"{status} [{todo['id']}] {todo['title']}")
                print(f"   Created: {created}")
                print("-"*60)
        except Exception as e:
            print(f"Error: Failed to list todos - {e}")

    def complete(self, todo_id: str) -> None:
        try:
            todo_id = int(todo_id)
        except ValueError:
            print("Error: Invalid todo ID. Must be a number.")
            return

        try:
            cursor = self.conn.execute(
                "UPDATE todos SET completed = 1 WHERE id = ?",
                (todo_id,)
            )

            if cursor.rowcount == 0:
                print(f"Error: Todo with ID {todo_id} not found")
            else:
                self.conn.commit()
                print(f"✓ Completed todo {todo_id}")
        except Exception as e:
            print(f"Error: Failed to complete todo - {e}")

    def delete(self, todo_id: str) -> None:
        try:
            todo_id = int(todo_id)
        except ValueError:
            print("Error: Invalid todo ID. Must be a number.")
            return

        try:
            cursor = self.conn.execute(
                "DELETE FROM todos WHERE id = ?",
                (todo_id,)
            )

            if cursor.rowcount == 0:
                print(f"Error: Todo with ID {todo_id} not found")
            else:
                self.conn.commit()
                print(f"✓ Deleted todo {todo_id}")
        except Exception as e:
            print(f"Error: Failed to delete todo - {e}")

    def close(self):
        self.conn.close()

def print_usage():
    print("""
Usage:
    python todo.py add <title>      - Add a new todo
    python todo.py list             - List all todos
    python todo.py complete <id>    - Mark a todo as completed
    python todo.py delete <id>      - Delete a todo
    """)

if __name__ == "__main__":
    app = TodoApp()

    if len(sys.argv) < 2:
        print_usage()
        sys.exit(1)

    command = sys.argv[1].lower()

    try:
        if command == "add":
            if len(sys.argv) < 3:
                print("Error: Please provide a todo title")
                print("Usage: python todo.py add <title>")
            else:
                title = " ".join(sys.argv[2:])
                app.add(title)

        elif command == "list":
            app.list()

        elif command == "complete":
            if len(sys.argv) < 3:
                print("Error: Please provide a todo ID")
                print("Usage: python todo.py complete <id>")
            else:
                app.complete(sys.argv[2])

        elif command == "delete":
            if len(sys.argv) < 3:
                print("Error: Please provide a todo ID")
                print("Usage: python todo.py delete <id>")
            else:
                app.delete(sys.argv[2])

        else:
            print(f"Error: Unknown command '{command}'")
            print_usage()
            sys.exit(1)

    finally:
        app.close()

import sqlite3
from datetime import datetime
import argparse
import sys

DATABASE = 'todos.db'

def init_db():
    conn = sqlite3.connect(DATABASE)
    c = conn.cursor()
    c.execute('''CREATE TABLE IF NOT EXISTS todos (
                id INTEGER PRIMARY KEY AUTOINCREMENT,
                title TEXT NOT NULL,
                created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
                completed BOOLEAN NOT NULL CHECK (completed IN (0, 1))
              )''')
    conn.commit()
    conn.close()

def add_todo(title):
    conn = sqlite3.connect(DATABASE)
    c = conn.cursor()
    c.execute("INSERT INTO todos (title, completed) VALUES (?, ?)", (title, False))
    conn.commit()
    conn.close()

def list_todos():
    conn = sqlite3.connect(DATABASE)
    c = conn.cursor()
    c.execute("SELECT id, title, created_at, completed FROM todos ORDER BY completed, created_at")
    rows = c.fetchall()
    conn.close()

    for row in rows:
        status = "[x]" if row[3] else "[ ]"
        print(f"{row[0]:<3} {status} {row[1]} (created at: {row[2]})")

def complete_todo(id):
    conn = sqlite3.connect(DATABASE)
    c = conn.cursor()
    c.execute("UPDATE todos SET completed = 1 WHERE id = ?", (id,))
    rows_affected = c.rowcount
    conn.commit()
    conn.close()

    if rows_affected == 0:
        raise ValueError(f"Todo with ID {id} not found.")

def delete_todo(id):
    conn = sqlite3.connect(DATABASE)
    c = conn.cursor()
    c.execute("DELETE FROM todos WHERE id = ?", (id,))
    rows_affected = c.rowcount
    conn.commit()
    conn.close()

    if rows_affected == 0:
        raise ValueError(f"Todo with ID {id} not found.")

def main():
    parser = argparse.ArgumentParser(description='CLI todo application')
    subparsers = parser.add_subparsers(dest='command')

    add_parser = subparsers.add_parser('add', help='Add a new todo')
    add_parser.add_argument('title', type=str, help='Todo title')

    list_parser = subparsers.add_parser('list', help='List all todos')

    complete_parser = subparsers.add_parser('complete', help='Mark a todo as completed')
    complete_parser.add_argument('id', type=int, help='TODO ID to mark as completed')

    delete_parser = subparsers.add_parser('delete', help='Delete a todo')
    delete_parser.add_argument('id', type=int, help='TODO ID to delete')

    args = parser.parse_args()

    init_db()

    try:
        if args.command == 'add':
            add_todo(args.title)
        elif args.command == 'list':
            list_todos()
        elif args.command == 'complete':
            complete_todo(args.id)
        elif args.command == 'delete':
            delete_todo(args.id)
        else:
            parser.print_help()
    except Exception as e:
        print(f"Error: {e}")
        sys.exit(1)

if __name__ == '__main__':
    main()

Speed Analysis

The performance numbers tell two very different stories depending on what you care about.

For interactive chat and streaming, TTFT is what matters. Cloud models dominated here. Sonnet 4.6 started streaming in 0.87 seconds. Opus in 1.23 seconds. You ask a question, you immediately see output. That responsiveness is a big part of why cloud models feel fast even when their total generation time is longer.

Local models have a fundamentally different cost model. TTFT includes model loading time, and on first request after a cold start, that loading time is significant:

Devstral: 2.24s TTFT (best local, model stays warm in VRAM)
DeepSeek R1: 11.74s (14B params loading into VRAM)
Codestral 22B: 15.81s (22B params, larger model footprint)
Qwen 3.5B: 28.20s (35B total params, 23 GB model loading from disk into VRAM despite only 3B active)

Qwen's 28-second TTFT is brutal for interactive use. You type a prompt and wait half a minute before anything appears. The MoE architecture means the full model weight file is enormous even though inference is fast once loaded.

For batch processing and code generation, total time and throughput matter more than TTFT. And here, the picture flips. Devstral at 10.26 seconds total beat both cloud models. Once the local models are loaded and generating, their token throughput is competitive:

Model	Tok/s	Context
Qwen 3.5B	1,510.2	MoE architecture, 3B active params
DeepSeek R1	191.7	Includes reasoning tokens
Sonnet 4.6	104.2	Cloud, shared infrastructure
Codestral 22B	98.5	Full 22B model on single GPU
Devstral	90.2	24B model, balanced speed/quality
Opus 4.6	74.3	Cloud, larger model

Devstral found the sweet spot: fast enough TTFT to feel responsive, fast enough generation to beat the cloud on wall-clock time, and high enough quality to score perfectly. It's the model that made me stop thinking of local inference as a compromise.

The Verdict

For production coding tasks: Sonnet 4.6 or Devstral. Sonnet if you're already in the Anthropic ecosystem and want sub-second TTFT. Devstral if you want the same quality with zero API costs, zero rate limits, and total data privacy. Both scored 100. Devstral was actually faster end-to-end.

Opus 4.6 is capable but slower and more expensive for no quality gain on this task. Its strengths show on harder problems: multi-file refactors, complex debugging, architectural decisions. For straightforward code generation, you're paying a premium for capability you don't need.

Codestral 22B and DeepSeek R1 aren't bad models. They wrote valid, working code. The "failure" was a prompt interpretation issue that a single clarifying word would have fixed. In a conversational coding session where you can follow up, both would have corrected course immediately.

Qwen 3.5B is a speed demon trapped by token limits. At 1,510 tok/s it's the fastest generator by an order of magnitude, but that speed is wasted if you cap output too low. With proper max_tokens settings and the right tasks (short functions, completions, refactors), it could be the best option for high-throughput local work. We'll retest with higher limits.

The real takeaway isn't about which model "won." It's that the prompt matters as much as the model. Two models scored 60 because of a single ambiguous word in the prompt. One model scored 28 because of a configuration parameter. The gap between cloud and local quality has effectively closed for focused coding tasks. The remaining differences are in speed characteristics, token economics, and how forgiving the model is when your prompt isn't perfectly specific.

Local LLMs on consumer hardware aren't a compromise anymore. They're a legitimate option. Devstral proved it.

What's Next: Round 2

This was a useful first benchmark, but it was also a simple one. A single-file todo app with a clear spec is the kind of task where every model should do well. The interesting question is what happens when you make it harder.

Round 2 will use a more complex task: multi-file, with tests, with ambiguous requirements that force the model to make architectural decisions. We'll also adjust based on what we learned here. The prompt will be more explicit (no more "CLI or web" ambiguity that tripped up two models), and we'll give every model a larger context window and higher token limits so no one gets cut off mid-line.

We're also adding two models that came as requests from the Coder dev team:

Kimi K2.6 (Moonshot AI). A 1T-parameter MoE model with 32B active parameters and 256K context. It's getting strong benchmark scores and has native tool-calling support. The catch: even the most aggressively quantized version needs ~240 GB of memory, which is well beyond what the homelab can handle locally. We'll need to test this one via API.
Gemma 4 (Google). We need to research the available sizes and quantizations to see what fits on 32 GB of VRAM. If there's a version in the 14B-27B range, it could slot in alongside Devstral and Qwen as another local contender.

Both additions will be interesting tests of whether the "local models are good enough" conclusion holds with a harder prompt, and whether the newer model generation has closed the gap further.

By the Numbers

6 models benchmarked
1 prompt, identical across all models
3 perfect scores (Sonnet 4.6, Opus 4.6, Devstral)
2 models that built the wrong kind of app
1 model that ran out of tokens mid-f-string
10.26 seconds for Devstral to write a complete, working todo app
1,510 tokens per second from Qwen 3.5B (fastest local generation)
0.87 seconds for Sonnet 4.6's first token (fastest TTFT)
28.2 seconds for Qwen's first token (slowest TTFT)
4,096 token limit that killed an otherwise promising run
32 GB of VRAM making all of this possible on a single GPU
0 API costs for the three local models

Model Showdown Round 2: Adding Gemma, Kimi, and 579 GB of Stubborn Optimism

Rob — Thu, 07 May 2026 23:28:23 +0000

At the end of Round 1, we promised a rematch. More models. Fixed settings. Harder questions about what "local inference" really means when you push past what fits in VRAM.

This is that rematch.

We added two models that the Coder dev team specifically requested: Gemma 4 from Google (27B parameters, fits comfortably on the RTX 5090) and Kimi K2 from Moonshot AI (1 trillion parameters, does not fit in anything reasonable). We also reran every model from Round 1 with fixes for the configuration issues that tripped up three of them.

The results changed the leaderboard significantly.

What We Fixed from Round 1

Round 1 had three avoidable failures:

Qwen hit the token limit — scored 28/100 because the output was capped at 4,096 tokens and the code got truncated mid-f-string. The model was generating at 1,510 tok/s. It wasn't slow. We just cut it off.
Codestral and DeepSeek built interactive menus — both interpreted "commands: add, list, complete, delete" as while True: input() loops instead of CLI argument parsers. The code worked perfectly if you used it interactively. Our automated test suite couldn't.
Context windows varied — each model had different settings, making the comparison uneven.

For Round 2:

Setting	Round 1	Round 2
`num_predict` (max output tokens)	4,096	16,384
`num_ctx` (context window)	Varied	16,384 for all
Prompt clarity	"Commands: add, list, complete, delete"	"using argparse or sys.argv, NOT interactive input"
Model management	Random loading	Auto-unload previous, preload next

Same prompt. Same task. Same validation. Just fair settings this time.

Adding Gemma 4

Google released Gemma 4 while we were writing the Round 1 results. The 27B parameter model downloads as a 9.6 GB file through Ollama — the smallest of our serious contenders.

ollama pull gemma4

That's it. Model pulled, loaded onto the 5090 in seconds, registered in Coder's admin panel as another OpenAI-compatible model on the existing Ollama provider. The entire setup was one command and two form fields.

After Round 1's configuration adventure with five different models, this felt almost anticlimactic. In the best possible way.

Adding Kimi K2 (The Hard Way)

Kimi K2 is a different story entirely.

The numbers: 1 trillion total parameters, 32 billion active per token (Mixture of Experts architecture), 256K context window. The quantized model (Q4_K_M) is 579 GB across 13 shard files. Our RTX 5090 has 32 GB of VRAM.

We knew this going in. Round 1's post explicitly said Kimi would need API testing because it's too large for local. But this blog is about pushing boundaries with consumer hardware, and "it probably won't work" isn't a reason not to try. It's the reason to try.

Step 1: Getting llama.cpp Built

Ollama doesn't offer Kimi K2 for local inference — only a cloud-hosted variant. So we went to llama.cpp, the C++ inference engine that supports loading models larger than VRAM via memory-mapped NVMe offloading.

Building it required installing half of Ubuntu's dev toolchain:

sudo apt install -y cmake build-essential nvidia-cuda-toolkit
cd ~ && git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release -j$(nproc)

First roadblock: cmake wasn't installed. Fixed with apt.

Second roadblock: CUDA toolkit not found. Fixed with nvidia-cuda-toolkit.

Third roadblock: nvcc fatal: Unsupported gpu architecture 'compute_120a'. The RTX 5090 is Blackwell architecture (compute 12.0), but Ubuntu's apt CUDA toolkit is version 12.0 — too old to know about it. The fix was targeting an older compatible architecture:

cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=89

Compute capability 89 (Ada Lovelace) runs fine on the 5090 via backward compatibility. Not ideal, but it builds.

Step 2: Downloading 579 GB

Next: the Hugging Face CLI. Which required pip. Which was externally managed. Which required --break-system-packages. Which installed but wasn't on PATH. Which turned out to be deprecated in favor of the hf CLI. Which required python3.12-venv. Which left behind a broken virtual environment that needed manual cleanup.

sudo apt install -y python3-pip python3.12-venv
pip install huggingface-hub[cli] --break-system-packages
rm -rf ~/.hf-cli
curl -LsSf https://hf.co/cli/install.sh | bash
source ~/.bashrc

Then the actual download:

~/.local/bin/hf download unsloth/Kimi-K2-Instruct-GGUF --include "*Q4_K_M*" --local-dir ~/models/kimi-k2

The download started reporting 384 GB, then revised upward to 432 GB, then 481 GB, then settled at 529 GB. The HF CLI discovers shards progressively — it didn't know the full file list upfront.

Kimi K2 mid-download — 327 GB down, revising the total upward as new shards are discovered.

3 hours and 27 minutes later, 13 shard files totaling 579 GB sat on the NVMe. At ~370 Mbps sustained throughput.

Step 3: The VRAM Math

First attempt: 10 GPU layers. Tried to allocate 94 GB on a 32 GB card. Dead.

The math: 94 GB / 10 layers ≈ 9.4 GB per layer. With 32 GB of VRAM, that's roughly 3 layers maximum. MoE architectures make each layer massive because every expert's weights live in the same layer.

We settled on 2 GPU layers (confirmed working, 3 was borderline). That means ~18 GB on the GPU, the remaining ~560 GB paging from NVMe via memory-mapped I/O. The OS's virtual memory system handles the page faults — when inference needs weights that aren't in RAM, it reads them from the NVMe on demand.

Step 4: The Conversation Mode Bug

Here's where it got interesting. llama.cpp's llama-cli has a --no-conversation flag that's supposed to run a single prompt and exit. It doesn't work. Every run dropped into an interactive > prompt, waiting for input. Our benchmark script would hang indefinitely.

We tried:

--no-conversation flag (ignored)
--no-display-prompt flag (still conversational)
Piping prompt via -p with -e flag (still conversational)

llama-cli ignoring --no-conversation and dropping into an interactive prompt, hanging the benchmark script.

Three benchmark attempts. Three hangs. The script captured zero timing data from Kimi because it was waiting for a conversation that would never end.

Step 5: The Fix — llama-server

Instead of fighting the CLI, we ditched it. llama.cpp ships with llama-server, which exposes an OpenAI-compatible HTTP API — the exact same interface Ollama uses. We wrote a standalone benchmark script that:

Starts llama-server as a background process
Polls /health until the 579 GB model finishes loading
Sends the benchmark prompt to /v1/chat/completions with streaming
Captures every metric programmatically — TTFT, total time, tokens, tok/s
Runs the full validation suite
Shuts down the server

No conversation mode. No stopwatch. No manual intervention.

server_cmd = [
    LLAMA_SERVER,
    "-m", MODEL_PATH,
    "--n-gpu-layers", str(N_GPU_LAYERS),
    "--mmap",
    "-c", str(CTX_SIZE),
    "--port", str(PORT),
]
server_proc = subprocess.Popen(server_cmd, ...)

# Wait for 579 GB to load into memory
wait_for_server(PORT, timeout=900)

# Hit the same API as Ollama
url = f"http://127.0.0.1:{PORT}/v1/chat/completions"

It worked on the first try. The model loaded in 375 seconds (6.3 minutes), then generation began.

Kimi K2 generating code at 0.6 tokens per second. Every character is paging through 579 GB of weights on an NVMe drive.

The Results: Performance

Raw benchmark output — Qwen finishing its run and Kimi K2 kicking off next.

Model	TTFT	Total Time	Output Tokens	Tok/s	Lines
Codestral 22B	1.75s	10.01s	826	82.5	80
Devstral 24B	2.11s	9.97s	703	70.5	98
Gemma 4 27B	3.92s	11.77s	1,966	167.1	171
DeepSeek R1 14B	7.21s	12.44s	1,451	116.7	84
Qwen 3.5 MoE 35B	27.00s	35.23s	5,020	142.5	144
Kimi K2 1T	68.90s	1,140.94s	686	0.6	87

Gemma 4 immediately stands out. 167 tok/s is the fastest generation speed of any model we've tested that also scored perfectly — faster than Sonnet 4.6's 104 tok/s from Round 1. It wrote 1,966 tokens (171 lines) in under 12 seconds.

Devstral remains the wall-clock champion at 9.97 seconds total, though Codestral edges it on TTFT (1.75s vs 2.11s).

Kimi K2 is in a different universe. 68.9 seconds before the first token appeared (that's prompt evaluation at 1.5 tok/s across 579 GB of weights). Then 19 minutes of generation at 0.6 tok/s. Total wall clock including model load: 25 minutes.

The Results: Quality

Model	Syntax	Features (X/10)	Functional (X/7)	Score
Gemma 4 27B	Yes	10/10	7/7	100
Devstral 24B	Yes	10/10	7/7	100
DeepSeek R1 14B	Yes	10/10	7/7	100
Qwen 3.5 MoE 35B	Yes	10/10	7/7	100
Codestral 22B	Yes	9/10	7/7	94
Kimi K2 1T	Yes	10/10	6/7	94

Five out of six models scored 100. That's up from three in Round 1.

The Fixes Worked

Qwen: 28 → 100. With the token limit raised from 4,096 to 16,384, Qwen wrote 5,020 tokens — a complete 144-line program with ANSI color codes, proper error handling, and clean argparse subparsers. The speed is still absurd (142.5 tok/s with a 27-second cold start), but now it finishes what it starts.

DeepSeek R1: 60 → 100. The clarified prompt ("NOT interactive input") worked. DeepSeek built an argparse-based CLI with proper subparsers, colorama integration, and structured error handling. It still uses <think> blocks (the 1,451 tokens include reasoning), but the final code is correct.

Codestral: 60 → 94. Also switched to argparse, passing all 7 functional tests. But it missed error handling entirely — no try/except blocks, no input validation. Its complete command also silently deletes the record instead of marking it done. Functional but sloppy.

The New Models

Gemma 4 wrote the most polished code of any model in either round. 171 lines with a dedicated Colors class for ANSI escape codes, emoji status indicators (✅, ⏳, 🎉, 🗑️), full try/except/finally blocks on every database operation, and a clean argparse architecture. It writes like a senior developer who actually cares about user experience.

Kimi K2 wrote clean, minimal code — 87 lines with with context managers for database connections (the most Pythonic approach of any model), proper sys.exit(1) on errors, and a formatted table output. It scored 94 instead of 100 because one functional test failed: the delete command reported "Task 2 not found" due to the model storing its database at ~/.todo.db (a global path) instead of a relative path. Stale data from an earlier test run interfered. The code logic is correct — it's a test isolation issue, not a bug.

Style Comparison: How Each Model Writes

The code style differences are telling:

Gemma 4 (171 lines): Enterprise polish. ANSI color class, emoji, docstrings on every function, defensive error handling everywhere. The code you'd put in a demo.

Qwen 3.5 (144 lines): Also polished — ANSI codes, structured table output, exit-on-error patterns. More defensive than Gemma but less decorative.

Devstral (98 lines): Minimal and correct. Flat functions, no class, CURRENT_TIMESTAMP in SQL. The code you'd actually ship.

Kimi K2 (87 lines): Even more minimal. Context managers everywhere, zero waste. Reads like it was written by someone who's read a lot of production Python.

DeepSeek R1 (84 lines): Compact with colorama dependency — the only model that imported an external library. Risky in an isolated test environment.

Codestral (80 lines): The shortest, and it shows. No error handling, buggy complete command. Brevity at the cost of correctness.

The Speed Tiers

Round 2 reveals three distinct performance tiers for local inference:

Tier 1: VRAM-Native (~10-35 seconds)

Models that fit entirely in the RTX 5090's 32 GB VRAM. Response times competitive with cloud APIs.

Model	Size	Total Time	Tok/s
Devstral 24B	14 GB	9.97s	70.5
Codestral 22B	12 GB	10.01s	82.5
Gemma 4 27B	9.6 GB	11.77s	167.1
DeepSeek R1 14B	9 GB	12.44s	116.7
Qwen 3.5 MoE 35B	23 GB	35.23s	142.5

Tier 2: NVMe-Offloaded (~19 minutes)

Models too large for VRAM, paging from NVMe via mmap. Functional but glacial.

Model	Size	Total Time	Tok/s
Kimi K2 1T	579 GB	1,141s	0.6

The gap between tiers is ~100x. Gemma 4 at 167 tok/s vs Kimi K2 at 0.6 tok/s. Both wrote correct code. One took 12 seconds, the other took 19 minutes.

This isn't a criticism of Kimi K2 — it's a 1 trillion parameter model running on hardware that costs less than a month of cloud API credits. The fact that it works at all is the story. The fact that it wrote correct, clean, well-structured code is the punchline.

Round 1 vs Round 2: Combined Leaderboard

Model	Round	Size	Tok/s	Score
Gemma 4 27B	R2	9.6 GB	167.1	100
Sonnet 4.6	R1	Cloud	104.2	100
Devstral 24B	R2	14 GB	70.5	100
Opus 4.6	R1	Cloud	74.3	100
Qwen 3.5 MoE 35B	R2	23 GB	142.5	100
DeepSeek R1 14B	R2	9 GB	116.7	100
Codestral 22B	R2	12 GB	82.5	94
Kimi K2 1T	R2	579 GB	0.6	94

Gemma 4 is now the fastest model with a perfect score — local or cloud. A 9.6 GB model running on consumer hardware, outperforming Anthropic's Sonnet 4.6 on raw throughput while matching it on code quality.

The local-vs-cloud gap hasn't just closed. On this task, local won.

What We Learned

Configuration matters more than model selection. Three models went from failing to perfect with two setting changes. If your local models are underperforming, check your token limits and prompt clarity before blaming the model.

The prompt is still the variable. Round 1's "ambiguous CLI" issue was a prompt problem, not a model problem. Six words ("NOT interactive input") fixed two models.

VRAM is the cliff. The performance difference between "fits in VRAM" and "doesn't fit in VRAM" is 100x. There's no gradual degradation — you're either generating at 70-167 tok/s or you're at 0.6. If your model fits, you're competitive with cloud. If it doesn't, you're watching paint dry.

Big models can still write good code slowly. Kimi K2 at 0.6 tok/s is impractical for interactive coding. But for batch processing, overnight code generation, or "I need an answer and I don't care when" use cases, a 1T model on consumer NVMe is a real option that didn't exist a year ago.

Gemma 4 is the new default. Fastest throughput, perfect score, smallest download, most polished output. If you're running a homelab with a single GPU, it's the model to install first.

What's Next: Gemma vs Opus — A Real Fight

Round 1 tested a toy todo app. Round 2 fixed the settings and added models. Both rounds answered a useful question: can local models write correct code for a well-defined task?

The answer is yes. Five out of six scored perfect. That question is settled.

The next question is harder: can a local model replace my daily driver on a real task?

My daily driver is Opus 4.6. It's what I use for everything on vibescoder.dev — features, refactors, debugging, the works. It's also a cloud model with per-token costs, rate limits, and a dependency on someone else's infrastructure.

Gemma 4 just beat every model in the benchmark on speed and matched the best on quality. It runs locally on my 5090 at 167 tok/s with zero API costs. The obvious question: can it actually do the job?

Round 3 will be a head-to-head. Gemma 4 vs Opus 4.6, same task, but not a toy. We're going to pick a real feature from the vibescoder.dev backlog — something that touches multiple files, requires architectural decisions, and has enough ambiguity to separate a good model from a great one. The kind of task I'd normally hand to Opus without thinking.

If Gemma holds up, local-first AI coding isn't just viable for benchmarks. It's viable for production.

By the Numbers

6 local models benchmarked (up from 4 local + 2 cloud in Round 1)
5 perfect scores (up from 3)
579 GB downloaded over 3 hours 27 minutes for Kimi K2
375 seconds to load 579 GB into memory-mapped NVMe
68.9 seconds for Kimi K2's first token
1,140 seconds (19 minutes) for Kimi K2's total generation
9.6 GB for Gemma 4 — smallest model, highest score + speed
167.1 tok/s from Gemma 4 — fastest perfect-scoring model across both rounds
0.6 tok/s from Kimi K2 — slowest, but correct
16,384 token limit that saved Qwen from another truncation
2 GPU layers out of ~60+ that fit in VRAM for Kimi K2
3 Round 1 bugs fixed by configuration changes, not model changes
1 llama-cli conversation mode bug worked around with llama-server
0 API costs for everything

DEV Community: Rob

From Idea to Infrastructure: Standing Up a Self-Hosted AI Dev Environment

Why Self-Hosted, Why Now

The Hardware Hack: Buy a Gaming PC

Standing Up Coder

GitHub Auth: The Long Way Around

Wiring GitHub Into the Workspace Template

System Instructions That Actually Work

The Vibe Coding Toolchain

Multi-User Setup

The Architecture

Gotchas Worth Knowing

What's Next: Local LLMs

By the Numbers

Putting the GPU to Work: Running Local LLMs on a Home Lab

Why VRAM Is the Only Spec That Matters

The Setup Script

What Happened When We Ran It

Phase 1: Hardware Detection

Phase 2: Ollama Install

Phase 3: Service Configuration

Phase 4: Model Downloads

Phase 5: Verification

Why These Models

Wiring It Into Dev Tools

Continue.dev (VS Code)

Environment Variables

Connecting to Coder Agents

Step 1: Add the Provider

Step 2: Add Models

Step 3: Use It

What Surprised Us

Ollama vs vLLM: When to Scale Up

Gotchas

What's Next

By the Numbers

The Agentic Gap: Claude Oneshots, Gemma Fails

The Experiment

Setting Up the Arena

Opus 4.6: The Quiet Professional

What Opus Built

Gemma 4: The Brilliant Planner Who Never Coded

The AGENTS.md Experiment

The Scoreboard

Technical Review: Opus Implementation

What We Learned

What's Next

By the Numbers

Thursday Thought: Chat is the New Source Code

The Paradigm Shift: From Code to Conversation

Storing Chat History in GitHub: A Game Changer

Intent Over Implementation

The Future of Version Control

What This Means for Developers

1. Documentation Becomes Native

2. Collaboration Changes

3. Debugging Gets Easier

The Big Picture

Your AI Strategy Has a Blind Spot: An SEO and AEO Audit of vibescoder.dev

The TL;DR for Non-Technical Readers

The TL;DR for Technical Readers

The Audit

The Cloudflare Gotcha (Yes, Again)

What Is AEO?

AEO-Specific Changes

llms.txt and llms-full.txt

Person Schema with sameAs

Full-Content RSS

Unblocking AI Crawlers

SEO-Specific Changes

Sitemap.xml

Canonical URLs

Homepage Caching

Custom 404 Page

Changes That Help Both

JSON-LD Structured Data

Heading Anchor IDs

RSS Feed Fix

Article Meta Tags

Image Improvements

Person Schema with `sameAs`