DEV Community: Ken Imoto

200ms Humans vs 700ms Voice Agents: What ACL IWSDS 2025 Says About the Turn-Taking Gap

Ken Imoto — Wed, 23 Sep 2026 13:00:00 +0000

The complaint that keeps showing up in every voice-agent beta test I've watched is not about the wrong answer. It's about the timing of the right answer.

"The AI cut me off." "It answered while I was still thinking." "I said 'um' and it started talking." Latency looks like a stopwatch problem, but users experience it as a manners problem, which is a much harder thing to fix.

The 200ms vs 700ms gap the field is trying to close

The ACL IWSDS 2025 survey on turn-taking in spoken dialogue puts a concrete number on the mismatch. In natural human conversation, a listener starts their reply roughly 200ms after the speaker's turn ends. Current dialogue agents take 700 to 1000ms to do the same thing.

Three to five times slower. If human dialogue is a tennis rally, current voice AI is playing chess by post.

Closing that gap is not one problem, it's two. And they pull in opposite directions.

Too slow to reply: awkward silence, dropped rapport, users start repeating themselves.
Too fast to reply: the agent talks over the user, interrupts mid-word, kills trust.

Both fail modes come from the same root cause: silence is not the same as end-of-turn.

Why plain VAD is not enough

Most current systems still decide the user is done talking with a Voice Activity Detector. VAD is cheap and fast, and it answers exactly one question: is there voice in this audio frame, yes or no.

That question is not the one the agent actually needs answered.

The 0.3–0.5s pause a speaker takes mid-sentence looks identical to end-of-turn.
The silence after "uh" or "let me think" looks identical to end-of-turn.
A cough or a sigh looks like voice.

You can tune the silence threshold, but you cannot tune your way out of the fact that VAD does not know what a sentence is. VoiceInfra's production data lands on 300–500ms of trailing silence as the least-bad setting: shorter and the agent chops off natural pauses, longer and the delay becomes obvious. The comfortable range depends on the use case. Call-center flows tolerate 400–500ms because callers pause to think; command interfaces want 200–300ms because utterances are short; long-form narration wants 500–600ms because the pauses inside the story are longer.

The threshold is a compromise, not a fix. The real fix has to know what the words mean.

Semantic endpointing: Deepgram Flux and the "one model does both" bet

The interesting move in 2026 is folding transcription and turn detection into the same model. Deepgram's Flux, launched as their first ASR built for voice agents rather than for general transcription, does exactly that. The model outputs turn boundaries directly from audio, using the same joint architecture it uses to output text. Because the same weights see both the acoustic signal and the emerging transcript, the model has a shot at answering the harder question: "is this utterance semantically complete?"

Flux exposes three knobs that make the tradeoff explicit rather than hidden inside a silence threshold:

eot_threshold — how confident the model has to be before it commits to end-of-turn.
eager_eot_threshold — a lower bar for tentative end-of-turn, useful for starting inference speculatively.
eot_timeout_ms — the maximum silence duration before the model forces end-of-turn even if the confidence never crosses the threshold.

You can also switch modes per-turn with a Configure message, so a barge-friendly interaction ("agent, wait") and a monologue turn ("read me the terms") do not have to share a threshold.

The pattern is a hint at where the field is going. Turn-taking will stop being a pipeline stage after ASR and start being a joint output of the ASR itself.

Krisp's 6M-parameter turn model: same idea, edge-shaped

Not everyone can afford a cloud round-trip on every silence check. Krisp ships two small audio-only models trained to handle the acoustic cues a naive VAD collapses — a 9M-parameter turn prediction model (~30MB) that scores end-of-turn from prosody and pausing patterns, and a 6M-parameter interruption model (~24MB) that separates real barge-ins from backchannels like "yeah" and "mhm". Together they cover the same failure modes VAD misclassifies:

intentional speech vs a thinking pause
filler words (um, uh, well) mid-sentence
backchannels vs interruptions

At that size it runs on-device in real time, which matters for privacy-sensitive deployments and for wearables where the round-trip to the cloud is itself the biggest source of latency.

The Krisp and Flux paths look opposite. One shrinks the model, the other gives the ASR the extra job. But they are attacking the same VAD failure mode from opposite sides.

Graceful abort: what to do when you're wrong

No detector, semantic or acoustic, is going to be right every time. The interesting question is what your pipeline does when it decides "the user is done" and turns out to be wrong.

Twilio's graceful abort pattern is the cleanest version I've seen written up. When STT signals end-of-turn early, the LLM starts generating a response. If fresh audio arrives before the response reaches the speaker, the pipeline kills the generation in flight and swallows the partial output. The window for this is the few hundred milliseconds it takes STT → LLM → TTS to produce audible speech. If you can revoke the guess inside that window, the user never hears it.

The economics matter here: you are paying for LLM tokens on turns that get thrown away. In exchange, you get to be aggressive on end-of-turn detection without punishing the user when you're aggressive-and-wrong. For most conversational products that tradeoff is worth it.

Barge-in: hearing the user through your own voice

The other half of turn-taking is the reverse case: the user interrupts while the agent is still speaking. Detecting that seems trivial until you realize the AI's own audio is leaking into the microphone, and any naive VAD will happily flag that leak as a barge-in and abort the agent's own utterance.

Sparkco's write-up on duplex barge-in handling is the clearest description of the fix I've read. Three moving parts:

Full-duplex audio — keep the mic hot the entire time the agent is speaking, never gate it on TTS output.
Echo cancellation — subtract the speaker output (as a reference signal) from the mic input, so what remains is the user's voice minus the agent's.
Nuisance rejection — filter environmental noise and short transients that a bare VAD would misclassify as speech onsets.

[agent is speaking]
  speaker out ──► reference signal ─┐
                                    ▼
  mic in ─────► echo canceller ────► user voice only ──► VAD / turn model

Without echo cancellation you get the failure mode users describe as "the AI got startled by its own voice and stopped talking." The dog scared of its own reflection.

The bot-feel paradox and the 200-300ms delay trick

Now the counterintuitive move. Once you've closed the acoustic and semantic gaps and the agent can reply in 300ms, it turns out that replying in 300ms feels worse. The response arrives before the user has finished processing their own sentence, and it registers as robotic rather than sharp.

The fix is to inject 200–300ms of intentional delay before the agent starts speaking, while continuing to run the LLM in the background. You get the "thinking for a moment" cue humans read as attention, without paying real latency for it. The tokens are already streaming; you're just holding the TTS start.

It's a UX trick, not a technical one. But it's the piece of the puzzle that gets forgotten when a team spends a quarter shaving milliseconds off ASR and then wonders why user ratings didn't move.

What actually closes the 500ms gap

There is no single component that takes you from 700ms to 200ms. You get there by stacking:

Layer	What it buys you
Semantic endpointing (Flux-class)	Stops chopping mid-sentence, stops waiting for silence that already means end-of-turn.
Small on-device turn + interruption models (Krisp-class)	Score end-of-turn and separate backchannels from real barge-ins without a round-trip.
Graceful abort	Lets you be aggressive on early end-of-turn without punishing the user when wrong.
Duplex barge-in with AEC	Lets you leave the mic hot without the agent interrupting itself.
200–300ms intentional delay	Buys back the "thinking" cue that pure speed removes.

The move that unlocks the stack is admitting that end-of-turn is a language problem, not a silence problem. Once you accept that, VAD-only pipelines look the way pre-BERT NLP pipelines look now: perfectly reasonable for their era, obviously incomplete in retrospect.

The gap to 200ms is not going to close by tuning thresholds. It closes when the model that hears the audio is the same model that understands the sentence.

The full 300ms-UX playbook (latency budgets by pipeline stage, when to pick Whisper vs Deepgram vs Piper, and how to design the wait states so users forgive the last 200ms you can't remove) is written up in Voice AI 300ms UX. Chapter 9 covers turn-taking end-to-end; chapters 4 and 5 cover the latency anatomy that decides whether closing the gap is even possible for your stack.

References

ACL IWSDS 2025. "A Survey of Recent Advances on Turn-taking Modeling in Spoken Dialogue."
Krisp AI. "Audio-only, 6M weights Turn-Taking model for Voice AI Agents." 2025.
Twilio. "Core Latency in AI Voice Agents." 2025.
Deepgram. "Flux: End-of-Turn Detection Parameters." 2026.
Sparkco. "Optimizing Voice Agent Barge-in Detection." 2025.
VoiceInfra. "Voice AI Prompt Engineering: Complete Technical Guide." 2025.

Claude Code + Playwright MCP + GitHub Actions: 3 Test Layers That Actually Kill Regressions

Ken Imoto — Mon, 21 Sep 2026 13:00:01 +0000

The first time I asked Claude Code to write tests for a feature it had just implemented, the tests passed. All of them. On the first try. I felt great about it for about ten minutes, until I realized the tests were describing the code, not the requirements. The code had a bug I had specified against, and the tests happily locked the bug in.

That is context pollution. It is the fundamental problem with letting one model both implement and verify. The New Stack put it best: letting the model that wrote the code write the tests is asking a student to grade their own homework.

The fix is architectural, not attitudinal. Split the loop across three isolated contexts, each running at a different point in the development lifecycle. Here is what those three layers look like when you build them out.

Layer 1: TDD by subagent (dev time)

Red-Green-Refactor works with Claude Code, but not the way most teams first try it. The naive setup is one Claude Code session doing all three phases. That is the setup that produces homework-grading.

The working setup uses three subagents, each blind to the others' work.

Skills (orchestrator)
  ├── tdd-test-writer   (RED)    — sees feature description, NOT implementation
  ├── tdd-implementer   (GREEN)  — sees test file, NOT the writer's reasoning
  └── tdd-refactorer    (REFACTOR) — sees passing code, must keep tests green

The critical bit is that each subagent runs in its own context. The test writer cannot peek at how the implementer plans to solve the problem. The implementer cannot see the notes the test writer wrote about "edge cases we should probably cover."

A minimal skill definition for the writer:

# .claude/agents/tdd-test-writer.md

You are a test writer. Your ONLY job is to write failing tests.

Rules:
- You CANNOT see or reference any implementation files
- Write tests based ONLY on the feature description
- Tests must cover happy path + edge cases (nulls, empty, negative, overflow)
- Run tests and confirm they FAIL before finishing

And the implementer, which is deliberately narrow:

# .claude/agents/tdd-implementer.md

You are an implementer. Your ONLY job is to make tests pass.

Rules:
- Write the MINIMUM code to make tests pass
- Do NOT refactor or optimize (that's the refactorer's job)
- Do NOT modify test files
- Run tests and confirm they PASS before finishing

Why this works: the "design intent" that a normal Claude Code session leaks between phases now cannot leak. The test author writes what the feature should do; the implementer writes what makes those tests pass; the refactorer cleans up without breaking anything. It is exactly the discipline TDD was invented to enforce, made mechanical by the fact that context boundaries are hard walls now.

The other nice property: you can CLAUDE.md this policy at the repo level and forget about it.

## Testing policy
- All new features go through the TDD subagent chain
- No test code and implementation code in the same Claude session
- Skip TDD only for one-line typo fixes and docs

Layer 2: Quinn, the Playwright MCP QA agent (PR time)

The second layer runs at pull-request time and does the thing no unit test can do: click around the UI like a person who has never seen the code.

Alexop's Quinn pattern is the reference here. Quinn is an AI QA engineer defined by a prompt: a 12-year-veteran QA persona, instructed to test only through the UI, never look at source, always exercise the mobile viewport (375x667), and file findings with reproduction steps.

The mechanism that makes Quinn possible in 2026 is Playwright MCP. The server ships dozens of browser-automation tools exposed over MCP, and the important detail is that it hands the agent an accessibility tree as structured JSON rather than a screenshot. No vision model needed. Claude Code reads the tree, decides what to click, tells Playwright to click it, reads the resulting tree, and so on. Deterministic, cheap, and stable across page redesigns.

A working .github/workflows/ai-qa.yml:

name: AI QA (Quinn)
on:
  pull_request:
    types: [labeled]

jobs:
  qa:
    if: github.event.label.name == 'qa-review'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: pnpm install && pnpm build
      - name: Start dev server
        run: pnpm dev &

      - uses: anthropics/claude-code-action@v1
        with:
          anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
          claude_args: |
            --mcp-config '{
              "mcpServers": {
                "playwright": {
                  "command": "npx",
                  "args": ["@playwright/mcp@latest", "--headless", "--no-sandbox"]
                }
              }
            }'
          prompt: |
            You are Quinn, a senior QA engineer with 12 years of experience.
            Test the application at http://localhost:3000.
            DO NOT look at source code. Test only through the UI.
            Test on desktop (1280x720) and mobile (375x667).
            Try edge cases: empty inputs, huge inputs, negatives, unicode names.
            Report each bug with a screenshot and a reproduction sequence.

Two flags in that config carry more weight than they look: --headless --no-sandbox on the MCP server side makes it run under GitHub Actions' container without display. Claude Code itself runs non-interactively under claude-code-action@v1, so the action handles the "exit when done" part for you.

Quinn is not a replacement for scripted E2E tests. Scripted E2Es catch known-bad regressions. Quinn catches the class of bug where "nobody thought to write a test for that particular sequence." Two different tools, two different failure modes.

The trigger being types: [labeled] matters. Quinn burns real tokens, and running her on every push to every PR gets expensive. Making her opt-in via a qa-review label means the team pays for her when the PR is close to ready, not while it is still churning.

Layer 3: claude-code-action on the CI gate (merge time)

The third layer is the boring one, which is why it works. Once a PR passes Quinn, one more Claude Code invocation runs on the CI gate, looking at the diff against the existing test suite.

# .github/workflows/claude-test.yml
name: Claude Code Test Gate
on:
  pull_request:
    types: [opened, synchronize]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '22'
      - run: pnpm install

      - uses: anthropics/claude-code-action@v1
        with:
          anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
          prompt: |
            Analyze the changes in this PR. Then:
            1. Run the existing test suite: pnpm test
            2. Identify code paths in this diff not covered by any test
            3. Propose specific test cases for the gaps (do not write them yet)
            4. Post the analysis as a PR comment

This is the "did we miss anything" pass. It does not write tests (that would push us back into homework-grading), it flags gaps. A human reads the comment, decides which gaps matter, and either accepts them or opens follow-up tickets.

Anthropic ships claude-code-action@v1 as the official integration, and the setup is one CLI call away — /install-github-app from Claude Code walks you through installation, permissions, and mentioning @claude on issues or PRs to get responses.

Between the three layers, the diff of who runs which tests when looks like this:

Layer	When	Who tests	What they use
1. TDD subagents	Dev time (per commit)	Claude Code, subagent per phase	Feature description + failing test
2. Quinn (Playwright MCP)	PR time (label-triggered)	Claude Code as QA persona	Browser via MCP, no source access
3. claude-code-action	Merge time (every push)	Claude Code as reviewer	Diff + existing suite + coverage report

Notice what each layer has that the others don't. Layer 1 has the requirements but not the implementation. Layer 2 has the running app but not the source. Layer 3 has the diff but is forbidden from writing tests. Each layer's blindness is what makes it useful.

Why selector reliability improves under this shape

Small side-benefit worth mentioning. When Quinn writes down what she clicked (for the reproduction steps she files with bugs), she writes it in Playwright MCP's accessibility-tree vocabulary. That means:

// The kind of selector Quinn produces
await page.getByRole('button', { name: 'Log in' }).click();

// Not the kind you get from asking Claude to "look at the DOM"
await page.click('.btn-primary.mt-4.px-6');

The role-plus-text form survives redesigns, works with screen readers, and doesn't break the first time someone renames a Tailwind class. When you promote Quinn's reproduction steps into permanent E2E tests (which you should, whenever the bug is worth guarding against), you get selector stability for free.

What each layer is doing that the others can't

The reason context pollution is a whole category of AI-testing bug is that most teams try to solve testing with more AI in one place. More prompts, better instructions, longer CLAUDE.md rules about "please check your own work." None of that helps. The problem is information leakage between phases, and stronger prompts inside one leaky pipeline still leak.

Three isolated contexts, three different data diets:

The test writer eats requirements, produces tests.
Quinn eats the running app, produces bug reports.
The CI reviewer eats the diff, produces gap analysis.

Roll any two of those together and you get a subtle regression somewhere. Keep them separate and each does its narrow job well.

The Anthropic tooling to make this work — subagents, Playwright MCP, claude-code-action — all landed in 2025-2026. The pattern was possible before, in the way most patterns are "possible" before the tooling; now it is a config-file exercise, not a research project.

If your team already has one Claude Code session doing all three jobs, split them next sprint. The tests will start finding things the single-session setup silently endorsed.

The full Claude Code testing playbook (subagent memory design, when to use skills vs commands vs subagents, how to structure a repo so the three layers stay independent, and what to do when Quinn files a false positive) is written up in Claude Code Mastery. Chapter 10 covers test automation end-to-end, including the TDD-subagent split this article is a distillation of.

References

The New Stack. "Claude Code and the Art of Test-Driven Development." 2025.
alexop.dev. "Building an AI QA Engineer with Claude Code and Playwright MCP." 2025.
alexop.dev. "Forcing Claude Code to TDD: An Agentic Red-Green-Refactor Loop." 2025.
Anthropic. "Claude Code GitHub Actions." Official docs, 2026.
Microsoft Playwright team. "Playwright MCP Server." github.com/microsoft/playwright-mcp, 2026.

MCP Resources vs Tools vs Prompts: 3 Layers That Cut My Agent's Tokens From 114K to 27K

Ken Imoto — Tue, 15 Sep 2026 13:00:01 +0000

Most MCP posts on this site stop at the Tools layer. That is the layer everyone builds first: get_weather(), send_email(), query_database(). It is also the layer that will eat your token budget if you leave it as the only thing you use.

The Model Context Protocol has three layers, not one. Resources, Tools, and Prompts. Each of them solves a different problem, each of them costs a different amount of tokens, and each of them is supported unevenly across clients right now. On the same browser-automation task, a CLI-style setup that pre-loaded the full tool documentation into the context ran at 114,000 input tokens. The MCP-native setup, where the server abstracted the low-level details away, ran at 27,000. A 4.2x cut without changing the model or the underlying browser — pulled from the case study in the Context Engineering book this article is adapted from.

Below is the concrete framework I use to decide which layer a piece of context belongs in, why the three layers cost so differently, and what to do when your MCP client only supports the layers it feels like supporting.

The 3 layers, side by side

Layer	What it is	When to use	Cost profile
Resources	Read-only information sources the model can pull on demand (`file://`, `db://`, `api://`)	Static or slow-changing context the agent may need	Cheap. Pulled only when referenced.
Tools	Executable operations with side effects (`create_file`, `send_email`)	Actions the agent takes in the world	Moderate. Definition sits in context every call.
Prompts	Reusable, parameterized templates the client can invoke	Repeated flows: code review, ticket triage, incident response	Cheap. Server-side; the client fetches on request.

Skip any of these three and you either lose expressiveness or you overpay in tokens. Both mistakes are common.

Why 114K → 27K happened

The 114K number came from a CLI-style approach to browser automation: dump the full command-line documentation, error handling, and usage examples for every primitive into the context, so the LLM knows how to compose them. Something like this:

# CLI-style design (114K tokens per run)
context = "Available CLI commands:\n"
context += generate_cli_documentation()   # 30 primitives, each with full docs
context += generate_cli_examples()        # example usage for each
context += generate_error_handling_docs() # every error code + retry strategy
# task itself: ~8K tokens sitting on top of ~106K of tooling documentation

The 27K number came from an MCP-native rewrite where the server hides those details behind a small set of high-level tools. The chapter this article adapts frames it as roughly this:

# MCP-native design (27K tokens per run)
tools = [
    "playwright_navigate(url)",
    "playwright_click(selector)",
    "playwright_type(selector, text)",
    "playwright_screenshot()",
    "playwright_extract_text(selector)",
]
# server holds the CLI-level docs, retry policies, and error mapping internally

The book credits three mechanisms for the drop:

Abstraction level. CLI-style is low-level detail. MCP is a high-level abstraction. The client sends intent; the server figures out how to execute.
Separation of context. The MCP server holds implementation details. The client only sees what it needs to decide the next action.
Dynamic resolution. MCP resolves details when the tool actually runs. CLI-style has to describe everything up front, in case it is needed.

Where the three MCP layers come in is that each of them is one of those separation strategies. Tools are the intent layer. Resources are how the "docs the agent occasionally needs" get moved off the always-loaded context. Prompts are how the "repeat this whole flow again" logic gets moved onto the server. Together they are what makes the 4.2x abstraction gain possible in the first place.

The one-line takeaway from the chapter: your token bill is the description of the tools, not the tools themselves.

Which layer does a piece of context belong in?

The decision is not that hard once you have a rule. Here is the one I use.

Ask: does the agent need to do something, read something, or repeat something?

Do something → Tool. Side effects, mutations, external calls.
Read something → Resource. Read-only, addressable by URI, pulled on demand.
Repeat something → Prompt. Templates the client can invoke by name.

If you feel yourself dumping a full API doc into a tool's description field, stop. That is a Resource. If you feel yourself teaching the model the same 6-step recipe over and over in the system prompt, stop. That is a Prompt.

Tools are the loud, expensive layer because their definitions live in the context on every call. Resources and Prompts are the quiet, cheap layers because their content is fetched only when the agent decides it needs them. Most teams over-invest in Tools and under-invest in Resources and Prompts, then wonder why their agents feel expensive.

What your client actually supports right now (September 2026)

Here is the part nobody warns you about. The MCP spec has three layers. The clients do not implement all three uniformly.

Claude Desktop supports Resources, Tools, and Prompts. Prompts appear in the slash-menu inside the chat.
Cursor supports Tools well. Resource support has been improving through 2026 but is uneven. Prompt support arrived in a recent update and is now working end-to-end in the mcp.json config.
ChatGPT has been rolling out MCP since 2025 (Developer Mode beta shipped September 2025), with the Enterprise and Business rollout continuing into 2026. Tool support is solid. Resources and Prompts are still shipping.
Custom SDK clients (via the Python or TypeScript SDK) implement whatever you write. The floor is Tools; adding Resources and Prompts is trivial in the SDK and worth doing.

The practical takeaway is unglamorous. Even if your server exposes all three layers, some fraction of your users are on a client that only invokes Tools. Build your server so that a Tools-only client still gets meaningful behavior, and Resources and Prompts stack on top for the clients that can use them. Do not gate the core action on a Prompt the client will never call.

The July 28, 2026 spec revision helped here: capability discovery is now handled through the server/discover method, so a well-behaved client can at least ask what the server offers instead of assuming. Whether it actually uses what the server offers is a different question.

The design mistake I keep watching people make

The most common failure mode is not "picked the wrong layer." It is "put everything in Tools because Tools is the layer they understood first." I did this on my first two servers before I noticed the pattern. It is a rite of passage I would rather you skip.

Symptoms:

Tool descriptions running 500+ words each.
30+ tools on the surface.
System prompt contains "when using tool X, remember to first check Y" instructions.
Token cost per turn is embarrassing.

The fix is almost always the same shape: collapse related low-level tools into an intent tool, move the docs the agent occasionally needs into Resources, and lift the repeated flows into Prompts. The MCP server picks up the orchestration; the agent stops paying for it in tokens.

The design principle behind this is the one line from the MCP guide I keep rereading: the description of the tool is context, and context has a price.

What to do this week

If you already run an MCP server, three concrete actions.

Count the tokens your tool definitions consume on a typical run. If they are more than 20% of the total input budget, you have room to compress.
Identify one thing you currently put in the system prompt or in a tool description that is really reference material. Move it to a Resource this week.
Identify one multi-step flow you invoke often. Lift it to a Prompt so it lives on the server instead of being rebuilt in the client each time.

None of these three moves require rewriting your agent. They are configuration changes on the MCP server side, and the client will start benefiting the next time it connects.

If you want the longer walkthrough that this article is based on, my Zenn book Context Engineering: Turn LLMs From Liars Into Experts has a full chapter on the MCP three-layer split, including the code snippets for the Playwright server refactor that got me from 114K to 27K.

References

Model Context Protocol — Official spec and getting started
Maxim AI — What Is Model Context Protocol (MCP)? A 2026 Guide
Descope — What Is MCP and How It Works
Prompt Architects — How to Use MCP to Manage Prompts Inside Cursor & Claude Desktop

Claude Code Hooks vs Skills vs Subagents: Three Ways to Extend the Agent, and When Each Backfires

Ken Imoto — Sun, 13 Sep 2026 13:00:01 +0000

The first time I tried to make Claude Code "always run the linter," I wrote a Skill. It worked most of the time. The rest of the time, Claude decided the task didn't need linting and skipped it. I spent an embarrassing afternoon tightening the Skill's description before I understood the actual problem: I had picked the wrong mechanism. "Always do X" is a Hook. "Decide whether to do X" is a Skill. They are not interchangeable, and the docs at the time were happy to let me confuse them.

Claude Code now ships four ways to extend the agent: Hooks, Skills, Subagents, and the Agent SDK. They overlap enough to look like alternatives and differ enough that using one where another belongs costs you reliability, tokens, or both. I've built real harnesses on the first three, and read the SDK docs closely enough to know where it takes over. Here is how each one actually fires, what it costs you in context, and the specific way each one bites back.

The one-line mental model

Before the table, the version I tell teammates:

Hooks are deterministic. They fire on an event whether Claude likes it or not.
Skills are probabilistic. Claude reads a description and decides to pull them in.
Subagents are isolation. They run in their own context window and hand you back a summary.
The Agent SDK is you taking the wheel: the same agent loop, running in your own process.

If you remember nothing else: Hooks remove a decision from the model, Skills add one, Subagents quarantine one.

The comparison

Mechanism	How it fires	Token cost	Best for	When it backfires
Hooks	Lifecycle events (`PreToolUse`, `Stop`, etc.) defined in `settings.json`. Not the model's choice.	Runs as a shell command outside the context window. Near zero, unless you deliberately inject stdout.	Non-negotiable rules: format on write, block secret files, run tests before stop.	A hook exiting `2` blocks the action. A bad `Stop` hook loops the agent forever.
Skills	Claude matches your prompt against the Skill's `description`, or you type `/name`.	Progressive disclosure: only `name`+`description` (a few dozen tokens) loaded per Skill at session start; body loads on invoke.	Reusable procedures Claude should choose when relevant: a review workflow, a release-notes drafter.	A bloated `description` taxes every turn and misroutes. The model can decline to fire at all.
Subagents	Auto-delegated when a task matches the agent's `description`, or via the `Agent` tool (the old `Task`).	Runs in its own context window. Only the final message returns to the parent.	Quarantining noisy work: codebase-wide greps, log trawls, parallel exploration.	It can't ask you follow-ups mid-task, and the summary drops the detail you needed.
Agent SDK	Programmatic. You call `query()` from TypeScript or Python and drive the loop yourself.	A separate program in your own process. Context is whatever your code feeds it.	Productizing the agent: CI bots, backend services, anything outside the CLI.	You own retries, sandboxing, session state, and cost. Nothing is automatic anymore.

Hooks: the mechanism that removes the model's vote

A Hook is a shell command wired to a lifecycle event in settings.json. The model does not get a say. This is the whole point and the whole danger.

{
  "hooks": {
    "PostToolUse": [
      {
        "matcher": "Write|Edit",
        "hooks": [{ "type": "command", "command": "prettier --write \"$CLAUDE_FILE_PATHS\"" }]
      }
    ]
  }
}

The 2026 event surface is much wider than the original set. Beyond the familiar PreToolUse, PostToolUse, UserPromptSubmit, SessionStart, Stop, and Notification, there are now session events like SessionEnd, agent events like SubagentStart/SubagentStop, and context events like PreCompact/PostCompact (hooks reference).

The footgun is the exit code. Exit 0 is success. Exit 2 is a blocking error: stderr goes back to Claude and the action is rejected. On a PreToolUse hook that's a feature: you can stop Claude from touching .env. On a Stop or SubagentStop hook, exit 2 means "you are not allowed to stop, keep going." I once wrote a Stop hook that checked for uncommitted changes and exited 2 if it found any. The agent dutifully refused to stop, tried again, still had uncommitted changes, refused to stop again. I had built a loop with my own hands. The lesson: a Hook that can block is a Hook that can deadlock. Test the exit-2 path before you trust it.

Because Hooks run with your shell privileges and outside the context window, they cost almost nothing in tokens. That's why "always run the linter" belongs here and not in a Skill. You are not asking Claude to remember; you are removing the choice.

Skills: the mechanism that adds the model's vote

A Skill is a SKILL.md file with frontmatter. Claude reads the description of every installed Skill at session start and decides, per turn, whether your prompt warrants pulling one in. You can also force it with /name.

The mechanic that makes Skills scale is progressive disclosure. Three tiers load at three different times:

Always loaded: the name and description only, a few dozen tokens per Skill. This is how one agent can know about hundreds of Skills without drowning.
Loaded on invoke: the SKILL.md body. Anthropic's own guidance keeps the body under ~500 lines (skill-development SKILL.md).
Loaded only when referenced: bundled scripts and detail files, zero tokens until Claude reaches for them.

So the failure mode is specific and quiet. Every Skill's description sits in context on every turn. Write a paragraph there and you've levied a permanent tax on the whole session, and you've made the routing decision harder, so the model misfires more often. My original "always lint" Skill failed because I was asking a probabilistic router to behave deterministically. The router did its job: it routed. Sometimes away from me.

Skills are right when you genuinely want Claude to choose. A code-review workflow, a "draft release notes from the git log" procedure, a triage routine. Things that should happen when relevant, judged by the model. Write the description like a routing key, not a summary (Agent Skills overview).

Subagents: the mechanism that quarantines context

A Subagent is a Markdown file in .claude/agents/ with frontmatter, and its body becomes that agent's system prompt:

---
name: codebase-explorer
description: Search the codebase for where a symbol is defined and used. Use proactively before refactors.
tools: Read, Glob, Grep
model: sonnet
---

You locate code. Report file paths and line numbers, not full file dumps.

The defining property: it runs in its own context window. Its system prompt, its tool calls, every grep result and log line it reads, none of that enters your main conversation. Only the final message comes back (custom subagents). This is the single best tool for a task that would otherwise flood your context: "find every caller of this function across 400 files." Run it in a Subagent and you get back a clean list instead of 400 files of noise.

Two backfires bit me here. First, a Subagent can't stop to ask you a question. It runs to completion and reports once. If it hits an ambiguous decision halfway through, it guesses, and you find out from the summary. Second, that summary is lossy by design. The detail lived in the isolated window and is gone unless the agent thought to surface it. I've had a Subagent confidently report "no usages found" when it had quietly searched the wrong directory. The fix is to write the agent's instructions to return evidence (paths, counts), not just conclusions.

One 2026 gotcha worth flagging: permission modes are inherited from the parent and can override the Subagent's own frontmatter. If your main session is running with relaxed permissions, the Subagent's careful tools allowlist may not save you. Treat the tool list as scoping, not a security boundary.

Agent SDK: the mechanism that hands you the keys

When the agent needs to live outside the terminal, in CI, a backend service, a scheduled job, you reach for the Agent SDK. It was renamed from the Claude Code SDK because the same loop powers far more than coding (migration guide). It ships for both TypeScript (@anthropic-ai/claude-agent-sdk) and Python (claude-agent-sdk).

from claude_agent_sdk import query, ClaudeAgentOptions

async for message in query(
    prompt="Find and fix the failing test in auth.py",
    options=ClaudeAgentOptions(allowed_tools=["Read", "Edit", "Bash"]),
):
    print(message)

You get the same agent loop, built-in tools, hooks, subagents, and MCP that power the CLI, but now running in your process. The backfire is everything the CLI used to do for free: retries, sandboxing, session persistence, and cost control are now your problem. One auth constraint to know before you build a product on it: third-party products on the Agent SDK can't use claude.ai subscription login, you authenticate with an API key (directly or via Bedrock/Vertex). Prototype on the SDK, then decide whether a hosted option fits production.

How I actually choose

The question I ask, in order:

Must this happen every time, no exceptions? Hook.
Should Claude decide when it's relevant? Skill.
Is this going to dump a pile of junk into my context? Subagent.
Does this need to run without me in the loop? Agent SDK.

Most of my mistakes came from answering question 2 when the honest answer was question 1. "Always" and "when relevant" feel similar when you're typing the config. They are opposites at runtime.

Takeaways

Hooks remove the model's choice. Use them for non-negotiable rules. Watch the exit-2 path: a blocking Stop hook can loop your agent.
Skills add a choice. Progressive disclosure keeps them cheap, but every description is taxed on every turn. Don't use a Skill to enforce something that must always happen.
Subagents quarantine context. Best for noisy exploration. They can't ask follow-ups and the summary is lossy, so make them return evidence.
The Agent SDK hands you the keys. Same loop, your process, your operational burden.
The biggest reliability win isn't picking the "best" mechanism. It's noticing when "always" is masquerading as "when relevant."

If you've been wiring Claude Code config long enough to feel the shape of a coherent harness, where Hooks, Skills, Subagents, and CLAUDE.md all sit in one place and reinforce each other, that's the book I wrote.

👉 Claude Code Mastery

References

Dijkstra, Knuth, Kernighan: 10 Quotes That Predicted the AI Coding Debate

Ken Imoto — Sat, 12 Sep 2026 13:00:01 +0000

In 1972, Edsger Dijkstra stood up to accept a Turing Award and gave a lecture called "The Humble Programmer." He had never seen an LLM. He had never watched an agent rewrite a file in his terminal. And yet what he said that day reads like a direct response to the way we argue about AI coding in 2026. So does a good deal of what his contemporaries wrote over the two decades that followed.

That's the strange thing about the current moment. We talk about vibe coding, generated-code trust, and "is learning to code even worth it" as if they're brand-new problems. They're old problems wearing a new model's clothes. Here are ten quotes, most of them decades old, that predicted the exact arguments we're having now. I'll give you the line, who said it, and why it lands harder in 2026 than the day it was written.

1. The whole case for AI coding, stated in 1972

"The competent programmer is fully aware of the strictly limited size of his own skull; therefore he approaches the programming task in full humility, and among other things he avoids clever tricks like the plague."
— Edsger Dijkstra, The Humble Programmer (ACM Turing Award Lecture, 1972)

This is the entire justification for handing work to an agent, written 50 years early. We reach for AI because our skulls are, in fact, strictly limited. The catch is the second half: Dijkstra's humility meant avoiding clever tricks, and a model prompted to be impressive will hand you the cleverest trick it can find. The humble move in 2026 isn't refusing AI. It's refusing the clever output it gives you when a boring version would do.

2. "It ran" is not "it's correct"

"Testing shows the presence, not the absence of bugs."
— Edsger Dijkstra, NATO Software Engineering Conference (1969)

The single most useful sentence to keep taped above your monitor when reviewing generated code. An agent produces something, the tests go green, and there's a powerful urge to call it done. Dijkstra's point is that green tests prove your bugs hid well, not that they're gone. With AI-written code the gap is wider, because the model also tends to write tests that confirm its own assumptions. Passing its own exam is not the same as being right.

3. Why vibe coding has an expiration date

"Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it."
— Brian Kernighan, The Elements of Programming Style (1978)

Now substitute the model for "you." If an LLM writes code at the absolute limit of its cleverness, and you accepted it without understanding it, then by Kernighan's arithmetic nobody in the room is smart enough to debug it. That's vibe coding's actual failure mode, and I've lived it: a two-hour prototype I didn't read, followed by half a day of re-reading my own project to add one feature. Fun until the bill comes.

4. Confidence is not correctness

"Beware of bugs in the above code; I have only proved it correct, not tried it."
— Donald Knuth (1977)

Knuth could prove his code correct and still warned you it might be broken. LLMs do the opposite: they can't prove anything, but they'll present output with the serene confidence of someone who can. If the most careful computer scientist alive hedged on code he'd formally verified, the right posture toward a model's "this should work" is somewhere south of trust.

5. The readability quote that aged into an AI strategy

"Programs must be written for people to read, and only incidentally for machines to execute."
— Harold Abelson & Gerald Sussman, SICP (1985)

For 40 years this was about your teammates. In 2026 it's also about the agent. The model is now one of the readers, and clean, well-named, well-structured code is exactly what lets it make correct edits instead of confidently wrong ones. The same code that's kind to a junior engineer is legible to an LLM. Readability stopped being a courtesy and became a performance feature.

6. There is still no silver bullet

"The amateur software engineer is always in search of magic."
— Grady Booch

Booch was writing in the lineage of Fred Brooks's "No Silver Bullet," and AI is the most convincing-looking silver bullet the field has ever produced. But Brooks's distinction holds: a tool can strip away accidental complexity (boilerplate, syntax, glue) and do nothing about the essential complexity of figuring out what to build. AI is a phenomenal accidental-complexity eraser. The amateur thinks that's the whole job. It never was. I'd know: I once spent a weekend wiring up an elaborate agent pipeline to dodge the fifty lines of logic that were the actual point, then wrote the fifty lines anyway.

7. AI lets you take on debt 10x faster

"Shipping first-time code is like going into debt."
— Ward Cunningham

Cunningham coined "technical debt" to explain code quality to people who think in money. The 2026 update writes itself: AI lets you ship first-time code at ten times the volume, which means ten times the debt if you never go back to pay it. Debt with no repayment plan is the problem. An agent that can generate a thousand lines before lunch is also an agent that can quietly max out your credit.

8. "Move fast and break things" already got revised once

"Move fast and break things."
— Facebook internal motto (~2009–2014)

Facebook itself retired this in 2014 for "Move fast with stable infrastructure." The motto is a complete lesson, limit included. It's right for a throwaway prototype and reckless for code with millions of users downstream. Autonomous agents make moving fast trivial; they don't make the breaking part any cheaper. The phase you're in still decides whether speed is a virtue or a liability.

9. The counterweight, from someone who'd know

"Saying that learning to code is unnecessary because of AI is some of the worst career advice ever given."
— Andrew Ng

Ng founded DeepLearning.AI and led AI at both Google and Baidu, so when he says don't stop learning to code, it isn't nostalgia. His logic is leverage: the better the tools get, the more value accrues to the person who can direct and verify them. You cannot review an output you couldn't have written. The people who get the most out of AI coding are precisely the ones who didn't skip the part where you learn to code.

10. The 2026 argument in two quotes

"Talk is cheap. Show me the code."
— Linus Torvalds

"The hottest new programming language is English."
— Andrej Karpathy (2023)

I'm ending on a collision because the whole debate lives in the gap between these two. Torvalds spent decades insisting that talk is cheap and only running code counts. Karpathy says the talk is the code now. They're both right, which is the uncomfortable part: English is how you instruct the machine, and the code is still the only thing that tells you whether the English worked. The skill of 2026 is doing both, describing intent precisely and reading the output critically. Karpathy gets you a draft. Torvalds tells you whether to trust it.

The point

None of these people were predicting AI. They were describing software, and software didn't change its nature just because the author did. The arguments we're treating as unprecedented (can you trust generated code, is cleverness a trap, is coding a dying skill) were settled, or at least well-framed, by people working in languages most of us have never touched. The tools are new. The wisdom is paid for.

If you enjoy this kind of thing, I collected 100 of these quotes and unpacked why each one stuck (the history behind it, the rhetorical structure that makes it memorable, and the lesson for working engineers today) in Engineering in 100 Quotes. The AI-era chapter is where the old lines and the new ones finally meet.

MCP Sampling: When the Server Gets to Prompt Your Model

Ken Imoto — Fri, 11 Sep 2026 13:00:00 +0000

Most of what I'd read about MCP framed the data flow one direction: I ask, the client picks a tool, the server runs it, I get a result. The server was a thing my model called. It did not occur to me that the server could call back.

Then I read the part of the spec that covers Sampling, and the arrow flipped. With Sampling, an MCP server can send a request up to the client that says, in effect, "run an LLM completion for me and hand me the text." The server is not returning data anymore. It is asking my model to think on its behalf, with a prompt the server wrote.

I spent an evening tracing exactly what that request contains, who approves it, and which clients even support it in 2026. This post is what I found: how Sampling works, the trust boundary it crosses, and why a feature designed to make servers smarter is also a clean new way to get into your model's context.

What Sampling actually is

In a normal MCP exchange, the model is the one with the LLM. The server is dumb plumbing: it exposes tools, runs them, returns text. Sampling inverts that. It lets a server that has no model of its own borrow yours.

Say a server is processing an expense and hits a transaction it can't categorize. Without Sampling it has to fail, guess, or hand the problem back. With Sampling it pauses mid-task and asks the client: "given this description, which account does this belong to?" The client runs that against its LLM and returns the answer, and the server keeps going. (Speakeasy, What is MCP sampling)

That is genuinely useful. It lets a server stay simple and still make judgment calls, instead of every server shipping its own model and API key. The MCP docs pitch it as the thing that makes "agentic" server workflows possible: the server can stop and reason at a step instead of running blind. (Model Context Protocol, Sampling concept)

But re-read that sentence. A third party's server got to write a prompt and run it through the model I authenticated, on context I own. That is a different relationship than "I called your tool."

The request shape

The method is sampling/createMessage, and the request the server sends back up to the client looks roughly like this. (MCP spec draft, client/sampling)

{
  "method": "sampling/createMessage",
  "params": {
    "messages": [
      { "role": "user",
        "content": { "type": "text", "text": "Categorize this transaction: ..." } }
    ],
    "modelPreferences": {
      "hints": [{ "name": "claude-3-sonnet" }],
      "costPriority": 0.3,
      "intelligencePriority": 0.8,
      "speedPriority": 0.5
    },
    "systemPrompt": "You are a helpful bookkeeping assistant.",
    "includeContext": "thisServer",
    "maxTokens": 100
  }
}

Three fields are doing more than they look:

messages and systemPrompt are a prompt written by the server. The server controls what the model is asked to do. The client did not write this; a remote party did.
modelPreferences lets the server nudge model selection with hints, costPriority, intelligencePriority, and speedPriority. The client still makes the final pick, but the server gets to lobby for an expensive model. (rust-mcp-schema, CreateMessageRequestParams)
includeContext is the one I kept staring at. It accepts "none", "thisServer", or "allServers". It tells the client how much of the conversation to fold into the prompt the server requested. The default is "none", and "thisServer"/"allServers" are soft-deprecated: a compliant client only honors them if it has declared a sampling-context capability. (MCP spec draft, client/sampling)

That includeContext field is where the convenience and the danger sit on the same line.

The approval gate the spec built in

The protocol authors clearly saw this coming, because they did not leave the server's request unsupervised. The spec puts a human in the loop at two points, not one.

Before the call. The client is supposed to show me the prompt the server wants to run, and I can edit it, approve it, or reject it. The server's text does not reach the model unless I let it. (Speakeasy, What is MCP sampling)
Before the result goes back. The client shows me the completion the model produced, and I can approve or block it before it returns to the server. (Model Context Protocol, Sampling concept)

So the design intent is: the server proposes, the model thinks, but a person sits on both the inbound prompt and the outbound answer. The client owns the actual LLM call, picks the real model, and is the gate. That is a reasonable threat model on paper.

The catch is the word supposed to. The spec describes the gate. Whether a given client implements it well, or implements Sampling at all, is a different question.

Who actually supports it in 2026

This is the part that reset my assumptions. Sampling is one of the older MCP primitives by spec, and it is still thinly implemented.

Claude Code acting as an MCP client still does not support Sampling. The feature request (Issue #1785, claude-code) has been open since June 2025 and was still open, at 58 comments, in August 2026. Other clients have moved: opencode's "Add MCP sampling support (createMessage)" request (Issue #11948, opencode) was closed as completed in April 2026, and VS Code's MCP client acts on modelPreferences.

Two things follow from that. First, if you are building a server and leaning on Sampling, your server fails or degrades on the clients most of your users run. Second, the security surface lives in how each client implements the gate. A primitive that is "emerging" and implemented unevenly across clients means the human-in-the-loop guarantee is only as strong as the specific client in front of you. The spec mandating an approval step does not mean the client you're on actually renders one. Which is its own small joke: I'd spent an evening auditing a gate that, on the client I use every day, doesn't exist yet.

The trust boundary, stated plainly

Here is the boundary I had been ignoring. When my model calls a tool, the data flows server-to-me and I treat the result as untrusted output. When a server uses Sampling, the request flows server-to-my-model, and the server is now upstream of my model's reasoning. It crossed from "thing I call" to "thing that prompts me."

In April 2026, Palo Alto's Unit 42 published an analysis of attack vectors specific to Sampling, and it names the failure modes cleanly. Sampling attacks slip past tool-integrity checks and sandboxing because they ride a legitimate protocol feature, not a malformed tool. They group the abuse into three classes: covert tool invocation that performs hidden file and system operations, conversation hijacking that injects instructions persisting across turns, and resource theft that drains your compute quota for the attacker's workloads. (Unit 42, Prompt Injection Attack Vectors Through MCP Sampling)

The mechanism for the second one is worth spelling out, because it's subtle. A malicious server's Sampling prompt instructs the model to append a directive to its next visible response. Because that text lands in the conversation history, the model keeps following it on later turns, long after the Sampling call finished. The same trick exfiltrates data by telling the model to slip extracted information into its next answer to you. (Unit 42, MCP Sampling attack vectors)

And includeContext? That is the cross-server problem. If a client isn't strict about scoping each server's Sampling request to that server's own context, a malicious server can ask for "allServers" and pull in conversation belonging to servers it was never meant to see. In a multi-server session, one untrusted server can read context from the trusted ones it shares the model with. (Unit 42, MCP Sampling attack vectors) The soft-deprecation of "thisServer"/"allServers" is the spec quietly walking that back, but only for clients that respect the capability flag.

A short threat list

If I'm reviewing a server that uses Sampling, or a client that implements it, this is what I check:

Prompt injection inbound. The messages/systemPrompt from a server are attacker-controlled text aimed straight at your model. Treat them like any other untrusted prompt.
Persistent hijack. A Sampling prompt can plant an instruction in the visible answer that survives into later turns. The damage outlives the one call.
Cross-server context leak. includeContext: "allServers" on a loose client hands one server the conversation of every other server in the session.
Quota theft (Denial-of-Wallet). A server can fire Sampling requests to burn your tokens and budget on its own work, not yours.
Silent or missing gate. If the client doesn't render the request-and-response approvals the spec calls for, the entire safety story collapses to "trust the server."

Takeaways

Sampling flips MCP's direction: the server asks your client to run an LLM call (sampling/createMessage) using a prompt the server wrote.
The spec puts a human in the loop twice, before the request reaches the model and before the result returns. That gate is the whole defense.
includeContext (none/thisServer/allServers) is the cross-server leak risk; thisServer/allServers are soft-deprecated and capability-gated for a reason.
Sampling is unevenly implemented in 2026: Claude Code still doesn't support it as a client, while opencode shipped it in April. The safety guarantee is only as real as the client's gate.
Unit 42 documents three live abuse classes: covert tool invocation, persistent conversation hijacking, and quota theft. Sampling prompts are untrusted input pointed at your model.

I went deeper on MCP's primitives, trust boundaries, and the OWASP MCP failure modes in my book if you want the long version: MCP Security Practice.

Nielsen's 3 UX Cliffs Mapped to Voice AI: 100ms Feels Instant, 300ms Alive, 800ms Dead

Ken Imoto — Thu, 10 Sep 2026 13:00:01 +0000

In 1993, Jakob Nielsen wrote three numbers that have quietly governed every UI ever since.

0.1 second. 1 second. 10 seconds.

Under 100 milliseconds, an interface feels instant. Under 1 second, thought doesn't break. Past 10 seconds, users are gone. Thirty-plus years later, those thresholds are still baseline material in every UX curriculum, and Nielsen Norman Group still publishes the same three cliffs when they're asked about response times.

They map cleanly to the web. They map badly to voice.

The missing screen is what breaks the mapping. Take away the loading spinner and every one of Nielsen's thresholds contracts.

Nielsen's thresholds are neurological, not screen-based

The numbers weren't invented for computers. Nielsen was consolidating perception research going back to the 1960s: Miller in 1968, Card and colleagues in 1991, all trying to pin down how long humans stay "in the loop" of an interaction.

100 ms: the ceiling for perceiving something as a direct response to your action. Below this, the effect feels like it belongs to you. Above it, cause and effect start to separate.
1 second: the ceiling for uninterrupted thought. Between 100 ms and 1 second, users notice the delay but stay in flow; past a second, they drop out of the task and start waiting.
10 seconds: the ceiling for holding attention at all. Past this, minds wander to email, phones, other tabs.

The numbers describe human cognition, and the hardware they were measured on has changed beyond recognition without moving them. The 100 ms figure is unchanged in 2026. The 1-second figure still describes when a web page starts feeling broken.

But those numbers assumed there was something to look at while you waited.

Voice has no spinner

Voice interfaces strip out the entire "we're working on it" channel.

There is no loading bar. No skeleton state. No progress percentage. No "typing..." indicator sitting under the previous message. The only signal the user gets between "I stopped talking" and "the agent starts talking" is silence, and silence is ambiguous. The agent might be thinking. The connection might have dropped. Or the microphone stopped listening halfway through the sentence.

Compare this to a chat app. If the reply takes 1.2 seconds, that's fine, because the "…" bubble tells you a human (or a bot) is actually on the other end. The waiting is annotated. Voice doesn't get to annotate.

That's the whole reason Nielsen's cliffs don't survive the port. In a channel with no fallback signal, silence gets expensive fast.

The voice-side numbers, and where they come from

Here's the practical translation. Two of the three cliffs move down. One stays put.

100 ms → still 100 ms (instant)

The perceptual limit doesn't care about the channel. A beep 80 ms after you finish speaking still reads as "the system heard me." A beep 250 ms after still works, but you notice it. This is why voice agents that emit an acknowledgement sound (a soft "mm", a chime, a barely audible breath) feel more responsive than agents that stay silent. The sound carries no information. Its only job is to occupy the 100 ms slot.

1 s → 300–500 ms (flow)

This is the cliff that moved the furthest. In GUI, 1 second of waiting is annotated: spinner, loader, progress. In voice, 1 second of silence after you finish speaking is unbearable. It reads as "the agent didn't hear me," or worse, "the agent is ignoring me."

The empirical floor lands somewhere around 300 milliseconds. That number keeps showing up:

Human conversational turn-taking gaps average roughly 200 ms across cultures, per Stivers et al. (PNAS, 2009). Anything past that pushes the exchange out of its normal rhythm.
Doherty and Thadani's 1982 IBM study identified 400 ms as the point where response and action fuse into a single perceived event.
AssemblyAI's "300ms rule" and Cresta's latency engineering writeup both treat 300 ms as the reference point and sub-second as the outer bound.

Which means: if your first audible token comes out more than about 500 ms after the user finishes their sentence, the user has already noticed. Past 800 ms, they start suspecting the connection.

10 s → 4 s (abandon)

The abandonment cliff also collapses, because there's nothing to do during the wait. On a web page, 10 seconds is skimmable. You glance at what's already loaded, you read the header, you can even open another tab. On a phone call with a voice agent, 4 seconds of dead air is the point at which most users say "hello?" or hang up. ACM CUI 2025 experiments put the perceived-quality collapse right around that same 4-second mark.

Where 2026's voice agents actually sit

The uncomfortable part: most production voice AI in 2026 doesn't hit the 300 ms target.

The ACL IWSDS 2025 turn-taking survey puts current spoken dialogue agents at 700–1,000 ms per turn, against the roughly 200 ms humans use with each other. Three to five times the human gap, and already past the 800 ms mark where users start wondering about the connection. Cresta puts the point of steep quality degradation at 1.5 seconds, which plenty of production stacks still cross under load.

The theoretical ceiling for a voice UI to feel alive is 300–500 ms. Shipping systems are landing at 700–1,000 ms. That 200–700 ms band between target and reality is where most of the practical work in voice AI happens right now.

What actually cuts the gap

Once you know where the cliffs are, the engineering has a shape. The whole pipeline does not have to run end-to-end in 300 ms. Every threshold has to be occupied as it arrives.

Under 100 ms: emit an acknowledgement sound the instant end-of-speech is detected. A single chime, a quiet "mm-hm," anything. This is the cheapest UX win in the entire stack, and most voice agents skip it.
100–400 ms: start streaming a filler or the first prosodic beat of the response. Even "one moment" said naturally buys you 800 ms of goodwill before the actual answer starts.
400–800 ms: this is where the real first token needs to arrive. Everything upstream (endpointing, STT, LLM first-token latency, TTS first byte) has to fit inside this window.
Beyond 800 ms: the user is now actively wondering whether something is wrong. You need to say something to prove the connection is alive: "let me check that," an explicit "still here," anything with words in it.

Most of the tricks in this space are scheduling tricks. Fire the TTS as soon as the first LLM token arrives. Start endpointing as soon as amplitude drops. Pre-fetch the likely reply while the user is still talking, when confidence is high enough.

But all of it starts with knowing where the cliffs are, and knowing that they sit two to three times closer than the GUI numbers suggest.

The one line to remember

Treat Nielsen's numbers as the GUI ceiling. Voice sits under all three, because silence is the only status bar voice users have.

Design the pipeline around the 300 ms target. Occupy the 100 ms slot with anything at all. Speak within 800 ms, or the user will assume you're gone.

This article expands on Chapter 2 of The 300ms Threshold: Voice AI UX for Sub-second Response. The book maps Nielsen's thresholds across the full voice AI stack, breaks down the TTFB budget for STT/LLM/TTS individually, and covers the streaming, filler, and turn-taking patterns that let real systems land inside the 500 ms cliff.

Qwen 3.6 vs 3.5: Same 37 tok/s on RTX 4070, +43% on Frontend Generation

Ken Imoto — Wed, 02 Sep 2026 13:00:01 +0000

The first number I saw on Qwen3.6-35B-A3B was 12 tok/s.

I almost hit publish on "Qwen regressed at generation speed" and moved on. The 3.5 baseline on the same RTX 4070 was 34.6 tok/s. A new generation running at a third of the old one would have been a hell of a headline. It was also completely wrong.

The culprit was not the model. Another process on the box was sitting on 9-11 GB of VRAM, so the layers that were supposed to live on the GPU were spilling to system RAM. The tell was that my sanity-check run of Qwen3.5 slowed down too. When two independent models degrade together, the model is not the variable.

I killed the offending process, re-measured, and got numbers that told a completely different story.

Model	Generation speed tg128 (tok/s)	Runs
Qwen3.6-35B-A3B	38.76 ± 0.82	avg of 3
Qwen3.5-35B-A3B	36.7 ± 1.4	avg of 3 (range 34.9-38.6)

Both models sit inside the ±1.5 tok/s band on the same RTX 4070. On the tokens-per-second axis, "the new generation" is not a story. Same architecture, same activated-parameter count (3B active out of 35B), same MoE routing pattern. The half-speed regression was a measurement bug, and it lived for about half a day before its own inconsistency killed it.

The lesson I keep re-learning: when the number you got is dramatically convenient for your narrative, measure it again before you write anything. The moment I could sell 12 tok/s as a regression, I should have been suspicious. The version of me that ran the second test earned the version of me that got to keep his self-respect.

So where did the generation move to?

If speed did not change, does the 3.5-to-3.6 bump mean anything? It does. The move lives on a different axis.

The official Qwen3.6-35B-A3B model card publishes benchmarks with a very lopsided shape:

Benchmark	Qwen3.5	Qwen3.6	Lift
Terminal-Bench 2.0	40.5	51.5	+27%
QwenWebBench (frontend generation)	978	1,397	+43%
SWE-bench Pro	44.6	49.5	+11%
LiveCodeBench v6	74.6	80.4	+8%
SWE-bench Verified	70.0	73.4	+5%
AIME26	91.0	92.7	+2%
GPQA	84.2	86.0	+2%

Terminal operations: +27%. Frontend generation: +43%. Repository-scale coding tasks: +11%. Single-question knowledge probes like AIME and GPQA: +2%.

The pattern is almost too clean. Every benchmark that rewards tool-calling, long-context reasoning, and multi-turn execution moves double-digit percentages. Every benchmark that fits in one problem statement and one answer barely moves at all.

Why did AIME and GPQA plateau at +2%?

Benchmark saturation is the boring answer, and it is probably the right one. AIME and GPQA at the 90-point range are already near the ceiling of what the eval format can measure. A model that gets 91.0 on AIME26 is not being asked to demonstrate reasoning it cannot do. It is being asked whether it can grind out the arithmetic without slipping.

Terminal-Bench 2.0 and QwenWebBench are not saturated. They score long-horizon behavior: does the model recover from a shell error, does it wire the CSS classes to the right components, does it complete the task instead of writing a plan and stopping. These are the axes where a real capability gap still has room to show up.

Which reframes the release. Qwen3.6 is not a faster 3.5. It is the same footprint with more of the model's weight thrown at agent behavior: tool call stability, long context, and thinking control. If your workload is one-shot QA, you will not see the gap. If your workload is "read this repo and land a PR," you will.

The 7-question test where nothing showed up

To sanity-check the model card claims on my own hardware, I ran the same 7-question standard set from Chapter 5 against both models. Both scored 7/7.

Zero delta. If I had squinted, I could have talked myself into "3.6 gives more polished answers," and I actually started drafting exactly that. Then I put the 3.5 answers next to the 3.6 answers, and 3.5 was often the more thorough one (the capital-cities question, the WebRTC explanation). The comparison collapsed.

The reason is the same reason AIME plateaus. My 7-question set is saturated. When both models nail every question, the "generation gap" gets swallowed by run-to-run sampling noise. If you want to see 3.6 win over 3.5, you need problems where both models can still fail: long agent traces, unfamiliar repos, frontend layouts under a spec.

What this changes if you run local Qwen

Two takeaways I would actually act on:

Do not upgrade to 3.6 for the tokens. On the same 12 GB VRAM budget with the same --cpu-moe MoE-offload setup, the tokens-per-second number is unchanged. If your bottleneck was throughput, 3.6 gives you nothing.
Do upgrade to 3.6 if you were about to hand the model a repo. The +43% on frontend generation and +27% on Terminal-Bench are the numbers that matter for coding-agent, IDE-plugin, and CLI-agent workloads. The gap is real, and it is exactly where you want it if you are treating a local 35B as a workhorse instead of a chatbot.

The generation went sideways on speed and forward on autonomy. That is a more interesting release than "3.6 is 5% faster," even if it makes for a worse tweet.

Notes

Setup: RTX 4070 (12 GB VRAM), llama.cpp with -ngl 99 --cpu-moe, GGUF quantization from the lmstudio-community mirror. Measurement via llama-bench tg128.
Model card benchmarks are quoted directly from the Qwen3.6-35B-A3B Hugging Face card. Released 2026-04-15 under Apache 2.0.
The original Japanese chapter this article is adapted from goes deeper on the VRAM debugging story and the standard 7-question quality set (see the canonical link at the top of this article).

Flash Attention 3 + nanochat: 1 Prompt Pass, N Parallel Samples via KVCache Prefill

Ken Imoto — Tue, 01 Sep 2026 13:00:01 +0000

If you want 8 different answers to the same prompt, the naive way costs you 8x the prefill.

Every one of those 8 forward passes re-reads the prompt, re-computes attention over the whole context, re-fills a KV cache from scratch. On a 4k-token system prompt with a 30B-parameter model, that is not a small tax. It is most of the wall-clock time before the first token comes out.

Andrej Karpathy's nanochat has one of the cleanest workarounds for this I have read. The core move is a 12-line method called prefill on the KVCache object, and it turns the 8x prefill bill into a 1x prefill plus an 8-way memcpy. This post walks through why that works, why it depends on Flash Attention 3's flash_attn_with_kvcache API, and where the design bites you.

What the KV cache looks like on disk

nanochat/engine.py allocates the cache up front, one tensor per attention side:

# k_cache and v_cache shape: (n_layers, batch_size, seq_len, n_kv_head, head_dim)

The tricky index is n_kv_head, not n_head. Under GQA (grouped-query attention), multiple query heads share a single KV head. nanochat only allocates enough cache slots for the reduced set. That is not an optimization the code goes out of its way to advertise, but it is why the cache stays inside VRAM budgets that would otherwise not fit.

cache_seqlens (shape (batch_size,)) tracks how far each row has written into the buffer. Flash Attention 3's flash_attn_with_kvcache API writes the new K/V slices in place at the right offsets and does attention against the full cache, all in a single kernel launch. Python does not touch the cache tensor except to increment cache_seqlens at the end of the step.

That single-kernel guarantee is what makes the whole design tractable. If Python had to re-assemble K/V from N shards between steps, the memory bandwidth alone would eat the win.

The prefill trick, 12 lines

Here is the method that does the interesting work:

def prefill(self, other):
    """
    Copy cached KV from another cache into this one.
    Used when we do batch=1 prefill and then want to generate multiple samples in parallel.
    """
    assert self.get_pos() == 0, "Cannot prefill a non-empty KV cache"
    assert self.n_layers == other.n_layers and self.n_heads == other.n_heads and self.head_dim == other.head_dim
    assert self.max_seq_len >= other.max_seq_len
    other_pos = other.get_pos()
    self.k_cache[:, :, :other_pos, :, :] = other.k_cache[:, :, :other_pos, :, :]
    self.v_cache[:, :, :other_pos, :, :] = other.v_cache[:, :, :other_pos, :, :]
    self.cache_seqlens.fill_(other_pos)

The flow is: run the prompt through the model with batch size 1, get a KVCache back that has other_pos tokens filled. Allocate a fresh cache sized for N sampling rows. prefill copies the batch-1 K/V state into every row of the batch-N cache, then updates cache_seqlens so all N rows agree they are already other_pos tokens into their history.

From that point, each of the N rows samples its own next token independently, writes its own K/V into its own row of the cache, and diverges from the others. The prompt itself was only ever processed once, and by an attention kernel that got the full luxury of contiguous batch-1 memory access.

The cost model changes shape:

Sampling mode	Prefill FLOPs	Decode FLOPs
Naive (N independent runs)	N x prompt_len^2	N x output_len
Prefill + clone	1 x prompt_len^2	N x output_len

For a 2000-token prompt and N=8, that is 32M prefill units against 4M. Almost an order of magnitude, and you get it back before the first decode token.

I know this because I spent an evening tuning the wrong kernel before finding it. Prefill was eating 40% of my TTFT budget and I was busy micro-optimizing the decode loop. An evening I would like back.

Where the trick would not work

Two conditions have to hold for this to be safe.

First, the KV cache tensors have to be the same shape across rows. That is the assert on n_layers == other.n_layers etc. If your inference engine hot-reloads models mid-flight (say, for a routing layer), you cannot reuse the prefill.

Second, no state that is per-row can leak into the prompt encoding. If your prompt embedding depended on, for example, a rotary embedding phase that was per-sample, the copy would put every row into the same phase and downstream tokens would drift. nanochat sidesteps this because rotary embeddings are position-based, and positions are the same for all N rows at prefill time.

The engine.py module actually copies prev_embedding alongside the K/V, exactly to preserve one piece of per-step state that would otherwise diverge. It is a small correctness detail that is easy to miss on a first read. I missed it. My second read was because my clone worked but the 8 outputs looked suspiciously like 8 copies of the same one.

The generator loop and one queue

Engine.generate is a Python generator that yields (tokens, mask) per step. Internally each row has a RowState object holding current_tokens, a completed flag, and a queue called forced_tokens.

That queue is where the tool-call machinery lives.

When the model emits <|python_start|>, the row switches into "collecting an expression" mode and stops emitting anything to the caller. When <|python_end|> comes out, the collected tokens get decoded into a string, passed to use_calculator, and if that returns a value, the encoded result gets pushed into forced_tokens wrapped in <|output_start|> / <|output_end|> markers.

if next_token == python_end and state.in_python_block:
    state.in_python_block = False
    if state.python_expr_tokens:
        expr = self.tokenizer.decode(state.python_expr_tokens)
        result = use_calculator(expr)
        if result is not None:
            result_tokens = self.tokenizer.encode(str(result))
            state.forced_tokens.append(output_start)
            state.forced_tokens.extend(result_tokens)
            state.forced_tokens.append(output_end)

On the next step, the loop pops from forced_tokens before checking what the model sampled. The mask value 0 means "we overrode the model's choice." The mask value 1 means "the model actually picked this." The forward pass still runs on every step (the model gets to keep contributing to its own internal state), but its output gets discarded while the queue drains.

Instead of a second event loop, a state machine, or an interrupt handler, tool calls are one queue check per token. That is roughly 40 lines of Python doing the work that Ollama or vLLM would split across three modules.

The calculator is a very small calculator

use_calculator is not a Python interpreter. It accepts numeric expressions and .count() string calls. Anything containing __, import, or eval gets rejected before it reaches eval. There is an eval_with_timeout wrapper that fires SIGALRM at 3 seconds to catch runaway expressions.

The full-Python execution path (execute_code in nanochat/execution.py) exists, but only gets called from tasks/humaneval.py for benchmark scoring. Even that one has a docstring that says explicitly it is not a security sandbox, just a crash-prevention wall. It spawns a subprocess, scrubs the environment, and limits memory to 256 MB. There is no network isolation and no kernel-level jail.

If you were expecting the chat CLI to run arbitrary Python for you, it does not. It runs a calculator. The gap between "calculator" and "code execution" is exactly the gap between "I trust this to run in-process" and "I do not."

What breaks between turns

One design choice worth flagging: the KV cache does not survive across turns of the same conversation.

scripts/chat_cli.py maintains a conversation_tokens list that grows by appending <|user_start|> / <|user_end|> / <|assistant_start|> blocks per turn. When you send turn 6, engine.generate gets handed the entire 5-turn history and re-prefills all of it. The KV state from turn 5 was garbage-collected the moment the generator returned.

For a chat CLI this is fine. Turn latency stays acceptable up to a few thousand tokens of history, and the code stays boring. For a serving system this would be a scaling wall, and you would want to hold cache blocks per conversation and reattach them. nanochat is explicit that it is not that system, and the choice is a good example of picking the boring option that ships.

The right way to read this file

If you are debugging your own inference engine and hit the "N samples of the same prompt" problem, engine.py is worth reading end-to-end. Roughly 350 lines. The interesting patterns compose:

KVCache.prefill for the batch-1-to-batch-N clone.
RowState.forced_tokens for tool call injection without a second loop.
sample_next_token for the top-k + temperature that is deliberately not top-p (the argument being that top-p is not worth the extra code for this use case).
The GQA-aware n_kv_head allocation, which is one of those details that only matters until it saves you 4 GB of VRAM.

Karpathy's design principle here reads as: put the complexity in the shape of the data, not in the control flow. The queue is one line. The clone is 4 assignments. The tool call system is a state variable and a check at the top of the loop. Everything that could have been an event system or a plugin architecture stayed as a plain function.

Notes

Repository: karpathy/nanochat. The file to read is nanochat/engine.py.
Flash Attention 3 requires SM90 (Hopper) GPUs. The flash_attn_with_kvcache API is documented in the Dao-AILab/flash-attention repo.
The chapter this article is adapted from also covers the tool-call state machine, use_calculator vs execute_code, and scripts/infer_bench.py (TTFT/TPOT/MBU/MFU measurement): 8000行でわかる大規模言語モデル, chapter 7 (Japanese).

CodeRabbit autoFix + Biome pre-commit: The 2-Layer Split That Stops Nitpick Round-Trips

Ken Imoto — Mon, 31 Aug 2026 13:00:01 +0000

Telling someone their shoelaces are untied is slower than tying them yourself.

That is roughly the whole thesis of this post. Every code review comment that says "unused import," "wrong quote style," "missing semicolon," or "add trailing comma" is a nitpick round-trip. Someone writes it, someone reads it, someone edits the file, someone re-runs CI, someone re-reads the diff. On a normal PR, that entire loop is 4 to 12 hours of wall-clock elapsed, spread across timezones. And every single one of those comments describes a fix that a machine could have applied deterministically.

The two-layer setup below deletes that entire category from human review queues. Layer 1 is Biome on a pre-commit hook. Layer 2 is CodeRabbit autoFix on the PR. The split matters, and putting them in the wrong order is worse than not having either.

The clean split: what belongs on which layer

Machine-fixable falls into two buckets. Getting them into the right bucket is the whole point.

Layer	Runs at	Fixes	Time-to-fix
Biome (pre-commit)	Before the commit hash exists	Formatting, import order, quote style, semi rules	<1 s local
CodeRabbit autoFix	Once PR opens, review-time	Unused imports, obvious null-checks, small refactors	~10 s in PR

Biome is the ESLint + Prettier merge that ships as one Rust binary and runs the whole check-and-write cycle in under a second on a typical file. The pre-commit hook catches formatting drift before the commit hash exists, which means the reviewer literally never sees a diff where the only change is " vs '.

biome.json is the whole configuration:

{
  "$schema": "https://biomejs.dev/schemas/1.9.0/schema.json",
  "formatter": {
    "enabled": true,
    "indentStyle": "space",
    "indentWidth": 2
  },
  "linter": {
    "enabled": true,
    "rules": {
      "recommended": true,
      "suspicious": {
        "noExplicitAny": "error"
      }
    }
  },
  "organizeImports": {
    "enabled": true
  }
}

Wired into .husky/pre-commit or lefthook.yml as biome check --apply ., this is the layer that never lets a formatting-only diff into a PR. Not "shouldn't." Cannot.

What's left after Biome is where autoFix earns its keep

Biome handles deterministic transforms. It does not, and should not, handle changes that require reading semantic context, like "this import is unused because the only usage got refactored out three commits ago." That is a whole-file analysis, not a line-level transform.

CodeRabbit's autoFix picks up exactly this class of change on the PR. When it comments a nitpick: (their prefix for low-severity findings), the comment ships with a committable suggestion block:

nitpick: Unused import.

--- a/src/app/page.tsx
+++ b/src/app/page.tsx
@@ -1,5 +1,4 @@
 import React from 'react'
-import { useState } from 'react'   // unused
 import { UserList } from '@/components/UserList'

[Apply suggestion]   <- one-click commit into the branch

The reviewee sees a fix, not a task. Click. Commit. Move on.

If you want to skip the click entirely, CodeRabbit's autoFix runs as a batch job that opens either a commit-to-branch or a stacked PR containing every accepted suggestion at once. Trigger it from a PR comment. The docs cover both flows: CodeRabbit Autofix documentation.

The autoFixable vs non-autoFixable line

The most useful mental model I have found is a rule-of-thumb table. If your team is arguing about whether something should be a lint rule or a human review point, this is the question to ask.

autoFixable	non-autoFixable
Prettier / Biome formatting	Design (multiple correct answers)
ESLint --fix / Ruff --fix	Fix for N+1 query (business logic)
`eslint-plugin-import` order	Naming (needs context)
TypeScript `organizeImports` (unused)	Bug in logic (needs spec knowledge)
`noExplicitAny` where obvious	API shape decisions

The split is: if there is exactly one correct answer that does not depend on business context, it belongs on a fix layer. Everything else is a real review conversation, and a human should have it.

The value of enforcing this line is not that machines are cheap. It is that human reviewer attention is expensive and finite. Every nitpick you route to a machine is a bug or a design flaw a reviewer noticed instead.

GitHub Actions, if you want the belt-and-suspenders version

Some teams do not want pre-commit hooks because contributors forget or bypass them. If you want the same guarantees enforced at PR time, you can run the fix pass in CI and commit the result back into the branch:

name: Auto Fix
on:
  pull_request:
    types: [opened, synchronize]

jobs:
  autofix:
    runs-on: ubuntu-latest
    permissions:
      contents: write   # required to push
    steps:
      - uses: actions/checkout@v4
        with:
          ref: ${{ github.head_ref }}
          token: ${{ secrets.GITHUB_TOKEN }}

      - uses: actions/setup-node@v4
        with:
          node-version: '22'
          cache: 'npm'

      - run: npm ci

      - name: Format + lint --fix
        run: npx biome check --apply .

      - name: Commit if changed
        run: |
          git config user.name  "github-actions[bot]"
          git config user.email "github-actions[bot]@users.noreply.github.com"
          git diff --quiet || (
            git add -A &&
            git commit -m "chore: auto-fix format and lint"
          )

Two gotchas.

The contents: write permission is not the default for GITHUB_TOKEN. If you skip that line, the workflow will run cleanly and silently fail to push. Look for "unable to push" in the logs on the first run. I know because I spent a solid afternoon assuming Biome was just skipping files.

The ref: ${{ github.head_ref }} matters. Without it, pull_request checks out the merge commit, not the branch head, and your push goes nowhere useful.

The measurement I would actually track

If you install this two-layer setup and want to prove it worked, one metric matters:

Ratio of PRs that hit human review with zero formatting or style comments.

Before, in my experience this ratio hovers around 30-50%. Every other PR ships with at least one comment about a linting fix. After Biome pre-commit plus CodeRabbit autoFix, the ratio moves to 90%+. The remaining 10% is contributors who bypassed the hook or files the linter did not cover.

The number does not need to be perfect. It needs to move enough that reviewers stop reading nitpicks reflexively. Once the signal-to-noise ratio flips, reviewers start noticing actual bugs faster because they are no longer filtering through style comments to find them.

Why the order matters

If you had to pick only one, pick Biome pre-commit. It runs before the commit hash exists, so it cannot fail asynchronously and the fix is deterministic. CodeRabbit autoFix can only fix what is already committed, and it costs a review-cycle even when applied by a click. I learned this the expensive way — shipping autoFix alone and watching the PR queue still fill up with format-only diffs because the hook wasn't there to catch them upstream.

But the real win is stacking them. Biome catches 80% of the noise deterministically. CodeRabbit picks up the semantic long tail that lint rules cannot see. Together, the reviewer sees exactly one class of comment: the ones that actually require their judgment.

The most under-priced skill in code review is knowing which comments should have been fixed automatically. Once you build the machinery, the reviewers who used to burn cycles on nitpicks are the same ones who now catch design flaws faster. The floor moved. The ceiling did too.

If you want the full walkthrough (the chapter also covers the non-autoFixable classification, a GitHub Actions auto-fix workflow, and where Biome fits inside the broader review pipeline), it is in the Zenn book below.

Book

If you are building your own AI-plus-human review pipeline and want the full framework, the design decisions in this post come from the harness-code-review book:

Harness Code Review: a two-layer machine + human review pipeline

Notes

Biome documentation: biomejs.dev. One binary, one config file, no npm plugin ecosystem to manage.
CodeRabbit autoFix documentation: docs.coderabbit.ai/finishing-touches/autofix. Supports GitHub, GitLab, Azure DevOps, and Bitbucket Cloud.
The chapter this article is adapted from walks through the CodeRabbit autoFix example, a GitHub Actions auto-fix workflow, and the autoFixable vs non-autoFixable classification tables in full: Harness Code Review, Chapter 12: autoFixable patterns.

Prompt Engineering Is Dead. Long Live Harness Engineering.

Ken Imoto — Sun, 30 Aug 2026 06:41:50 +0000

I spent 3 months perfecting prompts. Then I deleted half of them.

In late 2023 I had a directory called prompts/ with 47 carefully tuned templates. Few-shot examples, Chain-of-Thought scaffolds, a tiny ReAct loop I was very proud of. I'd A/B tested wording. I'd argued on Twitter about whether "Let's think step by step" still worked.

By mid-2025 I deleted 23 of them. They weren't wrong. They just weren't the bottleneck anymore.

The thing that broke my agents in production was never the prompt. It was the environment around the prompt — the tools they could call, the files they could see, the moment the loop should stop, the rollback when a tool returned garbage. The prompt was a polished doorknob on a house with no foundation.

That's the story of the last three years of AI engineering, compressed: we keep renaming the layer where the real problem lives.

A 40% failure rate, and it's not the model's fault

Here is the number that should embarrass us. In 2026, around 40% of AI agent projects fail in production. Y Combinator's DevTool Day surveyed CTOs and CPOs in March 2026 and found a strikingly consistent post-mortem: "the difference between success and failure isn't the model."

75% of YC enterprise companies have already deployed coding agents. Most of them hit the same wall: the demo works, the prod deploy collapses. Linear declared in March 2026 that "issue tracking is dead" — meaning if your coding agent gets the issue context directly, you don't need a human ticketing layer at all. Enterprise workflows are being redesigned around agents.

In that environment, shipping an agent without understanding the harness around it is like merging onto a highway without a seatbelt. You'll go fast. You'll go through the windshield on the first curve.

So how did we get here? Three stages.

Stage 1: Prompt Engineering (2022–2023)

Scope: one input string.

Prompt engineering optimized a single message. Few-shot examples. Chain-of-Thought. ReAct. The deliverable was the wording itself, and we treated it like poetry. Some of it was poetry. A lot of it was incantation.

What it solved: a single LLM call going from 60% useful to 85% useful, on a single task, with no tools and no loop.

What it couldn't solve: anything that needed the model to do something rather than say something. The moment you wanted the model to call a function, read a file, remember yesterday's conversation, or decide between three branches, your beautifully tuned prompt was a single-cell organism trying to run a marathon.

I remember the exact week I realized this. I'd written a prompt that scored 92% on my eval set and 11% on real customer tickets. The eval set didn't have the messy attached PDFs, the half-deleted Slack thread, the customer who said "you know, the thing." The prompt was perfect for an environment that didn't exist.

Stage 2: Context Engineering (2024–2025)

Scope: everything the model sees at inference time.

Andrej Karpathy reframed it: "it is a lot more than just the prompt itself." Philip Schmid (formerly Hugging Face, now Google DeepMind) was blunter: "the new skill in working with AI is not prompting, it is context engineering."

Context engineering treats the entire context window as the artifact. System prompt, retrieved documents (RAG), tool definitions, conversation memory, structured outputs from previous turns — all of it. Anthropic's own framing in their Effective Context Engineering post calls it "the natural progression of prompt engineering": you're still curating tokens, you're just curating a lot more of them, and most of them weren't typed by a human.

This is when MCP showed up, when vector databases stopped being a curiosity, when "what does the agent know right now?" became an actual debugging question with an actual answer.

What context engineering solved: agents that could read your codebase, recall a meeting from last week, and call a real API with the right schema.

What it didn't solve: what happens when that agent runs for six hours unsupervised, calls 200 tools, accidentally rm -rfs a sandbox, and there's no one watching. Context engineering tells you what the model sees. It doesn't tell you what happens when the model acts and something goes sideways.

I learned this the expensive way when an agent of mine, with a beautifully engineered context, spent four hours and $38 in tokens recursively summarizing its own summaries because nothing in the system told it to stop. The context was perfect. The environment was a fire.

Stage 3: Harness Engineering (2025–)

Scope: the entire environment the model operates inside.

Context + constraints + tools + lifecycle + feedback + observability.

Louis Bouchard puts it cleanly: "Context engineering is what you send to the model. Harness engineering is how the whole thing runs."

If prompts are the recipe and context is the ingredients, the harness is the kitchen — the fire suppression, the timers, the knife rack out of the toddler's reach, the voice that says "chef, table 4 just walked out."

Concretely, a harness includes:

Constraints — what tools the agent is not allowed to call, what files it can't touch, what it must escalate to a human.
Lifecycle — when does a task start, branch, retry, give up, hand off?
Feedback loops — how does the agent know its last action worked? Linter? Test run? User reaction? Another agent grading it?
Observability — when it fails at 3am, can you reconstruct why in under 10 minutes?
Permissions and isolation — can it write to prod, or only to a worktree?

The term went mainstream in early 2026 for a specific reason: every major lab made the same architectural bet at once. Anthropic shipped Managed Agents in April 2026 at $0.08 per session hour and introduced a three-agent harness separating planning, generation, and evaluation for long-running coding work. OpenAI updated its open-source Agents SDK with a model-native harness. Google and Microsoft followed. The New Stack summarized the moment with a headline I won't forget: "They agree the harness is the product. They disagree on the price."

When four labs that disagree on everything agree the harness is where the value is, that's a signal.

Replace, or stack?

There's a real disagreement in the field about whether harness engineering replaces prompt and context engineering or contains them.

The replace camp (Data Science Dojo, others) argues that the world agents now operate in wasn't anticipated by the prompt-and-context era, so we should retire the old vocabulary and start clean.

The stack camp (AnyTech, several others) argues there's no fundamental difference — the vocabulary just keeps growing because LLMs keep doing more, and we shouldn't throw away knowledge every time a new buzzword arrives.

I'm in the stack camp, with one caveat: stacking only makes sense if you understand the containment relationship.

Harness ⊇ Context ⊇ Prompt

Prompt engineering is still essential. A bad prompt inside a great harness still produces bad output. Context engineering is still essential. Garbage retrieved documents poison the smartest harness. But by 2026, prompt-only and context-only thinking has run out of altitude. The interesting bugs — the 40%-failure-rate bugs — live in the outer layer.

The reason is unromantic. Agents got long-running. Once an agent operates autonomously across hours and hundreds of tool calls, no single prompt can steer it and no static context can describe its world. The environment becomes the load-bearing thing.

Why it matters now (and not in 2024)

A Dev.to post from WonderLab put it well: "The timing isn't a coincidence. In 2025, AI agents went from 'fun demo' to 'actual productivity tool.'"

Demos forgive a lot. A demo agent runs for 30 seconds, in front of a sympathetic audience, on a path the demoer has walked five times. A production agent runs for hours, on inputs nobody anticipated, while you're asleep. The forgiving environment is exactly what made prompt engineering feel sufficient. The unforgiving environment is what made harness engineering necessary.

This is also why "harness engineering" suddenly has multiple competing definitions from multiple companies — I wrote a separate piece about five companies and five definitions of the term. Everyone agrees the layer matters. Nobody agrees yet on its boundaries. That's how new disciplines look in year one.

Where I see this going

Honestly? I expect the word "harness" to get embarrassing within 18 months. We'll either have absorbed it into "agent engineering" (the umbrella term gaining ground), or split it into four more specialized terms (orchestration engineering, eval engineering, permission engineering, runtime engineering — pick your poison).

The vocabulary will keep moving. The underlying problem won't. The problem is, and has always been: we are putting probabilistic systems in environments that punish probabilistic behavior, and the environment is the part we keep forgetting to design.

I'm fine being wrong about the term. I'd rather be right about the layer.

And if you're still maintaining a prompts/ directory of 47 templates, no judgment. I have one too. It's just a lot smaller now, and most of the file is comments explaining what the harness around it does.

Want the deep dive?

The full 2026 timeline, every definition from every company, and the patterns that actually keep agents alive in production are in the book.

Harness Engineering Guide (Kindle)

Sources:

Anthropic, OpenAI, Google, and Microsoft agree the harness is the product — The New Stack, 2026
Effective Context Engineering for AI Agents — Anthropic
Anthropic Designs Three-Agent Harness for Long-Running Full-Stack AI Development — InfoQ, April 2026
What Is an Agent Harness in the Context of LLMs? — Parallel Web Systems

I Replaced grep-Based Code Review with a Knowledge Graph + MCP. Here Are 3 Bugs Vector Search Missed.

Ken Imoto — Sun, 30 Aug 2026 06:41:41 +0000

For about a year, my AI code review setup looked like this: AI gets a PR, AI greps for related code, AI reads way too many files, AI says "looks fine."

It mostly worked. Until the bugs that didn't show up in grep started shipping.

The problem wasn't the model. It was the retrieval. Vector search and keyword grep are great at finding files that mention auth.py. They're terrible at finding files that depend on auth.py through three import hops, an event bus, and a decorator. That's where the bugs live.

I rewired the retrieval layer with a code knowledge graph plugged in through MCP. Three bugs surfaced in the first week that vector search had been quietly missing. Here's what changed and the bugs themselves.

Why grep + vector search missed these

Vector search retrieves by semantic similarity. "Find code about authentication" finds auth.py, login.py, password_validator.py. Useful.

Knowledge graphs retrieve by structural relationship. "What depends on auth.py?" returns the call graph -- including event_handlers/login_event.py, which never mentions auth in its variable names but listens to a login event whose payload changes when auth.py changes.

Both are valid. They answer different questions. The bugs that ship to production tend to live in the second question.

The setup: code KG as an MCP server

The Model Context Protocol (MCP), released by Anthropic in late 2024, lets you expose tools to a model in a standard way. By 2026 it's supported by Claude Code, Cursor, Windsurf, Zed, VS Code, and (as of GA in May 2025) the official MCP Registry hosts hundreds of servers.

I used code-review-graph, an open-source tool that builds a property graph of your codebase and exposes it as an MCP server. The setup is a three-line ritual:

pip install code-review-graph
code-review-graph build ./my-project
code-review-graph install      # auto-detects Claude Code / Cursor / Windsurf

The graph contains nodes for files, classes, functions, and tests, with edges for imports, calls, inheritance, decorates, listens-to, and tested-by. Once it's wired in, the AI can call MCP tools like:

Tool	What it answers
`blast_radius(file)`	Every file that depends on this one (N hops)
`flow_trace(func)`	Where a function's output flows
`semantic_search(query)`	Hybrid: vector + graph proximity
`community_detect()`	Tightly-coupled modules
`risk_score(diff)`	Numerical risk of a change
`dead_code()`	Unreachable from any entry point

Anthropic's MCP rollout in 2025 also brought OAuth, prompts, and resource subscriptions, so the graph can push updates when files change instead of being re-queried each turn. That detail matters at scale -- code KGs are not cheap to walk, and stale snapshots are how teams ship bugs.

The token math (the cheap reason to bother)

Before the graph, my AI reviewer was getting context like this:

PR diff: auth.py + 1 file
Reviewer context: grep for "auth" -> 50 related files
Tokens: ~150,000

After the graph, it gets this:

PR diff: auth.py + 1 file
Reviewer context: blast_radius("auth.py", hops=2) -> 7 files
Tokens: ~18,000

The first time I ran it, the AI answered the review question in two seconds with a 7-file context. I had spent 30 minutes the day before grepping the same answer by hand. That moment of "what was I doing with my career" is, I think, the actual product of harness engineering.

But cheaper context isn't the interesting part. The interesting part is which bugs the graph surfaced that grep + vector had missed.

The 3 bugs

Bug 1: the silent contract change (event handler)

The diff was small. auth.py added a device_id field to its login event payload.

Vector search retrieved login.py, auth_test.py, password_validator.py -- the obvious neighbors. The reviewer approved.

The graph retrieved one extra file: event_handlers/audit_log.py. It listens to login_event and serializes the payload to a fixed schema in S3. Adding a new field broke the schema validator on every login. Production caught fire 90 minutes after merge.

Why grep missed it: audit_log.py doesn't import auth.py. It listens to an event bus. There's no string match on "auth" in the file.

What the graph saw: auth.py --emits--> login_event --consumed-by--> audit_log.py. Three hops, zero string matches, but a clean structural path.

A code review pass that doesn't follow event subscriptions is a code review pass that doesn't review event-driven systems.

Bug 2: the decorator surprise (transitive change)

A teammate refactored @with_retry to add a backoff parameter. The default value was the same, so existing callers were "unaffected." Reviewer approved on the strength of the unit tests.

Vector search retrieved files that explicitly imported the decorator. About a dozen.

The graph retrieved 31 files. The 19 the graph added were files that applied @with_retry to functions that, three calls deep, ended up calling a function whose retry behavior had subtly changed under load.

One of those callers was a payments webhook handler. Under retry, it now waited an extra 800ms before raising. That 800ms put it past the webhook timeout. We started losing about 0.4% of webhook deliveries silently.

Why grep missed it: a decorator's effect propagates to every callsite of every decorated function. That's structural, not lexical.

What the graph saw: with_retry --decorates--> {19 functions} --called-by--> {31 files}. The caller of a decorated function inherits the decorator's behavior, even if it never mentions the decorator's name.

Bug 3: the orphan test (false confidence)

A migration changed the way one ID was hashed. The PR included a test that asserted the new hash. CI was green.

The graph showed something the test runner didn't: that test was in a file that no longer ran in CI because it had been moved out of the tests/ directory three weeks earlier and nobody had updated the path glob in the CI config. The test passed because the test file was never executed. The PR landed with a broken hash that corrupted the migration's first 8,000 rows.

The graph had dead_code() for unreachable functions and an inverse query for unreachable test files. I'd never asked it. After this PR, I added "run dead_code() on test files" to the postflight check.

Why grep missed it: grep doesn't know what CI runs. A test file's existence and a test file's execution are different facts.

What the graph saw: test_user_id.py had no incoming edge from any CI config and no tested-by edges from production code. It was a green file in a green repo that didn't actually test anything.

What the graph doesn't replace

I don't want to oversell this. Three things the graph is bad at, and you should keep using vector search or grep for:

Natural-language queries. "Find code about authentication" is still better answered by a sentence-embedding model. The graph wants a node name.
New code with no edges yet. If the function was added in the diff, the graph knows it exists but has no incoming edges. Grep is fine here.
Cross-repo retrieval. Most code KG implementations are per-repo. If your auth lives in another service, you need a cross-repo strategy (or, increasingly, a cross-repo MCP server -- this is where Sourcegraph and Context.ai are converging in 2026).

The right setup is hybrid: vector search for "find related concepts," graph for "find dependents," grep for "find this exact string." MCP makes that hybrid trivial because the model picks the tool per question.

The 2026 landscape, briefly

Three things have changed since I started doing this in late 2024:

GitHub Copilot Chat added repo-wide knowledge graph context in March 2026, walking the call graph for @workspace queries instead of relying purely on file embeddings.
Sourcegraph shipped an MCP server that exposes its long-standing code graph to any MCP-compatible IDE. They had this graph in 2017; the MCP wrapper is what makes it model-accessible.
Cursor integrated repomap for project-wide structural context. Not a graph, technically, but the same idea: structural retrieval beats lexical retrieval for cross-file changes.

The pattern is converging. By the end of 2026, "AI code review" without a structural retrieval layer is going to look the way "AI code review without a vector store" looked in 2023. Quaint.

What to wire up if you're starting

If you don't already have something like this, the pragmatic order is:

Build the graph for one repo. code-review-graph or equivalent.
Expose it as an MCP server so your AI tool of choice can call it.
Add blast_radius to the postflight check for every PR. Just print the list of 2-hop dependents next to the PR. Even without the AI doing anything, the human reviewer reads better.
Keep your vector search and grep. Don't rip them out. Add the graph alongside.
Wait for the bugs to surface. They will.

I had the graph for two weeks before the first bug it caught -- the audit log schema break. I don't think I would have shipped that bug without it. I do know I shipped six versions of it across my career before I had this tool.

Closing

For the last decade, code search has meant "embed the file and find similar embeddings." That's a fine answer to half the question. The other half -- "what does this change break?" -- is structural, and embeddings don't see it.

The graph sees it. MCP makes the graph addressable. Together they collapse most cross-file retrieval into a single, cheap query. The bugs that used to live in those gaps don't, anymore.

Want the full rationale and more graph patterns? I cover graph schema design, GraphRAG vs vector RAG, and code-as-graph patterns in Knowledge Graph Practical Guide: From RAG Limits to Graph-Native AI.

References

Model Context Protocol -- Anthropic, 2024
code-review-graph -- Open-source code KG with MCP server
GraphRAG: From Local to Global -- Microsoft Research, 2024
Sourcegraph MCP integration -- 2026 MCP server announcement
GitHub Copilot Workspace Context -- Repo-graph integration, March 2026