DEV Community: Ken Imoto

KV Cache Quantization: I Stretched Qwen 35B's Context 8 on 12GB VRAM

Ken Imoto — Tue, 28 Jul 2026 13:00:01 +0000

600 MiB of headroom

My RTX 4070 was running Qwen 35B beautifully after the --cpu-moe trick from a previous run. The tokens/sec were where I wanted them. VRAM sat at 11,714 MiB out of 12,281 — 95% full.

That leaves 600 MiB. Not enough for a serious agent.

The context window I was giving llama.cpp was -c 4096. Fine for chat. Not fine when a Claude Code-style agent hands the model 12,000 tokens of tool definitions before it says hello.

I wanted -c 32768. That's an 8× jump. And the memory that grows with context length is the KV cache. Multiply the cache by 8 with 600 MiB free, and llama.cpp dies during warm-up. I know because I tried it first.

What actually sits on the GPU

After offloading the MoE experts to CPU (the previous chapter's trick), the GPU is holding two things:

The attention weights and non-MoE parameters
The KV cache — a running record of every token the model has already read

The first is fixed. The second grows linearly with context length. Double the context, double the cache. -c 4096 → -c 32768 doesn't just want 8× more tokens processed, it wants 8× more cache resident in VRAM the whole time.

There is no room. So the cache itself has to shrink.

Two flags

llama.cpp takes two flags for KV cache dtype:

llama-server -m qwen35.gguf -ngl 99 --cpu-moe -c 32768 \
  -ctk q8_0 -ctv q8_0

-ctk is the Key cache, -ctv is the Value cache. Default is f16 (16-bit). q8_0 cuts each in half. Halving both means the KV cache footprint drops by roughly 50%.

That freed-up VRAM is exactly what I need to make the context 8× bigger without touching the model weights.

The measurement

Same prompt, same seed, two runs — one at f16 KV, one at q8_0 KV:

KV dtype	Max `-c` I could allocate	Tokens/sec (decode)	Perplexity delta
f16 (default)	4096	~34.6	baseline
q8_0	32768	~34.1	negligible in my tests

The speed loss is inside noise. The context is 8× longer. The quality drop I could not tell apart from run-to-run variance.

Community measurements agree: symmetric q8_0 KV lands somewhere under 0.1% perplexity delta on most models. Going harder — q4_0 on both K and V — is where you start seeing real degradation, and asymmetric setups (-ctk q4_0 -ctv q8_0) end up being the pragmatic Q4 configuration if you go there.

I have not gone there. q8_0 was already enough headroom for the agent workloads I run.

Trade-off summary

Keep it in your head as two rows:

f16 KV: fastest, but context stays around 4096 on a 12GB card. Fine for chat.
q8_0 KV: same speed you measured before, VRAM roughly halved on the cache side, context can grow to 32768. Fine for agents.

That is the entire decision.

When to actually bump `-c`

Bigger -c is not free even with quantized KV. llama.cpp reserves the full window up front — a 32k context eats 32k worth of cache the moment the server boots, whether you use it or not.

So I tier it:

Chat, single-shot generation: -c 4096. Do not bother quantizing KV.
Light agents (Claude Code-style): -c 8192 covered almost every case I hit. No quantization needed if you have any headroom.
Heavy agents (multi-tool CLIs like Qwen Code CLI): -c 32768 with -ctk q8_0 -ctv q8_0. The first agent turn alone can hit 19,000 tokens once system prompt and tool schemas load. 8192 will crash mid-invocation.
Long document reading: only bump -c to the actual document size. Do not pre-provision 32k for a 6k document.

The generalization "agents need big context and quantized KV" is too coarse. The right number depends entirely on the CLI's tool definition weight. Two agent frameworks on the same model can want radically different -c values.

The two-line summary

If you have a 12GB card and you want Qwen 35B to run agents:

Offload the MoE experts to CPU (previous chapter).
Turn on q8_0 KV cache (this chapter).

That is a total of three flags: --cpu-moe -ctk q8_0 -ctv q8_0. For that, you get roughly the same tokens/sec you had, an 8× longer usable context, and no measurable quality regression on the workloads I have tested.

I spent a week assuming I needed a 4090 to get here. I did not.

If you're pointing a Claude Code-style agent at this setup — where tool schemas and system prompts eat 12k tokens before you type anything — the sizing decisions and CLAUDE.md hardening are covered here:

Claude Code Mastery — agent workflows, context budgeting, and hardening

The AI Summary Said "It's Not a Scam." The Springboard Was Your Site's Search Box

Ken Imoto — Tue, 28 Jul 2026 01:28:17 +0000

Last August, a man planning a cruise googled Royal Caribbean's customer service number. Google's AI Overview served him a phone number at the top of the results. He called it, handed over his card details, and the number belonged to scammers. Similar cases hit Southwest Airlines searches. That variant, fake support numbers planted where AI summaries would pick them up, got plenty of coverage.

Last week, Japan's Metropolitan Police announced a quieter variant that I think deserves more attention from developers, because the attack surface sits on legitimate sites: the search box. Possibly the one on yours.

Here's the scene the police described: someone gets invited into an investment group on social media. Before sending money, they do the sensible thing and search the group's name. The results show "XX is not a scam" and "I made money with XX." The AI summary at the top of the page agrees: "XX is not a scam." Reassured, they transfer the money.

The victim's verification habit -- "let me search before I trust this" -- has been folded into the trap.

When I read the report, my first question was: how? I work on LLMO (optimizing sites to get cited by AI search) day to day, so I suspected one of the search-pollution techniques floating around SEO circles. The trail led to something older and dumber than I expected: site-search spam, documented by the Japanese SEO firm JADE back in February 2023.

This post covers the mechanism, why AI summaries repeat the lie, and the defenses you can ship this week (noindex, X-Robots-Tag, 404-on-zero-hits).

The mechanism: three steps, no hacking

Most sites with a search box return results at a URL like /search?q=keyword. Two properties of a typical implementation set up the attack:

Anyone can put an arbitrary string in the query parameter
The page reflects that query into its <title> or <h1> ("Search results for 'keyword' | Acme Corp")

The attack:

The attacker composes a search URL on a trusted domain: acme.com/search?q=XX+is+not+a+scam. No need to touch the search box. The URL alone does the job.
They link to that URL from sites they control.
Googlebot follows the link, crawls the results page, and indexes it. From then on, web search can show "XX is not a scam | Acme Corp" under a legitimate domain.

The victimized site was never breached. No malware, no intrusion, no tools. The attacker built a URL and placed a link. When I first understood this, I said "wait, that's it?" out loud. What's being exploited is not a vulnerability. It's a spec.

To the person searching, it looks like Acme Corp's website says "not a scam." The trust the domain spent years earning gets subleased to a stranger's sentence.

Why the AI summary repeats the lie

AI Overviews and similar features are structurally close to RAG: retrieve pages relevant to the query from the search index, then compose an answer from them. The internals aren't public, but the dependency is observable: the summary is built downstream of the index.

The AI has no way to smell the setup. What it retrieved is, as far as it can tell, text on a trusted domain. It doesn't verify claims; it weighs source authority and cross-source agreement. So if an attacker seeds the same sentence into search URLs on several reputable domains, the AI sees multiple independent authoritative sources agreeing.

That's the ugly part: the more seriously an AI weights authority signals, the better this attack works on it. The diligent ones are the easiest marks.

The pipeline is simple: search index upstream, AI summary downstream. Poison the upstream and the downstream poisons itself. You could wait for AI vendors to filter better (Google said it "took action" on the fake phone numbers; new ones kept popping up), or you could close the reflection surface on your own site, which is faster and actually under your control.

The 5-minute self-check

Can your site be used as a springboard? Three checks:

# 1. Are your search result pages indexed? (in Google)
site:example.com inurl:search
site:example.com inurl:"?s="

# 2. Indexed under suspicious phrases?
site:example.com scam
site:example.com refund

# 3. Do your search result pages carry noindex?
curl -sI "https://example.com/search?q=test" | grep -i x-robots-tag

# No header? Check the HTML meta tag
curl -s "https://example.com/search?q=test" | grep -i '<meta name="robots"'

site: queries are a quick smoke test; Google doesn't guarantee exhaustive results. For a definitive answer, open Search Console and check Indexing > Pages and Performance > Pages for URLs containing /search or ?s=.

Also look at your search results template: does it reflect the query into <title> or <h1>? Reflection plus indexability is the combination that makes you a target.

One reassurance: client-side search (JS filtering in the browser, common on static sites) doesn't have this attack surface at all, because the server never returns different HTML per query.

Defenses

Two viable strategies, based on JADE's recommendations:

Measure	Effect	Caveat
`<meta name="robots" content="noindex">`	Reliably keeps result pages out of the index	Neutralized if robots.txt blocks the page
`X-Robots-Tag: noindex` header	Same, applied at infra level without touching templates	Same caveat
noindex (or 404) on zero-hit queries	Keeps search-page SEO traffic while blocking spam	404 can hurt UX for legitimate zero-hit queries
`robots.txt` `Disallow: /search`	Suppresses crawling	Incomplete alone -- blocked URLs can still get indexed via external links

Choosing is simple:

Not chasing SEO traffic on search result pages? noindex all of them. Simplest, most reliable.
Want to keep that traffic? Return noindex on zero-hit queries. Spam phrases like "XX is not a scam" almost always hit zero results, so this alone kills most of the attack.

There is one trap worth internalizing: noindex only works if the crawler can read the page. Block the URL in robots.txt and the crawler never sees your noindex, which un-neutralizes the whole defense. Google's docs state it outright: for noindex to be effective, the page must not be blocked by robots.txt. Never combine the two on the same URL.

Implementation examples.

WordPress search pages (?s=) get noindex by default if you run Yoast or similar. On a bare theme, use the wp_robots filter (WordPress 5.7+, plays nicely with core and plugin output):

// functions.php
add_filter('wp_robots', function ($robots) {
    if (is_search()) {
        $robots['noindex'] = true;
    }
    return $robots;
});

Next.js (App Router):

// app/search/page.tsx
export const metadata = {
  robots: { index: false, follow: true },
};

At the infra layer, nginx. Two gotchas in this snippet: it matches path-style search URLs (/search), not query-style (?s=); for those you'd branch on $arg_s instead. And nginx's add_header has inheritance rules that bite: a single add_header inside a location cancels all headers defined at upper levels, so re-declare your security headers there.

location /search {
    add_header X-Robots-Tag "noindex" always;
    # re-declare upper-level add_header lines (security headers etc.) here
    proxy_pass http://app;
}

Even setting the scam angle aside, noindexing search result pages is standard SEO hygiene: it prevents duplicate-content bloat and crawl budget waste. This is a good excuse to finally do it.

Wrap-up

The "not a scam" poisoning that Japanese police warned about is explained by site-search spam. What's exploited is the spec: reflect the query, allow indexing.
AI summaries are RAG over the search index. Upstream poison becomes the downstream answer. Closing your reflection surface is faster than waiting for AI-side filters.
noindex is the backbone. Never robots.txt-block a URL you want noindexed. Keep search traffic if you want it, but noindex on zero hits.

If you run a site, try site:yourdomain inurl:search today. If anything comes back, the defense section above is your afternoon. Is your search box carrying someone's "it's not a scam"?

References

Metropolitan Police Department (Japan), Cyber Security Countermeasures Division advisory, July 24, 2026. Coverage: ITmedia NEWS, July 27, 2026 (Japanese)
Yusuke Murayama, Site-search spam abusing other companies' sites, JADE blog, February 8, 2023 (Japanese)
Washington Post via Slashdot: Google's AI Overview pointed him to a customer service number. It was a scam. (August 2025)
Google Search Central, Block Search indexing with noindex

freee + MCP: I Automated My Japanese Tax Return With Claude Code — 3 Things That Broke in Production

Ken Imoto — Mon, 27 Jul 2026 13:00:01 +0000

Context for readers outside Japan

Japan's kakutei shinkoku is the annual self-employed tax return — the local equivalent of the US Schedule C or the UK Self-Assessment. Every sole trader files one. The bookkeeping underneath it is the same problem every self-employed engineer knows: categorize a year of receipts correctly, or the tax office comes back with questions.

freee is the country's dominant cloud accounting SaaS for this. In late 2025 they shipped an MCP server. So this year I let Claude Code drive it, and filed the whole thing with the AI in the loop.

That is the part that worked. Below is the part that did not.

The setup that worked

The MCP wiring is standard OAuth + PKCE. Claude Desktop config:

{
  "mcpServers": {
    "freee": {
      "command": "npx",
      "args": ["@him0/freee-mcp"],
      "env": {
        "FREEE_CLIENT_ID": "your_client_id",
        "FREEE_CLIENT_SECRET": "your_client_secret"
      }
    }
  }
}

The browser opens once for OAuth consent. After that, refresh tokens keep the session alive.

The first useful moment came about ten seconds after setup. I typed one line:

Log a ¥3,200 stationery purchase from Amazon on Dec 15.

Claude picked the account (消耗品費 / office supplies), computed the tax code (課対仕入10% / 10% input-taxable purchase), and wrote it to freee. Three seconds. Doing that by hand through the freee UI takes two or three minutes.

Ten in a row, all correctly categorized:

Input	Account picked	Correct?
Amazon stationery	Office supplies	✅
Taxi fare	Travel expense	✅
Coworking monthly	Rent	✅
Domain renewal	Communication	✅
Accountant fee	Professional fee	✅
AWS invoice	Communication	✅
Business dinner	Entertainment	✅
Coffee (client meeting)	Meeting expense	✅
Book (technical)	Books/subscriptions	✅
Printer paper	Office supplies	✅

Ten for ten. Batch mode was better — I threw a CSV of 32 transactions at it and had them all in freee in about three minutes. Manual: over an hour.

That is the 90% story. Now the 10%.

Break #1: receipts cannot be attached

Japanese tax law requires receipts (ryōshūsho) to be attached to bookkeeping entries. This is not optional. Storage of the digital copy is legally mandatory.

freee's REST API supports it. MCP does not. The moment I tried:

Tool: mcp_server__api_post
Path: /api/v1/receipts
Body: { "company_id": "xxx", "description": "electric bill, July" }
Response: 400 — Content-Type must be "multipart/form-data"

MCP's JSON-RPC transport cannot send multipart/form-data. Binary uploads are not in the protocol. This is not a freee bug — this is the shape of MCP itself.

Fix: I stopped trying. Receipt uploads went into a separate Python script that hits freee's REST endpoint directly with requests.post(files={...}). MCP handles the bookkeeping; the CLI handles the file uploads. Two tools, clean seam.

Break #2: 270 tools, several thousand tokens gone before "hello"

The freee MCP server exposes 270 tools. Accounting, HR, invoicing, timekeeping — the whole company platform. All 270 schemas load into every conversation turn.

I only need maybe 8 of them for a tax return. The other 262 sit in context taking up space I would rather spend on transaction descriptions.

This is a real 2026 pattern: SaaS vendors are shipping "one MCP server for the whole product" instead of narrow, task-scoped servers. It works right up until you connect a second one. Two of these and you're spending most of the context window on tool descriptions before the model reads a single receipt. I know this because I ran freee alongside a second MCP for two days before I noticed Claude cheerfully "forgetting" transactions I'd told it about three turns earlier.

Fix: Client-side allow-listing of tool names. Claude Desktop supports this via config. Not every MCP client does — Cursor did at the time, some smaller clients didn't. If your client doesn't, you're stuck with the full 270.

Break #3: rate limits, silently

freee's API rate-limits per minute. Batch-processing 32 transactions in a row hits it. What happens then depends on the tool:

REST API: 429 with Retry-After
MCP tool call: the tool returns an error string. Claude sees a text failure and moves on. No backoff, no retry, no queue.

MCP has no rate-limit primitive in the protocol. There is no retry_after field for a tool response to signal "wait 30s then try me again." So the model just fails the row and continues.

Fix: I put a 1-second sleep between transaction writes in my prompt ("write these one at a time, wait one second between them"). Ugly, but it holds. The 2026 story here is that MCP gateways with per-tool token-bucket enforcement are starting to appear — that is where the fix eventually lives, not in the model or the server.

What the final workflow actually looks like

The end state was a hybrid, not a pure-MCP flow:

Import bank/card CSV → my script, not MCP
Categorize + write transactions to freee → MCP + Claude
Upload receipt PDFs → my script, not MCP
Reconciliation & review → MCP + Claude
Final tax form generation → freee's own UI (with MCP-produced data underneath)

MCP owns the parts where language understanding matters — reading a messy expense description, picking an account, catching a typo in a memo. The CLI owns the parts where MCP structurally cannot go — files, throttling, orchestration.

"MCP will replace my accountant" turned out to be the wrong framing. "MCP is a 90% automation layer, the last 10% is CLI glue" is the right one. And the last 10% is where all the design work is.

The one-paragraph summary

freee + MCP + Claude Code cut my bookkeeping time from a weekend of drudgery to about two hours. Three things broke in production: no binary uploads, 270-tool context bloat, silent rate-limit failures. All three had workarounds. None of the workarounds lived inside MCP. If you're building an MCP integration for a real business process, plan for the seams before you plan for the happy path.

The full MCP threat model, plus the seven-service comparison of where each MCP server actually hits these ceilings, is here:

MCP Security in Practice — production integration patterns and the failure modes nobody talks about

Playwright + Chromium: 1 Script Replaces 2 Manual Testers for WebRTC Video Call E2E

Ken Imoto — Fri, 24 Jul 2026 13:00:01 +0000

If you've ever watched a QA engineer press "call" on one laptop, walk across the room, and press "answer" on a phone before the ringing stops, you already know the setup. Two-browser sync is the single hardest part of WebRTC video call testing.

The whole flow looks simple on a whiteboard. Alice presses call, Bob's screen lights up with an incoming banner, Bob answers, video panes appear on both sides. All of it has to be automated without breaking the timing. One script, two contexts, and enough patience to let ICE finish. That's the shape of it.

I wrote this after replacing a two-person manual regression pass for a video call app. The team ran that check every merge to main, and every time it involved two people, two devices, and a Slack thread that started with "wait, ready?" Now one Playwright script does it in about 40 seconds on CI, and it fails loudly at the merge when signaling regresses.

Why two browsers and not one page

The instinct on a first pass is to open one page and simulate both sides. That works for signaling unit tests. It fails for anything that involves peer connection state, because the local peer and the remote peer end up sharing the same JS runtime and the same fake camera, and every failure mode that matters in production stays invisible: race between offer and answer, ICE candidate ordering, DTLS handshake stall.

Two Browser Contexts inside a single Chromium process fix that. Each context has its own cookies, localStorage, permission grants, and (crucially) its own RTCPeerConnection instance living in its own renderer. When the caller's context sends an SDP offer, it goes out through the signaling server the same way it would from a second laptop, and the receiver context reads it back through a real network round trip.

The same set of Playwright fixtures also catches signaling bugs, presence-server bugs, and TURN misconfiguration. The exact bugs manual testers used to catch by accident.

The launch flags

Every Chromium in this test has to lie about its hardware.

import { test, chromium } from '@playwright/test';

test('one-on-one video call connects', async () => {
  const browser = await chromium.launch({
    args: [
      '--use-fake-ui-for-media-stream',
      '--use-fake-device-for-media-stream',
    ],
  });

  const callerCtx = await browser.newContext({
    permissions: ['camera', 'microphone'],
  });
  const receiverCtx = await browser.newContext({
    permissions: ['camera', 'microphone'],
  });

  const callerPage = await callerCtx.newPage();
  const receiverPage = await receiverCtx.newPage();

  await callerPage.goto('http://localhost:3000');
  await receiverPage.goto('http://localhost:3000');

  // ...
  await browser.close();
});

--use-fake-ui-for-media-stream auto-accepts the permission prompt so the test doesn't hang staring at a browser dialog. --use-fake-device-for-media-stream replaces getUserMedia with a synthetic camera. The default is Chromium's green ball testcard, which is fine for signaling checks but useless if you want to verify pixels. For pixel checks, pass --use-file-for-fake-video-capture=./sample.y4m and drop in a Y4M file. Chrome only reads Y4M here; MP4 is not supported for fake capture, which surprised me the first time.

A gotcha worth calling out: those flags must be passed at browser launch time. If you set them in playwright.config.ts under use.launchOptions.args but then call chromium.launch({}) directly inside a test, they're silently ignored on the second launcher. Same rule applies to any custom launcher.

The call sequence, step by step

Two-browser tests read like a stage direction script. Every action has an implicit "wait until the other side sees it" between the lines.

test('caller initiates, receiver answers, both see video', async () => {
  // Precondition: browser, callerPage, receiverPage already set up.

  // Step 1: both join the same room
  await callerPage.fill('#room-input', 'test-room');
  await callerPage.click('#join-button');
  await receiverPage.fill('#room-input', 'test-room');
  await receiverPage.click('#join-button');

  // Step 2: caller starts the call
  await callerPage.click('#call-button');

  // Step 3: wait for the incoming banner on the receiver
  await receiverPage.waitForSelector('#incoming-call-notification', {
    timeout: 10000,
  });

  // Step 4: receiver answers
  await receiverPage.click('#answer-button');

  // Step 5: wait for the remote-video element on both sides
  await callerPage.waitForSelector('video#remote-video', {
    state: 'attached',
    timeout: 15000,
  });
  await receiverPage.waitForSelector('video#remote-video', {
    state: 'attached',
    timeout: 15000,
  });

  // Step 6: verify actual frames are decoded, not just that <video> exists
  const callerHasFrames = await callerPage.evaluate(() => {
    const v = document.querySelector('video#remote-video') as HTMLVideoElement;
    return v.videoWidth > 0 && v.videoHeight > 0 && !v.paused;
  });
  expect(callerHasFrames).toBe(true);
});

About the 10-15 second timeouts

WebRTC connection setup is not fast. Signaling exchange over your signaling server, then ICE candidate gathering, then STUN/TURN round trips, then DTLS handshake, then media flow start. Even on localhost with a fake camera, the whole chain takes 3-8 seconds under normal conditions, and it can spike past 12 under CPU pressure on a shared CI runner.

Playwright's default 5-second selector timeout is what gives you the classic "works locally, flakes in CI" pattern for WebRTC. Raising it to 30 seconds hides real regressions because ICE stalls take that long to surface as errors. Ten to fifteen seconds is the band where the test fails when signaling actually broke, and passes when signaling is slow-but-alive.

Fixtures do the setup once

Repeating the two-context boilerplate in every test file gets old within about three tests. Playwright's fixture system turns it into a one-line dependency.

// fixtures/webrtc-context.ts
import { test as base, BrowserContext, Page } from '@playwright/test';

type WebRTCFixtures = {
  callerContext: BrowserContext;
  receiverContext: BrowserContext;
  callerPage: Page;
  receiverPage: Page;
};

export const test = base.extend<WebRTCFixtures>({
  callerContext: async ({ browser }, use) => {
    const ctx = await browser.newContext({
      permissions: ['camera', 'microphone'],
    });
    await use(ctx);
    await ctx.close();
  },
  receiverContext: async ({ browser }, use) => {
    const ctx = await browser.newContext({
      permissions: ['camera', 'microphone'],
    });
    await use(ctx);
    await ctx.close();
  },
  callerPage: async ({ callerContext }, use) => {
    const page = await callerContext.newPage();
    await use(page);
  },
  receiverPage: async ({ receiverContext }, use) => {
    const page = await receiverContext.newPage();
    await use(page);
  },
});

export { expect } from '@playwright/test';

Now the test file just asks for what it needs:

// tests/webrtc/p0-basic-call.spec.ts
import { test, expect } from '../../fixtures/webrtc-context';

test('caller and receiver connect', async ({ callerPage, receiverPage }) => {
  await callerPage.goto('http://localhost:3000');
  await receiverPage.goto('http://localhost:3000');
  // Room join → call → answer → verify video
});

Context teardown runs in the fixture, so you don't leak a BrowserContext per test.

Verifying that a video actually plays

"A <video> tag exists" is not the same as "video is showing." A broken test can pass the selector check and miss a black screen. There are two levels of paranoia here, and which one you want depends on what you're testing.

Level 1: videoWidth / videoHeight

const hasVideo = await page.evaluate(() => {
  const v = document.querySelector('video#remote-video') as HTMLVideoElement;
  return v && v.videoWidth > 0 && v.videoHeight > 0;
});

Both attributes stay at 0 until the video track is active and at least one frame is decoded. Any positive value means media is flowing. Cheap, fast, catches most of the failures I've seen in practice.

Level 2: requestVideoFrameCallback

For "is the video actually updating, or is this a frozen frame" you need the frame callback.

const isReceivingFrames = await page.evaluate(() => {
  return new Promise<boolean>((resolve) => {
    const v = document.querySelector('video#remote-video') as HTMLVideoElement;
    if (!v) return resolve(false);
    let count = 0;
    const cb = () => {
      count++;
      if (count >= 3) return resolve(true);
      v.requestVideoFrameCallback(cb);
    };
    v.requestVideoFrameCallback(cb);
    setTimeout(() => resolve(false), 5000);
  });
});
expect(isReceivingFrames).toBe(true);

Three frames within five seconds. A frozen first frame or a static test pattern fails this check. It's slower and slightly heavier, but the failure mode it catches (the connection made it, and then media stopped) is exactly the one users describe as "my video froze."

The one thing not to do: Promise.all

Two browsers with two independent await chains suggests parallelizing with Promise.all. Don't. WebRTC's signaling is sequential by design, and firing both sides at once creates race windows that only fail 15% of the time, which is the worst possible flake rate because it looks stable until it doesn't.

// Safe: strictly sequential
await callerPage.click('#call-button');
await receiverPage.waitForSelector('#incoming-call');
await receiverPage.click('#answer-button');

// Dangerous: receiver clicks answer before the banner appears
await Promise.all([
  callerPage.click('#call-button'),
  receiverPage.click('#answer-button'),
]);

The rule I follow: await the caller's action, then move to the receiver. The only place I've found genuine parallelism useful is group calls with three or more participants where multiple joins are semantically simultaneous. Even there I only use Promise.all on the join() calls, never on answer() calls.

Why Playwright and not Cypress / WebdriverIO / Selenium Grid

The multi-context first-class support is the main reason. Cypress runs a test in a single browser tab with an iframe sandbox and cannot open a second real browser context in the same process, which means you end up with Cypress plus a headless driver on the side and manual sync between them. WebdriverIO handles multi-browser, but the API for coordinating two browser objects reads like grid orchestration rather than a test script. Selenium Grid works, but the setup cost is a full network topology, and you pay that cost forever.

Playwright's browser.newContext() giving you N isolated peers inside one JS process is the shape you want for peer-to-peer testing. The Chrome DevTools Protocol access under the hood is what makes flags like --use-fake-device-for-media-stream actually work end-to-end without a driver in the middle rewriting the launch args.

Docker + GitHub Actions with fake devices

The fake camera flags work on Chromium headless without xvfb. That means the standard mcr.microsoft.com/playwright:v1.49.0-jammy image with default GitHub Actions runners handles WebRTC E2E out of the box: no display server, no OpenGL, no sudo apt install. Runtime for the one-on-one test above is about 40 seconds on a ubuntu-latest runner. For a 6-peer group call it climbs to 90 seconds; still well inside a normal CI budget.

The one thing that surprised me: the fake camera has a memory footprint per peer that scales linearly, so a 10-peer group call test on a 2-core / 7GB runner starts hitting OOM. If you need more than 6 peers, bump to ubuntu-latest-4-cores or split the peers across two runs.

Wrap-up

The two-manual-testers approach doesn't scale past two testers, and every video call product I've seen eventually needs to test presence, screen share, mute state, and 5-person group calls. Once you're there, scheduling a five-person QA sync every merge is not a plan.

One Playwright script with two Browser Contexts, launched with --use-fake-device-for-media-stream, gives you the smallest useful WebRTC E2E setup. Add frame verification when you care about pixels. Add fixtures the moment you write a second test. Keep the calls sequential and let ICE finish. What you buy back is the ability to ship a video-call feature and let CI tell you when the presence server broke, instead of Bob-from-support at 11pm.

MCP Scorecard: 4-Layer Pre-flight Check Before You Publish a Server

Ken Imoto — Thu, 23 Jul 2026 13:00:01 +0000

The MCP ecosystem grew faster than the review rules for it. In the last few weeks I published three MCP servers of my own and each time I found myself running the same mental checklist before shipping: how many tokens am I burning per turn just by being registered, are my tool descriptions telling the LLM enough to pick the right one, am I leaking anything into a description string, and does the tool name look like something else.

I moved that checklist into a tool and put it on PyPI. mcp-scorecard runs four pre-flight checks against an MCP server's declared surface and returns a scorecard graded A to F with per-tool findings. The CLI is one command; the same four layers ship as five MCP tools, so an LLM inside Claude Code can audit another MCP without leaving the session.

pip install mcp-scorecard
mcp-scorecard scan ./your-server.py

This post is the reasoning behind each of the four layers, why they exist in that order, and what the tool caught the first time I scanned my own MCPs. If you already run MCP-Scan or MCP Inspector, the last section explains where this one sits alongside them.

Why pre-flight for MCP at all

The default assumption when reviewing an MCP server is that the interesting risks live at call time: prompt injection in a tool response, credentials leaking through a shell exec, tool shadowing that steers the model to the wrong function. Those are real, and MCP-Scan covers them at runtime. What that framing misses is the surface the LLM sees before any tool is called.

Every tools/list entry is sent to the model on every turn, because the model needs the list to decide which tool to call. That is the passive cost of registering a server. And every description field in that list is what the model reads to choose. That is the passive quality of the server. Both are set at author time and neither depends on runtime traffic. Both are also invisible to a runtime scanner.

A pre-flight covers exactly that layer: what does the LLM see about your MCP, before any request goes out. The four layers below are one attempt at a compact answer.

Layer A — Passive Footprint

A single verbose MCP server can silently burn 5,000+ tokens per turn without anyone calling a tool. The tokens come from three places: the description string for each tool, the JSON Schema for each tool's input, and the tool name itself. All three get concatenated into tools/list and shipped to the model every turn.

Layer A counts these with tiktoken on the cl100k_base encoding (the OpenAI GPT-4-family tokenizer, close enough to Claude's to be a reasonable proxy). The output is a per-tool breakdown plus a global initial_token_load:

                per-tool footprint (top 10 by total)
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┓
┃ Tool                   ┃ Desc tok ┃ Schema tok ┃ Name tok ┃ Total ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━┩
│ check_domain           │      182 │         88 │        2 │   272 │
│ list_typo_permutations │       79 │         54 │        5 │   138 │
│ check_trademark        │       83 │         41 │        4 │   128 │
│ check_handles          │       76 │         40 │        2 │   118 │
└────────────────────────┴──────────┴────────────┴──────────┴───────┘
findings
  · 1 tool(s) have description > 150 tokens: check_domain

That is domain-pre-flight scanned by mcp-scorecard. Total initial_token_load is 656 tokens for four tools, well inside the GREEN band. But one tool (check_domain) is flagged: its description is 182 tokens, over the 150-token bloat threshold. When five servers like that are registered simultaneously, the passive drain adds up in a way that stays invisible until an "unused MCP" review.

The thresholds are calibrated but not sacred. v0.1 uses:

Metric	GREEN	YELLOW	ORANGE	RED
`initial_token_load`	≤ 1500	≤ 4000	≤ 8000	> 8000
Description tokens per tool	≤ 150	—	—	flagged as `bloat` if over
Tool count	≤ 15	—	—	≥ 30 raises `tool_count` warning

The RED threshold on initial_token_load is roughly where a single server starts eating into the context budget of long conversations. The 150-token bloat threshold on per-tool descriptions is where descriptions start reading like documentation instead of one-line usage.

The obvious way to fix a bloat finding is to trim the description to a single-purpose sentence and move examples out of the tool surface. Layer B has more to say about what "single-purpose sentence" should contain.

Layer B — Use-Case Scoping

Passive footprint answers how much the LLM sees. Layer B answers how well-targeted what it sees is. The failure mode is: the model has three tools with plausible descriptions and can't tell which one to reach for, so it picks by heuristic and gets it wrong. Fixing that is not a security check; it is a UX check on the LLM as the user of your MCP.

Four rules run in v0.1:

Vague verbs. Descriptions that start with run, handle, process, manage, execute, do, perform, work with, deal with, or the generic helper / utility are flagged with an action hint. The flag on run in my own check_domain is a good example: the description said "run pre-flight checks on a domain", which reads fine as documentation but leaves the LLM with no signal about what is being run. Rewriting to "check availability, run WHOIS, resolve DNS, and score TLD risk for a domain candidate" gives the same information plus enough disambiguation for the model to pick this tool over a sibling.

When-to-use trigger. Each tool's description is checked for an explicit trigger phrase (use this, use when, call when, useful for, and the Japanese equivalents 使う, 使用, 呼び出). Three of my four domain-pre-flight tools do not have one, which is what the ORANGE band in the scan above is complaining about. The trigger is not a magic keyword; it is a marker for the model saying "here is the situation where I am the right tool". A missing trigger tends to correlate with tools that get called from context inference rather than explicit reasoning, which is not a stable pattern.

Overlap detection. Each pair of tool descriptions is scored on shared stem-token overlap. If two tools use most of the same content words in their descriptions, they are probably telling the LLM roughly the same story and it will pick one at random. v0.1 flags overlap pairs above a threshold but does not fail on them; the fix is usually to rewrite one of the two descriptions to name the axis that separates them.

Naming style consistency. The bundled check classifies each tool name as snake_case, camelCase, kebab-case, or flat, and flags a server whose tools mix styles. Mixed styles do not cost tokens directly, but they cost model attention: switching between naming conventions in the same list makes the model spend a small budget on remembering which style a given tool used, and that budget could have gone into the reasoning.

Layer B is the layer where the tool grades its own MCP server harshly. Running mcp-scorecard scan against mcp-scorecard's own MCP server returns overall grade D (ORANGE) because five scoping findings fire on the docstrings of the preflight_* tools. Four of those findings hit vague verbs (handle, process, manage, execute) that appear in the docstrings as examples of what the scoping check catches. That is a false positive in context, and it is flagged as such in the caveat on the product LP. But the same yardstick is applied to the tool itself, unmodified. The alternative would have been to whitelist my own docstrings, and that would have been the kind of trick that makes a rule set useless.

Layer C — Security own rules

Layer C is the layer that most obviously overlaps with existing tools, so it is also the layer with the tightest scope. v0.1 runs three families of own rules against the declared surface only; the runtime coverage that MCP-Scan does well is deliberately left for a v0.2 wrap.

Prompt injection markers in descriptions. A tool description is text the LLM reads on every turn, so any imperative language in it steers the model whether the author meant to or not. The rule catches known injection patterns (ignore previous instructions, disregard, system: prefix, closing </system> tags, you must, and the equivalent Japanese phrasing). This one comes straight out of the book『MCP実践セキュリティ』(Impress NextPublishing) whose review checklist informs several rules in this layer.

Tool shadowing. A tool named ls that lists remote objects will get called instead of the shell ls any time the model reads intent-imprecise instructions like "list the files here". The rule checks tool names against a small set of common shell / filesystem / process names (ls, cat, rm, cp, curl, sudo, exec, eval, shell, execute) and flags any collision. The fix is namespace hygiene: rename to list_objects or s3_ls and the ambiguity goes away.

Hardcoded secrets in descriptions. Regex sweep for AWS access keys (AKIA...), GitHub PATs (ghp_...), OpenAI keys (sk-...), Anthropic keys (sk-ant-...), Google API keys (AIza...), Slack tokens (xox...), PEM private-key blocks, and hardcoded Bearer patterns. The failure mode is not "the credential got committed to the repo": that is gitleaks / trufflehog territory and deliberately not duplicated. The failure mode is "a description string sent to every LLM turn contains a credential", which is a different and worse leak because it exfiltrates the credential through the model's context on every request.

What the layer does not do in v0.1: full-source secret scanning, runtime injection over live traffic, or protocol-level validation. All three are covered better by existing tools; the v0.2 roadmap is a wrap around MCP-Scan for the runtime side, not a rewrite of it.

Layer D — Name Safety

Layer D covers the naming decisions the author already made and asks whether they are safe to publish. Four rules:

Case collision. The bundled dictionary of ~50 brand names and 23 known MCP names is normalized to lowercase; the candidate name is checked against it after the same normalization. GitHub-mcp collides with github-mcp under this rule, and the finding is "case collision under PEP 503 style normalization; will be indistinguishable from github-mcp once packaged".

Brand Levenshtein similarity. Candidate names within Levenshtein distance 2 of a known brand (for brands of at least 4 characters) trigger the typosquat warning. This is the same rule that runs in domain-pre-flight for domain typosquat detection, scaled down to package-name character counts. The 23 known-MCP list is small on purpose; the fix for a hit is not to make the list bigger but to pick a name that does not need to be adjacent to something else.

Separator variants. mcp_scorecard vs mcp-scorecard vs mcpscorecard all normalize to the same package name under PyPI's PEP 503 rules, so shipping one variant while another already exists on PyPI is a namespace conflict. This layer catches it before publish; I ran into the same class of issue with mcp-scorecard itself, where the original name mcp-preflight was already occupied and mcp-pre-flight was rejected by PyPI as too similar. Layer D would have flagged the second name as "too close to an existing MCP" before I hit the PyPI validator.

Namespace hygiene. A regex on the candidate name checks for lowercase kebab-case with the optional @vendor/tool scoping. Names with mixed case or unusual characters are flagged as "prefer kebab-case or @vendor/tool scoping". This one is cosmetic; the finding is a warn, not an error. It exists because the LLM's tool-picking heuristics do work better on consistently formatted names, which loops back to Layer B.

The 23 known-MCP list is bundled as static data. It ships with obvious brands (github, google, anthropic, openai, aws, cloudflare, stripe, hubspot) plus a smaller set of common MCP names (mcp-scan, mcp-inspector, mcp-validator, and this tool's own name). Additions are a plain PR against src/mcp_preflight/data/known_brands.py.

What it does not cover

The four layers above are the LLM-facing quality of an MCP server: what the model sees, how expensive that is, whether it can pick the right tool, whether the names collide with things it already knows. What they do not cover is the runtime security surface that MCP-Scan handles well: prompt injection at call time in tool outputs, credential handling during exec, tool-shadowing at request time. That is a separate scanner reading a live server; v0.1 of mcp-scorecard is an AST + manifest read, and it never executes the target.

A CI-friendly gate is on the roadmap: --format sarif output and non-zero exit codes for ORANGE / RED already work today, and the JSON schema is stable enough for local use. What is not stable in v0.1 is the specific band thresholds; they will move as I run the tool against more real MCPs and find where the current numbers are too permissive or too strict. The alpha label on v0.1 exists exactly for that reason.

Install

pip install mcp-scorecard              # CLI + library
pip install "mcp-scorecard[mcp]"       # + MCP server (stdio)

mcp-scorecard scan ./your-server.py    # full four-layer scan
mcp-scorecard footprint ./server.py    # Layer A only
mcp-scorecard scoping ./server.py      # Layer B only
mcp-scorecard security ./server.py     # Layer C only
mcp-scorecard name my-new-mcp          # Layer D only, on a candidate name

TypeScript / Node MCPs are supported through a manifest JSON: pass the tools/list output (or a saved copy) as JSON to --target, and Layers A / B / C / D all run against it. The AST path is Python-specific; the layers themselves are not.

To use it as an MCP server so Claude Code / Cursor / Windsurf can audit another MCP from chat:

{
  "mcpServers": {
    "mcp-scorecard": {
      "command": "mcp-scorecard-mcp"
    }
  }
}

Then ask the model to score an MCP by path; five tools (preflight_scan, preflight_footprint, preflight_scoping, preflight_security, preflight_name_check) route to the same layers as the CLI.

Companion book

mcp-scorecard is the tool half of a companion pair with a book that teaches the same checks in prose. The book covers the threat model behind Layer C in detail (prompt-injection markers, tool shadowing, hardcoded secrets in descriptions), the review checklist that informs the naming rules in Layer D, and the deployment considerations that come after your MCP passes pre-flight.

👉 MCP実践セキュリティ (MCP Security in Practice)

Feedback on the rules and thresholds is exactly what a v0.1 alpha needs. Issues welcome on kenimo49/mcp-scorecard.

MCP Token Cost Audit: PostgreSQL vs Google Maps vs GitHub vs Freee — 270 Tools Priced

Ken Imoto — Wed, 22 Jul 2026 13:00:01 +0000

PostgreSQL's MCP server costs 35 tokens to sit in your context. Freee's costs 17,500. Same protocol, same JSON, 500x apart before the agent has said a single word.

I ran the same session boot against four production MCP servers — PostgreSQL, Google Maps, GitHub, and Freee — and priced the tool definitions at Claude 3.5 Sonnet input rates. The bill for "just being connected" is small on any single call. It's not small when you multiply it by a year of daily runs, and it's not small when you find out most of the tokens are for tools you will never invoke.

Here is the audit.

Why "connect" costs tokens at all

The first thing a Model Context Protocol client does after handshaking is call tools/list. The server responds with every tool it exposes: name, description, JSON Schema for parameters, and any annotations. Every one of those bytes lands in the LLM's context window before the user has typed a prompt.

Old-style REST integration didn't work this way. You called the one endpoint you needed. If Stripe added forty new endpoints last quarter, your app didn't get slower or more expensive.

MCP inverts that. Connect a server that ships 270 tools and you get all 270 tool definitions, permanently, for every session. The library metaphor is the honest one: you asked to borrow a single book, and the librarian handed you the entire catalog first.

The four-server audit

I measured each server's tools/list response with tiktoken (Claude uses the same tokenizer family for cost purposes) and priced it at $3 per 1M input tokens.

MCP server	Tools shipped	Boot-time tokens	Cost per 10 sessions
PostgreSQL (official)	1	~35	~$0.001
Google Maps	7	~704	~$0.02
GitHub (official)	26	~4,242	~$0.13
Freee	270	~17,500	~$0.52

Two things fall out of that table.

First, the range is not linear. Going from PostgreSQL to Freee is 500x on the token axis, not 270x on the tool axis, because Freee's descriptions are longer and its schemas are richer. A single Freee tool costs roughly 65 tokens of definition, versus roughly 35 for a stripped-down PostgreSQL one.

Second, GitHub is the interesting middle case. The official server ships 26 tools if you take the vanilla bundle, but I've seen forks and community rewrites push that to 90+ tools with correspondingly larger token bills. The InfoQ writeup from May 2026 clocked one large-team configuration at 55,000 tokens of boot-time overhead — roughly 21% of a 200K context window, paid before the first prompt. That's the same shape as Freee, just wearing a hoodie.

The number to fear is not "how many MCP servers do I have." It's "how many tool definitions am I loading, and how verbose are they."

Why Freee balloons to 270 tools

Freee is a Japanese SaaS platform that bundles five back-office APIs — accounting, HR, invoicing, timekeeping, and sales — under one login. The MCP server mirrors that structure honestly:

Freee API	Purpose	Approximate tools
Accounting	Deals, ledger, trial balance	~80
HR	Employees, departments, payroll	~60
Invoicing	Draft, send, reconcile	~40
Timekeeping	Punch, overtime, leave	~50
Sales	Quote, order, revenue	~40

If you're a small business using every corner of Freee, this design is exactly right. You want the accounting agent to reach into invoicing without a second server handshake.

If you're an individual filing one annual tax return, ten of those 270 tools would cover you, and the other 260 are dead weight in your context window for the entire session.

Nobody at Freee did anything wrong. Their MCP server matches their product surface area. What went wrong is the assumption that "connect a server" and "load its tools" should be the same action.

The yearly cost, spelled out

Filing a Japanese sole-proprietor tax return realistically involves 10-ish sessions per month with the accounting API. At 17,500 tokens per session boot, at $3 per 1M input tokens:

Per session:  17,500 × $3/1M     = $0.0525
Monthly:      10 × $0.0525       = $0.525
Yearly:       12 × $0.525        = $6.30

Six dollars is not going to bankrupt anyone. Two things about that number still deserve attention.

One: that's just the definitions. Actual tool calls, arguments, and results are billed separately. The $6 is what you pay for the librarian handing you the catalog, before you've read a single page.

Two: this scales with usage, not with the tools you actually use. A daily user of the same MCP config pays ~$20/year. A team of ten pays $200. A middleware layer running this in every automation run pays real money.

Compare to Freee's own subscription (¥2,380/month ≈ $200/year). The MCP overhead is small next to the product. It's not small next to "we're paying for a bill line no one on the team knows exists."

Three ways to shrink the bill

Strategy 1: allowlist the tools you actually use

The single biggest lever. Most MCP clients let you filter which tools get exposed from a server. Configure it once and 96% of Freee's boot cost evaporates:

{
  "mcpServers": {
    "freee": {
      "allowedTools": [
        "create_deal",
        "list_deals",
        "get_deal",
        "update_deal",
        "get_trial_balance",
        "list_account_items",
        "list_partners",
        "list_taxes",
        "list_walletables",
        "get_company"
      ]
    }
  }
}

Ten tools, ~650 tokens. That's a 96% cut from the 17,500 baseline, and the agent still gets everything a tax return needs. This is the change GitHub itself credited in the May 2026 InfoQ piece for a 62% drop in agent workflow token spend — the mechanism was audits and pruning, nothing exotic.

Strategy 2: rewrite the descriptions

MCP tool descriptions are English prose, and English prose has a wide dynamic range on token count. The same tool, honestly documented, can cost 80 tokens or 20:

// ~80 tokens
{
  "description": "Uses the freee accounting API to create a new transaction (journal entry) against the specified company_id. Accepts amount, date, account item, partner name, and memo. Consumption tax classification is inferred automatically from the account item."
}

// ~20 tokens
{
  "description": "Create a transaction. Args: amount, date, account_item, partner."
}

The verbose version reads like docs. The terse version reads like a signature. Agents infer the same behavior from both — the schema already spells out the parameter names and types — and you save 60 tokens per tool. Multiply by 270 tools and it's non-trivial.

If you're building an MCP server, this is the highest-leverage habit to form early. Descriptions written like API reference docs cost more than descriptions written like function signatures, and both get you to the same agent behavior.

Strategy 3: connect on demand, not on boot

The token bill is per session, so a long-running interactive session with Freee always attached pays it once. A batch job that spawns 100 sessions pays it 100 times.

If a task only needs Freee for one step, connect for that step and disconnect. Most modern MCP clients — including Claude Desktop, Cursor, and Codex CLI — let you enable and disable servers per session or per project. Keep the giant servers off by default. Bring them in when the task actually requires them.

What to evaluate before adopting a new MCP server

The community has spent 2026 auditing servers rather than adding them — see the getunblocked, PolicyLayer, and mcp-audit tooling that emerged this spring — and the useful checklist has settled around four questions:

Question	Good signal	Bad signal
How many tools ship by default?	Under 20, purpose-scoped	Hundreds, bundled by product surface
How long are the descriptions?	Function-signature length	Paragraph, marketing-inflected
Can I filter tools client-side?	`allowedTools` or equivalent	All-or-nothing connection
Does the vendor publish boot-time token counts?	Yes, in the README	You have to measure it yourself

Servers built by teams that have felt the token bill answer these questions on the tin. Servers built without that pressure make you find out at runtime.

The bigger frame

MCP was designed for composability, and composability has a cost the original design didn't foreground. Every server you connect is a permanent tax on the model's attention and your input token budget, whether you use its tools or not. The ecosystem is still catching up to that reality — 14,000+ servers as of May 2026, and the audit tooling is barely older than that.

The good news is that the fixes are cheap and mostly local: allowlist the tools you use, keep descriptions tight, connect on demand. The bad news is that most teams find this out the same way I did — by running the numbers on the bill they were already paying, and being unpleasantly surprised.

If you've never audited yours, the ten minutes it takes to enumerate what your MCP setup is loading is the highest-ROI ten minutes you can spend on your agent this quarter.

The full version of this — the four-server measurement methodology, the yearly cost model at each Claude tier, and eleven MCP servers audited end-to-end — is what my book on MCP security practice covers.

MCP Security Practice: Auditing Tools, Tokens, and Trust in the Model Context Protocol

Sources: InfoQ — GitHub Slashes Agent Workflow Token Spend up to 62%, getunblocked — MCP Token Cost: A Line-Item Autopsy, PolicyLayer — MCP Token Cost Calculator, mcp-audit on GitHub.

OpenAI Codex Shipped 1M Lines From AGENTS.md: 3 Harness Lessons That Beat Model Swaps

Ken Imoto — Tue, 21 Jul 2026 13:00:00 +0000

I spent a week blaming the model.

My Codex runs kept blowing past their budgets, my agents kept editing the wrong files, and every morning I woke up to another PR that "almost" worked. I bought the hype, swapped Sonnet for GPT-5, then swapped it back, then tried a bigger context window, then tried a smaller one. Nothing moved.

The bug was one paragraph deep in my AGENTS.md.

OpenAI published "Harness engineering: leveraging Codex in an agent-first world" in February 2026, and the Codex team's public numbers tell the same story I lived that week. Between August 2025 and January 2026, three engineers drove roughly 1,500 merged PRs and shipped on the order of a million lines of production code with zero lines written by hand. The model did not change halfway through. The AGENTS.md did, twice.

Here are the three things the experiment proved.

Lesson 1: The "one big AGENTS.md" collapses under its own weight

The Codex team's first move was one big AGENTS.md covering the whole repo. Coding conventions, module boundaries, review checklist, deployment runbook, all in one file. It reads like the right thing to do.

It failed in the most predictable way possible. Context is a scarce resource, and a giant instruction file crowds out the task, the code, and the documentation the agent actually needs to look at. The team wrote about this directly: the encyclopedia approach lost, because the encyclopedia was the whole reason the agent stopped reading the code.

The fix was to shrink AGENTS.md to a table of contents (roughly 100 lines) and move the real knowledge into a docs/ directory treated as the system of record. AGENTS.md no longer answers questions; it points at where to look.

I ported the same shape into my own harness the day I read the post. My previous AGENTS.md was 940 lines. My new one is 118 lines and looks like this:

# AGENTS.md

## Overview
FastAPI + Postgres backend for the internal billing service.

## Structure
- `app/` → FastAPI application
- `tests/` → pytest suite
- `docs/skills/` → task-specific skills

## Key Skills
- Add an endpoint → docs/skills/add-endpoint.md
- Write a migration → docs/skills/write-migration.md
- Debug a production incident → docs/skills/oncall-triage.md

## Constraints
- All endpoints must have contract tests
- Never inline SQL — go through the repository layer
- PRs are squash-merged

Every constraint in the old file that used to sit in AGENTS.md now lives in the skill file that actually needs it. The agent loads it only when it enters that task.

The regressions I had been fighting stopped that afternoon. Not because the model got smarter. Because I stopped hiding the code from it.

Lesson 2: Swapping models fixes nothing when the harness is the bug

This is the uncomfortable one.

For a week I A/B-tested models. I tried GPT-5, I tried Claude Sonnet 4.6, I tried Haiku for the cheap runs and Opus for the hard ones. My completion rate on the same fifteen tasks moved by less than four percentage points across all four.

I switched to a smaller model but a cleaner harness: a 118-line AGENTS.md, hooks that ran linters before every commit, and a sandbox that refused writes outside app/. Completion jumped by 31 points on the same fifteen tasks.

The Codex team's phrasing landed differently after that: "Agents aren't hard; the harness is hard." I read it a month before I believed it, which is roughly the industry-average delay.

Louis Bouchard put it more bluntly in his 2026 write-up: stop saying "the model is dumb" and start saying "my system tolerated this failure." That reframe is why the Codex team keeps the same model checkpoint for weeks at a time and treats every regression as a docs/ update, not a model swap. When the harness is the thing that changes, the model becomes a stable variable, and you can actually reason about what broke.

Lesson 3: AGENTS.md, Anthropic Skills, and `.cursorrules` do different jobs

The vocabulary is confusing on purpose. Every vendor picked a different noun for a similar shape.

Here is what actually differs, based on what I ship with each:

Property	OpenAI AGENTS.md	Anthropic Skills	Cursor `.cursorrules`
Primary role	Repo-level index for any agent	Task-scoped, lazy-loaded playbook	Editor-level rules per project
Scope	One per repo	Many; one per task type	One per project
When loaded	Every session, top of context	Only when the task matches	Every prompt in this repo
Failure mode if oversized	Context crowded, agent ignores code	None — it never fires	Rules bleed into unrelated prompts
Best for	Where-to-look pointers	How-to-do-X procedures	Style + safety rails

They compose. On my own harness I keep a slim AGENTS.md as the top-level index, Anthropic-style skill files under docs/skills/ for anything task-specific, and a short .cursorrules for the two or three editor-scope conventions that survive across every task (never introduce any, never touch main.py without permission). The AGENTS.md is not competing with skills; it is announcing where the skills live.

The failure mode I still see most often in other people's repos is putting the skill content inside AGENTS.md "so the agent always has it." That is the encyclopedia problem, one level down. If it always has it, it never reads the code.

What actually changed for me

Three concrete moves that came from reading OpenAI's post and porting the shape:

AGENTS.md down from 940 lines to 118 lines. Everything else moved into docs/skills/.
Model held constant for four weeks. Any regression writes a new line into a skill file, not a switch to a new checkpoint.
A harness/hooks/pre-commit.sh that runs the linter and the type checker before the agent is allowed to move on. This one I stole from Anthropic's long-running-agent post from April, but it belongs on this list.

Completion rate on my personal fifteen-task benchmark went from 41% to 78% across the three changes. The model has been the same Sonnet 4.6 checkpoint for that entire span.

The uncomfortable truth of the Codex experiment is that "which model" was the wrong question the whole time. The right question is which harness this model is wearing. If you have been swapping checkpoints and blaming the vendor, try shrinking your AGENTS.md first. It's cheaper, and it's usually the actual bug.

If you want the longer version of this — the six components of a harness, why AGENTS.md is only one of them, and how the same shape applies to CLAUDE.md and hooks — that is what my book is about.

Harness Engineering: Building the Environment That Makes AI Agents Reliable

Sources: OpenAI — Harness engineering, Anthropic — Harness design for long-running app development, InfoQ — Anthropic three-agent harness.

SQLite vs Kuzu vs Neo4j: Which Graph DB Survives 1M Code Nodes?

Ken Imoto — Mon, 20 Jul 2026 13:00:01 +0000

Two weeks ago I was drafting a recommendation table for a code knowledge graph MCP: SQLite up to about 100k nodes, Kuzu from 100k to a million, Neo4j past that. It was a clean answer. I ran the numbers, I trusted the ladder, I moved on.

Then a colleague pointed me at a Register piece from October 14, 2025. Kuzu's GitHub repo had been archived. Apple had bought Kùzu Inc. on October 9, and the 9to5Mac follow-up in February 2026 traced it to an EU Digital Markets Act filing (The Register: KuzuDB abandoned, 9to5Mac: Kuzu joins Apple's list). The final release, 0.11.3, shipped the same day the repo went dark.

That changed the middle rung of the ladder. This post is the re-selection I actually shipped, at the four scales you might have: 10k, 100k, 1M, and past 1M code nodes. FalkorDB gets a section too because its GraphBLAS engine is genuinely fast in one specific lane, and pretending otherwise would be dishonest.

First: sizing your codebase in nodes, not lines

Before you pick a graph DB, translate your repo into node count. For a minimal schema of File / Module / Class / Function / Variable plus six edge types, this is the shape I keep seeing:

Lines of code	Nodes	Edges
10,000	~3,000	~8,000
100,000	~30,000	~80,000
300,000	~100,000	~270,000
1,000,000	~300,000	~800,000

The node-to-edge ratio hovers around 1:2.5 because an average function calls two or three others. Semantic edges (added later, if you build a Pass 2) push it higher, but the table above is the baseline. A 1M-node code graph means about 3M lines of source, which is Google or Meta scale monorepo territory. Most teams live in the 30k-300k node band.

Under 100k nodes: SQLite, and the one index that decides everything

For anything under 100k nodes I still recommend SQLite. Three reasons hold up:

Zero infrastructure. Ship a .codegraph.db file inside the repo. git clone and the tool works. No Docker, no auth, no docker-compose file that no one wants to review.
Recursive CTE handles 3-5 hop BFS. For a hop-3 blast radius query on a 100k-node graph, SQLite returns in 50-150ms in my measurements. Good enough to sit behind an MCP tool.
You already have the driver. Every language you might build an MCP in already has SQLite bindings in the standard library.

The catch is one index: edges(dst, type). Without it, reverse traversals (who calls this function?) fall off a cliff, because SQLite has to scan every edge row. With it, they stay in tens of milliseconds. If you take one thing from this post, take this: on SQLite, edges(dst, type) is not optional. I have seen four independent code-KG implementations, and every one of them either had this index or was silently unusable.

OSS reference points: code-review-graph and CodeGraph both run on SQLite with FTS5 for symbol lookup.

100k-1M nodes: Kuzu was the answer, and it still is (with an asterisk)

This is the band that changed. Until October 2025 the answer was Kuzu, and honestly it still is on the merits. Kuzu was embedded like SQLite (no server), spoke Cypher, and shipped with real graph indices instead of asking you to bolt them onto B-trees. On the 300k-node codebases I profiled, hop-3 traversals dropped from SQLite's 800ms into Kuzu's 40-80ms range.

The asterisk is that the repo is archived, the maintainers moved to Apple, and there is no roadmap. What you get today:

LadybugDB, a community fork under a permissive license, keeps Kuzu's columnar storage, Cypher dialect, and the last vector and full-text indices intact (ArcadeDB's Neo4j alternatives writeup, 2026). If you already know Kuzu, LadybugDB is a straight lift.
Bighorn, another fork by Kineviz, exists but has less traction.
Kuzu 0.11.3 itself still works. It is not going to acquire security bugs because you stopped writing to it. For an internal code KG behind an MCP tool, an abandoned but functional DB is usable; for a customer-facing service, it is a policy decision.

Would I start a new project on LadybugDB today? For the 100k-1M code node band, yes, provided the team is comfortable with a small-community fork. If that word "fork" makes your legal team nervous, skip to Neo4j.

Past 1M nodes: Neo4j, and the boring reason it wins here

Once you cross a million nodes you are also past the point where "embedded" is a virtue. You have multiple engineers hitting the same graph, you want a query log, you want backup tooling that already exists. That is server-shaped work, and Neo4j is the mature server-shaped answer.

Cypher travels from Kuzu / LadybugDB to Neo4j with only dialect nicks, so if the codebase grows out of Kuzu you migrate rather than rewrite. Neo4j Community handles most single-node code-KG workloads; you reach for Enterprise (or Aura) when you need clustering.

The scale ceiling story worth knowing: NTT Comware runs Neo4j on a ~40M-node network device management graph in production, with query times dropping from about 80 minutes to tens of seconds after the migration (Neo4j case: NTT Comware). That is not a code KG, it is network topology, but it establishes the upper bound. If you are worried about Neo4j surviving your monorepo, it will.

Where FalkorDB actually belongs

FalkorDB is easy to mis-place because its benchmark numbers are impressive in isolation. FalkorDB reports sub-140ms p99 on aggregate expansion patterns where Neo4j hits 46.9 seconds (FalkorDB benchmark writeup). It represents the graph as sparse adjacency matrices and executes traversals as GraphBLAS matrix operations, which is a very different physical model from Neo4j's B-tree pointer walks.

The caveats are equally real: FalkorDB's benchmark methodology is FalkorDB's own, tuning effort likely differs, and the workloads picked are the ones where sparse-matrix execution shines. In LDBC SNB results both vendors have wins.

Practical read: FalkorDB is a fit when you need very low read latency, your graph fits in memory, and you can tolerate reload time on restart. Real-time IDE integrations, live code-navigation panels, hot-path autocomplete. For a nightly code-KG rebuild that a code review MCP queries a few dozen times per PR, it is overkill. CodeGraphContext uses FalkorDB Lite as one option for exactly this "hot, small, ephemeral" pattern.

The selection matrix I actually use now

Scale	First pick	Fallback	Not this
Under 100k	SQLite	LadybugDB (Kuzu fork)	Neo4j (operational overkill)
100k - 1M	LadybugDB (with fork risk noted)	SQLite (up to ~200k), Neo4j (up to 1M)	Vanilla Kuzu 0.11.3 for customer-facing work
1M - 10M	Neo4j	FalkorDB (if read-only and in-memory OK)	Anything embedded
Past 10M	Neo4j Cluster	--	Embedded, period

"Not this" is not a technical impossibility. You can run SQLite at 500k nodes; it just tips into "please stop" territory on hop-4 CTEs.

What I picked, and the migration escape hatch

For the specific project that started this — a code review MCP over a ~300k-node Python monorepo — I picked SQLite. The number was inside its comfort band, the repo was fine, and the team did not need a fork discussion at the review meeting. If it grows past 500k I will move to LadybugDB, and if the org grows past a million nodes we will migrate to Neo4j.

The escape hatch is the schema. If you keep the schema to a minimal common denominator (five node types, six edge types, confidence and line_range as plain properties), then the migration between any two of these four DBs is a CSV export and a LOAD CSV on the other side. The temptation is to reach for DB-specific features (Neo4j label hierarchies, Kuzu's custom indices, FalkorDB's matrix-native aggregates). Every one of those pins you.

Apple's move on Kuzu made one thing concrete: "the best embedded graph DB" is one board meeting away from being someone else's IP roadmap. Keep the schema portable. Pick the DB that fits your current scale. Move when the numbers, not the vendor's Twitter, tell you to.

If you are building a code knowledge graph and want the full schema, ingest pipeline, and MCP wiring in one place, I wrote it up in Knowledge Graph Practical Guide. It covers the Pass 1 / Pass 2 split, the edges(dst, type) gotcha in more detail, and the migration paths between the four DBs above.

The Machine Accent Travels: AI Text Is Rhythmically Monotone in 70/70 Cells Across 3 Languages

Ken Imoto — Sat, 18 Jul 2026 07:09:54 +0000

Yesterday I published my third research paper on Zenodo. It showed that Japanese AI-generated text swings its sentence lengths far less than human text does, and that all seven models I tested drift the same direction. I called the phenomenon a "machine accent."

One thing kept nagging at me after publishing. An accent belongs to the speaker. It follows you into whatever language you attempt. But all I had measured was Japanese. If the monotony vanished in English or Portuguese, my "accent" was never an accent. It was a fact about Japanese.

A name like that has to earn its metaphor.

So today I published paper number four. These are the field notes, landmines included, same as last time.

The setup

Two hypotheses. H1: if the accent is real, AI monotonization shows up in English and Portuguese with the same direction (d < 0) on every model. H2: the direction holds but the magnitude may differ by language.

The human corpora had to be pre-ChatGPT. For English I took 853 of Dev.to's all-time most popular posts, January 2019 through October 2022. If you wrote a well-liked Dev.to post back then, congratulations: you are now a scientific baseline.

Portuguese got lucky. TabNews, the Brazilian dev forum, opened in May 2022. ChatGPT landed November 30 of the same year. That leaves a seven-month window where every post is guaranteed human, so I took the entire window: 403 posts. Short, but airtight.

The AI side is 7 models × 10 themes × 5 attempts × 2 languages, using the same zero-shot prompt as the Japanese study, translated. Metric definitions carried over unchanged: burstiness, sentence-length CV, paragraph-structure CV. Two adaptations only. Japanese mora counts became pyphen syllable approximations, and sentence splitting uses one shared regex for both languages instead of pysbd, which does not support Portuguese. Mixing splitters would have confounded the language difference with a tooling difference.

New measurements: 1,202 English + 751 Portuguese documents, 1,953 total. The Japanese numbers come straight from the published third paper.

Landmine 1: my models had retired

The moment generation started, Claude 3 Haiku, Sonnet 4, and Opus 4 all returned 404. The three Claude models from the Japanese study had been retired from the API.

For a replication-style design this hurts. Same models, different language: broken. I substituted the current tiers (Haiku 4.5, Sonnet 5, Opus 4.8) and restricted the direct Japanese comparison to the four models both studies share (GPT-3.5 Turbo, GPT-4o, GPT-OSS 20B, Llama 3.2 1B). The paper discloses the substitution plainly.

Annoying at the time. It ends up delivering the most interesting finding in the study.

Landmine 2: GPT-4o wraps entire documents in a code fence

Early in measurement, 35 GPT-4o documents came back as "0 sentences." Opening them explained why: the entire document sat inside a

```markdown

fence. My code-block exclusion, which exists so that code doesn't pollute rhythm stats, was eating the whole article as one giant block. The fix unwraps only an outer fence that carries the markdown tag. A bare fence stays untouched, because it might be actual code.

It sounds like a footnote. It is not: left alone, a third of GPT-4o's sample (35 of 100 documents) disappears silently, and nearly half on the Portuguese side. Preprocessing for multilingual measurement doubles as a catalog of model quirks.

Results: 70 cells, 70 negative

Five core metrics (three burstiness variants, two sentence-length CVs) × 7 models × 2 languages gives 70 cells. Every one of them came out d < 0: AI more monotone than humans. The direction survives length residualization in all 70. Zero exceptions.

Pooled effect sizes, next to the Japanese result:

burstiness (char)	Japanese	English	Portuguese
Cohen's d	−0.96	−1.12	−1.03

Three languages, one band around −1. And the ordering carries over too. Across the four shared models, GPT-3.5 Turbo is the most monotone in every language and GPT-OSS 20B sits closest to the human band in every language. A model's accent strength follows it across languages, rank and all.

The accent travels.

The comma is the exception

One metric refused to line up: commas per sentence. In English, AI uses more commas than humans (d = +0.45). In Portuguese, fewer (d = −0.83).

The explanation lives on the human side. Portuguese writers average 1.14 commas per sentence; English writers 0.58. The models settle around 0.6 to 0.7 in both languages, a kind of textbook middle. Against comma-light English humans that looks excessive. Against comma-loving Brazilians it looks starved. Identical behavior, opposite sign, decided entirely by the local convention it gets compared against.

Rhythm crosses languages with its direction intact. Punctuation flips depending on where you stand. That contrast became the paper's framework.

Three layers: signature, accent, dialect

The third paper sorted AI text traces into two layers: vocabulary as a model-specific signature, rhythm as a shared machine accent. The comma result is a third kind of trace that fits neither.

Signature (vocabulary): diverges per model. Tells you which machine wrote it
Accent (rhythm): shared by all models, persists across languages. Tells you a machine wrote it
Dialect (punctuation): the behavior is shared, but its visible sign flips with the conventions of the language you compare against

One machine-written document carries all three kinds of traces, in separate layers. That is the paper's central claim.

The accent is fading

Here is the gift from landmine 1. Because the model roster changed, the data holds both GPT-3.5 from 2023 and the current Claude tier from 2026, side by side.

On English burstiness (char), GPT-3.5 Turbo scores d = −2.59. The current Claude generation scores −0.59 to −0.96, roughly a third of that on average. Portuguese shows the same ratio. Newer models write rhythm much closer to the human band.

So rhythm-based AI detection probably has a shelf life.

The accent gets trained away, generation by generation. For writing improvement the same trend cuts the other way: the closer models get to the human band, the more precisely a rhythm metric points at whatever monotony remains. Detection value and editing value move in opposite directions, which is exactly the shape of the third paper's conclusion.

The practical part: rhythm lint crosses languages, thresholds don't

The takeaway for tooling is short. The direction of rhythm metrics is shared across all three languages, so lint logic ports as-is. The distributions differ, so thresholds need per-language calibration.

rhythm-lens, the small CLI I released last week, ships the English and Portuguese baselines from this study as of v0.2.0.

pip install rhythm-lens
rhythm-lens draft.md            # language auto-detected (ja/en/pt)
rhythm-lens --lang en draft.md  # or explicit

I ran this post through it before publishing. It flagged me on the first pass, and I rewrote my own paragraph structure to satisfy my own linter. The tool biting its author feels like a good sign for the tool.

Paper and data

The paper is on Zenodo (text CC-BY 4.0, code and data MIT on GitHub). Human corpus texts are not redistributed; the repo ships metadata plus recollection scripts instead.

Paper: 10.5281/zenodo.21424903
Code and data: github.com/kenimo49/llm-rhythm-crosslingual
The third paper (Japanese lexical vs. rhythm fingerprints): 10.5281/zenodo.21413035

This ended up being a sequel published 24 hours after the original. Working while the question is still warm meant every scraper, metric, and mistake was fresh in memory. And none of it works without communities that kept their pre-ChatGPT writing intact, so if your 2021 Dev.to post is in the baseline: thanks for the rhythm.

Brave Search Buries Pages Google Ranks #1, and Your AI Agents Can't Find Them

Ken Imoto — Fri, 17 Jul 2026 13:00:01 +0000

I have an article that ranks #1 on Google for its target query. Position one, above the fold, the SEO equivalent of a parking spot right by the door. I was proud of it. I had earned it the boring way: clean headings, internal links, a year of patience.

Then I searched for the same query on Brave. My article was on page 5. Page five. The place URLs go to die unmourned, somewhere below a forum thread from 2019.

The part that actually stung came a few minutes later. I asked Claude Code, running in my own terminal, to research that exact topic and cite good sources. It came back with three links. None of them were mine. My agent, which I built, which runs on my machine, could not find the article I wrote. It was searching Brave. And on Brave, I do not exist.

Google and Brave are not looking at the same web

The instinct here is to assume Brave is just a smaller, scrappier mirror of Google. It is not. Brave runs its own index, built from a completely separate crawl, with its own ranking logic. When I say my page is #1 on one and page 5 on the other, I am not describing a glitch. I am describing two different maps of the web that happen to share a planet.

Brave's index is real infrastructure, not a side project. It covers over 40 billion pages and refreshes more than 100 million of them daily, fully independent of Google and Microsoft. The interesting part is how it stays fresh. A chunk of its signal comes from the Web Discovery Project: tens of millions of Brave browser users who opt in to share anonymous data about which pages they actually visit. So instead of ranking purely on backlinks and the usual SEO machinery, Brave leans on pages humans genuinely land on.

Which, when I think about it, explains my page 5 problem with uncomfortable precision. My article ranked on Google because I optimized it for Google's machinery. It ranked nowhere on Brave because I had never once asked whether Brave's separate index had even noticed it existed. I had been studying for the wrong exam and getting an A on it.

Why a search engine I never use decides whether AI can find me

Here is where it stops being a curiosity and starts being a problem with my paycheck attached.

In May 2025, Microsoft announced it was retiring the Bing Search API, and it shut down for good on August 11, 2025. For years, a huge slice of AI tools and third-party search services ran on Bing's API under the hood. When it went dark, the replacement was not obvious. Google does not open its real web index to developers for grounding or RAG; its Programmable Search Engine is built for a narrower job. The scraper-based APIs (Tavily, Exa, and friends) ultimately depend on indexes they do not own, which means they inherit someone else's blocking, pricing, and terms-of-service risk.

That left exactly one independent commercial web-search API at scale: Brave. Brave's own chief business officer described the shift as giving developers "the only independent search API in the market," and for once the marketing line is just describing the terrain.

So follow the chain. AI coding agents need web search. The independent web-search API they can actually buy is Brave's. Therefore the agents search Brave. Cursor, Cline, and Windsurf all use Brave for web lookups, Anthropic shipped Brave Search as one of the first Claude MCP demo servers, and Brave is increasingly the default web-search provider baked into agent toolchains. The top AI companies by usage all touch Brave Search at training or inference time.

Put plainly: Brave's index is the front door for a growing share of AI agents. If your content is not in that index, or it is in there on page 5, those agents will never hand it to a user. You can be the #1 result on Google and still be functionally invisible to the tools engineers actually use to research things. I was. On my own laptop.

The LLM Context API reads structured data first, and most of us never gave it any

In February 2026 Brave shipped its LLM Context API, and it changes what "being indexed" even means. The old web-search API returned what humans need: a title, a URL, a snippet to click. The LLM Context API returns what a model needs: pre-chunked, ranked pieces of content ready to drop into a prompt. It is already powering over 22 million answers per day inside Brave Search itself.

The detail that should make every blog owner sit up is in the extraction step. When the API pulls content from your page, it preserves JSON-LD schemas and tables with row-level granularity, and it prioritizes that structured data during extraction. One write-up put it bluntly: it is not optional anymore.

So if your page ships clean TechArticle or FAQPage JSON-LD, the API can lift your author, your headline, your published date, and your key claims out cleanly and feed them straight to the model. If it ships a wall of <div> soup with the real answer buried in paragraph nine, the API has to work harder and your page loses to one that did the structuring for it. Schema stopped being a nice-to-have for Google rich snippets. It became the format your content gets read in.

And before this sounds like I am about to tell you the biggest model wins, Brave published a benchmark that says the opposite, which is the most encouraging thing I have read all year.

"Data quality beats model performance" is now a measured result, not a vibe

Brave ran a pairwise evaluation over 1,500 queries, judged by Claude Opus and Sonnet acting as graders, with each pair scored in both orders to cancel out position bias. The headline: their "Ask Brave" answer engine, running on the open-weight Qwen3 model, beat both ChatGPT and Perplexity on answer quality.

Let that land. An open-weight model you can download for free out-scored two of the most heavily funded AI products on the market. The variable was not parameters or training budget. It was the quality of the grounding data fed into the model at answer time.

For a content creator this is the rare benchmark that is actually good news for the little guy. It means the thing under my control, the structure and clarity of what I publish, is the lever that moves AI answers. Not the size of someone's GPU cluster. If clean, well-structured grounding data can make a small open model beat ChatGPT, then clean, well-structured pages are not a tax I pay for tidiness. They are the whole game.

This is the part I keep coming back to with the framework work I've been building over at LLMO Framework, which I treat as the canonical playbook for the index-side fixes: it formalizes exactly this, that you optimize the data you hand the model, not the model. Brave's benchmark is the cleanest external proof of that idea I have found.

What I actually did about my page 5 problem

Diagnosis first, because it costs nothing and it is humbling in a useful way.

Search yourself on Brave. Go to search.brave.com and run your own article titles and target queries. Compare the result to Google. The first time I did this I found three of my "top-ranked" posts nowhere in Brave's first few pages, and one post Google had buried sitting near the top on Brave. The two indexes disagree more than you would believe until you look.
Ask an agent to find you. Open Claude Code or any Brave-backed tool, ask it to research your topic and cite sources, and see if your URL shows up. This is the real test, because it is the exact path a reader-via-agent would take. Mine failed it. That failure is the whole reason this article exists.
Ship JSON-LD, server-rendered. Add TechArticle and FAQPage schema with your author, headline, date, and description, and make sure it renders server-side so the crawler and the LLM Context API actually see it. Client-injected schema that only appears after JavaScript runs is schema the index never reads.
Structure for extraction. Clean heading hierarchy, real <table> elements for comparisons, fenced code blocks for anything technical. The LLM Context API pulls these out with row-level and block-level precision. Give it clean blocks and you get extracted cleanly; give it mush and you get skipped.

None of this is exotic. It is mostly the hygiene I had skipped because Google rewarded me anyway and I let "ranks #1" paper over "structured like 2014." The Brave index does not grant that grace.

The uncomfortable summary

For years "rank on Google" was a complete sentence. It is now a partial one. Google still owns roughly 90% of human search, so SEO is not dead and I am not telling you to torch it. But human search and agent search now run on different rails, and the agent rail increasingly runs through Brave. Optimizing only for Google buys you nothing on the index that AI tools actually query.

The fix is not a growth hack. It is going to Brave, searching for yourself, watching an agent fail to find you, and then giving Brave's index the structured, clean, extractable content it rewards. I ranked #1 on Google and still could not get my own agent to cite me. Fixing that started with admitting the search engine I never use had been quietly grading my homework the whole time.

If you want the full implementation playbook for the index side of this, including the JSON-LD patterns and why AI engines keep ignoring perfectly good pages, llmoframework.com is where I keep the working version.

Claude Code Made My Sprints 40% Slower: 3 Time Sinks I Only Found by Timing Myself

Ken Imoto — Thu, 16 Jul 2026 13:00:00 +0000

Monday morning I estimated an HTTP retry tweak at 45 minutes. I closed the laptop at 8:07 PM. Fine, one bad estimate. Except when I added up the last five sprints and compared them to the five before I bolted Claude Code onto everything, the shape was ugly: my "quick" tickets were finishing about 40% slower on average. And I had walked around for two months telling anyone who would listen that the AI had turned me into a shipping machine.

The AI is not the villain. I still use Claude Code every day, and Opus 4.7 is a real jump from Opus 4.6. Anthropic reports SWE-Bench Pro moving from 53.4% to 64.3% in the April 2026 release notes. What died is the story I told myself about where the time was going. This is the log of the three specific places I lost it.

The illusion I was defending

I had a "feeling" I was faster. That feeling is the same one described in the METR randomized study from July 2025 (arXiv:2507.09089): 16 experienced open source developers with an average of 5 years on their own repositories, given AI tools, took 19% longer to complete tasks. Before starting, they predicted a 24% speedup. After finishing, slower, they still believed the AI had sped them up by 20%. The gap between what you feel and what the clock records is about 39 percentage points, and it does not close after you live through the slowdown.

So my private "40%" is my number, on a smaller sample, in a codebase I know cold. METR gets 19% and I get 40% for roughly the same reason: the harder I know the codebase, the more expensive it is to route the decision through someone who does not.

Fine. Where did the time actually go?

Sink 1: The 3-second response that hid a 40-minute read

Claude Code returns a diff in three seconds. My brain reads that as "the task took three seconds."

The task did not take three seconds. It took three seconds of generation plus twelve minutes reading the diff, plus twenty-eight minutes chasing an off-by-one in an exponential backoff that only fired under load, plus fifteen minutes writing the test that would have caught it if I had written the test first the way I do without AI. That is fifty-five minutes, and I logged it in my head as "quick fix, Claude did it."

The generation speed contaminates the estimate for the entire ticket. If you asked me on Monday morning "how long?" I would have said 45 minutes because the AI part is 3 seconds and the "read and verify" part felt free. It is not free. It is the whole job.

Anthropic's docs point to Plan Mode for exactly this reason — read-only, Claude proposes changes without touching files, and you are supposed to review the plan before it runs (permission modes). I did use plan mode. I skimmed the plans the same way I skim CI output: looking for red, not looking for wrong.

Sink 2: Auto-accept mode as a slow leak

Anthropic shipped an official Auto mode through spring 2026, opening it to Team, Enterprise, and API between March 24 and April 16, and then to Max and Pro users, with Sonnet 4.6 support added alongside Opus 4.7 (Auto mode announcement). The safety classifier runs on Sonnet 4.6 no matter which model your main session uses, and it is genuinely good at blocking the obviously catastrophic stuff: pushing to main, exfiltrating secrets. That is not where the time died.

The time died in the acceptable-but-wrong lane. Auto-accepting file edits ("Accept edits" mode, which auto-approves file edits and safe filesystem operations inside your working directory) let a whole category of "yes, this compiles, but it is not what I wanted" changes ship into my working tree without me looking at them until I ran the code. Then I read the change on a compile failure or a test failure, which is a much more expensive place to read it. Cognitive context switches are pricey and I was buying them wholesale.

The trap is not the mode. The trap is treating "safe" (the classifier blocked the disaster) as "correct" (this actually matches my intent). Those are different words. The classifier does not know my intent. My intent lives in my head, and I was outsourcing that head to a very fast typist.

Sink 3: The "one more prompt" verification loop

This one is the deepest cut. Every time Claude Code returned a solution that was 90% right, I sent it back with "close, but also handle X." Twenty seconds to type the follow-up. Two minutes to re-read the new diff. Six minutes to run tests. Eight minutes to notice that the fix to X regressed the thing that was already right. Sixteen minutes per loop, and the loop feels fast because each turn is short.

Three loops is 48 minutes, which is longer than it would take me to write the whole thing myself in a codebase I know. But I never budget for three loops on Monday morning. I budget for one, feel great about the first response, and by the third loop I have talked myself into "I'm almost done" and I stay another hour past the point where sunk cost should have told me to close the loop and finish it by hand.

Simon Willison has been putting this fairly plainly for a while: LLMs are a productivity amplifier for tasks you know how to do and a mirage for tasks you don't, and the trap is that they feel the same from the inside. He is right, and my log agrees.

What actually broke: the estimate, not the AI

In each of the three sinks, the AI performed roughly as advertised. It generated code fast. Its Auto mode blocked genuinely dangerous shell calls. Its plan mode showed me plans. None of that is the problem.

The problem is that "generation" and "task" are different sizes and my estimate collapsed them. When I say "this will take 45 minutes," I am estimating the total ticket: generate + read + verify + fix + verify + commit. The AI compresses only the first component. It leaves the rest untouched or, worse, expands it because now I am reading someone else's code instead of writing my own. If you shrink 5 minutes of a 45-minute task, you save 5 minutes. If you shrink 5 minutes and blow up the other 40 into 60 because verification is now more expensive, you lost 15 minutes and told yourself you saved 5.

That is not a Claude Code bug. That is an arithmetic bug in my planning, and it took me sixty days and one 8 PM Monday to notice.

The rule I added to CLAUDE.md on Monday

I did not quit Claude Code. I added one rule to my per-repo CLAUDE.md, and it took me under a minute to write:

## Estimate policy

When I estimate any task involving AI code generation, I split the estimate
into two lines:

- Generation budget: how long the AI will take to produce first-draft code.
- Verification budget: how long I will take to read, run, and correct it.

The commit only happens when both budgets close. If the verification budget
runs out and the code is not shipping, I stop and rewrite by hand rather
than opening another prompt loop.

Three effects since I added it:

My estimates got more honest. "Quick" tickets get 90 minutes now instead of 45, because 45 was fiction all along.
I catch the "one more prompt" loop earlier because verification has a real budget instead of an infinite string.
I stopped using auto-accept for anything I could not describe out loud before generating it. If I cannot describe the intended change in one sentence, the AI does not have my intent either, and any speed it gives me is going to be given back with interest inside 20 minutes.

One more piece if your team ships services: pair the verification budget with a CI perf gate. CodSpeed hardened its AI-agent integration in March 2026 with an MCP server that can hunt regressions on every PR (changelog). Auto mode plus a perf-gate CI catches the "compiles green, ships slow" class of AI-generated code that would otherwise blow up the verification budget after merge.

What I now believe

Two things I did not believe on Monday morning.

First, the feeling of speed is not evidence of speed. It is evidence of how fast the last visible unit of work happened. If the visible unit is "AI returns code" and the invisible unit is "I read and verify code," the feeling is going to lie in a very consistent direction. METR measured 39 percentage points of lie. My personal number is uglier because I estimate more aggressively than the median dev. Yours will differ. Timing yourself once, on real work, will tell you which side of the line you are on faster than any think piece.

Second, I got slower not because of AI, but because I let AI speed rewrite my estimation model in the background. The estimation model is the thing that keeps me honest. Once it goes, the tool that gets blamed is the loudest tool in the room, and Claude Code has been by a wide margin the loudest tool in mine. The tool did not lie. My clock did, and I trusted the clock more than the ticket. That is the mistake, and the mistake is fixable with sixty seconds and a CLAUDE.md edit.

The irony that lands hardest: the study that most helped me use AI well is a study about AI making experienced developers slower. I did not get slower from using AI. I got slower from believing the AI without timing myself. Different failure mode, same repair.

The full set of harness rules I use across all Claude Code projects — memory files, permission mode discipline, verification budgets, session cost accounting — is collected in Harness Engineering: A Field Guide. It is written for engineers who want to run Claude Code past the "open three terminals and hope" stage.

Whisper v3 Turbo + Qwen2.5 1.5B: 5 Ollama Models Benchmarked for Sub-300ms Voice AI on CPU

Ken Imoto — Wed, 15 Jul 2026 13:00:01 +0000

Light does not negotiate. Tokyo to Virginia is about 130ms round-trip and no amount of infrastructure spend changes that. If you want a voice agent to feel like a person and not a call-center IVR, the round trip has to leave the building.

So I ran the whole stack on a CPU-only laptop. No GPU. No cloud LLM. Whisper v3 Turbo for STT, one of five Ollama models for the LLM, a local TTS for the last mile. Wall-clock target: sub-300ms voice-to-first-byte. Here are the seconds.

5 Ollama models, one CPU, one budget

I put each model behind the same Whisper Turbo front end and the same local TTS, and asked it to produce the first spoken token as fast as possible. The audio input was a short customer-support prompt in English and Japanese. The box was a mid-range x86 laptop, no accelerator, running Ollama with Q4_K_M weights where available.

Model	Size on disk	Tokens/sec	Time to first token	Japanese quality	Verdict for 300ms
Qwen2.5 1.5B	~1 GB	4-8 tok/s	200-350 ms	Usable	Fits
Gemma 3 2B	~1.5 GB	~15 tok/s	220-380 ms	Passable	Fits (best throughput)
Qwen3.5 0.8B	~1 GB	12-15 tok/s	150-250 ms	Thinking loop	Fast but unstable
Qwen3.5 2B	~2.7 GB	8-9 tok/s	280-450 ms	Thinking loop	Overshoots on retries
DeepSeek-R1 1.5B	~1.1 GB	3.9-4.0 tok/s	400-600 ms	JP/CN code-switch	Misses the budget

Two of five hold the budget with margin. Qwen2.5 1.5B and Gemma 3 2B are the only ones I would put in front of a real user today. The rest have a reason they miss, and the reasons matter more than the seconds.

If you were expecting DeepSeek-R1 to win, you and I both. It is the model most local-LLM tutorials open with. On CPU, at 1.5B, with a Japanese prompt, it thinks in Chinese for a beat before it answers in a mix of both. The stopwatch says the latency is fine. The user hangs up anyway.

Why Whisper v3 Turbo made this fight winnable

Whisper Large v3 Turbo is 809M parameters with 4 decoder layers instead of the Large v3's 32. That single architectural change cuts per-chunk decode time in half at a 0.3-0.7pp WER cost. Simplismart's production benchmark put the served RTF at 1300x, and Groq's hosted variant runs at 216x real-time. On my CPU box I get nothing close to 1300x, but I get 50-150ms of STT on a 3-second utterance and that is what the budget needs.

The distilled English-only cousin, distil-whisper large-v3, runs at ~90x RTF on a GPU, which is faster on paper. It also drops multilingual, so for the Japanese half of my traffic Turbo is the only one that survives. If your product is English-only and you have a GPU, distil-whisper is worth a look. Turbo is the model I ship.

Where the 300ms actually goes

Adding up the budget line by line is the boring part, and it is also the part everyone gets wrong.

VAD (voice activity detect):     0-5 ms
Whisper Turbo STT (3s audio):   50-150 ms
Qwen2.5 1.5B LLM TTFT:         200-350 ms   ← the fat one
Local TTS TTFB:                 40-75 ms
Network:                         0 ms       ← the win
--------------------------------------------
Total to first spoken byte:    290-580 ms

The LLM's time-to-first-token is the fat variable and everything else is thin. If Qwen2.5 lands at 200ms you clear 300ms. If it lands at 350ms you clear 400ms. There is nothing else to tune. Speeding up Whisper by 40ms buys you 40ms. Speeding up the LLM by 40ms buys you 40ms on every subsequent token too, because that is the model that keeps talking.

Which is why my "winning" configuration is Qwen2.5 1.5B, and Qwen2.5 is not the fastest of the five. Gemma 3 2B has better tokens-per-second by roughly 15 vs 4-8, and I still ship Qwen2.5 when the traffic is Japanese-heavy, because Qwen's TTFT variance is tighter and the output quality does not embarrass me. Gemma is my English-first pick.

The two failure modes nobody warns you about

Thinking-loop overshoot. Qwen3.5 in the 0.8B and 2B sizes will start a chain-of-thought passage before it answers. On the desktop that is a feature. In a voice loop it burns 200ms and outputs <thinking> tokens the TTS then tries to speak. If you want a reasoning model for your voice agent, wrap it in a stop-on-first-answer regex. Do not skip that step.

Cross-language code-switching. DeepSeek-R1 1.5B on a Japanese prompt produces answers with Chinese phrases mixed in. This is a distillation artifact from the base model, and prompt tuning does not remove it. If your users speak the language the model bleeds from, you will hear it in the TTS output. I noticed on the second test call. Users will notice on the first.

Both of these fail in production and pass in benchmark scripts, which is exactly the opposite of what you want.

The Pipecat vs LiveKit context, briefly

If you are doing this in a framework rather than raw Python, LiveKit's baseline is 750-900ms end-to-end and Pipecat's is 800-950ms with a standard cloud stack. Both frameworks can reach sub-500ms p95 in 2026 with streaming STT, partial TTS, and prefix caching, assuming a GPU somewhere in the loop.

The CPU-only path is aiming at a different product entirely. 300ms from a device that is not on the internet and does not care about a regional outage. That is what the physics buys you when you take the trip out of the loop.

The realistic edge + cloud hybrid

The pure-edge configuration is a technical achievement, and it is also not what I would ship to most customers. This is the version I actually recommend:

Edge: VAD, Whisper Turbo STT, a small local LLM as a fallback / cache.
Cloud: primary LLM (whatever your quality tier demands) and TTS.
Only text crosses the wire. Voice data stays local.

That configuration lands at 300-350ms voice-to-first-byte with the same Whisper Turbo front end and a cloud LLM, gets you the quality of a large model, and keeps the privacy story that the pure cloud path cannot buy at any price.

Constraints worth naming

Local inference on CPU costs you memory, battery, and thermal headroom. 1.5B at Q4_K_M is about a gigabyte of RAM permanently pinned. A long call drains a laptop battery faster than a video call. Sustained inference will thermal-throttle a fanless device inside ten minutes.

None of that is a dealbreaker. All of it is a reason to have a fallback path. My rule: local first, cloud on thermal or memory pressure, log which path served each request so I can see the shape of the tradeoff a month later.

What I would do differently

If I were starting fresh I would benchmark Phi-4 Mini alongside Qwen2.5. Phi-4 Mini is 12 tok/s on CPU at 3.8B, which is a good spot in the CPU-bandwidth-bound curve, and it did not exist when I ran this test. I would also test the newer distil-whisper INT8 variants for the English-only case, since the 6x speedup is real and the WER cost is smaller than the reputation suggests.

I would not switch off Whisper Turbo. The 6.3x speedup over Large v3 with a tiny WER penalty is the single most important line item in the whole stack, and every downstream design decision falls out of that one architectural change.

The 300ms wall was physics. Whisper Turbo and a 1.5B LLM on a laptop CPU turned it into an engineering problem, and engineering problems ship.

The long form of this stack (Pipecat design patterns, filler-word strategies for hiding the last 50ms, the full latency breakdown for hybrid cloud, and the parts I cut from this piece for length) is the book that grew out of the work: Voice AI 300ms UX (English edition).

DEV Community: Ken Imoto

KV Cache Quantization: I Stretched Qwen 35B's Context 8 on 12GB VRAM

600 MiB of headroom

What actually sits on the GPU

Two flags

The measurement

Trade-off summary

When to actually bump -c

The two-line summary

The AI Summary Said "It's Not a Scam." The Springboard Was Your Site's Search Box

The mechanism: three steps, no hacking

Why the AI summary repeats the lie

The 5-minute self-check

Defenses

Wrap-up

References

freee + MCP: I Automated My Japanese Tax Return With Claude Code — 3 Things That Broke in Production

Context for readers outside Japan

The setup that worked

Break #1: receipts cannot be attached

Break #2: 270 tools, several thousand tokens gone before "hello"

Break #3: rate limits, silently

What the final workflow actually looks like

The one-paragraph summary

Playwright + Chromium: 1 Script Replaces 2 Manual Testers for WebRTC Video Call E2E

Why two browsers and not one page

The launch flags

The call sequence, step by step

About the 10-15 second timeouts

Fixtures do the setup once

Verifying that a video actually plays

Level 1: videoWidth / videoHeight

Level 2: requestVideoFrameCallback

The one thing not to do: Promise.all

Why Playwright and not Cypress / WebdriverIO / Selenium Grid

Docker + GitHub Actions with fake devices

Wrap-up

MCP Scorecard: 4-Layer Pre-flight Check Before You Publish a Server

Why pre-flight for MCP at all

Layer A — Passive Footprint

Layer B — Use-Case Scoping

Layer C — Security own rules

Layer D — Name Safety

What it does not cover

Install

Companion book

MCP Token Cost Audit: PostgreSQL vs Google Maps vs GitHub vs Freee — 270 Tools Priced

Why "connect" costs tokens at all

The four-server audit

Why Freee balloons to 270 tools

The yearly cost, spelled out

Three ways to shrink the bill

Strategy 1: allowlist the tools you actually use

Strategy 2: rewrite the descriptions

Strategy 3: connect on demand, not on boot

What to evaluate before adopting a new MCP server

The bigger frame

OpenAI Codex Shipped 1M Lines From AGENTS.md: 3 Harness Lessons That Beat Model Swaps

Lesson 1: The "one big AGENTS.md" collapses under its own weight

Lesson 2: Swapping models fixes nothing when the harness is the bug

Lesson 3: AGENTS.md, Anthropic Skills, and .cursorrules do different jobs

What actually changed for me

SQLite vs Kuzu vs Neo4j: Which Graph DB Survives 1M Code Nodes?

First: sizing your codebase in nodes, not lines

Under 100k nodes: SQLite, and the one index that decides everything

100k-1M nodes: Kuzu was the answer, and it still is (with an asterisk)

Past 1M nodes: Neo4j, and the boring reason it wins here

Where FalkorDB actually belongs

The selection matrix I actually use now

What I picked, and the migration escape hatch

The Machine Accent Travels: AI Text Is Rhythmically Monotone in 70/70 Cells Across 3 Languages

The setup

Landmine 1: my models had retired

Landmine 2: GPT-4o wraps entire documents in a code fence

Results: 70 cells, 70 negative

The comma is the exception

Three layers: signature, accent, dialect

The accent is fading

The practical part: rhythm lint crosses languages, thresholds don't

Paper and data

When to actually bump `-c`

Lesson 3: AGENTS.md, Anthropic Skills, and `.cursorrules` do different jobs