DEV Community: Ken Imoto

4 Vector DBs Compared: Pinecone, Weaviate, Qdrant, pgvector on Enterprise RAG

Ken Imoto — Mon, 09 Nov 2026 13:00:00 +0000

The enterprise RAG vendor selection meeting always starts in the wrong place.

Someone pulls up a benchmark. The latency numbers go on the whiteboard. An hour later, the decision is made on p99 at 1M vectors, and nobody has asked what happens at the eighteen-month mark, when the index is 30M vectors, three teams depend on it, and the vendor just raised their per-query price.

Four names come up in almost every one of those meetings: Pinecone, Weaviate, Qdrant, and pgvector. They are not an exhaustive list. They are the shortlist that falls out of "we need production scale, we need metadata filtering, and the person sitting here has to be able to operate it on Monday." This post compares them on the four axes that actually move that Monday-morning decision, based on each vendor's public documentation.

The 10,000-foot view

Before the axes, a single table to orient:

Axis	Pinecone	Weaviate	Qdrant	pgvector
Category	Managed-only cloud	OSS (self-host or cloud)	OSS (self-host or cloud)	PostgreSQL extension
Primary language	Opaque (managed)	Go	Rust	C (PG extension)
Index algorithm	Proprietary (no tuning)	HNSW	HNSW (multiple variants)	HNSW + IVFFlat
Pricing	Per-pod or per-read/write	Infra + optional cloud	Infra + optional cloud	PostgreSQL infra only
Default scale ceiling	Billions (managed)	~10B (self-host)	Billions (self-host)	~5-10M before memory pressure

Four choices, four different bets about where the operational pain will show up. Let's walk the axes.

Axis 1: Scaling model

Pinecone is managed-only. You cannot self-host it. In exchange, the capacity planning problem turns into a dropdown: pick a pod tier, pay the bill. The ceiling is high (Pinecone's public docs describe deployments into the billions of vectors), but every scale knob is behind their control plane. When you hit a performance issue at 2 AM, your only lever is to open a support ticket.

Weaviate publishes both a self-hosted binary and a managed cloud. Self-hosted scales horizontally by sharding across nodes, and the public docs describe deployments up to roughly 10 billion vectors. The cost: you own the Kubernetes. The payoff: you can see the memory profile, add nodes, debug the thing.

Qdrant is OSS first, and the Qdrant Cloud managed tier is a second option. Self-hosting is a single Rust binary that runs on a few hundred MB of RAM for small indexes and scales up by adding shards. The operational surface is smaller than Weaviate's, which is partly why Qdrant has been quietly eating mindshare in the self-host crowd.

pgvector is not a separate system. It is a PostgreSQL extension, which means your scaling model is your existing PostgreSQL scaling model. If you already run managed Postgres (RDS, Cloud SQL, Neon, Supabase), adding pgvector is a CREATE EXTENSION away. That is also its limit: pgvector performs well up to a few million vectors, but above roughly 5-10M vectors the HNSW index starts to feel the memory-boundedness of a single-node Postgres, and you are back to sharding Postgres, which is nobody's favorite afternoon.

Rule of thumb. If your projected 24-month vector count is under 5M, pgvector is almost always the cheapest answer. Over 50M, either Qdrant or Weaviate self-hosted will cost less than Pinecone. Between those, it depends on the team.

Axis 2: Index algorithm (and what you're allowed to tune)

Three of the four — Weaviate, Qdrant, and pgvector — use HNSW (Hierarchical Navigable Small World) as their primary ANN index. Pinecone runs its own proprietary stack (per Pinecone's docs, HNSW is not used). The differences are in which parameters you're allowed to touch.

pgvector has had HNSW since version 0.5.0 (shipped 2023), with two main tuning parameters: m (max connections per node) and ef_construction (candidate list size during build). At query time you set hnsw.ef_search. The public pgvector GitHub README documents all three. Simple, exposed, nothing hidden.

Qdrant's HNSW implementation is one of the fastest in the open benchmarks. It exposes m, ef_construct, and full_scan_threshold, and the engine does a lot of work behind the scenes to keep the index memory-mapped efficiently. Public benchmarks (the ANN Benchmarks project and the vendors' own) consistently put Qdrant in the top tier for recall-at-latency at 1M vectors.

Weaviate also uses HNSW with efConstruction and maxConnections, both documented in the public docs under "vector index parameters." Weaviate's differentiator is hybrid search: you can combine the vector score with BM25 keyword scores in the same query without wiring up a separate Elasticsearch, which matters for any RAG system where the question has named entities.

Pinecone is the outlier. Its index is a proprietary black box — Pinecone's documentation describes algorithm selection as automatic, and parameters like m, ef_construction, or ef_search are not exposed because the underlying engine is not HNSW to begin with. You pick a pod type, you get Pinecone's choice of tradeoff. For most teams that is fine. For a team that just saw their p99 regress and wants to buy back latency by lowering a search-time knob, it is not.

The implication is organizational, not technical. A team that likes knobs will outgrow Pinecone eventually. A team that wants the knobs invisible will outgrow self-hosted Qdrant eventually, in the opposite direction.

Axis 3: Metadata filtering (pre vs post)

This axis is where the actual RAG quality lives, and it is also where the vendors diverge hardest.

Every production RAG query looks something like: "give me the 10 nearest chunks, but only from documents owned by tenant_id=42, created after 2026-06-01, with status=published." The vector search is the fast part. The filter is the part that decides whether the result is correct.

Two design patterns exist:

Post-filter. Run the ANN search first, get top-K candidates, then apply the metadata filter. If the filter rejects most candidates, you get fewer than K results. Fix: fetch K×N and over-retrieve. This is what most naive systems do.

Pre-filter (filtered search). Push the filter into the HNSW graph traversal itself. The graph walk skips nodes that fail the filter. You get exactly K results that match the filter.

Qdrant's documentation is unusually explicit about this: its payload index supports filtered ANN where the filter is applied during graph traversal, not as a post-process. In practice, this means Qdrant degrades gracefully on highly-selective filters (tenant_id=rare_customer) where a post-filter engine would miss results.

Weaviate's documentation describes a similar approach with "filtered vector search" that integrates with its inverted index for the metadata side.

Pinecone supports metadata filtering, and their docs describe it as pre-filtered search, though the specific internal strategy isn't exposed (per Axis 2).

pgvector's situation is different: the filter is a WHERE clause in SQL, and the Postgres planner decides whether to use the HNSW index or a sequential scan. For narrow filters on indexed columns, Postgres can filter first and then do a brute-force vector scan over the small candidate set, which is often faster than HNSW over everything. For broad filters, it goes through the HNSW index. The developer doesn't pick the strategy; the query planner does.

If your RAG system has highly-selective tenant or permission filters, this axis matters more than latency. On a 100M-vector index where each query is restricted to one tenant's 10,000 vectors, a pre-filter engine returns the right 10 results. A post-filter engine might return 2 or 3 and silently drop the rest.

Axis 4: Pricing model

The axis that gets undercounted in the first meeting and overweighted in the second.

Pinecone is pay-per-use (serverless) or pay-per-pod (classic). Public pricing for serverless is listed per million read units, per million write units, and per GB stored. On a RAG system that does 100K queries/day against a 10M-vector index, the monthly bill at the time of writing lands in the mid-four-figures USD depending on query complexity. Predictable. Linear. Not cheap at scale.

Weaviate Cloud has a tiered pricing model (free sandbox, then paid tiers differentiated by SLA, replication, and dedicated-infrastructure guarantees). Self-hosted is infra cost only: your Kubernetes, your bandwidth, your on-call.

Qdrant Cloud publishes per-cluster pricing by instance size. Self-hosted is infra cost only, same as Weaviate. Qdrant's single-binary footprint tends to make self-host cheaper than Weaviate's at small scale.

pgvector has no license cost. The only cost is whatever Postgres is already costing you. If you have a managed Postgres, pgvector is approximately free. If you don't, pgvector isn't actually free; you just moved the vendor lock-in to the Postgres host.

The honest comparison: at the 1-10M vector scale, pgvector on an existing Postgres is often 10-30× cheaper than managed Pinecone. At the 100M+ scale, operated Qdrant or Weaviate self-hosted usually beats Pinecone by 2-5×, but with operational overhead that is not free. The crossover depends on how expensive your SRE hours are. Those hours never cost what the capacity plan said they would.

Three decision paths

With the four axes, a decision on vendor usually falls into one of three profiles.

SaaS startup, 1-10M vectors, no dedicated infra team. Pinecone serverless or pgvector. Pinecone if your Postgres is already stressed and you want zero additional operational surface. pgvector if your Postgres has headroom and your team knows SQL.

Internal RAG for a mid-size company, 10-100M vectors, Kubernetes team available. Qdrant self-hosted. The ceiling is high enough, the operational surface is small enough, and the filtered-search behavior is correct on tenant-scoped queries. Weaviate if you need hybrid search without wiring Elasticsearch next to it.

Embedded product (RAG-as-feature in a shipped app), unknown scale. Start with pgvector if your product already has Postgres. Move to Qdrant if you cross 10M. Do not start with Pinecone; the per-query cost at the embedded-feature scale tends to dominate the unit economics.

Why not Milvus, Chroma, LanceDB

Three honorable mentions that often come up:

Milvus is excellent at the billions-of-vectors scale and has enterprise deployments at that tier. For the 1M-100M range this post targets, it is more operational weight than Qdrant for similar performance.
Chroma is optimized for local development and small-scale deployments. Scales beyond a few million vectors are a stretch; this is a local-first system, not an enterprise one.
LanceDB has a strong story for vector-plus-analytics workloads sitting on object storage. Worth a look if your RAG system wants to query vectors and OLAP columns in the same query. For pure RAG, the four above are more battle-tested.

What the book covers that this post doesn't

The book this post is condensed from spends most of its length on the pieces that matter before vendor selection:

A 3-axis evaluation framework for enterprise RAG (faithfulness 50%, relevancy 25%, correctness 25%)
The security and access-control design patterns (RBAC, pre-filter on tenant, prompt-injection detection)
The information-classification matrix for deciding which internal docs to even put in a RAG system in the first place

Vendor selection is downstream of all three. If you pick the vendor first and the evaluation framework second, you tend to end up with the vendor your evaluation framework can't honestly measure. The chapter walks the full ordering in Japanese; this post is the vendor-level summary.

If you want the full version of the Context Engineering playbook (RAG, MCP, CLAUDE.md, Agentic RAG, with benchmarks), it's a book: Turning LLMs from Liars into Experts. Original benchmarks show a 4.6x quality gain across five context strategies, with 96 production-grade Python files published on GitHub.

3 UI Slop Patterns in AI UI Generators: Purple Gradients, Inter Fonts, Uniform Cards

Ken Imoto — Fri, 06 Nov 2026 13:00:00 +0000

Open v0, Lovable, Bolt.new, or Claude Artifacts, and ask any of them for a modern SaaS landing page. They converge. Four independent tools produce pages that look like they came from one studio, and that studio really likes purple.

The convergence comes down to three patterns. Once you can name them, you can design around them. Until then, anyone who has seen ten AI-generated pages will spot yours in about two seconds.

Why "modern" means the same thing to every generator

Before the three patterns, the mechanism.

These tools don't invent styles. They sample from a training corpus of what the open web calls "modern." The open web, between 2020 and 2024, decided "modern" meant three specific things, and now every generator reproduces them because the gradient-descent floor underneath every one of them points to the same local minimum.

v0 rebranded from v0.dev to v0.app in August 2025 and shipped full-stack. Lovable writes the backend too now. Bolt expanded its stack. Claude Artifacts generates React inside the chat window. The engines got smarter. The aesthetic did not move.

Pattern 1: Purple Gradient Syndrome

Prompt any 2026 generator for a landing page hero and you get the same background. A diagonal gradient from purple to blue, sometimes with a floating geometric shape in the top right. Left-aligned heading, right-side mockup image. Sans-serif body text. A rounded white-text CTA that gradients again.

The lineage is traceable:

iOS/macOS popularized the aesthetic (Apple-style surface treatment).
Dribbble reinforced it (gradient-heavy posts upvote well).
CSS frameworks shipped it as a default (purple→blue linear-gradient works on every browser).
AI training data inhaled all three.

A gradient is fine on its own. The trouble is that this particular gradient has become the default placeholder for "we haven't decided what our brand is." Many generators, same prompt, same color. If your SaaS launch page gradient-matches your competitor's launch page, your positioning work has leaked into the stylesheet.

Escape path: Pick a background that is not a gradient and not pure white. Textured off-whites, warm earth tones, deep single-color navy. Anything that one generator cannot produce for free is worth more than one that twelve of them produce for free.

Pattern 2: Inter Font Dependency Syndrome

Open the devtools on a v0-generated page. Open them on a Lovable-generated page. Open them on your competitor's new homepage. All three load Inter.

Inter is a well-made typeface. Rasmus Andersson released it in 2017 and has shepherded it since, Tailwind UI templates leaned on it heavily through 2020, Next.js tutorials settled on it around 2021, Figma switched its default UI font from Roboto to Inter in March 2022. By 2023 it was in nearly every piece of UI training data on the open web. By 2026 it had stopped registering as a choice at all.

That ubiquity has turned into a signal of its own. Shipping an Inter-only product in 2026 is a tell: "we did not spend one hour on type."

Escape path: Pair Inter with something that announces intent. Serif + mono for a developer tool. Display + grotesque for consumer. Even if you keep Inter as the body font, letting the headings carry a different voice turns a generated layout into a designed one.

Pattern 3: Uniform Rounded Card Syndrome

Three columns. Four columns. All cards the same height. All cards border-radius: 8px or rounded-lg. All cards the same box-shadow: 0 1px 3px rgba(0, 0, 0, 0.12). All cards featuring icon + title + 2-line description.

The convergence has a provable origin:

<!-- The exact Tailwind pattern every generator reaches for -->
<div class="rounded-lg shadow-md p-6">
  <Icon class="h-10 w-10 text-indigo-500" />
  <h3 class="mt-4 font-semibold">...</h3>
  <p class="mt-2 text-sm text-gray-600">...</p>
</div>

Cards themselves are fine. A row of six identical ones, though, tells the reader that nothing on the page matters more than anything else. Real design communicates by making one card bigger, or weirder, or placed on its own. Generated design cannot. It has no model of which card the user is supposed to care about.

Escape path: Break the grid. One hero card at 2× size. One row of three, then one row of two, then one standalone. The visual hierarchy is what the generator doesn't know, which means it is what you still have to add.

The Slop Score that catches all three

If you want a number, the source for this piece defines a weighted score across the five AI-UI tells:

Slop Score = (Gradient_penalty × 0.30)
           + (Inter_penalty × 0.25)
           + (Grid_uniformity × 0.20)
           + (Color_convergence × 0.15)
           + (Element_predictability × 0.10)

Score band interpretation:

Range	Verdict
0–30	Distinctive. Escape successful
31–60	Mostly generic, some character
61–80	Typical AI slop
81–100	Pure template

Run it against your current landing page before you run it against a competitor's. The number tends to be higher than most teams expect.

What stays recognizable in 2026

Expect the three patterns to get more common over the next couple of years. Newer models are increasingly trained on pages that earlier generators produced, so the defaults feed back into themselves.

The practical implication for engineering teams: every AI-generated UI deserves a second pass that specifically removes the three signatures above. If you ship the first draft, you are shipping a photograph of the training corpus. If you spend fifteen minutes replacing the gradient, pairing the font, and breaking the card grid, you ship something that reads as yours.

The three patterns above are drawn from Chapter 7 of AI Slop Escape Guide. The full Zenn edition (Japanese) covers 17 chapters including before/after experiments, the full Slop Score definition, and other UI signatures beyond the three above. Canonical link to the source chapter is in the frontmatter.

AGENTS.md Review Policy: 4 Sections CodeRabbit, Copilot, and Claude Code All Read

Ken Imoto — Thu, 05 Nov 2026 13:00:00 +0000

Three AI code reviewers, one file. That shift happened faster than most teams noticed.

AGENTS.md was formalized in August 2025 as a shared config format co-promoted by OpenAI, Google, Sourcegraph, Cursor, Factory, and others. By September 2026, Claude Code v2.1.277 shipped native AGENTS.md support — if no CLAUDE.md is present, Claude Code falls back to AGENTS.md. GitHub Copilot CLI reads it. Cursor reads it. The spec at agents.md is now the closest thing to a universal standard for AI coding agents.

If one file configures three review tools, the question becomes: what should actually be in it for code review?

Why "review" belongs in AGENTS.md, not in a separate `.review.md`

The temptation is to split. One file for "how the agent writes code" (AGENTS.md), another for "how the agent reviews code." Resist it. Two files drift: the reviewer ends up approving patterns the writer config told the agent never to produce, and nobody notices until a bug ships.

The agent that writes the code is the agent that reviews the next PR. CodeRabbit reviews what Codex just generated. Claude Code reviews what Copilot just committed. The same set of conventions needs to be enforced on both sides of the slash. Splitting them into two files means they drift, and when they drift, your reviewer approves what your writer is told to avoid.

So AGENTS.md needs a Review Policy block alongside the code-writing block. Four sections are enough.

Section 1: Review Policy — the Conventional Comments labels

The first section defines the comment vocabulary. Without it, every reviewer invents their own.

## Code Quality

### Review Policy

Code review uses Conventional Comments labels:
- `issue:` must-fix (Approve blocked until resolved)
- `suggestion:` improvement (optional, reviewee's call)
- `nitpick:` small polish (optional, address if time)
- `question:` ask, not an attack
- `praise:` call out good code (don't skip this)

Why five labels and not three: the gradient between issue and nitpick is where review threads get stuck. If every comment is either "block this merge" or "do whatever you want," your senior reviewers end up relitigating style on every PR. The middle three labels (suggestion, nitpick, question) absorb the arguments that are not worth blocking a merge over.

What each tool does with this: CodeRabbit will parrot the label scheme if you reference it from .coderabbit.yaml. Copilot CLI will adopt the vocabulary if the review prompt references AGENTS.md. Claude Code picks it up automatically once AGENTS.md is in the repo root.

Section 2: Approval Criteria — the four conditions

The second section says what "ship it" means.

### Approval Criteria

Approve when all four hold. Don't chase perfection:
1. Functional requirements are satisfied
2. Tests exist (happy path + failure modes)
3. No type errors
4. Design aligns with the team's direction

The reason to write this down is that AI reviewers over-flag. CodeRabbit will raise a style issue on a two-line fix. Codex will ask for more test coverage on a hotfix. Without an Approval Criteria section, the reviewer has no stopping condition. Every PR gets more comments until someone manually overrides them.

Four conditions are enough. The fourth one (team direction) is deliberately subjective. That is what Section 3 is for.

Section 3: Code Direction — the patterns with reference paths

This is the section that makes AGENTS.md load-bearing instead of decorative.

### Code Direction

**Patterns to increase:**
- Repository Pattern for data access
  → See `src/repositories/user-repository.ts`
- Result-type error handling
  → See `src/types/result.ts`
- Pure functions for business logic (no side effects)

**Patterns to decrease:**
- Business logic in controller layer
- `any` type (use `unknown` instead)
- Direct DB queries (go through the ORM)
- Files over 1000 lines (split responsibilities)

The reference-path convention is the whole point. Abstract rules like "use Repository Pattern" don't survive contact with an AI agent. Ask three agents to "use Repository Pattern" with no reference path and you can get three different ideas of what a repository is. A path to an actual implementation (src/repositories/user-repository.ts) collapses the interpretation to one.

Compare:

Rule style	What the agent does
`Use Repository Pattern`	Generates something it calls a repository. Could be anything.
`Use Repository Pattern → see src/repositories/user-repository.ts`	Reads the file. Copies the structure.

Every bullet in Code Direction should have a → see <path> if a reference implementation exists. If no reference exists yet, the first time someone implements the pattern, their file becomes the reference. That is how the section ratchets up without you writing a style guide.

Section 4: PR Guidelines — the mechanical rules

The fourth section is the smallest and the most skipped.

### PR Guidelines

- Target 300 lines or fewer per PR
- Separate refactors from feature changes (different PRs)
- PR description: what changed, and why

The 300-line target is where review quality collapses in practice. Above it, the reviewer scans. Below it, the reviewer reads. The "refactor separate from feature" rule is the one that pays for itself fastest: when a bug ships, you can bisect by PR without having to untangle a renamed function from a changed behavior.

Wiring AGENTS.md into CodeRabbit (`.coderabbit.yaml`)

AGENTS.md on its own gets read by Claude Code and Copilot CLI. CodeRabbit needs one more step. Point it at the Code Direction section explicitly.

# .coderabbit.yaml
reviews:
  path_instructions:
    - path: "**"
      instructions: |
        Reference the "Code Direction" section in AGENTS.md.
        Code that drifts from "patterns to increase": comment with `suggestion:`.
        Code that matches "patterns to decrease": comment with `issue:`.

        Always use Conventional Comments labels.

With this in place, CodeRabbit comments arrive pre-labeled, matched against your real patterns, and quoting the AGENTS.md rules by reference. The maintenance cost of CodeRabbit drops to editing one file.

The feedback loop that AGENTS.md unlocks

The reason to put review policy in AGENTS.md and not in a separate review-config file is that it closes the loop:

A review surfaces a new pattern (say, you decide to use Result<T, E> everywhere).
The pattern gets added to Code Direction with a reference path.
The next PR Claude Code or Codex generates follows the new pattern.
CodeRabbit flags any drift from the pattern on later PRs.

That is a four-step feedback loop operating across three AI tools. No single-tool config file can do that. The moment the config fragments, the loop breaks.

When one AGENTS.md isn't enough

A monorepo with a frontend subfolder and a backend subfolder will outgrow a single root AGENTS.md within weeks. The spec supports nesting. An AGENTS.md inside packages/frontend/ overrides the root for that subtree. Claude Code and Codex walk the tree down from the root; Copilot CLI only picks up AGENTS.md on the path from the current working directory up to the git root (nested-subtree discovery is tracked in copilot-cli#3051), so run it from the subfolder you want to scope.

Scope the Review Policy section to the repository root. Scope the Code Direction section to each sub-folder. Frontend and backend will have different "patterns to increase" lists, and that is exactly what the subfolder files are for.

The 4-section layout above is drawn from Chapter 6 of the Zenn book harness-code-review. The full book covers the three-layer review harness: pre-commit hooks, CodeRabbit tuning, GitHub Actions review pipelines, and PR metrics that make a review culture quantifiable.

Systematizing AI Code Review — the full harness

Conway, Brooks, Parnas: 6 Architecture Quotes Still Deciding Code-Review Fights in 2026

Ken Imoto — Wed, 04 Nov 2026 13:00:00 +0000

Three names keep showing up in modern code reviews, and none of them wrote their best lines about code review. Conway wrote about committees. Brooks wrote about schedules. Parnas wrote about Fortran modules in 1972. Half a century later, the pull requests you merged this morning are still arguing with them.

The six quotes below are the ones that keep landing in review threads when a merge stalls on the same old fight. Each one has an original context from decades ago. Each one points at a 2026 problem that didn't exist when it was written.

1. Conway, 1968: the quote your multi-agent system is secretly obeying

"Organizations which design systems [...] are constrained to produce designs which are copies of the communication structures of these organizations."
— Melvin Conway, Datamation 14(4):28-31, April 1968

Conway was writing about committees inside aerospace contractors. The paper was a complaint, not a theory. Someone's design-by-committee had produced a compiler whose stage boundaries matched the subcommittee boundaries, and the result was ugly.

Fifty-eight years later, the quote has found a new target: multi-agent architectures. In 2026, teams routinely spin up five, ten, twenty sub-agents to handle a workflow. The silent default is that each team ships its own sub-agent for the piece it already owns. The result is a copy of the org chart with LLM calls in it — hub bottlenecks, lossy handoffs, no shared standards. Anthropic quietly reversed course in late 2026 and started telling teams that one capable agent beats a crowd of narrow ones. That is not a modeling argument. That is Conway speaking from 1968.

Code-review application: When a PR adds a new service, agent, or module, ask whether its boundary matches a problem shape or a team shape. If the only reason it exists is "that is the team that was free," the merge is reinforcing your org chart, not your architecture.

2. Parnas, 1972: the quote that still kills export statements

"The connections between modules are the assumptions which the modules make about each other."
— David L. Parnas, Information Distribution Aspects of Design Methodology, IFIP Congress 1971, Proceedings vol.1, pp.339-344

Parnas was arguing with Fortran programmers. His famous paper from the same stretch — On the Criteria To Be Used in Decomposing Systems into Modules, CACM 15(12), December 1972 — makes the sister point: don't carve a system by processing step. Carve it by what each module can hide.

The sentence above is the one you cite when you reject a PR that leaked an internal data type into a public interface. Every export type Internal... that drifts into a public barrel file is a Parnas violation. In TypeScript 5.x, this is where structural typing bites hardest: an "accidentally public" shape becomes load-bearing before anyone notices, and the API can no longer evolve without a breaking change.

Code-review application: When reviewing a new public surface, list what the caller must assume about the implementation to use it. If the list is longer than three items, the module's boundary is wrong, not its implementation.

3. Brooks, 1975: the quote your "we'll just add another agent" moment deserves

"Adding manpower to a late software project makes it later."
— Fred Brooks, The Mythical Man-Month, Addison-Wesley 1975, p.25

Brooks was writing about OS/360. Half a floor of IBM programmers, one slipping deadline, more programmers added, more communication overhead, more slip. He gave the pattern a name: Brooks's Law.

In 2026, the same shape appears without any humans involved. "The agent is too slow — let's split it into five sub-agents." "The pipeline is too long — let's add a planner on top." Every added agent is a communication edge, a context handoff, a chance for information to decay. Brooks's n²-ish communication cost shows up as token usage: the sub-agent fleet that promised speedup burned 2× the tokens of a single agent and shipped the same answer later. I have approved that PR. More than once.

Code-review application: Any PR that adds a new coordinator, planner, router, or sub-agent needs a specific sentence of justification. "Without this layer, the request fails because X." If the sentence is "it would be cleaner," reject it. You are hiring Brooks's extra programmers.

4. Fowler, 1999: the quote your lint rule can't enforce

"Any fool can write code that a computer can understand. Good programmers write code that humans can understand."
— Martin Fowler, Refactoring, Addison-Wesley 1999, p.15

Fowler was writing a book about renaming variables for a living. The sentence lands because it compresses the whole premise of refactoring into one line. Compile-passing is the floor, not the ceiling.

The 2026 twist is that the "computer" now writes most of the code, and the "human" is both the next engineer and the next AI agent reading AGENTS.md. When Claude Code or Codex re-enters a codebase six weeks later, it reads the names, the comments, the file layout. Code that was legal but unloved — a function called handleData, a parameter called opts, a 300-line file called utils.ts — now penalizes you twice. The human maintainer and the AI pair-programmer both get lost in it.

Code-review application: Rename and relocate before you ship. In 2026, your reviewer is not just the human on the PR. It is the agent that will inherit this file at 3 a.m. in six weeks.

5. Beck, 1999: the quote that keeps "we'll optimize later" honest

"Make it work, make it right, make it fast."
— Kent Beck, Extreme Programming principles

Beck's three-step ordering has been quoted inside out ever since XP shipped. The point was not that performance comes last. The point was that each step needs to be finished before the next starts. "Work" means a passing test. "Right" means code the next engineer can live inside. "Fast" means a measured bottleneck, not a hunch.

In 2026, the step most teams skip is still "right." AI pair-programmers happily jump from "work" to "fast" — they will generate a benchmark harness, propose a parallelized rewrite, and never once ask whether the function should exist. I have hit merge on exactly that PR, which is how I know. "Make it right" is the step where a human reviewer still has to sit down and ask: does this belong in this module? Does this abstraction pay rent? Does the function name match what it actually does?

Code-review application: When a PR comes in with a passing test and a 10% speedup, don't merge it. The missing step is "right." Ask the author to show the version that is clean, small, and obvious. Then compare.

6. Hickey, 2011: the lesser-known quote that keeps winning

"Simple is actually an objective notion. [...] Easy is relative. [...] easy for whom, or hard for whom? It's a relative term."
— Rich Hickey, Simple Made Easy, Strange Loop 2011

Hickey, the creator of Clojure, delivered this talk at Strange Loop in 2011. The core distinction: simple means one role, one task, one axis. It braids no other concerns. Easy means familiar, near-at-hand, matching what you already know.

The talk has had a second life since 2024. Every "we're adopting X because it is easy" decision in the AI stack has run into Hickey's framing within a year. LangChain was easy, until it braided prompts, tools, state, and routing into one object. "Easy" let you ship the demo. "Simple" would have let you refactor it without a rewrite.

Code-review application: When a PR picks a library "because everyone uses it," ask what the library braids together. If pulling it out requires touching four unrelated concerns, you have imported ease, not simplicity.

Why these six, in this order

These aren't the six most quoted. "Premature optimization is the root of all evil" (Knuth, 1974) is more quoted than any of them. The six above are chosen for a different reason: each one points at a 2026 argument that is still live in your pull-request queue this week.

Conway fires when a sub-agent split copies the team split.
Parnas fires when a public interface starts carrying implementation leak.
Brooks fires when the fix for a slow agent is a bigger agent fleet.
Fowler fires when the next reader is now an AI.
Beck fires when "right" gets skipped between "work" and "fast."
Hickey fires every time a stack picks a library for familiarity.

If a code review stalls, drop the matching quote into the thread. The author either rebuts it or revises the PR. Either outcome is useful.

The six above are a thin slice of a larger shelf. If you want ~100 engineering quotes with the original context next to the modern application — not a generic business quote book, but one that puts lesser-known voices like Hickey next to Fowler — the full collection is here:

Why Do Some Words Stay With You Forever? 100 Engineering Quotes Decoded

5 PR Metrics on 303 Merged PRs from pytest and pydantic: 2 Point at Review Bottlenecks

Ken Imoto — Thu, 22 Oct 2026 13:00:01 +0000

Engineering-productivity dashboards tend to show the same five pull-request metrics: Time to First Review, Time to Merge, Comments per PR, Review Cycles, and AI Resolution Rate. Each gets a card and a trend line.

I wanted to know which of them can answer one narrow question: "is review the slow part of shipping a change?" My own repositories are no help here. I work alone, and my busiest repo merged four PRs in the last 90 days. So I pointed a script at two public projects with real review traffic:

pytest-dev/pytest: 107 merged PRs from humans (166 merged, 59 from bots), 46 authors
pydantic/pydantic: 196 merged PRs from humans (221 merged, 25 from bots), 40 authors

The window is PRs merged between 2026-07-21 and 2026-10-19 (UTC). All numbers below come from the GitHub REST API, and you can rerun the script to check them.

The script

# review_metrics.py
import json, os, sys
from datetime import datetime, timedelta, timezone
from statistics import mean, median
from github import Auth, Github

REPO = sys.argv[1] if len(sys.argv) > 1 else "pytest-dev/pytest"
END = datetime(2026, 10, 19, tzinfo=timezone.utc)
START = END - timedelta(days=90)

g = Github(auth=Auth.Token(os.environ["GITHUB_TOKEN"]))
repo = g.get_repo(REPO)

rows = []
for pr in repo.get_pulls(state="closed", sort="updated", direction="desc"):
    if pr.updated_at < START:
        break  # sorted by updated: nothing older can have merged in the window
    if not pr.merged_at or not (START <= pr.merged_at < END):
        continue
    if pr.user.type == "Bot" or pr.user.login.endswith("[bot]"):
        continue

    author = pr.user.login
    reviews = [r for r in pr.get_reviews()
               if r.user and r.user.login != author and r.submitted_at]
    first = min((r.submitted_at for r in reviews), default=None)
    approvals = [r.submitted_at for r in reviews if r.state == "APPROVED"]
    hours = lambda a, b: (b - a).total_seconds() / 3600

    rows.append({
        "pr": pr.number,
        "author": author,
        "created_at": pr.created_at.isoformat(),
        "merged_at": pr.merged_at.isoformat(),
        "first_review_h": hours(pr.created_at, first) if first else None,
        "merge_h": hours(pr.created_at, pr.merged_at),
        "last_approval_to_merge_h": hours(max(approvals), pr.merged_at) if approvals else None,
        "review_comments": pr.review_comments,
        "cycles": sum(1 for r in reviews if r.state in ("CHANGES_REQUESTED", "APPROVED")),
        "changed_lines": pr.additions + pr.deletions,
    })

json.dump(rows, open(f"{REPO.replace('/', '__')}.json", "w"), indent=1)

def summary(key):
    v = [r[key] for r in rows if r[key] is not None]
    return f"n={len(v):3d}  median={median(v):7.1f}  mean={mean(v):7.1f}"

print(f"{REPO}  merged {START:%Y-%m-%d}..{END:%Y-%m-%d}  human PRs: {len(rows)}")
print(f"PRs with no review from anyone but the author: {sum(r['first_review_h'] is None for r in rows)}")
for k in ("first_review_h", "merge_h", "last_approval_to_merge_h", "review_comments", "cycles"):
    print(f"{k:26s} {summary(k)}")

Three definitions matter. A "review" is any submitted GitHub review (approve, request changes, or comment) from someone other than the PR author. review_comments counts inline code comments only, not the conversation tab. A "cycle" is one approve or request-changes event.

Raw output:

pytest-dev/pytest  merged 2026-07-21..2026-10-19  human PRs: 107
PRs with no review from anyone but the author: 0
first_review_h             n=107  median=    9.2  mean=  284.6
merge_h                    n=107  median=  143.4  mean=  806.0
last_approval_to_merge_h   n=107  median=    1.8  mean=  128.3
review_comments            n=107  median=    0.0  mean=    3.3
cycles                     n=107  median=    1.0  mean=    1.5

pydantic/pydantic  merged 2026-07-21..2026-10-19  human PRs: 196
PRs with no review from anyone but the author: 100
first_review_h             n= 96  median=    0.2  mean=   38.2
merge_h                    n=196  median=    1.6  mean=   63.4
last_approval_to_merge_h   n= 41  median=    0.3  mean=   28.7
review_comments            n=196  median=    0.0  mean=    0.8
cycles                     n=196  median=    0.0  mean=    0.2

Two things in that output are worth more than any single metric.

Problem 1: the average describes five PRs

pytest's mean Time to First Review is 284.6 hours. The median is 9.2 hours. A dashboard that shows the mean says "nearly 12 days"; nearly two thirds of the PRs got one within 24 hours.

The gap comes from a handful of old PRs. Five of the 107 were opened between January and early July 2026 and only merged inside the window. Their first-review waits (5,568h, 5,415h, 4,261h, 3,305h, 1,650h) add up to 66% of all first-review hours in the sample. Drop those five and the mean falls to 100.5 hours.

The spread is wide even without them:

pytest, Time to First Review	Hours
25th percentile	0.6
median	9.2
75th percentile	95.4
90th percentile	507.5

43 of 107 PRs (40%) got a first review within 4 hours, 69 (64%) within 24 hours, and 20 waited more than a week. "Share of PRs reviewed within N hours" plus the 90th percentile says far more than an average.

Problem 2: the metric quietly skips half the PRs

In pydantic, 100 of 196 human PRs were merged without any GitHub review from someone other than the author. 75 of those 100 came from a single maintainer. Unreviewed PRs merged in a median of 0.3 hours, reviewed ones in 7.5 hours.

Time to First Review cannot be computed for a PR that never got a review, so the 0.2-hour median above describes only 96 of the 196 PRs. Whether merging without a review is fine depends on the project's rules, and I am not judging pydantic's. The dashboard problem is that the number looks complete when it covers 49% of merges. Put the coverage next to the metric: "median 0.2h, 96 of 196 PRs reviewed".

The two metrics that point at review

Time to First Review, reported as a median and percentiles with coverage, measures the stretch where a PR waits and no reviewer has acted. That wait is a review-process delay by definition.

Published research gives a reference point. Google's 2018 case study of its own code review, based on logs of about nine million reviewed changes, reports a median wait for initial feedback of under an hour for small changes and about 5 hours for very large ones, with an overall median under 4 hours for the whole review process (Sadowski et al., ICSE-SEIP 2018). That is a large company with dedicated tooling, so treat it as a reference point rather than a target for a volunteer project.

The lever with the strongest evidence is explicit assignment. Meta suspected that diffs assigned only to a team were slowed by the bystander effect ("someone else will probably review this") and tested it. In an A/B test on 12.5k authors in spring 2023, it randomly assigned one of the top three recommended reviewers on team-only diffs. TimeInReview fell 11.6%, with no significant change in the time reviewers spent on each diff (Improving Code Reviewer Recommendation, ACM TOSEM 2026, arXiv:2312.17169). You do not need Meta's recommender to try the idea. GitHub can request code owners automatically, and team review requests can be routed to individual members by round robin or load balance.

Review Cycles measures whether review converges. In pytest, 68 of 107 PRs merged after one approve-or-request-changes event, 26 after two, and 13 after three to five. Cycles barely tracked PR size (Spearman 0.19 against changed lines), so a high count is less about big diffs and more about how a review gets run, for example comments arriving in several rounds. That makes it a review-process signal you can act on.

The three that answer other questions

Time to Merge mixes three intervals: waiting for a first review, the back-and-forth until the last approval, and the wait from last approval to merge. I split every pytest PR into those three segments and looked at which one was longest:

Longest segment of Time to Merge (pytest)	PRs
Waiting for first review	44
Approval to merge	34
First review to last approval	29

Summed over all PRs, the split was 35% waiting, 49% back-and-forth, 16% after approval. One number covers three different problems, and the approval-to-merge part (CI, merge queues, release timing) is outside review entirely. Time to Merge is still a fine throughput metric. It just cannot tell you review is the bottleneck.

Comments per PR mostly tracks size. In pytest, inline comment count correlated with changed lines at Spearman 0.53, and 67 of 107 PRs (63%) had zero inline comments. In pydantic, 135 of 196 had none. A month-over-month rise in the average usually reflects a shift in PR mix. What it can flag is a sustained zero on large PRs, which is worth a look.

AI Resolution Rate cannot be computed from the GitHub API alone, because GitHub does not record whether a bot comment was "resolved" in the sense the metric needs. You need data from the AI review tool itself, so I did not measure it here. One structural caution applies anyway: the rate depends on what the bot chooses to flag, so changing the tool or its rules moves the number without any change in how your team responds. Compare it only within one tool and one configuration.

What I would put on the dashboard

Time to First Review: median, 90th percentile, share reviewed within 4h and 24h, and how many merged PRs had a review at all
Review Cycles: distribution (1 / 2 / 3+) alongside the mean
Time to Merge, Comments per PR, AI Resolution Rate: on a separate throughput or tooling page

Rerun the script on your own repo before you trust any of these. The two projects here behaved so differently (pytest reviews every PR, pydantic merged half without one) that a single benchmark would have misled both.

The longer version, with the three-layer review model (hooks, AI review, human review), the CodeRabbit configuration, and a GitHub Actions workflow that writes a weekly metrics report, is in my book Systematizing AI Code Review. The English edition is on Kindle.

Sources:

pytest-dev/pytest and pydantic/pydantic, GitHub REST API, PRs merged 2026-07-21 to 2026-10-19
Sadowski, Söderberg, Church, Sipko, Bacchelli. Modern Code Review: A Case Study at Google. ICSE-SEIP 2018
Improving Code Reviewer Recommendation: Accuracy, Latency, Workload, and Bystanders. ACM TOSEM 35(1), 2026
GitHub Docs: About code owners, Managing code review settings for your team

RTX 4070 + logprobs: 96 of 100 Claude Code Requests Needed the Cloud. 97 of 200 Steps Could Go Local

Ken Imoto — Wed, 21 Oct 2026 13:00:00 +0000

Routing whole requests to a local LLM did not work for my Claude Code workload. I took 100 jobs from my work log (March to September 2026), rewrote each one as the instruction I would have typed, and labelled where it could run. 96 needed a frontier model. The other 4 had to stay local because they contained personal or financial details, not because a local model could do them. Zero were simply "a local model is good enough."

Then the unit went down to single steps inside agent sessions. Out of 200 tool-call steps pulled from my Claude Code logs, 97 were labelled "a 4B to 9B local model is enough if you hand it the material." That is 48.5%.

The step-level router that came out of this is on GitHub under MIT: kenimo49/local-step-router. This post covers the numbers behind it, what the router code actually looks like, and where a logprobs judge on an RTX 4070 helped and where it did not.

Why request-level routing failed

A request like "fix the flaky test" is a label, not a payload. Nothing in the text tells a local model what to do until something reads the repo. In the 100 reconstructed requests, the task mix was mostly agent work (74 of 100), and agent work is exactly what you hand Claude Code in the first place.

Raw instructions told the same story. Of 150 instructions sampled from interactive sessions, 138 only made sense with the earlier conversation ("send it", "do the daily log"), and only 5 were labelled local-capable. Of 100 fixed prompts that my cron pipeline sends to claude -p, also 5.

Unit	n	Local is enough	Must stay local (sensitive)	Frontier
Requests rebuilt from the work log	100	0	4	96
Raw instructions	150	5	2	143
Fixed pipeline prompts	100	5	0	95
Single steps	200	97	4	99

Which steps go local

The 200 steps came from three tool types in the logs. The split is lopsided:

Step type	n	Local is enough	Must stay local	Frontier
`WebFetch` (extract something from a page)	90	86	0	4
`git commit` message	50	10	4	36
Sub-agent delegation	60	1	0	59
Total	200	97	4	99

Almost all of the local share is WebFetch. A fetch prompt usually says what to pull out ("the price", "the release date"), and once the page text is in hand, that is extraction. Commit messages go local only when they are one-line summaries of a diff. Sub-agent delegation is research and multi-step tool use, and it stays frontier.

Two caveats matter here. The labels are judgments, not runs: "local is enough" means "given the page body or the diff." And they were assigned by my AI assistant, with my review planned only for disagreements.

So I ran a sample. 20 of the 86 local-labelled WebFetch steps went to qwen3.5:9b on Ollama (think=false, temperature 0), with the page re-fetched and extracted by trafilatura. 11 could be compared with the original output: 8 matched, 2 were partly wrong (a merged-cell spec table read into the wrong column, a title reported as missing), and 1 was wrong (it quoted "about 1 to 3 months" as evidence and still answered yes to "does it say 2 to 3 months?"). The other 9 never reached a fair comparison: JS-rendered pages with no extractable text, pages that had disappeared, or an original fetch that had already failed. Local processing took 6 to 66 seconds per item, median about 18. Most of the loss happened before the model, in fetching and extracting the page.

The router: rules first

The repo's Router checks rules top to bottom, first match wins. A probability judge only runs for steps no rule matched, and it is off by default. This is the shipped examples/routing.yaml, rules part:

rules:                      # step rules, top to bottom, first match wins
  - match: {tool: WebFetch}
    route: local
  - match: {tool: [Agent, Task]}
    route: frontier
  - match: {kind: commit_message}
    route: frontier         # frontier until a probability judge is added

probability: null           # off until measured with the same judge model (chapter 14)
default_route: frontier

And from Python:

from steprouter import config
from steprouter.router import Router

router = Router.from_config(config.load("examples/routing.yaml"))

router.route({"tool": "WebFetch", "prompt": "Extract the price table"})
# Decision(route='local', by='rule', p_local=None)
router.route({"tool": "Edit"})
# Decision(route='frontier', by='default', p_local=None)

Decision.by tells you whether a rule, the probability judge, or the default made the call.

Why rules first: on the 444 non-sensitive items across all three sets (107 labelled local-capable), I compared a single rule against asking a local model for P(local is enough):

Router	Local-capable items caught (of 107)	Frontier items sent local by mistake (of 337)
qwen3.5:9b probability, threshold set for at most 5% wrong-locals (AUROC 0.809)	59	17
gemma4:e4b probability, same calibration (AUROC 0.618)	8	19
One rule: `WebFetch` goes local	86	4

The 9b judge's AUROC of 0.809 was mostly the judge recognising the step type. Within one type it could barely separate items: 0.095 on the pipeline prompts, 0.411 on raw instructions. If the step type decides, write a rule. Keep the probability for what the rules leave open.

I also tried qwen3.5:4b as the routing judge. It timed out under GPU contention at 414 of 450 items, so it has no routing result.

Reading logprobs from Ollama

Where the local model does earn its place is the entrance gate: "does this request contain actual sensitive data?" The judge asks for a one-letter answer (Y or N), generates exactly one token, and reads the candidate letters from top_logprobs. In the repo this is steprouter/judge.py, using the native /api/chat with think: false, num_predict: 1 and top_logprobs: 20. It returns the probabilities renormalised over the candidates, plus mass: how much probability landed on the candidates at all.

mass is the field to watch, because two things make it collapse without any error:

The first token is not the answer. A model that reasons before answering spends token 0 on its reasoning. With think: false, qwen3.5 answers in the first token. gpt-oss:20b did not put a label there and gave mass 0 on my setup, so I could not use it this way. The other models I tried stayed between 0.89 (qwen3.5:0.8b) and 1.00.
A candidate outside the top 20 reads as 0. Ollama rejects top_logprobs above 20 (the check in server/routes.go; a PR to raise it to 100 was closed without merging). If a candidate is not in that list, its probability is simply missing, and the others share 100%.

The repo's judge marks a result unusable when mass is below 0.5. A first test on any new judge model: send one known question and check that mass is close to 1.

One correction to something you may have read, including in an earlier version of my own notes: the OpenAI-compatible /v1/chat/completions endpoint does return logprobs. The v0.12.11 release notes say both APIs support them, and in issue #16117 a maintainer showed it working before the reporter closed it as their own client bug. The compatibility page in the docs still marks logprobs unsupported. The repo uses the native endpoint, so it does not depend on which of those is current.

The threshold is not 0.5

qwen3.5:4b ranked the 60 English test requests almost perfectly for sensitive content (AUROC 0.999), but its P(Y) stayed low. With a fixed 0.5 line it missed all 20 sensitive requests. Drawing the line from labelled examples with 5-fold cross-validation brought that to 2 misses and 1 false alarm. On the Japanese version of the same set, the fixed line missed 9 of 20 and the calibrated one missed 1.

The repo's CLI does this from your own judged examples:

python3 scripts/judge_set.py qwen3.5:4b en > judged.jsonl
steprouter-threshold judged.jsonl --margin 0.3

It prints JSON with auroc, the line (lowest P among your positive examples times the margin), and 5-fold cv counts of misses and false alarms, and it warns when AUROC is under 0.95. The data set is 60 made-up requests, so treat these as small-sample numbers, and redraw the line on your own labels.

Where the RTX 4070 fits

All of the judging above ran through Ollama on one RTX 4070 12 GB (version 0.20.2 for the 60-request tests). Small judges were not faster on that box: qwen3.5:0.8b through 9b all took about 0.5 to 0.7 seconds for two questions, mostly network round trip, while qwen3.5:35b-a3b took about 3.3 seconds because it does not fit in 12 GB. If you want a 35B MoE model on the same card for heavier local work, the --cpu-moe flag in llama.cpp took it from 12.2 to 34.6 tok/s in a separate benchmark. That benchmark is about speed; the routing numbers above did not use it.

What I would copy

Count your own steps before tuning anything. My 48.5% is almost entirely WebFetch. A workload with fewer fetches will get a smaller share.
Decide by step type first. One WebFetch rule beat both probability judges in my data.
Check mass, then draw the threshold from your own labels. A fixed 0.5 missed every sensitive English request with qwen3.5:4b.

The code: kenimo49/local-step-router. MIT, Python 3.10+, PyYAML as the only dependency, Ollama 0.12.11 or later for the judge. The tests run without Ollama.

The full write-up, with the step labelling criteria, the sensitive-data gate, and the typed return values, is the book this repo accompanies: Local LLM or Claude Code? Kindle, included in Kindle Unlimited.

GitHub Stars Lied: 3 Tech-Selection Traps Senior Devs Still Walk Into

Ken Imoto — Tue, 20 Oct 2026 13:00:00 +0000

In the Stack Overflow Developer Survey 2025, 17.6% of respondents said they had done extensive work in Vue.js over the past year. On W3Techs (checked 2026-10-19), Vue.js runs on 0.6% of all websites. That is a 29x spread.

Angular: 18.2% in the survey, 0.2% of websites. About 90x. React, the safe pick: 44.7% against 6.0%, about 7x.

Neither number is wrong. The survey counts developers who answered a question about frameworks. W3Techs counts every website it can detect, and most of those are not hand-built single-page apps. The spread mostly says that the two denominators have nothing to do with each other.

That is the useful part. Every popularity number answers a question about some population, and in a tech-selection meeting nobody asks which population. GitHub stars are the most quoted popularity number of all, and they have a population problem of their own.

Trap 1: the bandwagon effect, dressed as "community size"

The bandwagon effect is the tendency to judge something as right because many people have adopted it. Part of that is rational. A widely used tool has more documentation, a more mature ecosystem, and a bigger hiring pool. The trouble starts when that advantage outweighs every other axis.

GitHub stars, npm download counts, Stack Overflow answers. All three signal community size. None of them signal fit for your project. As of 2026-10-19 (GitHub API), apache/echarts has 67.5k stars, chartjs/Chart.js has 67.7k, and d3/d3 has 113.8k. ECharts and Chart.js look like twins on a comparison table. D3 looks like the winner. D3 is a low-level toolkit for building visualizations; the other two hand you finished chart types. The star column cannot show that.

Some of the stars are not people

A study by He et al., published at ICSE 2026, built a detector called StarScout and ran it over all GitHub event data from July 2019 to December 2024. It flagged 6.0 million suspected fake stars and 18,617 repositories with fake-star campaigns. Fake stars stayed at or below 1% of all stars in any month, but campaigns grew from 2022 and surged in 2024. Most of them promoted short-lived phishing malware repositories. The rest went mostly to AI/LLM, blockchain, tool, and tutorial repos.

The paper also measured whether buying stars works. The effect on real attention lasts under two months, is about 5x smaller than the effect of real stars, and turns negative over the long term.

So for a big, old project, fake stars are a rounding error. For a young repo in a hyped category, the star count can be partly bought. Either way, stars are a weak input, and the bandwagon effect makes them the loudest one.

When the effect gets stronger

The bandwagon effect strengthens under three conditions, and all three describe a normal tech-selection meeting:

Your own expertise in the domain is shallow (so you lean on "what everyone picks")
The decision carries visible blame if it fails (so "we picked the popular one" is the safe story)
You are under time pressure (so detailed comparison gets compressed into "which has more stars")

Fifty thousand stars on one framework and five hundred on a competitor is the setup that triggers all three, even when the smaller one fits your use case better.

What catches it: a requirements matrix

Telling the room to "be careful about stars" does nothing. Changing the table does.

List the project's requirements before you look at candidates: performance targets, the team's existing skills, licensing, long-term support, learning cost. Then add community size as one more row, weighted like the others. Score every candidate on every row. Stars now contribute one row out of six, and the five rows that actually describe your project get filled in before anyone opens a GitHub tab.

Trap 2: the IKEA effect on "the thing we built ourselves"

The IKEA effect is the tendency to overvalue things you put labor into. Norton, Mochon, and Ariely described it after experiments in which people assembled IKEA boxes, folded origami, and built Lego sets, then valued their own results above equivalent ready-made ones. Code is labor you can see, so the effect carries over:

An in-house framework that two engineers wrote in a hackathon five years ago, defended with "it fits our needs exactly"
A custom build config that only one person can modify, kept because "replacing it is risky"
A fork of a library that has drifted three major versions, maintained because "we invested a lot of time understanding it"

Not Invented Here syndrome is the acute version. The chronic version is quieter: a steady preference for the thing that has your team's hours in it.

Market numbers will not prove the effect for you. npm counted 76.6 million Vitest downloads and 26.0 million Jest downloads in the week of 2026-10-10 to 10-16. Jest is still a very large installed base. Some of those projects stay because Jest fits them; some stay because of years of local config. The download count cannot tell you which, and neither can you, until you ask the question below about your own stack.

The point to hold on to: "we built it" is a fact about past labor. It says nothing about quality. The question is what the next twelve months of maintenance will cost.

The question that defuses it

One line, on a sticky note by the monitor:

If this system did not exist and we were choosing today, would we pick this?

A no does not mean "migrate now". Migration costs are real. A no means the current choice may be held in place by the IKEA effect plus the next trap, and deserves an actual comparison.

Trap 3: sunk cost, also known as the Concorde fallacy

Everyone can define the sunk cost fallacy, and almost nobody applies it. "We have already invested three months in this refactor" is the shape. The three months are gone. They do not come back if you continue, and they do not vanish if you stop. The only live comparison is the cost of continuing from here against the cost of stopping and switching from here.

The nickname comes from Concorde. Britain and France signed the development treaty in 1962, and costs overran the original estimate many times over. The British government privately regarded the project as a commercial disaster that should never have been started, yet both governments kept paying. Political and legal commitments made withdrawal hard, which is a lesson in itself: sunk cost rarely acts alone. Twenty aircraft were built, fourteen entered airline service, and the fleet retired in 2003.

In engineering, sunk cost shows up as:

Deferred refactors ("throwing this code away wastes three months of work")
A struggling project that keeps going ("we are 60% of the way there")
Unused features kept alive ("we went to the trouble of building it")

Each of those sentences is a sunk-cost claim in disguise.

The forward-only decision frame

The frame that works gives "already spent" zero weight. Three questions:

Set all past investment to zero. Looking only at future cost and return, which option wins?
What does staying on this choice cost over the next year (maintenance hours, learning cost for new hires, effect on hiring)?
What is the one-time cost of switching, and how much does it save per year afterward? Does the saving pay back the switch within a horizon that matters?

"Already spent" is the loudest voice in a technical meeting. It is also the one voice that cannot change the future.

The hype cycle chains all three

Gartner's hype cycle describes technology maturity in five phases, and a different bias dominates each part of it:

Innovation Trigger to Peak of Inflated Expectations: the bandwagon effect at maximum. "Do not get left behind" overrides every other signal.
Trough of Disillusionment: negativity bias takes over, and the technology gets written off below its real value.
Slope of Enlightenment to Plateau of Productivity: calm evaluation finally becomes possible, but whatever was adopted at the peak is now defended by sunk cost ("we cannot throw it away now") and the IKEA effect ("we worked hard to make it run").

Whatever you adopted during the current AI tooling peak will meet this chain on schedule. When the plateau arrives, the choice will be defended by two biases at once, and the forward-only frame is the tool for that review.

Three sentences for the wall

GitHub stars are one row out of six in the requirements matrix. Not a tiebreaker.
"If this did not exist, would we pick it today?" Ask it for every stack you maintain, once a quarter.
Money already spent is zero on the future-cost line. The only live comparison is forward.

Tech selection is a judgment you keep revisiting, and the three biases keep running the whole time. The sentences do not switch them off. They make the room notice when a bias is making the decision, and that is usually enough to get the actual comparison onto the whiteboard.

These three traps and the hype-cycle chain are Chapter 5 of Cognitive Bias for Software Engineers. Chapter 6 applies the same lens to debugging, starting with confirmation bias and the habit of holding several hypotheses at once. It is on Kindle in English, Japanese, and Portuguese.

Sources (all checked 2026-10-19):

Stack Overflow Developer Survey 2025: Technology: web frameworks, all respondents (React 44.7%, Angular 18.2%, Vue.js 17.6%)
W3Techs: React vs. Vue.js vs. Angular: share of all websites (React 6.0%, Vue.js 0.6%, Angular 0.2%), W3Techs data of 19 October 2026
GitHub REST API, gh api repos/<owner>/<repo>, 2026-10-19: apache/echarts 67,543 stars, chartjs/Chart.js 67,746, d3/d3 113,767
He, Yang, Burckhardt, Kapravelos, Vasilescu, Kästner, Six Million (Suspected) Fake Stars in GitHub: A Growing Spiral of Popularity Contests, Spams, and Malware, ICSE 2026 (doi:10.1145/3744916.3764531)
npm downloads API, last week (2026-10-10 to 2026-10-16): vitest 76,554,353, jest 26,009,195
Norton, Mochon, Ariely, "The IKEA effect: When labor leads to love", Journal of Consumer Psychology (2012)
Wikipedia: Concorde and Sunk cost

Claude, Codex, Kimi K3, Qwen: I graded their web search 120 times

Ken Imoto — Fri, 16 Oct 2026 13:00:02 +0000

The Kimi API usage in this post was paid for with credits Moonshot AI provided for evaluation. Moonshot AI had no part in designing, running, or concluding the test.

Four months ago I wrote that my own AI agents search Brave, not Google. Like most people, I thought about AI search in terms of which search engine sits underneath.

This time I reread the official docs to compare web search in Claude, Codex, Kimi K3 and Qwen. That framing no longer fits. None of the four vendors names its search engine. OpenAI makes its own maintained index the default in Codex. Kimi now sells search itself as a standalone API. The question has moved from "which engine?" to "where does search run, and who calls it?" I did not expect that much change in four months.

If the mechanics changed, the results should too. Before measuring, I assumed live search would beat a cache whenever the question needs fresh information.

Codex CLI's web search is not live by default. It runs in "cached" mode against an index OpenAI maintains. The docs give the reason: it lowers prompt injection risk (Codex: Web search). I read that as trading some freshness for safety.

The measurements said the opposite. I asked the same 8 questions 3 times under 5 setups: Claude, Codex live, Codex cached, Kimi K3 and a local Qwen. Only cached search got every answer right. It even found the title of an article that had been online for about three hours.

The other surprise: the same search API produced 21 correct answers with one model and 9 with another. I gave Kimi K3 and a local Qwen identical access to Kimi's search API. The results were the same. How each model read them was not.

Where web search runs differs by vendor

Before comparing anything, I checked the mechanics in public docs. The biggest difference is where the search actually runs.

	Claude	Codex / OpenAI	Kimi (Moonshot AI)	Qwen
Where search runs	Anthropic's side (server tool)	OpenAI's side (hosted tool)	New API: your code calls it and passes results to the model	Alibaba's side (DashScope). A local Qwen has no search
Freshness control	None	Codex defaults to cached; `--search` for live	Time window on Pro search	7-365 day window
Price per search	$0.01	$0.01	$0.002-0.003	Varies by region
Search engine	Not disclosed	Not disclosed ("OpenAI-maintained index")	Not disclosed	Not disclosed

None of the four names its search engine officially. For Claude, a March 2025 report said Brave Search had been added to Anthropic's list of subprocessors (Simon Willison). Today's docs only say "Anthropic's web search backend."

Claude and OpenAI run the search on their own servers and hand the results to the model. With Claude, results come back encrypted, and you must send them back unchanged to continue the conversation (Claude: Web search tool).

Kimi is mid-transition. The built-in $web_search tool retires on 2026-10-20. Its replacements are standalone REST APIs for search and page fetch, such as /v1/tools/search (Kimi API: Web Search). With the new APIs, you decide when and how often to search. They cost a third to a fifth of the others, billed only when a call succeeds and returns results (Kimi: pricing).

Qwen the model has no search of its own. Alibaba Cloud's API offers search, but a Qwen running locally can only look things up if you wire in an external search tool.

Two details in the docs you will want to know

Claude Code's WebSearch returns only titles and URLs. It does not read the pages. Reading happens in WebFetch, which summarizes the page with a separate model call first. The docs call this "lossy by design" (Claude Code: Tools reference)
Qwen's search on Alibaba Cloud silently skips searching above 15 requests per second. There is no error; the search just does not happen. The FAQ also says English questions often fail to trigger search, and recommends asking in Chinese or forcing search. Both notes appear only in the Chinese docs (阿里云: 联网搜索)

I hit the "lossy" part myself while researching this post. I had Claude Code's WebFetch read the Qwen docs, and it reported finding neither the 15 RPS note nor the English FAQ. Running grep over the raw HTML (about 460 KB) found both. Once a summary sits in the middle, "it doesn't say that" may only mean "the summary dropped it."

How I measured

Five setups, the same 8 questions, 3 runs each: 120 graded answers in total, collected on the evening of 2026-10-14.

Setup	What it is
Claude	Claude Code (`claude -p`) with WebSearch / WebFetch, Opus 5.5
OpenAI live	Responses API `web_search`, gpt-5.6-terra (my Codex CLI's default model)
OpenAI cached	Same, plus `external_web_access: false`
Kimi K3	kimi-k3 with my own search function backed by Kimi's `/v1/tools/search`
Local Qwen	qwen3.5:35b-a3b on Ollama with the same search function

For Codex, I called the API the CLI uses under the hood. In Codex's source, cached mode maps to external_web_access=false and live to true (codex hosted_spec.rs). So the API calls reproduce the CLI's two modes.

These were the 8 questions. I only used questions whose answers I could verify from primary sources or my own site.

#	Question	Answer	What it tests
q1	Kimi K3 API release date	2026-07-16	News after the training cutoff
q2	Kimi K3 open-weights release date (asked in English with "release date")	2026-07-27	The "date" trap
q3	Release date of a Japanese book, MCP実践セキュリティ (Impress)	2026-10-16	Japanese-language sources
q4	Japanese title of a Kimi K3 article posted on kenimoto.dev that day	Kimi K3の動画レビュー、Claudeと互角	Freshness: a page a few hours old
q5	Date the MCP 2026-07-28 spec release candidate came out	2026-05-21	A dated fact
q6	Retirement date of Kimi's `$web_search`	2026-10-20	A detail buried in docs
q7	Kimi K3 total and activated parameters	2.8T / 104B	Numbers
q8	Kimi K4 release date	Not announced	Does it invent something that doesn't exist?

q3 is my own book and q4 is my own article, chosen because I know the answers for certain. That choice carries my own bias, and I'd rather say so. Both are in Japanese, so they also test how well each setup reaches Japanese-language sources.

Results

Setup	Correct (of 24)	Partial	Wrong or no answer	Median time	Searches (avg)
Claude	21	1	2	22.0 s	3.5
OpenAI live	22	1	1	5.7 s	2.3
OpenAI cached	24	0	0	6.2 s	2.7
Kimi K3	21	0	3	25.9 s	4.0
Local Qwen	9	2	13	31.1 s	1.7

Correct answers per question, out of 3 runs:

Setup	q1	q2	q3	q4	q5	q6	q7	q8
Claude	3	2	1	3	3	3	3	3
OpenAI live	3	2	2	3	3	3	3	3
OpenAI cached	3	3	3	3	3	3	3	3
Kimi K3	3	3	3	0	3	3	3	3
Local Qwen	0	2	0	0	0	2	3	2

On the nonexistent Kimi K4 (q8), every setup said it hasn't been announced. Qwen once added, without a source, that K4 had been "mentioned as part of plans for 2027 or later."

Finding 1: the cache wasn't stale

Cached search answered all 24 correctly. The q4 article went live a little after 5 p.m. that day, and I measured about three hours later. Cached search returned its title in all three runs. Honestly, that surprised me. At least for my site, "an index OpenAI maintains" was refreshed within hours.

Source quality also favored the cache at times. On q6 (when $web_search retires), cached search cited Kimi's official docs. Live search got the same answer but, in one run, cited an unofficial roundup site. A wider reach also means picking up more unofficial pages.

Live search missed q3, the Japanese book, once, saying it couldn't confirm a book by that title.

Finding 2: same results, different answers

Kimi K3 and the local Qwen got exactly the same search API in exactly the same way. They still scored 21 and 9. That's more than double.

Qwen's mistakes followed a few patterns:

Its belief that it's 2024 beat the search results. On q4 it said: "As of now (May 2024), an article dated October 14, 2026 does not exist. That is a future date." On q1 it found the July 16 date, then concluded it was "not released as of 2025"
Its sources and its answers disagreed. On q5, all three runs cited the release-candidate announcement dated May 21, then answered with the final release date, July 28
It invented a source. On q3 it cited an Impress press release URL that doesn't exist (404) and gave a wrong release date. Each run gave a different wrong date
It barely searched. 1.7 searches on average, less than half of K3's 4.0

Model size differs, so this isn't a verdict on Qwen as a whole. What it does show is that web search accuracy is not decided by the search API alone.

Finding 3: Kimi's search API has two gaps

All three of K3's misses were q4. It rephrased the query 12 times and never found the same-day article. The page simply wasn't in Kimi's search index yet. The cause wasn't the model. It was index freshness.

The second gap: queries containing the site: operator return HTTP 502. Both K3 and Qwen tried site:kenimoto.dev when hunting for a specific page. Each time, the search API returned a non-JSON error page, and on the first run it crashed my script. My tool now returns the failure to the model so it can rephrase and try again.

Hitting the search API directly also showed how much wording matters. Searching "Kimi K3 release date Moonshot AI" let "date" take over: I got a time-zone site and JavaScript Date docs. Searching just "Kimi K3" put the official site, the paper and GitHub on top. None of the models fell into this trap during the actual runs, though. Both K3 and Qwen rephrased their own queries around it.

By the way, the soon-to-retire $web_search tool didn't work with K3. The second call, the one that returns search results, failed every time with 400 tokenization failed. Passing search through regular function calling, as in this test, works fine with K3 (more in Kimi K3 vs Claude: Video Review, Same Findings).

Claude struggled with a Japanese book

On q3, Claude said in 2 of 3 runs that it couldn't find the book and that it may not exist. The book is listed, with its release date, on the publisher's NextPublishing page and on Rakuten Books.

It was persistent elsewhere, though. In one q4 run, WebSearch didn't surface the article, so Claude found it through the sitemap and opened the page to confirm the title and date. Because WebSearch returns only titles and URLs, fetching pages when needed is part of the design, and here it paid off. Claude also often noted that it had only checked secondary sources, and avoided firm claims.

The 22-second median includes Claude Code's agent loop. Calling the search tool directly through the API should be faster.

How I'll use them

Here's what I'm taking from this:

Leave cached search as the default for research. If you use Codex, there's no need to pass --search every time. The safer setting didn't lose on accuracy
Don't trust "not found" for non-English sources. Not found is not the same as not existing. When Claude says so, I check another way
If you wire in search yourself, the model matters more. With the same search API, K3 and Qwen differed by more than 2x. Pick a model that can read search results before picking the cheapest search API
Make your own search tool return failures. Kimi's search API returns 502 on site:. Pass the failure back instead of crashing, and the model will rephrase and search again

Limits

This is a small test: 8 questions, 3 runs. Topics lean toward Kimi K3 and MCP, and q3 and q4 are my own book and article
Claude ran through Claude Code; Codex results come from calling the API the CLI uses
The Qwen setup is a local 35B model wired to Kimi's search, not Qwen's official search
Each setup works differently, so times aren't directly comparable. I didn't total the costs in this test

Next, I want to call Claude's search tool directly through the API and try Qwen's official search on Alibaba Cloud, to close the gaps between setups.

References

Claude: Web search tool: pricing, citations, encrypted results, versions
Claude Code: Tools reference: how WebSearch and WebFetch behave
Codex: Web search: cached / live modes
OpenAI: Web search tool: external_web_access
openai/codex hosted_spec.rs: how the modes map to external_web_access
Kimi API: Web Search / best practices / pricing
阿里云百炼: 联网搜索 (Chinese; the 15 RPS note and the English-question FAQ appear only here)

Kimi K3 reads video natively. Claude still caught the same issues

Ken Imoto — Thu, 15 Oct 2026 13:00:02 +0000

Moonshot AI provided the Kimi API credits used here ($50) for evaluation. They had no part in designing, running, or concluding the test. Everything K3 couldn't do, and the features that didn't work, are in this post too.

Moonshot AI's Kimi K3 takes video files straight through the API. Claude has no video input, so when I want Claude to look at a video, I cut it into frames first. Codex CLI doesn't have a video channel either, even when you point it at K3 (Kimi API: Use K3 in Codex).

I wanted to know how much that difference matters in real work. My test material was a pair of 40-50 second hands-on videos I made for the SSH series on kenimoto.dev /learn/ (Japanese).

The short version:

K3 and Claude agreed on what was wrong. Where viewers would drop off, the six fixes, where to insert diagrams, even which video to post first: all the same
The differences were timestamp precision and effort. K3 pinpointed frozen-screen intervals to fractions of a second, and all I did was hand it the file. Claude estimated from frames sampled every 2 seconds, so its timestamps drift by about ±1 second
K3 doesn't understand audio. It can't hear the narration or catch audio sync problems
Claude was faster. The same request took K3 363 seconds and Claude 185

The test material: auto-generated hands-on videos

The videos come from handson-cast, a CLI I'm building. You give it a JSON file of steps. It runs each command for real inside Docker containers while vhs records the terminal, then adds narration from OpenAI's text-to-speech and burns in subtitles. It's Python and ffmpeg underneath. The narration and subtitles are in Japanese.

I made two videos:

Key authentication (55 s): create a key, install the public key, confirm key-only login works, then disable password auth
Bastion host (47 s): the same key gets rejected by the bastion with ssh -J, but works through ~/.ssh/config

Every line of terminal output was produced by OpenSSH 10.0p2 running in a container. Nothing is staged. When the rejection message changes from (publickey,password) to (publickey), that really happened on camera.

The first cut (v1) was not watchable, though. The first second shows me typing an internal wrapper script, bash /lab/ssh-config.step-01.sh. You can't tell what the video is about. (I made it, and I couldn't tell either.)

Here's v1 (47 s, with audio):

Click the image to open the mp4 (with Japanese narration).

I showed v1 to K3 and Claude and asked what was wrong with it.

Sending video to K3, and what it costs

The K3 API is OpenAI-compatible. You upload the video as a file first, then reference the returned ID with an ms:// prefix.

from openai import OpenAI
client = OpenAI(api_key=KEY, base_url="https://api.moonshot.ai/v1")

f = client.files.create(file=Path("learn-ssh-key-auth.mp4"), purpose="video")
r = client.chat.completions.create(
    model="kimi-k3",
    max_tokens=32000,
    messages=[{"role": "user", "content": [
        {"type": "video_url", "video_url": {"url": f"ms://{f.id}"}},
        {"type": "text", "text": "Why would viewers drop off in the first 3 seconds?"},
    ]}],
)

Public URLs aren't accepted. The official docs describe video as a set of key frames, and higher resolution means more tokens (Kimi API: Vision).

My first measurement used one 55-second, 1920×1080 video:

Item	Measured
Input tokens	40,659 (almost all video)
Output tokens	2,500 (1,912 of them reasoning)
Cost	about $0.16
Time	83.5 s

Costs use $3 per million input tokens and $15 per million output tokens.

Two practical tips from actually running it:

Assume it can't hear anything

For this run I made a version without burned-in subtitles and asked K3 to transcribe the last sentence of the narration. It answered that it couldn't recognize any audio. Not pretending to hear something it can't is exactly what you want from a reviewer. But it means K3 can't see narration content or subtitle-audio drift. It sees the screen and any subtitles burned into it, nothing else.

It reads the screen well, and accurately. It listed the commands and output for all four steps in order, including the 600 / 644 permissions, the change in the rejection message, and an on-screen comment. The only misread was the wrapper script's filename.

Give it a generous output budget

K3 always reasons before it answers. With max_tokens at 300, the reasoning used everything up and the answer came back empty. It thought hard and then left without saying a word. At 8,000, a full video review got cut off in the middle of the first question. For long answers like reviews, I use 32,000.

The comparison setup

K3 and Claude each reviewed the same two v1 videos, under matched conditions:

	K3	Claude
How the video goes in	The file, once	Frames extracted and read as images
Audio	Not understood	Not used (transcription turned off)
Narration script	Step JSON attached	Same JSON attached
Questions	Same wording (5 questions)	Same wording (5 questions)

The Claude side ran as a subagent that never saw K3's answers. I had already read them, so if I'd done the review myself, I would have been anchored. That's not a comparison.

For frame extraction I used /watch from claude-video, and left the frame count and resolution up to Claude. It ended up re-extracting 4 times. For the opening it pulled frames at 0 / 0.5 / 1 / 2 / 3 seconds, then one every 2 seconds for the rest: 34 frames in total.

The five questions:

Where would viewers drop off in the first 3 seconds?
Up to six improvements, biggest impact first (what to change / why it works / which pipeline stage implements it)
If you combined the article's diagrams with the terminal recording, which diagram goes where, for how long?
Which video should go to social media first?
What's already good?

Result: nearly identical findings

They mostly matched. Lined up side by side, the answers overlap almost completely:

Finding	K3	Claude
Seconds 0-2 show the wrapper script being typed; the topic is unclear	✓	✓
The first narration line introduces the site, not a reason to watch	✓	✓
`Could not resolve hostname` in the bastion video is a strong hook, buried	✓	✓
Open with the conclusion or a question (cold open)	✓	✓
Cut the typing and the waits between steps	✓	✓
Bigger font, highlight the key output lines	✓	✓
Make a vertical version	✓	✓
Insert the article's diagrams: permissions at `ls -l`, the before/after at the climax	✓	✓
End on the article's OG image	✓	✓
Post the bastion video first	✓	✓
Show a progress bar along the top	✓
Rewritten narration lines, as actual sentences		✓
The four `ssh-copy-id` INFO lines are noise		✓

The fixes themselves are standard short-video advice: open with the conclusion, cut silence, keep subtitles short, put the call to action at the end. Either model will give you that. This comparison didn't surface anything only K3 could find. I expected a model that reads video directly to catch something Claude couldn't see. It didn't.

Where they differed: timestamps

The biggest gap was in how precisely each one placed things in time.

K3 wrote things like "the first meaningful output appears at 3.7 seconds" and "the error at 00:03.2 in the bastion video gets buried," sub-second throughout. When I later had it review an improved version, it came back with intervals: "from 30.5 to 36.1 seconds the screen is frozen after the output appears."

Claude's timestamps were estimates from frames 2 seconds apart, and Claude said so at the end of its review: interval boundaries could be off by about ±1 second. You can sample more densely, but then there are more images to read. It's a trade-off.

"Where exactly is the screen frozen?" is hard to answer from a handful of stills. That's where reading the video directly paid off.

Effort and speed

	K3	Claude
My effort	Hand over the file	Decide how to sample frames (Claude redid it 4 times)
Time	363 s	185 s
Size	84,461 input / 10,863 output tokens, about $0.20 (part of the input billed at the cache rate)	111,313 tokens, 40 tool calls

K3 took less effort; Claude was faster. Each has its edge. Because K3 spends time reasoning, each review took 6-8 minutes. Enough time to make a coffee and come back.

The videos really did get better

I made the fixes. Following the reviews, I added six changes to handson-cast:

Open with a card: "Same key, but ssh -J gets rejected. Why?"
Cut the wrapper-script typing and the waits between steps
Type commands out character by character, with a larger font
Right after the rejection, show the article's diagram full-screen for 4.5 seconds
Once the output is complete, slowly zoom into the top-left of the terminal
End with the article's OG image and URL

Here's the final cut (v4, 41 s, with audio):

Click the image to open the mp4 (with Japanese narration).

Frames from the same timestamps, side by side:

The bastion video went from 46.8 to 41.0 seconds, about 6 seconds shorter. I had K3 grade the improved cut against a 10-item checklist for technical short videos. It covers things like a hook in the first 3 seconds, something changing on screen every 1-3 seconds, and a short call to action only at the end. It scored 7 pass, 2 borderline, 1 pass (estimated). The two borderline items were a frozen stretch after the output appears and an overlong subtitle. Both are fixed now.

I'd meant to have K3 research that checklist too, but that didn't work. The Kimi API has a built-in web search tool ($web_search). With K3, though, the second call, the one that returns the search results, failed every time with 400 tokenization failed. With kimi-k2.6 the first search went through, but after that it stopped searching and wrote text that only looked like search calls into its answer. The docs say this feature is scheduled for retirement on 2026-10-20 (Kimi API: Web Search). I gave up. Claude's web search did the research, and I passed the results to K3.

How I'll split the work

Based on this, here's how I plan to use them:

K3 when I need to know where in a video something happens. Frozen intervals, the moment the screen gets cut off, how long a caption stays up: questions whose answer is a span of time. I don't have to think about how to sample frames
Either one for deciding how to fix it. The improvement ideas didn't differ
Not K3 for anything involving audio. For narration content or sync, I prepare a transcript separately and compare against it
No confidential screens. The video goes to Moonshot AI's servers. These videos were going public anyway, so that was fine; internal screen recordings need different handling

It's a small comparison: two videos, one set of questions, nothing statistical. Still, it gave me a working answer. Reading video directly didn't change the judgment; it changed timestamp precision and effort. Next I want to plant deliberate defects in a video and count how many each model finds.

References

MoonshotAI/Kimi-K3 (GitHub): model card. 2.8T total parameters, MoE with 104B active per token, 1M context
Kimi API: Vision models: how video input works
Kimi API: Web Search: $web_search and its retirement date
bradautomates/claude-video: the /watch tool used for Claude's frame extraction
charmbracelet/vhs: terminal recording
SSH: five ways to reach a remote machine, compared on seven axes (Japanese): the /learn/ series these videos were made for

Shai-Hulud: 796 Packages, 1 Yes-Mash, and the Claude Code Auto-Approve Risk

Ken Imoto — Wed, 14 Oct 2026 13:00:01 +0000

On November 24, 2025, JFrog Security Research disclosed that a self-replicating npm worm had quietly backdoored 796 unique packages. The industry called the second wave Shai-Hulud 2.0, or "The Second Coming." It followed the September 16, 2025 original wave, which had already poisoned over 1,150 packages (JFrog, Datadog Security Labs).

Over 25,000 GitHub repositories had credentials leaked by the campaign (Sysdig). The 796 is the count of poisoned npm packages. The 25,000 is the count of victim repositories whose tokens and secrets the worm harvested; the attacker-side drop points were separate public repos (random 18-character names, description Sha1-Hulud: The Second Coming) created on victim accounts.

The infection vector was the same thing every developer does fifty times a day. Run a package install. Hit Enter.

Now add Claude Code, or any other coding agent, with "always allow" turned on for npm install. The fifty-times-a-day moves to 500-times-a-day. The attacker now needs one of those 500 to land on a compromised version.

That is the article.

What Shai-Hulud 2.0 actually did

The worm's loop was simple:

Infect. A victim ran npm install on a poisoned version of a popular package. The payload fired as a pre-install hook. (V1 used post-install; V2 moved it earlier, so it ran even on failed installs.)
Steal. The payload scanned the victim's machine for ~/.npmrc tokens, GitHub tokens from ~/.config/gh/, SSH keys, AWS and GCP credentials, and loose .env files. Credentials were exfiltrated by creating a public GitHub repository on the victim's account (description stamped Sha1-Hulud: The Second Coming) that acted as the drop point.
Self-replicate. Using the stolen npm token, the worm published poisoned versions of every package the victim had publish rights to. The victim became the next publisher.
Persist. In some cases it enrolled the victim's host as a self-hosted GitHub Actions runner named SHA1HULUD, so the attacker could execute code on the machine later (Netskope, Zscaler).

Three details matter for the agent-usage angle:

The payload ran on install, not on first use. There was no runtime signal.
The exfiltration channel was public GitHub, not an exotic webhook. Egress filters that block weird DNS wouldn't help.
The runtime was Bun, not Node. Signature-based scanners trained on Node payloads lagged (Endor Labs).

By the time npm removed the compromised versions, the token mesh was already seeded. The affected packages had roughly 132 million monthly downloads between them.

Why "always allow" is the exploited behavior

Claude Code's permission prompt is the exact friction Shai-Hulud needs you to remove.

Default behavior: Claude Code asks before running npm install <pkg>. You read the package name, you decide. One prompt per install. Annoying when you're installing twelve dependencies for a Next.js app, which is why /permissions exists and why most Claude Code users have Bash(npm install:*) on "always allow" by their third week. Mine was on by week two. I know this because I checked after reading the JFrog post.

The attacker's threat model relies on this. The payload does not need to defeat your judgment. It needs you to never be asked.

The equivalent risk on other agents:

MCP servers with auto-approve: true for all tools, so a server you installed last month can shell out and you would never see it.
Codex, Aider, or Cursor running under --yes or an auto-run shell wrapper.
CI: a bot account running npm install without lockfile integrity, auto-publishing on merge.

In every case the pattern is the same. A human decided once, early, then stopped looking. The worm does not need to be smarter than Claude Code. It needs the human in the loop to have left the loop.

A defensive checklist that isn't "write a CLAUDE.md"

CLAUDE.md helps against prompt-injection and model behavior. It does not stop a signed, published, poisoned package from running exec in a pre-install hook. By the time install scripts fire, the model is no longer in the loop.

The checklist below targets the install surface itself.

Per-session, before you say "always allow":

Run /permissions and read what is on always-allow right now. If Bash(npm:*) or Bash(*) is on the list, that is your exposure.
Scope allow rules to specific commands, not globs. Bash(npm ci) or Bash(npm install --package-lock-only) is survivable; Bash(npm install:*) is not.
For MCP: review auto-approve arrays per server. New MCP servers get a sandbox project first, not production.

At the install layer:

Pin everything. package-lock.json + npm ci + --ignore-scripts on CI where you can. Install scripts are the exact Shai-Hulud entrypoint.
Flip --ignore-scripts on globally in local dev (npm config set ignore-scripts true), then whitelist the handful of packages that legitimately need scripts.
Verify the registry you resolve from (npm config get registry) and the integrity hashes in your lockfile. The worm published to the real npm registry, so this does not save you from the attack itself, but it does catch tampering between your lockfile and your node_modules.

At the detection layer:

Socket, Snyk, or GitHub Dependabot wired into every repo. Socket in particular flags behavioral risks like install scripts and unusual network calls, and was fast on both Shai-Hulud waves.
Token hygiene. ~/.npmrc and ~/.config/gh/ are what the worm wants. Rotate npm tokens quarterly, use npm token create --read-only for CI, scope GitHub PATs down to the minimum.
Watch your account for the Nov 2025 exfiltration markers: public repos with randomly-generated 18-character names and description Sha1-Hulud: The Second Coming, workflow files at .github/workflows/discussion.yaml or formatter_*.yml, and self-hosted runners registered as SHA1HULUD. (The V1 wave in Sept 2025 used repos literally named Shai-Hulud and a workflow at .github/workflows/shai-hulud-workflow.yml.)

At the agent layer:

Do not leave always allow on across restarts for Bash, WebFetch, or anything that writes files outside the project. Per-session is survivable, forever is not.
Have the agent list the package name and the latest publish date before any install. A fresh version of a popular package deserves a human look, even when you are tired.

None of these are novel. All of them are standard. What Shai-Hulud 2.0 showed is that the "I'll just hit Enter" discount was always being subsidized by someone else's attention budget. The second wave took the subsidy away.

Trust, but verify

Reagan used it on arms control. It fits here. Claude Code is in the loop. Your other agents are in the loop. Both are useful. Neither replaces the one-second pause before you widen a trust boundary.

The industry has, so far, caught Shai-Hulud waves inside 48 hours of publish (CISA alert, Trellix). The vulnerability window is small, but it is the exact window your "always allow" paves over.

For a longer walk-through of Claude Code security patterns, including CLAUDE.md templates and data-retention policy per plan, I pulled these together in Practical Claude Code on Kindle.

Deepgram Nova-3 + Cartesia Sonic 3 + GPT-4o: Where 750ms of Voice AI TTFB Actually Hides

Ken Imoto — Tue, 13 Oct 2026 13:00:00 +0000

Voice-AI conversations die the moment the agent takes too long to say the first word. Not the full answer. The first byte of audio.

The industry calls this TTFB (time to first byte), or TTFA (time to first audio). I measure it from the moment the user stops speaking to the moment the first audio frame plays. Everything else, including total answer length and follow-up quality, happens after the user has already decided whether the agent feels alive.

Below is a ~750ms streaming budget on a Deepgram Nova-3 + GPT-4o + Cartesia Sonic 3 stack, broken into what each hop actually costs. And where almost every vendor-quoted number stops matching the number you will measure in production.

1,200ms to 750ms is the easy part

Batch pipelines wait for the full response before TTS starts:

[User stops speaking] → STT → LLM (full answer) → TTS (full) → [play]
                                                                 ^ >1,200ms

Streaming pipelines hand sentence 1 to TTS the moment the LLM emits it, while sentences 2 and 3 are still generating:

[User stops speaking] → STT → LLM (sentence 1) → TTS (sentence 1) → [play]
                               LLM (sentence 2) → TTS (sentence 2) → [continue]
                               LLM (sentence 3) → TTS (sentence 3) → [continue]

On the stack above, the streaming budget breaks down roughly like this:

Hop	Time	Notes
STT (Deepgram Nova-3 or Flux)	350ms	includes end-of-turn detection on Flux
LLM first sentence (GPT-4o)	150ms	first-token latency, best-case
TTS first chunk (Cartesia Sonic 3)	170ms	first audio frame
Network round-trips	80ms	user ↔ server + server ↔ vendor
Total (streaming TTFB)	~750ms

The 1,200ms to 750ms move pays for itself: no code change beyond "do not wait for the full response." The hard part is everything below 750ms.

STT: Nova-3 is 300ms, Flux adds end-of-turn for about 260ms

Deepgram's public numbers, cross-referenced with Softcery's 2026 comparison:

Deepgram Nova-3: streaming latency under 300ms, 36 languages. General ASR; does not detect end-of-turn.
Deepgram Flux: around 260ms end-of-turn, English only. Flux folds ASR together with turn detection into a single model and emits StartOfTurn, EagerEndOfTurn, EndOfTurn events (Deepgram launch post).

The combination matters. Nova-3 transcribes, but you still need a separate endpointing step to know when the user is done. Flux folds endpointing in, which cuts a 400-500ms round-trip, but costs you multi-language support.

For reference across other vendors: OpenAI gpt-4o-transcribe is around 320ms with 100+ languages. AssemblyAI Universal-3 Pro Streaming is under 600ms. ElevenLabs Scribe v2 Realtime is around 150ms with 90 languages.

Each of these is a vendor-quoted number. None of them is what you will measure from Tokyo over LTE to a US-East endpoint.

LLM: GPT-4o claims 200-400ms, measured 719-1,608ms

This is the hop where the vendor-vs-real gap gets ugly.

Deepgram's own blog post listing reasons they chose GPT-4o for voice agents cites a first-token latency of 200-400ms. The same post's measured table, on a MacBook M1 Pro over Wi-Fi, reports GPT-4o first-token times of 719ms to 1,608ms. That is 2x to 5x the vendor's number, on a laptop, on consumer Wi-Fi.

A separate Dev.to benchmark from CloudX tested 30+ stacks and found GPT-4.1-nano landing at 0.30-0.40 seconds on 2nd-turn-onward, where the KV cache is warm. First turn is always worse.

What the numbers teach: first-token latency is dominated by TTFT variance, which is dominated by model size, GPU type, quantization, input length, KV cache hit rate, inference backend load, and cold starts. A table of "model X equals N ms" is strictly less useful than one measurement from your own region to your own endpoint.

Running a small LLM locally removes the LLM-side network round-trip entirely. On a tuned setup, that is where sub-500ms TTFB actually lives.

TTS: Cartesia Sonic 3 says sub-90ms, measured median is 166-190ms

Cartesia's two models get swapped in half the voice-AI blog posts:

Cartesia Sonic 3: sub-90ms (vendor claim, measurement definition undisclosed), 40+ languages, emotion support.
Cartesia Sonic Turbo: 40ms (vendor claim, speed-specialized model), 9 languages.

The widely-circulated 40ms number is Sonic Turbo, not Sonic 3. Real Sonic 3 measurements:

Cartesia's own launch disclosure: median around 190ms.
Vapi's Humanness Index (50-run test, round-trip included): median 166ms.

Both numbers are roughly 2x the vendor-quoted sub-90ms. Neither 166 nor 190 is "wrong." They measure different things: vendor model-inference vs client-side round-trip.

ElevenLabs Flash v2.5 quotes around 75ms, with their own docs noting that 75ms refers to model inference only, and that end-to-end latency will vary with your location and endpoint type (ElevenLabs latency docs).

Research on streaming TTS reaches sub-50ms TTFA (arXiv 2508.04721). But the same paper's own end-to-end telecommunications voice agent, with the LLM in the loop, measured 678ms average TTFA. The TTS was fast. The stack still was not.

The 3-level optimization ladder

Treating these numbers as a budget, the ceiling moves in three steps.

Level 1, 500ms target: fast TTS + sentence-level streaming + streaming STT. This is the "do not wait for the full response" move above; gets you from 1,200ms+ to roughly 500-750ms.

Level 2, 300ms target: Level 1 plus Flux-style end-of-turn detection, speculative LLM invocation on partial ASR, a smaller LLM, and a TTS whose measured (not quoted) first byte is under 100ms. The emphasis is on measured.

Level 3, 200ms target: edge inference, a unified STT+LLM+TTS model, hardware tuning. At this point you stop composing vendor APIs and start building.

The 300ms figure is not arbitrary. It is the UX cliff below which a voice agent starts feeling like a human conversation partner, as opposed to a very fast IVR. Everything above 500ms feels like an echo.

Three rules before you trust any TTFB number in a blog post

Was the network included? ElevenLabs says the quiet part out loud (75ms is model inference only). Most vendor numbers do not.
Which model? Cartesia's 40ms is Sonic Turbo; the Sonic 3 you probably want is 2x that.
Who measured it? Vendor, third-party, or user-run on their own hardware? Each of the three routinely disagrees by a factor of 2.

Build your own measurement. Pick the stack above, or three alternatives on each row of the table, point at your actual region, record a 50-run distribution. The p95 is the number you should budget against. The vendor's median is marketing.

The full chapter this article compresses, including the TTS vendor-vs-measurement table and the 3-level ladder in more detail, is in The 300ms Threshold on Kindle. The book is built around voice-AI UX cliffs at 300, 500, and 800ms, and how to budget inside them.

Kernighan, Fowler, Torvalds: 7 Debugging Quotes Every Senior Still Repeats

Ken Imoto — Mon, 12 Oct 2026 13:00:00 +0000

Some engineering lines refuse to fade.

They survive because every senior ends up repeating them in code review, year after year, usually on a Thursday afternoon when a junior ships something that works and shouldn't.

I spent the last few months decoding a hundred of these for a short Kindle book. Below are seven, from three names that keep surfacing when the topic is debugging: Brian Kernighan, Martin Fowler, Linus Torvalds. Each quote has its original public source. Each one has a modern code-review move I use when I quote it.

1. "Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it?"

Brian Kernighan & P. J. Plauger, The Elements of Programming Style, 2nd ed., ch. 2, 1978.

Half a century later, this is the single question I ask when a PR shows up with a one-line regex, a nested ternary, or a reduce that collapses six operations into one line. Not "is it correct." How will you ever debug it?

The AI-pair-programming era rediscovers this one every quarter. Give Claude Code or Copilot a prompt like "make this shorter" and it will happily compress four lines into one with a currying trick. The code works. Six months later, nobody (human or model) can rewrite the same function from scratch, because the compressed form hides the steps.

My review rule: if the author needs more than one sentence to explain what the line does, split it. I know, because I've been that author more than once.

2. "The most effective debugging tool is still careful thought, coupled with judiciously placed print statements."

Brian Kernighan, Unix for Beginners, 1979.

I used to be embarrassed by println debugging. Then I watched enough senior engineers reach for it before a debugger that I stopped apologizing.

The quote still holds, with one update. The tool today is console.log, fmt.Println, print(), or whatever your runtime offers, plus a tail of structured logs. Debuggers help when you can reproduce the bug and the state is static. Print statements help when the bug lives in a flow, in production, or at a boundary the debugger can't easily cross: gRPC, serverless cold start, worker queues, flaky async tests.

The actual skill the quote names is careful thought. The prints are only useful once you've guessed where to put them.

3. "Controlling complexity is the essence of computer programming."

Brian Kernighan & P. J. Plauger, Software Tools, p. 319, 1976.

A junior and a senior will both call code "complex." The junior usually means "I don't understand this yet." The senior usually means "nobody will ever understand this again, including the person who wrote it."

I keep this quote pinned when reviewing refactors. The question isn't "is this clever." It's "does the next on-call engineer, at 2 a.m., with no context, have a chance?" Most production bugs I've chased started as a cleverness-driven refactor that added coupling the diff didn't name.

Complexity accrues. Most PRs make it worse. The ones that make it better are rare and should be celebrated.

4. "Any fool can write code that a computer can understand. Good programmers write code that humans can understand."

Martin Fowler, Refactoring: Improving the Design of Existing Code, p. 15, Addison-Wesley, 1999.

This is the one line of Fowler's that engineers who've never read Refactoring can still recite. Fair, because it compresses the whole book: tests exist so humans can change code; names exist so humans can read it; small methods exist so a human can hold one in their head.

The habit I lifted from this: whenever I'm tempted to leave a comment explaining what a block does, I rename or split instead. If the code reads without the comment, the comment was a smell.

LLM code generators love long comments. Rename the function.

5. "If it hurts, do it more often."

Martin Fowler, martinfowler.com, FrequencyReducesDifficulty.

Fowler surfaces this as one of his favorite soundbites. The surface reading sounds like masochism. The real reading: if deploys are painful, deploy daily; if merges are painful, merge every hour; if integration tests are painful, run them on every push.

The pattern survives the current stack. Painful AI-code reviews? Review each PR as soon as it opens, not once a day in a batch. Painful dependency bumps? Automate them nightly instead of quarterly. Painful incident post-mortems? Hold them within 48 hours while people still remember.

Every time I've watched a team avoid a painful activity, the pain compounded. Fowler's rule flips the compounding.

6. "Talk is cheap. Show me the code."

Linus Torvalds, Linux Kernel Mailing List, 25 Aug 2000.

Linus wrote this in the middle of a kernel discussion where someone was proposing changes without an implementation. The adage has outlived its original thread.

Modern application: architecture debates on Slack where no participant has opened the file in question. Design reviews on a tool nobody has prototyped. "We should migrate to X" threads with no benchmark attached. The quote doesn't ban design. It bans debating before anyone has sampled the code. Open a branch. Commit a 30-line proof of concept. Attach the diff to the thread.

I keep a text expander snippet with this quote. I pull it out maybe once a month, and every time it ends the thread in under an hour.

7. "Bad programmers worry about the code. Good programmers worry about data structures and their relationships."

Linus Torvalds, Git mailing list, 2006. (git mailing list via LWN, 2006-07-27)

The underrated debugging quote on this list. Most bugs that reach production aren't in the control flow. They're in the shape of the data: a nullable field someone assumed was present, a one-to-many relationship someone modeled one-to-one, a cache that lies about freshness, a queue that quietly drops ordering guarantees.

When a bug surfaces, the first question I ask is what the data actually looks like right now, and what the code assumed it would look like. More than half the time, the fix is a schema change or a type refinement, and the logic was never wrong. I lost a weekend last year "fixing" logic that was fine — the schema was lying about a nullable column.

Modern echo: in LLM pipelines, the hardest bugs don't live in the prompt. They live in the context layout the prompt silently depends on: what the retriever returned, in what order, with what metadata.

Why these seven, out of a hundred

The engineering world has more quotes than it has reviewed PRs. These seven survive translation across stacks: a 1978 line is still live in a 2026 Claude-Code-generated diff.

If this slice landed, the full set (100 quotes across 10 chapters — code aesthetics, bugs and failure, design, teams, innovation, learning, craftsmanship, open source, the AI era, and an engineer's life) is in 100 Engineering Quotes Decoded. One decoded quote at a time, so you can read a decoding on a coffee break and bring it into code review the same day.

DEV Community: Ken Imoto

4 Vector DBs Compared: Pinecone, Weaviate, Qdrant, pgvector on Enterprise RAG

The 10,000-foot view

Axis 1: Scaling model

Axis 2: Index algorithm (and what you're allowed to tune)

Axis 3: Metadata filtering (pre vs post)

Axis 4: Pricing model

Three decision paths

Why not Milvus, Chroma, LanceDB

What the book covers that this post doesn't

3 UI Slop Patterns in AI UI Generators: Purple Gradients, Inter Fonts, Uniform Cards

Why "modern" means the same thing to every generator

Pattern 1: Purple Gradient Syndrome

Pattern 2: Inter Font Dependency Syndrome

Pattern 3: Uniform Rounded Card Syndrome

The Slop Score that catches all three

What stays recognizable in 2026

AGENTS.md Review Policy: 4 Sections CodeRabbit, Copilot, and Claude Code All Read

Why "review" belongs in AGENTS.md, not in a separate .review.md

Section 1: Review Policy — the Conventional Comments labels

Section 2: Approval Criteria — the four conditions

Section 3: Code Direction — the patterns with reference paths

Section 4: PR Guidelines — the mechanical rules

Wiring AGENTS.md into CodeRabbit (.coderabbit.yaml)

The feedback loop that AGENTS.md unlocks

When one AGENTS.md isn't enough

Conway, Brooks, Parnas: 6 Architecture Quotes Still Deciding Code-Review Fights in 2026

1. Conway, 1968: the quote your multi-agent system is secretly obeying

2. Parnas, 1972: the quote that still kills export statements

3. Brooks, 1975: the quote your "we'll just add another agent" moment deserves

4. Fowler, 1999: the quote your lint rule can't enforce

5. Beck, 1999: the quote that keeps "we'll optimize later" honest

6. Hickey, 2011: the lesser-known quote that keeps winning

Why these six, in this order

5 PR Metrics on 303 Merged PRs from pytest and pydantic: 2 Point at Review Bottlenecks

The script

Problem 1: the average describes five PRs

Problem 2: the metric quietly skips half the PRs

The two metrics that point at review

The three that answer other questions

What I would put on the dashboard

RTX 4070 + logprobs: 96 of 100 Claude Code Requests Needed the Cloud. 97 of 200 Steps Could Go Local

Why request-level routing failed

Which steps go local

The router: rules first

Reading logprobs from Ollama

The threshold is not 0.5

Where the RTX 4070 fits

What I would copy

GitHub Stars Lied: 3 Tech-Selection Traps Senior Devs Still Walk Into

Trap 1: the bandwagon effect, dressed as "community size"

Some of the stars are not people

When the effect gets stronger

What catches it: a requirements matrix

Trap 2: the IKEA effect on "the thing we built ourselves"

The question that defuses it

Trap 3: sunk cost, also known as the Concorde fallacy

The forward-only decision frame

The hype cycle chains all three

Three sentences for the wall

Claude, Codex, Kimi K3, Qwen: I graded their web search 120 times

Where web search runs differs by vendor

Two details in the docs you will want to know

How I measured

Results

Finding 1: the cache wasn't stale

Finding 2: same results, different answers

Finding 3: Kimi's search API has two gaps

Claude struggled with a Japanese book

How I'll use them

Limits

References

Kimi K3 reads video natively. Claude still caught the same issues

The test material: auto-generated hands-on videos

Sending video to K3, and what it costs

Assume it can't hear anything

Give it a generous output budget

The comparison setup

Result: nearly identical findings

Where they differed: timestamps

Why "review" belongs in AGENTS.md, not in a separate `.review.md`

Wiring AGENTS.md into CodeRabbit (`.coderabbit.yaml`)